ignored_paths does not prune the .gitignore discovery walk at startup (35 s on a large tree)

[wallrj]

Preconditions:

• I have made sure it's an actual issue, not a question (use GitHub Discussions instead).
• I have consulted the user guide and verified that the issue cannot be resolved by adjusting configuration/following recommended workflows.
• I have looked for similar issues and discussions, including closed ones.

Issue details:

• I have provided a meaningful title and description.
• I have explained how the issue arose and, where possible, added instructions on how to reproduce it.
• I have added details on my setup: Serena version, MCP client, OS, the programming language(s), relevant configuration adjustments and project specifics.
• If the issue relates to an application of Serena to an open-source project, I have added the link.

Summary

ignored_paths does not prune the .gitignore discovery walk that runs at project activation. On a large tree that walk alone took 35 seconds on every start, even though the directories it spent that time in are listed in ignored_paths. The only way to prune the walk today is a .gitignore file at the project root.

What happens

At activation, Project._gather_ignorespec builds a GitignoreParser with only the project root:

serena/src/serena/project.py

Line 93 in 13ac8c5

gitignore_parser = GitignoreParser(self.project_root)

GitignoreParser._iter_gitignore_files then walks every directory under the root. It skips a directory only when it matches a .gitignore file it has already loaded. It never sees the global or project ignored_paths:

serena/src/serena/util/file_system.py

Lines 237 to 273 in 13ac8c5

	def _iter_gitignore_files(self, follow_symlinks: bool = False) -> Iterator[str]:
	"""
	Iteratively discover .gitignore files in a top-down fashion, starting from the repository root.
	Directory paths are skipped if they match any already loaded ignore patterns.
	:return: an iterator yielding paths to .gitignore files (top-down)
	"""
	queue: list[str] = [self.repo_root]
	def scan(abs_path: str | None) -> Iterator[str]:
	try:
	entries = os.scandir(abs_path)
	except PermissionError as ex:
	log.debug(f"Skipping unreadable directory {abs_path}: {ex}")
	return
	for entry in entries:
	try:
	if entry.is_dir(follow_symlinks=follow_symlinks):
	queue.append(entry.path)
	elif entry.is_file(follow_symlinks=follow_symlinks) and entry.name == ".gitignore":
	yield entry.path
	except PermissionError as ex:
	log.debug(f"Skipping entry due to permission error: {entry.path}", exc_info=ex)
	continue
	except FileNotFoundError as ex:
	log.debug(f"Skipping entry due to file not found error (possibly a broken link): {entry.path}", exc_info=ex)
	continue
	while queue:
	next_abs_path = queue.pop(0)
	if next_abs_path != self.repo_root:
	try:
	rel_path = os.path.relpath(next_abs_path, self.repo_root)
	except ValueError:
	# If the path is on a different drive (Windows) or cannot be made relative for another reason, we ignore it
	continue
	if self.should_ignore(rel_path):

So ignored_paths correctly excludes files from indexing and from gather_source_files, but the walk that discovers .gitignore files still descends into every ignored directory, including .git, node_modules and build output.

Expected

Directories matched by ignored_paths are skipped by the .gitignore discovery walk, in the same way directories matched by an already loaded .gitignore are skipped. One way to do that is to pass the combined ignored_paths patterns into GitignoreParser as an initial spec.

Reproduction

Any large tree with a directory you do not want scanned. Mine is a folder holding all the cert-manager repositories, with several worktree checkouts each (6.5 GB, 34,639 Go files, 36 checkouts of https://github.com/cert-manager/cert-manager).

1. Set ignored_paths in .serena/project.yml to exclude the extra checkouts, .git/, _bin/ and node_modules/.
2. Start the MCP server, or run serena project index.
3. Read the log line Loading of .gitignore files completed in N seconds.

Measurements

Serena MCP server log, before adding a root .gitignore:

serena.util.file_system:stop:336 - Loading of .gitignore files completed in 35.845 seconds
serena.project:stop:336 - Gathering ignore spec for project cert-manager completed in 36.097 seconds

After adding a root .gitignore with the same patterns (the parent folder is not a git repository, the file exists only for Serena):

serena.util.file_system:stop:336 - Loading of .gitignore files completed in 1.270 seconds
serena.project:stop:336 - Gathering ignore spec for project cert-manager completed in 1.276 seconds

A direct check of the walker with the root .gitignore in place:

$ uvx --from serena-agent python -c '
import time
from serena.util.file_system import GitignoreParser
t = time.monotonic()
p = GitignoreParser("/home/richard/projects/github.com/cert-manager")
print(f"{time.monotonic()-t:.1f}s, {len(p.ignore_specs)} .gitignore files")'
1.0s, 21 .gitignore files

Without it the same walk visits 155 .gitignore files and every .git and node_modules directory.

Workaround

Put the patterns in a .gitignore at the project root. _iter_gitignore_files loads that file first and then prunes on it. Note that a pattern with an inner slash is root-anchored, so nested directories need **/, for example **/.claude/worktrees/.

Setup

• Serena 1.7.0 (uvx --from serena-agent), LSP backend, Go (gopls)
• Claude Code 2.1.261, started through headroom wrap claude
• Fedora Linux 44, Python 3.14
• Project config: default project.yml generated by Serena, ignore_all_files_in_gitignore: true

Related: #1624 hit the same walker from a different direction (an unreadable directory listed in ignored_paths still crashed the walk).

[Claude Fable 5.1]