Skip to content

Bound background fetches and run at most one per repository [patch] - #518

Merged
matt-edmondson merged 2 commits into
mainfrom
limit-auto-fetch-concurrency
Nov 4, 2026
Merged

matt-edmondson merged 2 commits into
mainfrom
limit-auto-fetch-concurrency

Conversation

@matt-edmondson

Copy link
Copy Markdown
Contributor

Fixes #448

What changed

New FetchScheduler, in the same style as CloneTracker: the rule is pulled out so it can be tested without ImGui.

  • Bounded: fetches go through a queue, and at most 4 run at once (DefaultMaxConcurrent). It owns no disposable, so ProjectDirector does not have to become IDisposable.
  • One per repository: a repository whose fetch is running or still waiting for a slot is refused. A second git fetch can no longer race the first for ref locks, and a stale repository is queued at most once.
  • Out of phase: when a fetch is queued, LastFetchTime is backdated by a random jitter of up to a quarter of the interval. Repositories fetched together then go stale at different moments instead of re-firing on the same frame every minute. LastFetchTime and the options save are now only touched when a fetch was actually queued.

FetchRepo routes both the automatic stale fetch and the manual "Fetch All" through the scheduler, so both are bounded.

Tests

New FetchSchedulerTests:

  • 40 repositories never run more than 4 fetches at once, and all 40 complete.
  • A second fetch in the same repository is refused while the first runs. Another repository is not blocked.
  • A fetch that throws releases its slot and its repository, so it can be retried.
  • Jitter stays in [0, interval/4), varies between calls, and is zero for a disabled interval.

To show the tests fail without the fix, I put the old behaviour back inside the scheduler (no bound, no in-flight check). The bound test and the same-repository test then fail. With the fix restored, all four pass, and five repeated runs gave no flakes.

The full suite: 183 passed, 1 skipped, 1 failed. The failure is GitCliTests.CloningAnLfsRepositoryRestoresTheFileContentRatherThanThePointer, and it also fails on unmodified main in this container (an LFS smudge environment issue). The whole solution builds with 0 warnings. CLAUDE.md now lists the scheduler alongside the other extracted rules.

Related: #446 (the log panel). Most of the log lines that push failures out of view came from this burst.


Generated by Claude Code

Every stale clone started its own git fetch on the same frame, so forty
clones meant forty concurrent fetches every minute, in lockstep, and a fetch
slower than the interval was started again in the same repository while the
first still ran. FetchScheduler now queues fetches and runs at most four at a
time, refuses a second fetch for a repository that is running or queued, and
each fetch time is backdated by a random jitter of up to a quarter of the
interval so the repositories drift apart.

Fixes #448

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017vfJW5526rrxQXBzaZBBJE
…urrency

# Conflicts:
#	ProjectDirector/ProjectDirector.cs
@sonarqubecloud

sonarqubecloud Bot commented Nov 4, 2026

Copy link
Copy Markdown

@matt-edmondson
matt-edmondson merged commit 8e82987 into main Nov 4, 2026
16 checks passed
@matt-edmondson
matt-edmondson deleted the limit-auto-fetch-concurrency branch November 4, 2026 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto-fetch starts a git fetch for every cloned repository at the same moment every minute, with no concurrency limit and no in-flight check

2 participants