Skip to content

Cache the git objects of CodeTracker and RefactoringMiner in CI - #218

Merged
tsantalis merged 1 commit into
masterfrom
ci-cache-git-objects
Nov 4, 2026
Merged

tsantalis merged 1 commit into
masterfrom
ci-cache-git-objects

Conversation

@tsantalis

Copy link
Copy Markdown
Collaborator

The CI checkout of CodeTracker (with the oracle submodule) and the clone of RefactoringMiner download every object at each build. In the last successful master build (run 41630453109), the checkout took 168 s and the RefactoringMiner clone + publishToMavenLocal took 330 s.

This applies the same optimization as the RefactoringMiner workflow (tsantalis/RefactoringMiner@aeaea55fc):

  • CodeTracker: the .git of an earlier master build (including .git/modules/oracle) is restored before actions/checkout, which reuses it, so only the objects changed since then are fetched.
  • RefactoringMiner: its master commit is resolved with git ls-remote, a .git cached for an earlier RefactoringMiner commit is restored, and the commit is fetched with --depth 1 into it, instead of a fresh clone. The cache is keyed by the RefactoringMiner commit, so builds of the same RefactoringMiner commit reuse it as is.
  • On master builds, the objects of the older snapshots are dropped (reflog expire, repack -a -d, prune, also in the submodule) before the caches are saved, so they do not grow with every build. prune also removes the loose objects that small fetches leave behind.
  • persist-credentials: false keeps the token out of the cached .git/config. actions/checkout is updated from v3 to v4.

Notes:

  • The caches are only saved on master builds (branch caches are not visible to other branches), so this PR's own run takes the uncached path; the speed-up shows from the second master build after the merge.
  • The fetch script was tested locally (fresh fetch, fetch of another commit into an existing .git, cleanup of stray files, repack + prune leaving only the new snapshot).

🤖 Generated with Claude Code

The checkout of CodeTracker with its oracle submodule and the clone of
RefactoringMiner downloaded every object at each build (168 s and part
of 330 s in the last master build). Like the RefactoringMiner workflow,
the .git of an earlier master build is restored before the checkout,
so that only the objects changed since then are fetched. RefactoringMiner
is fetched at its master commit into a restored .git, keyed by that
commit, instead of being cloned.

On master builds, the objects of the older snapshots are dropped
(reflog expire, repack, prune, also in the submodule) and the caches
are saved. The credentials are not kept in .git/config, so that the
cached .git does not contain the token.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tsantalis
tsantalis merged commit 0d7493a into master Nov 4, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant