CI: cache Windows compiles with ccache by Gin890 · Pull Request #1611 · PokemonAutomation/Arduino-Source

Gin890 · 2026-10-17T23:53:09Z

Description

#1610 added ccache to the Linux and macOS builds but left out Windows, where each job spends 19–27 minutes compiling. Three things blocked it there:

ccache can't wrap the compiler under the Visual Studio generator.
It can't cache /Zi, which writes debug info to a shared PDB.
It can't cache a compile that writes an assembly listing (/FAs).

Changes

Windows CI builds with "Ninja Multi-Config". It keeps Visual Studio's output layout (bin/RelWithDebInfo/), so the upload and test steps are unchanged. A new step runs vcvars64.bat from the newest Visual Studio, the same one the Visual Studio generator used, and passes its environment to the later steps. For the clang job, it puts that Visual Studio's own clang-cl (22.1.3) first on PATH, so the compiler and linker (lld-link) stay the same as before.
No debug info in CI. Nothing in CI uses it: the Windows artifacts are for testing, and their error reports stay on the tester's machine.
- Windows: CI passes an empty CMAKE_MSVC_DEBUG_INFORMATION_FORMAT (with CMAKE_POLICY_DEFAULT_CMP0141=NEW), so there's no /Zi or /Z7. It links RelWithDebInfo with /INCREMENTAL:NO instead of /debug /INCREMENTAL, so there's no PDB or .ilk.
- Ubuntu clang: keeps RelWithDebInfo's -O2 -DNDEBUG but drops -g.
- Releases: the defaults in CMakeLists.txt are unchanged, so release builds made outside CI keep full debug info.
New PA_ASSEMBLY_LISTINGS CMake option (ON by default, OFF in CI) for the /FAs listings, so local builds keep them.
clang-cl detection uses CMAKE_CXX_COMPILER_ID instead of CMAKE_GENERATOR_TOOLSET, which is empty under Ninja. This applies to the -march and -Wno-unused-function flags in CMakeLists.txt and CoreLib.cmake.
/MP and /external:anglebrackets go to cl only. clang-cl ignores them, and /WX turns its unused-argument warning into an error. Visual Studio's ClangCL toolset never passed them on.
Windows installs ccache-action's own ccache (4.13.6). Otherwise the action uses the 4.9.1 that Strawberry Perl puts on the runner's PATH.
ccache depend mode on every OS. On a miss, ccache hashes the includes the compiler reports (/showIncludes, -MD) instead of running the preprocessor first.
Windows builds run nproc + 2 jobs instead of 10, close to the one compile per core that Visual Studio's /MP ran.
*.lib is left out of the "Serial Programs" artifact. The static libraries were already in it, and SerialProgramsLib.lib alone was 1.2 GB uncompressed with /Zi. Only the build itself uses them.

Results

All from workflow_dispatch runs on this branch. Runs from branches other than main don't save the cache, so every compile was a miss. These runs were made before the "build without debug info" commit, so they still built with /Z7 and uploaded a PDB.

	Before (Visual Studio)	This PR
Windows Clang total	22–32 min	20 min
Clang: compile SerialProgramsLib	24.6 min	15.5 min
Windows Default (MSVC) total	22–31 min	29 min, 29 min
MSVC: compile SerialProgramsLib	17.6 min	22 min (34 before the ccache and depend-mode fixes)
MSVC artifact	592 MB	458 MB

Cacheable: 1,589 of 1,589 compiles on both compilers. The saved caches are 0.5 GB (MSVC) and 0.2 GB (clang).
Cold MSVC looked slower: about 25% slower than before. Windows runner speed varies widely, though (see below), so that difference may be noise.

Without debug info

Runs on the "build without debug info" commit: a workflow_dispatch run and this PR's own CI, run at the same time. Every compile was a cache miss.

	main, Visual Studio with `/Zi`, same hour	This PR, no debug info
Windows Default "Generate binaries"	1,832 s	1,954 s, 1,953 s
Windows Clang "Generate binaries"	1,595 s	1,702 s, 1,658 s
Saved ccache entry	—	0.1 GB on both, down from 0.5 GB (MSVC) and 0.2 GB (clang) with `/Z7`
Windows artifacts	592 MB (MSVC), 483 MB (clang)	242 MB, 243 MB, with no PDB

Timings: over the last day, main's own unchanged Visual Studio builds took 1,159–2,080 s (MSVC) and 1,227–1,623 s (clang). The tests slowed down by the same factor, and debug info doesn't affect how fast they run. So these cold runs can't measure what dropping debug info saves in compile time. Next to main's run from the same hour, a build where every compile misses is now about as fast as the old Visual Studio build.
Cache hits on this PR: the Mac job took 5 minutes in total, down from 21–25. libdpp came from its cache, and 1,588 of 1,589 compiles hit the cache main saved after CI: cut repeated work in the C++ builds #1610, so compiling took 35 s.

Not verified yet

Cache hits: only pushes to main save the cache, so hits first show on the second push to main after merge, in each job's "CCache Statistics" summary.

🤖 Generated with Claude Code

The two Windows jobs spend 19-27 minutes each compiling, and #1610 left them out of ccache because three things blocked it: - ccache can't wrap the compiler under the Visual Studio generator, so Windows CI now configures with "Ninja Multi-Config". It keeps Visual Studio's output layout (bin/RelWithDebInfo/), so the upload and test steps are unchanged. A new step runs vcvars64.bat from the same Visual Studio and passes its environment to the later steps. For the clang job it puts that Visual Studio's own clang-cl first on PATH, the one the ClangCL toolset used. - ccache can't cache /Zi, which writes debug info to a shared PDB. CI now builds with /Z7 (CMP0141 and CMAKE_MSVC_DEBUG_INFORMATION_FORMAT =Embedded); the linker still writes the PDB that gets uploaded. - ccache can't cache a compile that writes a /FAs assembly listing. The listings are now behind a PA_ASSEMBLY_LISTINGS option, ON by default so local builds keep them, and CI turns it off. CMakeLists.txt and CoreLib.cmake picked the clang-cl flags (-march, -Wno-unused-function) by checking for the ClangCL toolset, which is empty under Ninja. They now check CMAKE_CXX_COMPILER_ID, which is "Clang" for clang-cl under either generator. Ninja runs nproc + 2 compiles at once on Windows instead of 10, close to the one compile per core that Visual Studio's /MP ran, so memory use stays near what the runner handled before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The first Windows runs with ccache turned up three problems: - Windows Clang failed on its first compiles: under Ninja, /MP and /external:anglebrackets reach clang-cl, which ignores them, and /WX turns its unused-argument warning into an error. Visual Studio's ClangCL toolset never passed them on, so they now go to cl only. - ccache-action's default install: detect picked up the ccache 4.9.1 that Strawberry Perl puts on the Windows runner's PATH. Windows now installs the action's own build (4.13.6). - With every file a cache miss, the MSVC build took 34 minutes to compile SerialProgramsLib, against 18 with the Visual Studio generator. On a miss, ccache runs the preprocessor before compiling by default. Depend mode instead hashes the includes the compiler reports (/showIncludes, or -MD on Linux and macOS, which CMake already passes), so it's now on for every build. With the newer ccache and depend mode, the same cold build took 22 minutes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The "Serial Programs" artifact uploads the whole bin/RelWithDebInfo folder, which includes the static libraries. SerialProgramsLib.lib was 1.2 GB uncompressed with /Zi and is 2.3 GB with /Z7, since it now holds every object's debug info. Ninja also writes the incremental-link file SerialPrograms.ilk (308 MB) next to the exe, where Visual Studio kept it in its intermediate folder. With both, the MSVC artifact grew from 592 to 972 MB. Neither is needed to run the program, so leave *.lib and *.ilk out of the upload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Nothing in CI uses debug info. The uploaded Windows builds are for testing; error reports from them stay on the tester's machine, and release builds, which need the PDB to read crash dumps, are made outside CI. Leaving it out makes compiles and links cheaper, the ccache entries smaller, and the artifacts smaller. - Windows: an empty CMAKE_MSVC_DEBUG_INFORMATION_FORMAT (with CMP0141) drops the /Z7 flag, and linking RelWithDebInfo with /INCREMENTAL:NO instead of /debug /INCREMENTAL writes no PDB or .ilk. So the artifact no longer needs the .ilk exclusion. - Ubuntu clang: keep RelWithDebInfo's -O2 -DNDEBUG but drop its -g. Only CI passes these flags, so CMakeLists.txt and its defaults are unchanged: other RelWithDebInfo builds still get full debug info. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Dalton-V · 2026-10-18T05:49:21Z

+        run: |
+          $vs = & "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath
+          $vcvars = cmd /c "`"$vs\VC\Auxiliary\Build\vcvars64.bat`" >nul && set"
+          if ($LASTEXITCODE -ne 0) { throw "vcvars64.bat failed with exit code $LASTEXITCODE" }


nit: inline if statements can be annoying to read.

Dalton-V · 2026-10-18T05:51:39Z

            fi
+            # Visual Studio ran about one compile per core (/MP). Keep Ninja near that instead of
+            # 10 at once, so memory use stays close to what the runner handled before.
+            BUILD_PARALLEL=$(( $(nproc) + 2 ))


I think this is okay to be merged as is. We may want to come back to profile this to see if there is a reason to have this additional parallel limit.

Gin890 force-pushed the ci-windows-ccache branch from 82957cc to f331bf6 Compare October 17, 2026 23:54

Gin and others added 3 commits October 17, 2026 21:46

Gin890 force-pushed the ci-windows-ccache branch from f331bf6 to 22feb3a Compare October 18, 2026 04:46

Gin890 marked this pull request as ready for review October 18, 2026 04:48

Dalton-V reviewed Oct 18, 2026

View reviewed changes

Dalton-V approved these changes Oct 18, 2026

View reviewed changes

Navigation Menu

Repository navigation

CI: cache Windows compiles with ccache - #1611

CI: cache Windows compiles with ccache#1611
Gin890 wants to merge 4 commits into
mainfrom
ci-windows-ccache

Gin890 commented Oct 17, 2026 •

edited

Loading

Uh oh!

Dalton-V Oct 18, 2026

Uh oh!

Dalton-V Oct 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Gin890 commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes

Results

Without debug info

Not verified yet

Uh oh!

Dalton-V Oct 18, 2026

Choose a reason for hiding this comment

Uh oh!

Dalton-V Oct 18, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Gin890 commented Oct 17, 2026 •

edited

Loading