Repository navigation
perf(build): skip unchanged links, write compile_commands.json, key the hot pieces by the profile's contents - #2893
Open
jordanella wants to merge 1 commit into
Open
jordanella wants to merge 1 commit into
jordanella wants to merge 1 commit into
Conversation
…he hot pieces by the profile's contents Three changes to nx2/build.py: - A4: a link whose inputs are the last successful one's is not run again (run_link). Its key, beside the output as <output>.linkkey: the command, the compiler and the work folder's system-headers stamp, each object's key (.o.sha1), and the contents of the other files it names, inside options too (the hot symbol order). A build with nothing to do went from 16 s to 6 s on Windows. Players' builds let sa/build go, so it helps a work folder that keeps its files (the checkout's work/, HOOKSHOT_KEEP_BUILD_FILES=1). - A8: <build>/compile_commands.json, every compile as compile_one runs it, for clangd and clang-tidy (git-ignored at the checkout's top). - The hot pieces' keys name the play profile by its contents (-DHOOKSHOT_PROFILE_SHA1), not its path: a build from a checkout other than the work folder's compiled all eight again (~10-20 minutes). This changes their keys once. tests/test_link_skip.py (a stand-in linker: skipped, relinked on a changed object, library, symbol order or output, a failed link forgets the key; the compile commands) and tests/test_profile_path_key.py. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A build with nothing to compile still relinked the 440 MB executable and the renderer, a build from a second checkout of the same tree recompiled all eight hot pieces (~10-20 minutes) because their keys named the play profile by its path, and clangd had no compile database. Three small changes to
nx2/build.pyfrom the dev-loop proposal (#2636, A4 and A8); one PR because they touch the same lines, and I can split them if you prefer.Change
nx2/build.py:A4, no link when nothing changed:
run_linkskips a link whose inputs are the last successful one's. Its key is kept beside the output (<output>.linkkey) with the output's size:.o.sha1; its size and time without one);A failed link, or an output of another size or gone, links again. The executable's, the renderer's and the game DLL's links go through it.
A8,
compile_commands.json: every compile ascompile_oneruns it, written to the build folder for clangd and clang-tidy (compile_commands.jsonat the checkout's top is git-ignored, for a copy there).The hot pieces' keys name the profile by its contents:
flag_textleaves-fprofile-instr-use=<path>out of the key, since-DHOOKSHOT_PROFILE_SHA1beside it already says what the key needs of the profile. This changes their keys once.Measured
Windows 11, Ryzen 9 3900X, a dev work folder:
maincompiled them again.In the game
Every build I ran in the game today used this
build.py. The latest: Windows 11, RTX 3080 (NVIDIA 621.14), a dev-profile build, from a save, headless through the control bridge: in play in Kokiri Forest, 28-31 fps, nothing in the log about a lost device or a crash. A skipped link leaves the same binary in place.Testing
tests/test_link_skip.py(new): a stand-in linker through the realrun_link: the same inputs aren't linked again; a changed object, library or symbol order is; so is an output of another size or gone; a failed link forgets its key; the files inside options are inputs; the compile commands are the compiles; the build's links go through it.tests/test_profile_path_key.py(new): two paths, one profile give one key; another profile, another key; every use of the profile's path names its contents too.run_all.py, rebased on today'smain, Windows: pass, apart from four that fail the same way onmainhere (test_driver_cache_folder,test_fpx_sticky,test_linux_update,nx2/test_dll.py). Rebasing found thatmain'sbuild.pydoesn't importre(perf(build): a renderer header edit compiles the files that read it, not the whole back end #2706 adds it); this PR imports it too.See #2636 (A4, A8).
🤖 Generated with Claude Code