Repository navigation
perf(build): lifted chunks of ~4 MB, by guest address, besides chunks.txt's boundaries - #2890
Open
jordanella wants to merge 1 commit into
Open
jordanella wants to merge 1 commit into
jordanella wants to merge 1 commit into
Conversation
….txt's boundaries Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A small update that changes a few of the game's functions still recompiles every lifted chunk that holds one, and a chunk is ~24 MB of C: a typical one takes ~227 s of compiler time, and the few heavy ones far more. Splitting the lifted C into ~4 MB pieces by guest address makes that ~41 s, with no cost in play or in a full build. Part of the dev-loop proposal's work on small updates (#2636).
Change
nx2/chunks.py:image/chunks.txt's stable boundaries,sub_startsadds one everySUB_STRIDE(0x40000) of guest address, at the first function at or after it. They are guest addresses, so they stay put when a function grows: a change moves no boundary but its own chunk's.chunks.txtkeeps only the boundaries it had (the sub-boundaries are worked out again each lift from the same functions), so dropping this change later is clean.split_big,split_hotandbuild.pymatch three-digit chunk names.The first build after this compiles every lifted chunk once (all boundaries are new): for a player, that update takes about as long as a full build, once. Updates after it recompile only the ~4 MB pieces that changed.
Measured
Windows 11, Ryzen 9 3900X:
mainagainst 32:30, at the same total compiler time (25,700 against 25,500 s) over 698 files instead of 486. The smaller pieces spread better over the cores.In the game
No change in play: the 4 MB-chunk game against the 24 MB one, Kokiri Forest from a save (the game logic bounds its frame on the CPU there), RTX 3080 (NVIDIA 621.14), Windows 11, two interleaved pairs of 30 samples: median 29.6 / 30.0 fps against 31.0 / 30.0. No errors in either.
Testing
tests/test_sub_chunks.py(new): the first function past each stride; a long function spans strides without a boundary inside it; a function found since moves one boundary at most; chunks end at them as well as at the saved boundaries;writeuses them and saves onlychunks.txt's; the stride makes chunks of a few MB. (The 1000-chunk guard isn't tested: the game's code makes ~330 pieces.)run_all.py, rebased on today'smain(with perf(lift): emit the lifted C in several processes on Windows too #2748's spawned emit), Windows: 8 pass;test_spawn_emitfails the same way onmainhere.See #2636.
🤖 Generated with Claude Code