Repository navigation
feat(build): nx2/dev.py says why the next build translates again, shows the builds' times, and builds again - #2909
Open
jordanella wants to merge 4 commits into
Open
jordanella wants to merge 4 commits into
jordanella wants to merge 4 commits into
Conversation
…e lifted C The key took every A64_, OOT_ and RX_ variable, so an RX_VK_ switch left set in the shell, or RX_RELEASE, which the lift does not read, translated the game again (~5 minutes on Windows), and so did the same game in another work or game folder: OOT_WORK, OOT_EXEFS, OOT_ROMFS, OOT_DUMP and A64_HOT_FUNCS are paths, and what the lift reads through them is hashed by its contents already. The emitter's own SUYU_RECOMP_ switches were not in the key at all. This is the dev-loop proposal's A2 (NotRexed#2636), with hckrman101's point that the paths keep two checkouts of one tree from sharing a lift. lift_key now hashes the variables lift_reads_env() names: A64_ and SUYU_RECOMP_, and RX_TARGET, but not the paths (LIFT_PATH_ENV) nor the compile's own (COMPILE_ONLY_ENV). tests/test_lift_cache.py checks that the same game and inputs in other folders give the same key, that RX_RELEASE, an RX_VK_ switch and RX_MARCH leave it and SUYU_RECOMP_STRICT, RX_TARGET and an A64_ switch change it, and that every variable the lift's sources read is in the key, a path, or in LIFT_ENV_NOT_OUTPUT with why it changes nothing. The key's form changes, so the first build after this translates once. Measured on Windows: after that first lift, a build with RX_VK_NO_PROGRAM_CACHE=1 in the shell skipped the lift (15 s). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ws the builds' times, and builds again The build's environment was made in make_game.py alone, so a build started any other way had other object keys and lift key, and nothing could say why a build translated the game again (~5 minutes on Windows) beyond "the key changed". This is the dev-loop proposal's A9 (NotRexed#2636). - nx2/profiles.py makes the environment the lift and build.py run with (the dictionary make_game.py built in place), in two profiles: release, the builders', and dev (HOOKSHOT_PROFILE=dev: no RX_RELEASE, so a console and no shader preparation at start; the lift is the same since the key no longer hashes RX_RELEASE). - make_game.py takes it from there, and records how it was started in <work>/last_build.json: its arguments, its Python, the profile, the toolchain's variables. - lift_key keeps what each key was made of (the Python, the HLE functions, each file's hash by name, the variables), and write_lift_key writes the stored key's parts to <work>/sa/lift_key_parts.json, outside gen/. - nx2/dev.py status: whether the next build skips the lift, and if not which file, function, variable or Python changed; dev.py times: the last builds' phases side by side (build_times.json); dev.py build [--dev]: the last build again. tests/test_dev_tool.py is new; test_lift_cleanup_error.py takes write_lift_key's new globals. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… build's --log out Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…n at the checkout's top For clangd and clang-tidy; the file is nx2/build.py's (NotRexed#2893). Without one, it says to build first. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a build translates the game again (about 5 minutes on Windows) there was no way to see why beyond "the key changed", and a build started any other way than
make_game.py's own had another environment, so other object and lift keys. This adds one entry point for developers,nx2/dev.py, and puts the build's environment in one place. The dev-loop proposal's A9 (#2636); its later pieces (scenario runs,where, the bench lock, warnings) build on it.Stacked on #2874 (the lift key's environment, its first commit here): review the last three commits; this rebases to them alone once #2874 is in.
Change
nx2/profiles.py(new): the environment the lift andbuild.pyrun with, the dictionarymake_game.pybuilt in place, in two profiles:release(the builders') anddev(HOOKSHOT_PROFILE=dev: noRX_RELEASE, so a console and no shader preparation at start; the lift is the same, since feat(build): the lift key hashes only the environment that changes the lifted C #2874's key no longer hashesRX_RELEASE).nx2/make_game.py: takes it from there, and records how it was started in<work>/last_build.json: its arguments, its Python, the profile, the toolchain's variables.lift_keykeeps what each key was made of (the Python, the HLE functions, each file's hash by name, the variables), andwrite_lift_keywrites the stored key's parts to<work>/sa/lift_key_parts.json, outsidegen/.nx2/dev.py(new, standard library only):status: whether the next build skips the lift, and if not, which file, HLE function, variable or Python changed;times [-n N]: the last builds' phases side by side (feat(build): build_times.json and the build's last lines say where the time went #2693'sbuild_times.json);build [--dev]: the last build again, with its arguments, Python and tools (its--logleft out);compdb: the last build'scompile_commands.json(perf(build): skip unchanged links, write compile_commands.json, key the hot pieces by the profile's contents #2893) at the checkout's top, for clangd.Nothing it prints comes from the game's files: names, times and hashes of the tools' own inputs.
In use
Windows 11, in a work folder with a translation:
dev.py statusafterA64_NO_VFPX=1in the shell:lift: the next build translates the game again:thenenvironment A64_NO_VFPX: (unset) -> 1.dev.py buildreplayed the last build (the same arguments and Python, shown quoted).dev.py times -n 3showed the last builds' phases side by side.Testing
tests/test_dev_tool.py(new): the differencesstatusnames (a file, a variable, the Python, an HLE function), the work folder it picks,times' table,build's replayed command (without--log) and its environment for each profile,compdb.tests/test_lift_cleanup_error.py: takeswrite_lift_key's new globals.run_all.py, rebased on today'smain, Windows: 54 pass;test_linux_updatefails the same way onmainhere.See #2636 (A9), #2874.
🤖 Generated with Claude Code