Repository navigation
build(release): adult areas and dungeons added to jarynclouatre-most-of-the-game, 1406 pipelines - #2064
Conversation
…of-the-game, 1406 pipelines
|
Checked against main with the tour list (#2065): |
hckrman101
left a comment
hckrman101
left a comment
There was a problem hiding this comment.
Game-files check (CONTRIBUTING.md): clean. The only files are the contributed list and its note. I decompressed the .bin.xz: it is HKSTATE1, record size 528, exactly 8,657 fixed-size records (12-byte header, no trailing bytes), which matches the Pipelines: 8657 in the note. So it holds only the hashes, formats and draw state that the format describes, nothing else, and the change is data only. The .md edit is consistent with the PR description.
One thing to fix: CI is red on tests/test_pipeline_state_sources.py (test_split_list_lost_nothing): AssertionError: 8657 != 7251 : jarynclouatre-most-of-the-game. That test pins MIGRATED["jarynclouatre-most-of-the-game"] = 7251 and a sha256 of the three lists joined (MIGRATED_SHA256), as a one-time check that the split of the old assets/pipeline_states.bin.xz lost nothing. Growing this list breaks both the count and the hash, so the PR needs to touch the test as well (it is not data only, and the description says the tests pass, which was not true of this one). It can't just bump 7251 to 8657, because the hash would still fail. Options: check that the original 7251 records are all still in the file (keep them in the test as a hash of the sorted original records, not of the whole list), or assert >= the original counts. Please don't weaken it beyond that, per AGENTS.md. Note the same will apply to anything else that grows these lists, so a form that tolerates growth is better than another number to bump.
Python source checks (ubuntu-24.04) also fails tests/test_make_game_ngx.py ('SimpleNamespace' object has no attribute 'switching'). That is in a builder DLSS test this PR does not touch, so I'm not blaming the PR for it, but the windows jobs are red too and I did not check whether they are the same test or this one.
On the content: the author already checked the overlap with the tour list in #2065 (1,057 of these are not in the tour either), so the dedupe looks fine. Still open from the description: first-start shader preparation time with this list is not measured. With 14k+ pipelines it is worth a number before this ships, not a blocker for the data itself. The decision to merge stays with @NotRexed.
…'s first pipelines
|
Thanks, and sorry, I missed that test. e1720ae takes each migrated list's first n pipelines ( |
|
Preparation time, on the same PC (Windows 11, RTX 4070 Ti SUPER, driver 621.14), prepare-only runs on main f915af9's merged list, without and with this file. With the pipeline cache and the driver's shader cache emptied first, as on a first start: 15,323 graphics pipelines precompiled in 108.1 s, and 15,763 in 110.9 s. So the 440 this adds on top of current main cost about 3 s at a first start. With the caches an earlier session left, 115.1 s and 115.9 s. |
…, 921 pipelines (#2429) A new contributed list for #2029: the pipelines my test runs and a short play recorded that no list holds yet. The runs are scripted test routes on a test save (teleports through overworld areas and dungeons, among them Gerudo Valley, Dodongo's Cavern and Ganon's Castle, as child and as adult, with a short walk or stand at each), plus a short adult play in Ganon's Castle and Dodongo's Cavern. Made with: ``` python -I nx2/pipeline_states.py add jarynclouatre-test-routes <28 lists> --who ... --gpu ... --build ... --settings ... --route ... ``` Of 12,606 pipelines in those recordings, 1,296 were in no other contributed list (counted with #2064's version of my other file in place, so the two do not overlap). 375 of those had no render targets (samples 0); the game's reader skips such records (`targets_ok` in `vk_draw.cpp`), so I left them out, which leaves 921. Against main as it is, 542 of them are in no list main ships: the merged list goes from 25,251 to 25,793 pipelines, and from 9,948 to 9,963 programs. Recorded on Windows 11, RTX 4070 Ti SUPER (driver 621.14), no mods, on main between fc375e9 and 49f75dc; some runs had renderer changes from open pull requests, none of which touch shaders. The file is version 2 (HKSTATE2), so it holds what the game asked for, not my GPU's choices. As with the other lists it holds only hashes, formats and draw state, no game code. **Checked:** - `nx2/pipeline_states.py check` passes. - `tests/test_pipeline_state_sources.py`, `tests/test_pipeline_states.py` and `tests/test_ship_state_list.py` pass. `check` does not flag records with samples 0, so one of my test lists carried them through it. I have not timed the first start's shader preparation with this list. #2064's comment measured about 3 s for 440 more pipelines on this PC.
Adds the three pipeline lists posted in #2034 that aren't in `main` or in an open pull request yet. I added each one with `nx2/pipeline_states.py add`, so each list keeps only the pipelines that no other contributed list already has. Each one has a note beside it. ## What's added | List | From | GPU, OS | Build and settings | Route | Pipelines sent | Kept in the list | New to the shipped list | |---|---|---|---|---|---|---|---| | `contrib/yeetbot90-amd-rx-9070-xt-fsr3-4k` | @yeetbot90 (second list, "MORE PIPELINES") | AMD RX 9070 XT, driver 32.0.32015.7029, Windows 11 | v2.1.0, 3840x2160, FSR 3.1 Auto, frame generation 3x, HDR on | Kokiri Forest through Jabu-Jabu's Belly | 2,047 | 1,013 | 992 | | `contrib/jabbaxipil-rtx-4070-linux-dlaa` | @jabbaxipil | NVIDIA RTX 4070, driver 615.71.09, Ubuntu 26.04.1 (Wayland) | nightly v2.1.0-rc.3-46, 2560x1440, DLAA (model K), frame generation 3x | child, by day: Kokiri Forest out to Gerudo Valley | 960 | 778 | 710 | | `contrib/disgustingusting-web-intel-uhd-intro` | @disgustingusting-web | Intel UHD (i3-1115G4, integrated), driver 32.0.101.7088, Windows 11 | v2.1.0, 1920x1080, no frame generation | the intro only | 9 | 6 | 0 (the tour's list already has all 6) | All three are version 2 lists (`HKSTATE2`). The merged list grows from 25,793 to 27,441 pipelines (1,648 new; the two big lists share 54). I kept the Intel list even though it adds nothing today: the tour's list is replaced whole each time, and this one holds pipelines no other contributed list has. ## What's left out - @pmwork-personal's first list, @yeetbot90's first list and @Imasvpp's list: already in `main` (#2075 and #2082). - Lists from @Brett-D (#2143), @pmwork-personal (#2133) and @jarynclouatre (#2064): each is in its contributor's own open pull request. - @disgustingusting-web's `hookshot_log.txt`: it's a log, not a list. ## How it was checked - I downloaded each zip into its own empty folder outside the repository and listed it without extracting. Each one held a single `hookshot_pipeline_states.bin`: no paths, no symlinks, nothing encrypted, and sane sizes and compression ratios. Only that one file was extracted, with a size cap. - Each file starts with `HKSTATE2`, uses 552-byte records, has a whole number of records, and has no duplicates. - I also checked every field of every record: mesh, sample, depth and flag fields in range, no attributes or answers past their counts, padding zero, and each variant matching its answers' hash. The only exceptions were records written with no answers, which `tour.bin.xz` has in the same form. - `python -I nx2/pipeline_states.py check`: passes. The merge gives the same bytes twice. - Tests: `tests/test_pipeline_state_sources.py`, `test_ship_state_list.py`, `test_pipeline_states.py`, `test_pipeline_notes_open.py`, `test_pipeline_state_counts.py` and `test_pipeline_state_portable.py` all pass. ## Not tested - I didn't run the game: no first-start shader preparation with the new shipped list, no game build, and no check that these pipelines remove stutter on these GPUs. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…s) (NotRexed#2445) Adds the three pipeline lists posted in NotRexed#2034 that aren't in `main` or in an open pull request yet. I added each one with `nx2/pipeline_states.py add`, so each list keeps only the pipelines that no other contributed list already has. Each one has a note beside it. ## What's added | List | From | GPU, OS | Build and settings | Route | Pipelines sent | Kept in the list | New to the shipped list | |---|---|---|---|---|---|---|---| | `contrib/yeetbot90-amd-rx-9070-xt-fsr3-4k` | @yeetbot90 (second list, "MORE PIPELINES") | AMD RX 9070 XT, driver 32.0.32015.7029, Windows 11 | v2.1.0, 3840x2160, FSR 3.1 Auto, frame generation 3x, HDR on | Kokiri Forest through Jabu-Jabu's Belly | 2,047 | 1,013 | 992 | | `contrib/jabbaxipil-rtx-4070-linux-dlaa` | @jabbaxipil | NVIDIA RTX 4070, driver 615.71.09, Ubuntu 26.04.1 (Wayland) | nightly v2.1.0-rc.3-46, 2560x1440, DLAA (model K), frame generation 3x | child, by day: Kokiri Forest out to Gerudo Valley | 960 | 778 | 710 | | `contrib/disgustingusting-web-intel-uhd-intro` | @disgustingusting-web | Intel UHD (i3-1115G4, integrated), driver 32.0.101.7088, Windows 11 | v2.1.0, 1920x1080, no frame generation | the intro only | 9 | 6 | 0 (the tour's list already has all 6) | All three are version 2 lists (`HKSTATE2`). The merged list grows from 25,793 to 27,441 pipelines (1,648 new; the two big lists share 54). I kept the Intel list even though it adds nothing today: the tour's list is replaced whole each time, and this one holds pipelines no other contributed list has. ## What's left out - @pmwork-personal's first list, @yeetbot90's first list and @Imasvpp's list: already in `main` (NotRexed#2075 and NotRexed#2082). - Lists from @Brett-D (NotRexed#2143), @pmwork-personal (NotRexed#2133) and @jarynclouatre (NotRexed#2064): each is in its contributor's own open pull request. - @disgustingusting-web's `hookshot_log.txt`: it's a log, not a list. ## How it was checked - I downloaded each zip into its own empty folder outside the repository and listed it without extracting. Each one held a single `hookshot_pipeline_states.bin`: no paths, no symlinks, nothing encrypted, and sane sizes and compression ratios. Only that one file was extracted, with a size cap. - Each file starts with `HKSTATE2`, uses 552-byte records, has a whole number of records, and has no duplicates. - I also checked every field of every record: mesh, sample, depth and flag fields in range, no attributes or answers past their counts, padding zero, and each variant matching its answers' hash. The only exceptions were records written with no answers, which `tour.bin.xz` has in the same form. - `python -I nx2/pipeline_states.py check`: passes. The merge gives the same bytes twice. - Tests: `tests/test_pipeline_state_sources.py`, `test_ship_state_list.py`, `test_pipeline_states.py`, `test_pipeline_notes_open.py`, `test_pipeline_state_counts.py` and `test_pipeline_state_portable.py` all pass. ## Not tested - I didn't run the game: no first-start shader preparation with the new shipped list, no game build, and no check that these pipelines remove stutter on these GPUs. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Joey-Magnus
left a comment
Joey-Magnus
left a comment
There was a problem hiding this comment.
I merged this into today's main, which has gained the #2075, #2082, #2429, #2445 and #2133 lists since you rebased. nx2/pipeline_states.py check passes. Your file goes from 7,251 to 8,657 pipelines, and the merged list from 28,902 to 29,688, so 786 of the new ones are still not in any other source. test_pipeline_state_sources, test_pipeline_states and test_ship_state_list pass, and so does the full run_all (376 passed, 0 failed, Linux).
The new test_split_list_lost_nothing is the growth-tolerant form that was asked for. It checks the first n records of each migrated list against the original hash and count, so a grown list passes and a reordered or changed original part still fails. The preparation-time numbers (about 3 s more on a first start) answer the last open point.
The red checks are from the run on e1720ae's base. actionlint's SC2012 and test_make_game_ngx come from main at the time, not from this diff. A re-run on current main should clear them.
More play added to my contributed list from #2029, mostly adult areas and dungeons I had not reached before. The data, plus one test change.
The list file was re-made with the tool from #2040:
It keeps only pipelines no other contributed list has. Of 8,021 pipelines in my recordings, 1,406 are new, so my file goes from 7,251 to 8,657 pipelines and the merged list from 12,800 to 14,206. The
.mdnote lists the new areas: Graveyard, Hyrule Field, Kokiri Forest, Lost Woods, Sacred Forest Meadow, Forest Temple, Death Mountain Trail and Crater, Goron City, Fire Temple and its boss, Zora's River, Domain and Fountain, and the Ice Cavern, all as adult.Recorded on Windows 11, RTX 4070 Ti SUPER (driver 621.14), no mods, 2560x1440 with dynamic resolution, on main with #1961 (some runs also with #2020 and #2050, which change no shaders). As before it holds only hashes, formats and draw state, no game code.
Checked:
nx2/pipeline_states.py checkpasses.tests/test_pipeline_states.py,tests/test_ship_state_list.pyandtests/test_pipeline_state_sources.pypass.test_split_list_lost_nothingpinned each migrated list's exact size, so growing a list failed it. It now takes each list's first n pipelines, sinceaddkeeps a list's own pipelines first, and still checks their count, that they don't overlap, and their hash. The same file grown byaddpasses, and main's form of the test fails on it.I have not timed the first start's shader preparation with this list yet.