Skip to content

build(release): a pipeline list converted from Fritchii's shader reci… - #2133

Merged
hckrman101 merged 1 commit into
NotRexed:mainfrom
pmwork-personal:feat/fritchii-recipe-list
Nov 11, 2026
Merged

hckrman101 merged 1 commit into
NotRexed:mainfrom
pmwork-personal:feat/fritchii-recipe-list

Conversation

@pmwork-personal

@pmwork-personal pmwork-personal commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Fritchii ships a recipe of the pipelines Ocarina of Time draws, and 1,461 of them are pipelines no list here holds that convert to whole HKSTATE2 records. This adds them as contrib/pmwork-personal-fritchii-recipe, as discussed in #dev-work.

What the recipe is. Fritchii (a Lichii fork) ships OoT v131072 (cache v54).nx2recipe: the pipelines its testers' sessions drew plus the states it predicts. Each shader is named by a fingerprint, and the recipe holds no code (its format is in nx2_recipe.h / nx2_shader_extract.h on Fritchii's oot-gather-fix branch).

How it maps.

  • Shaders. The fingerprint is SHA-256 of the stage's control section (first 0x48 bytes, +0x30..0x33 zeroed), then its code area. code_key hashes the same control and code bytes with FNV, so a player's hookshot_shaders.bin names the recipe's shaders by code key: 41,165 of 41,216.
  • Render state. It maps by meaning:
    • target and depth formats (Maxwell render-target formats to NVN's, one to one for every format OoT uses)
    • samples and topology class
    • blend factors and equations, into nvn_blend_st
    • channel mask, polygon mode, depth clamp and alpha to coverage

What it can't hold. The recipe was recorded with dynamic vertex input, so it has no vertex formats or divisors. It has no texture answers either. So a pipeline is in this list only when:

  • main's lists fix both for its program (one vertex input for the vertex stage and one variant), or
  • it needs neither: mesh draws take no vertex input, and a program none of whose stages asks about a texture or image has variant 0. All 40 such programs in main's lists have variant 0 there.

Point-topology states that the recipe predicts for programs main's lists draw as triangles are left out: 728 of the 734 checkable ones disagreed with main.

Checked against main. I converted every recipe pipeline main's lists already hold the same way. 13,323 of 19,738 match one of their records field for field, including blend, masks and clamp. The rest are:

  • other states of the same draws, e.g. additive vs alpha blend; or
  • blend enables past the last target, which Lichii clears and Hookshot keys only on a GPU without dynamic blend state.

There was no systematic difference in any field.

Files.

  • assets/pipeline_states/contrib/pmwork-personal-fritchii-recipe.bin.xz and .md, written by nx2/pipeline_states.py add. The note says how the list was made.
  • assets/pipeline_states/README.md: one paragraph saying the lists named pmwork-personal-fritchii-recipe* are converted, not recorded. They hold the same records and nothing more.

The converter is a small script (recipe + hookshot_shaders.bin + main's merged list → HKSTATE2). It isn't in this PR; happy to add it under scripts/ if you want the list reproducible.

Tested.

  • nx2/pipeline_states.py check: merged list 26,712 pipelines (main 25,251 + 1,461).

  • Tests: test_pipeline_states, test_pipeline_state_sources, test_pipeline_state_counts, test_pipeline_state_portable, test_ship_state_list.

  • python -I -B tests/run_all.py on main with both PRs (Windows): 342 passed, 2 failed, 30 skipped (Linux-only). The two (test_precompile_compute_guard.py, test_recorder_sync.py) fail the same on main without these changes on this PC.

  • In the game's preparation: a build of main with this list and the follow-up's (feat(vk): a state list record with no variant prepares the plain one; 11,985 pipelines from Fritchii's recipe #2134) ran the Builder's shader preparation (RTX 5090, driver 621.14):

    vk: state list: 38697 pipelines (38697 shipped, version 2), 14866 programs (1200 translated now, 0 with stages not in the seed), 953 variants by their textures (953 made), 4798 by nothing bound (4798 made), 38584 noted (2438 native mesh, 0 mesh fallback, 11985 by nothing bound), 113 whose program variant was not made, in 10.2 s

    Every record of this list was noted. The 113 not made are records main's lists already hold that were noted without answers (1,285 such records there, none here).

Credit: the recipe is Fritchii's (Frizti), built from its testers' sessions.

🤖 Generated with Claude Code

…pe, 1461 pipelines

Fritchii (a Lichii fork) ships a recipe of the pipelines Ocarina of Time draws: its
testers' sessions and the states it predicts, each shader named by a fingerprint, with
no code. 1,461 of them are pipelines no list here holds that convert to whole version 2
records. They are contrib/pmwork-personal-fritchii-recipe.

The fingerprint (SHA-256 of the stage's control section with four bytes masked, then its
code area) and code_key hash the same bytes, so hookshot_shaders.bin names the recipe's
shaders by code key (41,165 of 41,216). Target and depth formats, samples, topology class,
blend, channel mask, polygon mode, depth clamp and alpha to coverage map by meaning. The
recipe has no vertex formats (recorded with dynamic vertex input) and no texture answers,
so a pipeline is here only when the lists here fix both for its program, or it needs
neither (mesh draws; programs that ask about no texture).

Of the recipe's pipelines the lists here hold too, converted the same way, 13,323 of
19,738 match one of their records field for field; the others are other states of the
same draws (blend, masks, clamp) or blend enables past the last target, which Lichii
clears.

The README says this one list is converted, not recorded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hckrman101 added a commit that referenced this pull request Nov 11, 2026
Adds the three pipeline lists posted in #2034 that aren't in `main` or
in an open pull request yet. I added each one with
`nx2/pipeline_states.py add`, so each list keeps only the pipelines that
no other contributed list already has. Each one has a note beside it.

## What's added

| List | From | GPU, OS | Build and settings | Route | Pipelines sent |
Kept in the list | New to the shipped list |
|---|---|---|---|---|---|---|---|
| `contrib/yeetbot90-amd-rx-9070-xt-fsr3-4k` | @yeetbot90 (second list,
"MORE PIPELINES") | AMD RX 9070 XT, driver 32.0.32015.7029, Windows 11 |
v2.1.0, 3840x2160, FSR 3.1 Auto, frame generation 3x, HDR on | Kokiri
Forest through Jabu-Jabu's Belly | 2,047 | 1,013 | 992 |
| `contrib/jabbaxipil-rtx-4070-linux-dlaa` | @jabbaxipil | NVIDIA RTX
4070, driver 615.71.09, Ubuntu 26.04.1 (Wayland) | nightly
v2.1.0-rc.3-46, 2560x1440, DLAA (model K), frame generation 3x | child,
by day: Kokiri Forest out to Gerudo Valley | 960 | 778 | 710 |
| `contrib/disgustingusting-web-intel-uhd-intro` | @disgustingusting-web
| Intel UHD (i3-1115G4, integrated), driver 32.0.101.7088, Windows 11 |
v2.1.0, 1920x1080, no frame generation | the intro only | 9 | 6 | 0 (the
tour's list already has all 6) |

All three are version 2 lists (`HKSTATE2`). The merged list grows from
25,793 to 27,441 pipelines (1,648 new; the two big lists share 54). I
kept the Intel list even though it adds nothing today: the tour's list
is replaced whole each time, and this one holds pipelines no other
contributed list has.

## What's left out

- @pmwork-personal's first list, @yeetbot90's first list and @Imasvpp's
list: already in `main` (#2075 and #2082).
- Lists from @Brett-D (#2143), @pmwork-personal (#2133) and
@jarynclouatre (#2064): each is in its contributor's own open pull
request.
- @disgustingusting-web's `hookshot_log.txt`: it's a log, not a list.

## How it was checked

- I downloaded each zip into its own empty folder outside the repository
and listed it without extracting. Each one held a single
`hookshot_pipeline_states.bin`: no paths, no symlinks, nothing
encrypted, and sane sizes and compression ratios. Only that one file was
extracted, with a size cap.
- Each file starts with `HKSTATE2`, uses 552-byte records, has a whole
number of records, and has no duplicates.
- I also checked every field of every record: mesh, sample, depth and
flag fields in range, no attributes or answers past their counts,
padding zero, and each variant matching its answers' hash. The only
exceptions were records written with no answers, which `tour.bin.xz` has
in the same form.
- `python -I nx2/pipeline_states.py check`: passes. The merge gives the
same bytes twice.
- Tests: `tests/test_pipeline_state_sources.py`,
`test_ship_state_list.py`, `test_pipeline_states.py`,
`test_pipeline_notes_open.py`, `test_pipeline_state_counts.py` and
`test_pipeline_state_portable.py` all pass.

## Not tested

- I didn't run the game: no first-start shader preparation with the new
shipped list, no game build, and no check that these pipelines remove
stutter on these GPUs.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

@Joey-Magnus Joey-Magnus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice piece of cross-checking: matching 13,323 of 19,738 field for field against main's lists is good evidence that the mapping is right. The merge is clean and CI is green. Before this goes in, I think it needs a maintainer decision and a couple of changes, because it is the first list in assets/pipeline_states/ that Hookshot did not record.

question (maintainer) assets/pipeline_states/README.md:5-7: the folder's rule until now was "only lists written by Hookshot and this folder's tool go in here". This PR adds an exception for one named list. That deserves an explicit yes from the maintainer, since it opens the door to converted lists in general. Related: the recipe is Fritchii's data, built from its testers' sessions. Credit is given, but the PR doesn't say under what terms it may be redistributed inside a GPL-3.0 repo. Was permission given (the #dev-work discussion?), or is the recipe published under a licence that allows this? A link or a line in the .md note would settle it. The list holds only hashes and draw state, so this isn't a game-data problem. It's a question of whose data it is.

should fix: the converter isn't in the PR (the body offers to add it). For a list that is converted rather than recorded, the script is the only way to review or regenerate it. That matters when main's lists change (inclusion depends on "main's lists fix one vertex input and one variant"), and when code_key or the record format changes. Please add it under scripts/ (standard library only, per CONTRIBUTING), and have the note name the command that produced this file.

should fix: predicted states. The recipe holds "the pipelines its testers' sessions drew plus the states it predicts". Your own check found that the point-topology predictions disagreed with main in 728 of 734 checkable cases. That is a strong signal that the prediction isn't reliable, and the other predicted states are kept. A wrong record costs a pipeline compiled for nothing during the Builder's preparation, on every player's PC. Could you split the 1,461 into drawn and predicted? Then either ship only the drawn ones, or show what share of the predicted ones match main where they overlap.

minor: the note says that blend enables past the last target are cleared by Lichii, and that Hookshot keys on them only on GPUs without dynamic blend state. On those GPUs, such records will not match what the game asks for. They are harmless but wasted there. Worth a sentence in the note so nobody later reads the mismatch as a bug.

I didn't review the .bin.xz record by record. CI's test_pipeline_state* checks validate its format, and they pass.

@Joey-Magnus Joey-Magnus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the first list in assets/pipeline_states/ that Hookshot didn't record, so it needs a maintainer yes or no on converted lists, and a word on whether Fritchii's recipe may be redistributed under GPL-3.0. There's no game data in it.

If it goes in:

  • The converter should be under scripts/, so the list can be reviewed and regenerated when code_key changes.
  • Predicted states should be split from drawn ones. 728 of the 734 checkable point-topology predictions disagreed with main, though 13,323 of the 19,738 overlapping pipelines match field for field.

The title is cut off with "…".

Erfboom pushed a commit to Erfboom/longshot that referenced this pull request Nov 11, 2026
…s) (NotRexed#2445)

Adds the three pipeline lists posted in NotRexed#2034 that aren't in `main` or
in an open pull request yet. I added each one with
`nx2/pipeline_states.py add`, so each list keeps only the pipelines that
no other contributed list already has. Each one has a note beside it.

## What's added

| List | From | GPU, OS | Build and settings | Route | Pipelines sent |
Kept in the list | New to the shipped list |
|---|---|---|---|---|---|---|---|
| `contrib/yeetbot90-amd-rx-9070-xt-fsr3-4k` | @yeetbot90 (second list,
"MORE PIPELINES") | AMD RX 9070 XT, driver 32.0.32015.7029, Windows 11 |
v2.1.0, 3840x2160, FSR 3.1 Auto, frame generation 3x, HDR on | Kokiri
Forest through Jabu-Jabu's Belly | 2,047 | 1,013 | 992 |
| `contrib/jabbaxipil-rtx-4070-linux-dlaa` | @jabbaxipil | NVIDIA RTX
4070, driver 615.71.09, Ubuntu 26.04.1 (Wayland) | nightly
v2.1.0-rc.3-46, 2560x1440, DLAA (model K), frame generation 3x | child,
by day: Kokiri Forest out to Gerudo Valley | 960 | 778 | 710 |
| `contrib/disgustingusting-web-intel-uhd-intro` | @disgustingusting-web
| Intel UHD (i3-1115G4, integrated), driver 32.0.101.7088, Windows 11 |
v2.1.0, 1920x1080, no frame generation | the intro only | 9 | 6 | 0 (the
tour's list already has all 6) |

All three are version 2 lists (`HKSTATE2`). The merged list grows from
25,793 to 27,441 pipelines (1,648 new; the two big lists share 54). I
kept the Intel list even though it adds nothing today: the tour's list
is replaced whole each time, and this one holds pipelines no other
contributed list has.

## What's left out

- @pmwork-personal's first list, @yeetbot90's first list and @Imasvpp's
list: already in `main` (NotRexed#2075 and NotRexed#2082).
- Lists from @Brett-D (NotRexed#2143), @pmwork-personal (NotRexed#2133) and
@jarynclouatre (NotRexed#2064): each is in its contributor's own open pull
request.
- @disgustingusting-web's `hookshot_log.txt`: it's a log, not a list.

## How it was checked

- I downloaded each zip into its own empty folder outside the repository
and listed it without extracting. Each one held a single
`hookshot_pipeline_states.bin`: no paths, no symlinks, nothing
encrypted, and sane sizes and compression ratios. Only that one file was
extracted, with a size cap.
- Each file starts with `HKSTATE2`, uses 552-byte records, has a whole
number of records, and has no duplicates.
- I also checked every field of every record: mesh, sample, depth and
flag fields in range, no attributes or answers past their counts,
padding zero, and each variant matching its answers' hash. The only
exceptions were records written with no answers, which `tour.bin.xz` has
in the same form.
- `python -I nx2/pipeline_states.py check`: passes. The merge gives the
same bytes twice.
- Tests: `tests/test_pipeline_state_sources.py`,
`test_ship_state_list.py`, `test_pipeline_states.py`,
`test_pipeline_notes_open.py`, `test_pipeline_state_counts.py` and
`test_pipeline_state_portable.py` all pass.

## Not tested

- I didn't run the game: no first-start shader preparation with the new
shipped list, no game build, and no check that these pipelines remove
stutter on these GPUs.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@hckrman101
hckrman101 merged commit b8fd5fd into NotRexed:main Nov 11, 2026
13 checks passed
elkarbeenus pushed a commit to elkarbeenus/LakuenRecompBuilder that referenced this pull request Nov 11, 2026
NotRexed#2133)

Fritchii ships a recipe of the pipelines Ocarina of Time draws, and
1,461 of them are pipelines no list here holds that convert to whole
HKSTATE2 records. This adds them as
`contrib/pmwork-personal-fritchii-recipe`, as discussed in #dev-work.

**What the recipe is.** Fritchii (a Lichii fork) ships `OoT v131072
(cache v54).nx2recipe`: the pipelines its testers' sessions drew plus
the states it predicts. Each shader is named by a fingerprint, and the
recipe holds no code (its format is in `nx2_recipe.h` /
`nx2_shader_extract.h` on Fritchii's `oot-gather-fix` branch).

**How it maps.**
- **Shaders.** The fingerprint is SHA-256 of the stage's control section
(first 0x48 bytes, +0x30..0x33 zeroed), then its code area. `code_key`
hashes the same control and code bytes with FNV, so a player's
`hookshot_shaders.bin` names the recipe's shaders by code key: 41,165 of
41,216.
- **Render state.** It maps by meaning:
- target and depth formats (Maxwell render-target formats to NVN's, one
to one for every format OoT uses)
  - samples and topology class
  - blend factors and equations, into `nvn_blend_st`
  - channel mask, polygon mode, depth clamp and alpha to coverage

**What it can't hold.** The recipe was recorded with dynamic vertex
input, so it has no vertex formats or divisors. It has no texture
answers either. So a pipeline is in this list only when:
- main's lists fix both for its program (one vertex input for the vertex
stage and one variant), or
- it needs neither: mesh draws take no vertex input, and a program none
of whose stages asks about a texture or image has variant 0. All 40 such
programs in main's lists have variant 0 there.

Point-topology states that the recipe predicts for programs main's lists
draw as triangles are left out: 728 of the 734 checkable ones disagreed
with main.

**Checked against main.** I converted every recipe pipeline main's lists
already hold the same way. 13,323 of 19,738 match one of their records
field for field, including blend, masks and clamp. The rest are:
- other states of the same draws, e.g. additive vs alpha blend; or
- blend enables past the last target, which Lichii clears and Hookshot
keys only on a GPU without dynamic blend state.

There was no systematic difference in any field.

**Files.**
-
`assets/pipeline_states/contrib/pmwork-personal-fritchii-recipe.bin.xz`
and `.md`, written by `nx2/pipeline_states.py add`. The note says how
the list was made.
- `assets/pipeline_states/README.md`: one paragraph saying the lists
named `pmwork-personal-fritchii-recipe*` are converted, not recorded.
They hold the same records and nothing more.

The converter is a small script (recipe + `hookshot_shaders.bin` +
main's merged list → HKSTATE2). It isn't in this PR; happy to add it
under `scripts/` if you want the list reproducible.

**Tested.**
- `nx2/pipeline_states.py check`: merged list 26,712 pipelines (main
25,251 + 1,461).
- Tests: `test_pipeline_states`, `test_pipeline_state_sources`,
`test_pipeline_state_counts`, `test_pipeline_state_portable`,
`test_ship_state_list`.
- `python -I -B tests/run_all.py` on main with both PRs (Windows): 342
passed, 2 failed, 30 skipped (Linux-only). The two
(`test_precompile_compute_guard.py`, `test_recorder_sync.py`) fail the
same on main without these changes on this PC.
- In the game's preparation: a build of main with this list and the
follow-up's (NotRexed#2134) ran the Builder's shader preparation (RTX 5090,
driver 621.14):

`vk: state list: 38697 pipelines (38697 shipped, version 2), 14866
programs (1200 translated now, 0 with stages not in the seed), 953
variants by their textures (953 made), 4798 by nothing bound (4798
made), 38584 noted (2438 native mesh, 0 mesh fallback, 11985 by nothing
bound), 113 whose program variant was not made, in 10.2 s`

Every record of this list was noted. The 113 not made are records main's
lists already hold that were noted without answers (1,285 such records
there, none here).

Credit: the recipe is Fritchii's (Frizti), built from its testers'
sessions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit b8fd5fd)
NotRexed pushed a commit that referenced this pull request Nov 12, 2026
… 11,985 pipelines from Fritchii's recipe (#2134)

Lists converted from records that hold no texture answers, like
Fritchii's shader recipe, cannot name a program variant. 11,985 of the
recipe's pipelines that no list here holds have vertex input the lists
here fix but no known variant; a build preparing them with the variant
nothing bound makes builds most of them as the draws will ask.

state_list_prepare: a record with variant 0 and no answers or questions
stands for the variant the program's translation makes with nothing
bound, each texture as its instruction declares it. When the record's
own variant is not found (a program without textures has variant 0
already and is found as before), it is made with seed_plain_of and the
pipeline noted with it. Draws whose textures differ make their own
variant as before; the prepared pipeline is just not used. Not
seed_variant_of with empty answers: that variant would hold no answers
and stand for any textures. The log's state list line counts them ("by
nothing bound").

seed_plain_of shares seed_variant_of's body (seed_variant_made), which
hands the translation questions and answers only when given; without
them it answers what nothing bound answers and notes the questions, as
the seed's own translation does.

contrib/pmwork-personal-fritchii-recipe-plain: those 11,985 records
(11,111 graphics, 874 mesh, 4,785 programs). A build without this rule
leaves them out ("whose program variant was not made").
test_state_list_plain.py checks the rule in the source.

**Since review** (hckrman101):
- The plain variant's own variant is the hash of its answers, not 0, so
`program_find_portable(key, 0)` never found it, and each prepare
translated the list's ~4.8k plain programs again and appended them to
`hookshot_programs.bin`. Now `program_find_plain(key)` finds the loaded
variant by its answers (an answer for every slot, each its declared
type; one `declared_answer` helper shared with `record_answers`), and
`state_list_prepare` queues a plain record only when that finds nothing.
The prepare's log before this fix: a first prepare in a fresh folder
made "4798 by nothing bound (4798 made)", and every later prepare made
them again. With it, even a cold first prepare shows "0 by nothing bound
(0 made)": a plain variant is the program's own first translation, so
the 11,985 pipelines are noted "by nothing bound" with no extra
translation (cost measured in the comments).
- Disjoint lists: by pipeline (the record without its variant, answers
and questions), none of the 11,985 is in #2133's list, any list on main
as of 34f0578, or #2143's lists.
- This lands after #2133 (the contrib note and README text come from
there).
- Tests: `test_state_list_plain.py` checks the lookup and both its uses;
`test_program_answer_domains.py` copies `declared_answer` with
`record_answers`; main's new `test_seed_variant_cleanup.py` (#2323)
copies `seed_variant_made` with `seed_variant_of`. `tests/run_all.py` on
Windows after merging main at 34f0578: 379 passed, 31 skipped, 2
failed, both failing on untouched main here too (`test_recorder_sync`,
`test_vram_budget_row`). A Windows build compiles.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants