[Joey-Magnus]

Movies stutter and fall behind their sound on Windows (#2316, now tracked in #2503; on an RTX 4060; a GTX 1050 in Discord). The Windows FFmpeg has no AV1 decoder, so the 1080p AV1 movies go to the AV1 Video Extension's software decoder on every GPU. This PR decodes them with dav1d instead, the same decoder Linux uses.

Why it was Media Foundation's software decoder everywhere

• The FFmpeg in the Builder's cpm part (9.0.2) is configured with --disable-everything plus h264, vp8, vp9 and opus. It has no AV1 decoder of any kind, and not even the AV1 parser.
• host/nn_video.c therefore hands AV1 to host/mf_av1.c. That file asks Media Foundation for its synchronous decoder first, which is the extension's software one; the hardware decoder is used only when no synchronous one exists. So an RTX 4060 decodes the movies in software too, and a PC without the extension gets the Store dialog (#380).
• The catch-up skips pictures that are late, but it still has to decode them. A decoder slower than real time leaves the picture behind the sound whatever the catch-up does.

What changed

• host/dav1d_av1.c wraps dav1d behind the same open, decode, flush and close interface as mf_av1.c, handing back NV12, which nn_video.c keeps as it is. It fetches one picture per submission: a second dav1d_get_picture with no data sent in between makes dav1d drain, which waits for every frame in flight and leaves its frame threads decoding one picture at a time. At the end of the stream it drains everything. HOS_DAV1D_THREADS=<n> sets the thread count (default: dav1d's, one per logical core). HOS_NO_DAV1D=1 goes back to Media Foundation.
• host/nn_video.c turns "our own AV1 decoder" into a small table of functions: dav1d first when it is built in, then Media Foundation. Nothing changes when FFmpeg has its own AV1 decoder (Linux).
• One log line per movie (always on): the decoder, the mean and worst decode time per submission, the time between pictures, and how many pictures the catch-up skipped. Its format: video: a movie of <n> pictures on <decoder>: <mean> ms to decode each (worst <ms>), one shown every <ms> ms; <n> skipped to keep up with its sound. A slower-than-real-time decoder is flagged "(slower than it plays)". The next IN GAME MOVIES STUTTER OR OUT OF SYNCH AUDIO #2316 log will show whether decoding is what falls behind.
• scripts/windows/dav1d.sh builds dav1d with llvm-mingw from the same release and source-tree hash that scripts/linux/deps.sh pins (1.5.4; it reads them from there). It installs into the CPM layout <cpm>/dav1d/1.5.4/{include,lib/libdav1d.a,COPYING}. The output is reproducible: two builds give the same libdav1d.a (SHA-256 e6f1adec…7e98 with llvm-mingw 20261104). That took clang's -mno-incremental-linker-compatible and nasm's --reproducible; without them the COFF objects carry timestamps.
• nx2/build.py compiles dav1d_av1.c and links libdav1d.a when the CPM cache has dav1d, on the llvm-mingw route only (it is a MinGW library). Without it, or on the Visual Studio route, nothing changes and Media Foundation decodes the movies as before.
• CI: a new Linux job in windows-builder.yml builds dav1d with the pinned Linux-hosted llvm-mingw and runs the wrapper's test on the Windows build under Wine. It uploads the CPM folder as the dav1d-windows artifact. The Linux test job installs libdav1d-dev for the new test.

How dav1d reaches players

Players build the game on their own PC, so dav1d has to reach them in the Builder's cpm part. The cpm part is normally reused from the latest release. This PR makes the release workflow add dav1d to it: the package job takes the dav1d-windows artifact and runs release_parts.py package --cpm-add. If the latest release's cpm part lacks dav1d/1.5.4, the job unpacks that part into tools/cpm with dav1d added. make_setup_parts.py then makes a new cpm part, and the release job uploads it to builder-parts like any other new part. Once a release carries it, later runs reuse it. No manual step is needed.

I checked this against the real v2.1.0 setup list and the dav1d built here. The new part is HookshotBuilder-cpm-73d3c9f5a31d.zip (19.6 MB, up from 18.6 MB). It holds the 451 files of HookshotBuilder-cpm-c30c77e4cf00.zip byte-for-byte, plus dav1d's 8 files. Two runs gave the same SHA-256, because the zip and libdav1d.a are both reproducible.

Testing

• tests/test_dav1d_av1.py (new) uses a 3.8 KB synthetic clip in tests/dav1d_av1/: 12 pictures of a formula pattern, encoded losslessly with libaom, so no game data. Every picture must come out in order, with its pts, as the exact NV12 bytes of the pattern. That includes the pictures dav1d holds back until the end, and a second pass after a flush. It runs at 1 and 4 threads, and checks that HOS_NO_DAV1D=1 leaves the decoder closed. It passes on Linux with clang, clang++, gcc and g++ against the system dav1d. It also passes for the Windows build (llvm-mingw, the script's libdav1d.a) under Wine at 1 thread, 4 threads and the default. Breaking the end-of-stream drain makes it fail.
• tests/test_cpm_discovery.py: dav1d is used only on the llvm-mingw route, never on the Visual Studio route or Linux, and only with both its header and library present.
• tests/test_release_cpm_add.py: --cpm-add adds dav1d to a cpm part that lacks it and keeps every file the part had. It leaves a part that already has dav1d to be reused as it is, and adds a new version beside an old one.
• tests/test_vcore_video_acquire.py and tests/test_run_all_declarations.py were updated for the new type and tool.
• python -I -B tests/run_all.py: 420 passed, 0 failed on Linux, rebased on current main.
• host/nn_video.c compiles for Windows with llvm-mingw and the cpm part's own FFmpeg headers, with and without HOOKSHOT_DAV1D, and for Linux.
• Speed, on a synthetic 1080p AV1 clip (SVT-AV1, 300 pictures). The Windows build under Wine, on a Ryzen 7 9800X3D: 2.8 ms per picture on 1 thread, 1.9 ms on 4, 1.4 ms on all 16. A 30 fps movie allows 33 ms per picture. The real movies are harder to decode than this clip.

Not tested

• The game on Windows. I'm on Linux with no Windows install, so I need a Windows tester to build with dav1d in the CPM cache and play the intro movie. Ideally on a PC that stutters today (IN GAME MOVIES STUTTER OR OUT OF SYNCH AUDIO #2316's RTX 4060, or a GTX 10/16), with the new log line before and after.
• No timing comparison against Media Foundation, since Wine has no AV1 Video Extension.
• The new CI job runs for the first time on this PR.
• 10-bit and 4:4:4 conversion paths (OoT's movies are 8-bit 4:2:0).

No screenshots: nothing new shows on screen. The movies should just play in sync, and the extension's Store dialog no longer appears once dav1d is in the Builder.

[Gonzxlx]

wouldn't it be better to make use of hardware acceleration instead of relying on software decoding? just curious