Skip to content

fix(build): check Linux prerequisites before reading the dump - #2793

Open
mooki000 wants to merge 3 commits into
NotRexed:mainfrom
mooki000:fix/linux-build-preflight
Open

mooki000 wants to merge 3 commits into
NotRexed:mainfrom
mooki000:fix/linux-build-preflight

Conversation

@mooki000

@mooki000 mooki000 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

What this is for: Linux source builders should see all missing build prerequisites before their game files are read or translated, rather than fixing separate errors minutes into the build. What is measured: real builder starts on Arch Linux (the missing-FFmpeg error appeared after 0.141 s, and a normal start passed preflight after 0.141 s), one complete source build with the preflight in place (493 files, 0 failed) whose game then reached the title screen, and authored dependency and ordering tests. Not measured: gameplay beyond the title, an installed AppImage, or Windows execution.

Refs #1000, specifically the source-build prerequisite report. This does not fix or close the tracker's other Linux/Steam Deck reports.

Problem

  • make_game.main() currently reaches extraction and translation before the Linux compiler check.
  • Missing Python modules can stop extraction. Renderer-header discovery and FFmpeg use happen later, so another missing prerequisite can stop a subsequent attempt.
  • The existing Windows compiler preflight already demonstrates the intended ordering: fail before opening the chosen game.

What changed

  • nx2/make_game.py: check Linux prerequisites before the read stage and report the complete missing set in one @@error message. Mandatory Python module names come from the top-level pins in requirements.txt; the included button-prompt requirements remain optional.
  • nx2/build.py: share renderer-header and prebuilt-renderer selection with the preflight. Keep the existing lookup precedence: CPM caches, explicit header roots, then system prefixes. Extend the existing FFmpeg resolver to explain missing pkg-config entries or files in its selected static prefix.
  • Compiler selection still uses the build's toolchain resolver. The C compile/link probe uses lld and the build's target/sysroot flags. A source checkout still needs clang++ for its native translator even with a prebuilt renderer; the packaged prebuilt path does not require renderer headers or a C++ compile.
  • Windows cross builds keep their Windows toolchain checks, rather than acquiring requirements for Linux FFmpeg development packages. NGX continues to select a source renderer instead of the prebuilt one.
  • Add authored prerequisite/ordering controls, preserve the existing NGX wiring assertion through the shared helper, and document the behavior in docs/development.md.

The check installs nothing and calls no package manager.

How it was tested

Environment: 2026-11-12, Arch-based Omarchy 4.5.0, Linux 7.2.15-1-omarchy, i9-13900K; GCC 16.2.1 and Clang 23.1.3. Base 51da0464b4cf0c7f56b81ebd6bb309325835e175; head 9fc8038b. The real builder starts below ran c8bccacedb9e5f5306aee0d8dac889086873fd1b; the only subsequent commit changes tests, not product code. The PC has an RTX 4090 with NVIDIA 615.78.08, and the builder starts below use only the CPU; the complete build's title run further down is the one part that used the card. Load averages at all three real starts were 34.45 / 56.47 / 51.74; these are startup observations under other work, not frame-rate benchmarks.

Real builder starts

Each start ran the unmodified make_game.py from a temporary archive of its named commit, with separate output/work folders and the existing extracted-game paths. For the missing case, HOOKSHOT_FFMPEG, PKG_CONFIG_LIBDIR and PKG_CONFIG_PATH pointed to an empty temporary directory. Nothing was removed from the system.

Build / setup Observed result Time from process start Exit
Base, FFmpeg hidden Reached Reading your game files without reporting the missing libraries 0.062 s Intentionally stopped at that stage
c8bccace, FFmpeg hidden One prerequisite error naming libavcodec, libswscale, libswresample and libavutil, with their resolution route 0.141 s; process ended at 0.150 s 1
c8bccace, normal dependencies Linux build prerequisites: OK, then the read stage 0.141 s; read stage at 0.157 s Intentionally stopped at the read stage

The failing head start emitted neither a read nor a lift stage and created neither the extracted-code directory nor translation output. No game data was read or translated on that failure path. The normal start's total time through successful preflight was below one second. The controls were deliberately stopped before doing a full build.

The error was:

@@error Missing Linux build prerequisites: FFmpeg: libavcodec, libswscale, libswresample, libavutil (HOOKSHOT_FFMPEG with lib/libavcodec.a first, else pkg-config --cflags/--libs for libavcodec libswscale libswresample libavutil; install the development libraries or set HOOKSHOT_FFMPEG). Command '['pkg-config', '--cflags', 'libavcodec', 'libswscale', 'libswresample', 'libavutil']' returned non-zero exit status 1.

A complete source build, and the game it made at the title

Built 9fc8038b completely from source into a fresh game folder (Clang 23.1.3, the renderer compiled too, eight lift and eight compile workers). The build log begins:

Linux build prerequisites: OK
@@stage read Reading your game files

The preflight passed 0.094 s after the start and the read stage began at 0.104 s. All 493 planned files compiled (compiled in 746s, 0 failed), both binaries linked, exit 0, 14 min 55 s in all. This is one build under other work on the PC (load average 11.5 falling to 11.0), not a build-speed figure.

That game then ran 90 seconds without a window on the RTX 4090 (NVIDIA 615.78.08) with a fresh profile and RX_VK_PRECOMPILE=0. It reached the title and was still there at the end; the log has 0 fatal, crash, device-loss or driver-fault lines. No frame rate is claimed.

The title screen of the game built from source with the prerequisite check in place: logo, New Game and Options

Game-free checks

  • New test: 17 cases pass with GCC and Clang. The production C probe is compiled without linking, so this game-free test needs no lld; the real builder runs above exercise linking. Controls cover valid CPM, precedence, explicit package/include roots, both system prefixes, the selected work-folder cache, prebuilt/NGX selection, optional prompt packages, bundled/partial static FFmpeg, linker failure and Windows cross-target selection. A new guard verifies that cache-style fixtures use their existing injected toolchain-check boundary without querying host dependencies; no production skip switch was added.
  • Fail-before: the same all-missing fixture against base source fails with both compiler selections: ('game', ['read', 'lift']) != (1, []). Main requests the authored extraction and lift before checking the deliberately missing module, headers, pkg-config entries and compilers. The head reports the missing set first.
  • Existing C++ preflight: 4 cases pass with g++ and clang++.
  • CI-like environment: pkg-config's search paths point to an empty directory, HOOKSHOT_FFMPEG, RX_CPM and the VMA override are unset, and only CI's pinned Vulkan/SPIR-V include overrides remain. The first strict whole-suite comparison had no preflight-caused failures: main 475 passed, 0 failed, 20 skipped; c8bccac 475 passed, 1 failed, 20 skipped, with the sole failure a shader-fixture compiler killed by SIGKILL in nx2/test_depth_sample_types.py, which never invokes the builder. Its cause is not established, and the failed output is retained.
  • Absent-linker control: with the real Clang directed at an absent linker in a temporary directory, the original positive fixture fails; the compile-only fixture passes. The separate authored linker-error test still requires the real preflight to reject a failed link before reading the dump.
  • Final strict suites on 9fc8038b, four workers: CI-like 476 passed, 0 failed, 20 skipped in 289 s; normal dependencies 476 passed, 0 failed, 20 skipped in 274 s. Neither final run has a failure, including the shader fixture that was killed in the earlier eight-worker run.
  • Windows-target syntax: the captured production C probe compiles for x86_64-w64-mingw32, preserving its compile flags: one translation unit, no diagnostics. Automatic Windows-selected fixture capture reached no compile because the real Linux probe is explicitly skipped there; the separate capture exercised that exact case. This is not Windows execution.
  • Branch/title checks pass. No core dumps were found after the checks.

Not tested

  • Gameplay beyond the title screen, and full shader preparation (the run above had RX_VK_PRECOMPILE=0); no game area or visual comparison is claimed.
  • An actual packaged AppImage build or a full Windows cross build. Those selection paths are covered with authored fixtures only.
  • Windows execution of the generated compiler probe or the new Python tests; the Windows-target result above is syntax only.
  • Other distributions, AMD/Intel runtime behavior, or physical input devices.

Risks

The new rejection happens earlier, so an incorrectly shared resolution rule could reject a working environment. The tests use the real build resolvers and cover their alternate roots and prebuilt/cross-build routes. Module presence is checked, not package-version compatibility; this does not attempt to certify every possible SDK or library incompatibility.

Overlaps

The open-PR comparison found one additional textual conflict: #1872, the macOS draft, changes FFmpeg's package list/static-library handling and renderer selection in nx2/build.py. Its thread and hunks were read; it is not a competing Linux prerequisite fix. An eventual integration must retain its platform-specific package choices. Other PRs sharing these code files merge cleanly, including the build-loop changes #2675, #2686, #2693 and #2706.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra noreply@openai.com

mooki000 and others added 3 commits November 12, 2026 06:47
Collect missing mandatory Python modules, compiler and linker, renderer headers and FFmpeg development libraries in one diagnostic. Reuse the build resolvers and keep packaged and Windows cross-build selection intact.

Refs NotRexed#1000

Assisted-by: GPT-6 Astra

Co-Authored-By: GPT-6 Astra <noreply@openai.com>
A source checkout builds a64emit with clang++ even when it uses a prebuilt renderer. Keep that prerequisite for source builds while the packaged prebuilt-renderer path needs neither C++ compile. Extend the authored controls and declare the compiler needed by the real probe.

Assisted-by: GPT-6 Astra

Co-Authored-By: GPT-6 Astra <noreply@openai.com>
Assert that cache-style injected build steps use the existing toolchain-check boundary without querying host prerequisites. Compile the production C probe without linking so the game-free fixture does not require lld; authored linker failures and the real CLI starts retain link coverage. No production bypass switch is added.

Assisted-by: GPT-6 Astra

Co-Authored-By: GPT-6 Astra <noreply@openai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant