Skip to content

Monorepo step 3: vendor JsRuntimeHost and UrlLib with full file history - #1947

Open
CedricGuillemet wants to merge 250 commits into
BabylonJS:masterfrom
CedricGuillemet:cedricguillemet-monorepo-step-3-jsruntimehost-urllib
Open

CedricGuillemet wants to merge 250 commits into
BabylonJS:masterfrom
CedricGuillemet:cedricguillemet-monorepo-step-3-jsruntimehost-urllib

Conversation

@CedricGuillemet

@CedricGuillemet CedricGuillemet commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

JsRuntimeHost and UrlLib are maintained in separate repositories even though their development is moving into BabylonNative. Bringing them into this repository lets changes to the host, URL handling, and their consumers be reviewed together, continuing the monorepo migration from #1944.

The import retains the complete reachable histories: 151 commits from JsRuntimeHost and 53 from UrlLib. These revisions match the previous dependency pins, so moving the sources does not intentionally upgrade either component. Rewriting paths changes commit IDs and invalidates original signatures, but preserves authorship, messages, and ancestry.

This needs a history-preserving merge, not a squash merge. Squashing would discard the imported ancestry that makes file history and blame useful. The repository's squash-only merge setting therefore needs maintainer coordination for landing, as with step 2.

Worth a look

  • Public targets, include paths, and default consumer build options are intended to remain compatible. Embedded component tests follow BABYLON_NATIVE_BUILD_APPS and the same supported-platform condition as BabylonNative's unit tests, without separate component switches. Apps-disabled consumers do not build tests; Universal Windows Platform (UWP) and visionOS remain excluded.
  • The bgfx wrapper's upstream CMake entrypoints stay separate from BabylonNative's source-selection and packaging policy. The existing bgfx, bx, and bimg revisions are retained, without the local GL-context patch; local and disconnected source builds must keep working.
  • Mobile CI runs only JsRuntimeHost and UrlLib, consuming named artifacts from the classic platform builds without rebuilding in emulator jobs; BabylonNative unit-test applications are neither uploaded nor run. Packaging and sequential execution live directly in YAML, using standard artifact/cache actions, Android Emulator Runner, and the Marketplace Launch iOS Simulator action. Android caches a clean emulator snapshot before installing test apps, keys it by toolchain and configuration, and does not save test state back into the cache. Boot and active-network readiness are checked after restore. The final CI check requires successful execution and rejects failed, cancelled, or unexpectedly skipped dependencies; iOSTests gates its iOS build transitively.
  • Imported license and notice obligations remain covered by the repository's consolidated attribution. The original files remain available in the imported history.

Verification limits

Historical verification covered every imported commit's metadata, full message, parent mapping, file modes, and blob identities, including empty commits. This evidence is not represented by the ordinary build checks.

The mobile checks cover component behavior, not BabylonNative's renderer; desktop results do not establish mobile renderer coverage. iOS simulator execution requires CI's macOS runner and cannot be validated on this Windows host.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Drop redundant local wrapper existence precheck and load the shared helper from HeadlessScreenshotApp, with a global include guard.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Comment thread Install/Test/CMakeLists.txt Outdated
Comment thread Dependencies/CMakeLists.txt Outdated
Comment thread Dependencies/CMakeLists.txt Outdated
Comment thread Dependencies/CMakeLists.txt Outdated
Comment thread Dependencies/CMakeLists.txt Outdated
Comment thread .github/workflows/build-android.yml Outdated
Comment thread .github/workflows/build-android.yml Outdated
Comment thread Dependencies/bgfx.cmake/README.md Outdated
Comment thread Dependencies/bgfx.cmake/UPSTREAM.md Outdated
Comment thread Dependencies/CMakeExtensions/CMakeLists.txt
Load CMakeExtensions once in the root build while preserving independent component builds. Restore the original bgfx pin, centralize bgfx child pins, use direct subdirectories, remove duplicate component notices and update affected CI executable paths.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Run JsRuntimeHost desktop and mobile tests for JsRuntimeHost changes, and UrlLib desktop tests for UrlLib changes. Keep both suites enabled on master pushes and for local builds by default; validate each desktop target can be enabled independently.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Drop redundant empty submodule selections for child repositories without gitmodules. Use conditional includes in independent component builds while the BabylonNative root provides the shared helpers directly.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Move UrlLib and JsRuntimeHost build/test steps into per-component platform actions, including the runtime Android/iOS harnesses. Preserve separate PR source-change gates, trigger checks for action changes, and document the action layout for AI contributors.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Replace composite actions with top-level JsRuntimeHost and UrlLib workflow_call workflows. Configure and build components on independent runners after platform builds, preserve per-component change gates and engine/compiler coverage, and make ci-complete validate intentional skips. Update AI placement guidance and build-system documentation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7

@bghgary bghgary left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Reviewed by Copilot on behalf of @bghgary]

Concerns inline.

Comment thread .github/instructions/component-unit-tests.instructions.md Outdated
Comment thread Dependencies/AndroidExtensions/CMakeLists.txt Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread Dependencies/JsRuntimeHost/CMakeLists.txt Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread Documentation/BuildSystem.md Outdated
@@ -4,8 +4,12 @@
design. Most external sources are now acquired with CMake FetchContent.
`Dependencies/bgfx.cmake` is tracked wrapper code with imported file history;
its bgfx, bx and bimg sources remain pinned external FetchContent acquisitions.
AndroidExtensions and CMakeExtensions are vendored with imported history under
`Dependencies/`; root source overrides make all FetchContent declarations use them.
AndroidExtensions, CMakeExtensions, JsRuntimeHost and UrlLib are vendored with

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update BUILDING.md as well. It still advertises the removed wrapper override and _deps/bgfx.cmake-build directory, describes JsRuntimeHost as fetched, and says the root sets FetchContent overrides for the in-tree components.

Comment thread Dependencies/CMakeLists.txt Outdated
else()
set(JSRUNTIMEHOST_TESTS OFF)
endif()
add_subdirectory(JsRuntimeHost "${CMAKE_BINARY_DIR}/_deps/jsruntimehost-build")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please retain EXCLUDE_FROM_ALL on these add_subdirectory calls. The earlier request was to remove the explicit binary directory, not EXCLUDE_FROM_ALL.

Comment thread Dependencies/bgfx.cmake/CMakeLists.txt Outdated
include("${CMAKE_CURRENT_SOURCE_DIR}/cmake/dependencies.cmake")
include(FetchContent)

FetchContent_Declare(bx

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please keep the imported wrapper identical to upstream and put BabylonNative-specific source selection and packaging outside it. Upstream already accepts BGFX_DIR, BX_DIR and BIMG_DIR; if the version changes fix a case upstream cannot handle, please show that case and propose the fix upstream.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: make sure same as upstream

CedricGuillemet and others added 20 commits October 21, 2026 19:34
Guard CMakeExtensions in its own script and remove caller-side checks. Copy the upstream bgfx entrypoint and fallback pins unchanged, moving BabylonNative source selection and staging outside the wrapper while preserving root pins.

Move component matrices into jsruntimehost-ci.yml and urllib-ci.yml with explicit completion gates. Remove runtime-source-changes and the component instruction file. Keep the unified required check reliable by running both callable suites on every CI event: native paths filters do not apply to workflow_call.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
…orepo-step-3-jsruntimehost-urllib

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7

# Conflicts:
#	.github/workflows/build-android.yml
Detect component, component-workflow and shared-helper changes before calling the component orchestrators. Accept intentional skips only for unchanged components, and reject filter failures, invalid outputs or unexpected test results in both component checks and ci-complete. Keep master pushes fully covered.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Build named Android and iOS unit-test application artifacts for BabylonNative, JsRuntimeHost and UrlLib. Run each platform's artifacts sequentially on one emulator or simulator and require both jobs in ci-complete. Add real shared mobile harnesses, remove component path filtering and obsolete standalone desktop workflows, and retain embedded desktop and sanitizer coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Reproduce missing Android SDK CMake and npm workspace installation ordering regressions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Install the Gradle harness's pinned Android SDK CMake and install Apps npm workspaces before child configuration invokes webpack. Fix the nine observed Android CI failures and the equivalent clean iOS ordering issue.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Keep component build workflows and shared Android/iOS execution directly gated by ci-complete; remove duplicate status-only jobs and their needs entries.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Disable the iOS toolchain's global ARC default before project initialization and clean Android packages after failures. Keep the bgfx revision pinned while correcting context-local VAO lifetimes, headless EGL config retry, and depth/stencil framebuffer blits. The Android suite now completes; two further GPU test failures remain visible rather than being suppressed.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Produce all three named test applications inside each existing Android or iOS build job. Remove separate component matrices and artifact-producer jobs, and depend only on classic platform builds before shared emulator execution. Preserve the nine Android and three iOS artifact identities and fail-closed final gate.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Remove the unpublished native Canvas regression now outside the requested mobile CI scope.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Exclude BabylonNative mobile unit-test builds and artifacts while retaining classic build dependencies and strict component execution gates.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Key snapshots by runner image, emulator and system-image revisions, and workflow configuration. Save before installing tests, restore without snapshot saving, and wait for a validated active network after boot.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Remove the local bgfx GL patch and Python mobile CI helpers. Use native packaging tools and YAML runner steps, retain isolated sequential component tests and clean Android snapshots, and let iOSTests transitively gate the iOS build.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Enable UrlLib and JsRuntimeHost tests with the existing app and unit-test platform gate, including Android. Remove redundant per-component BabylonNative options and CI arguments, keeping apps-disabled consumers and unsupported app-test platforms excluded.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e36eed22-9b7f-4098-b915-8a84e714ebf7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.