Generate a static module entrypoint behind a staticEntrypoint setting - #28
Draft
eunomie wants to merge 8 commits into
Draft
Generate a static module entrypoint behind a staticEntrypoint setting#28eunomie wants to merge 8 commits into
eunomie wants to merge 8 commits into
Conversation
Generate a manifest v2 entrypoint whose types() is written at dagger generate time, by importing the module in its own container and reading the runtime registry, gated by a staticEntrypoint SDK setting rolled out in two phases with the dynamic path kept. Signed-off-by: Yves Brissaud <yves@dagger.io>
Module.describe() walks the registry into plain dataclasses, and _typedefs() and to_typedef() materialise them into API calls. A static entrypoint renders the same description at generate time. Signed-off-by: Yves Brissaud <yves@dagger.io>
python -m dagger.mod entrypoint imports the module, describes it and writes types.dang (a literal TypeDef list) and main.dang (the ModuleEntrypoint that calls the module's container). main.dang bakes the content digests of the module's sources, so a stale entrypoint refuses to run instead of serving wrong types. Signed-off-by: Yves Brissaud <yves@dagger.io>
python -m dagger.mod call reads the request a ModuleEntrypoint forwards, runs it through the registry and writes the JSON result to a file, so the generated entrypoint never depends on stdout. Signed-off-by: Yves Brissaud <yves@dagger.io>
PythonModuleBuild in runtime/build.dang builds a module's container from its committed files; the runtime module adds runtime.py on top of it, and the generated static entrypoint carries a copy of the same build. Signed-off-by: Yves Brissaud <yves@dagger.io>
…setting With staticEntrypoint = true, `dagger generate` builds the module's container, renders its types into sdk/entrypoint/types.dang and writes a manifest version 2 that points at that entrypoint. The dynamic runtime stays the default and a module switches back by turning the setting off. Settings a version 2 manifest cannot carry (module clients, include, disableDefaultFunctionCaching, another runtime) are refused instead of silently dropped. Signed-off-by: Yves Brissaud <yves@dagger.io>
Signed-off-by: Yves Brissaud <yves@dagger.io>
The static path is in the repository, green in CI, and loads on a dev engine with the version 2 loader. Flipping the default waits for a released engine. Signed-off-by: Yves Brissaud <yves@dagger.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Static module entrypoint (opt-in)
With the SDK setting
staticEntrypoint = true,dagger generatewrites a manifest version 2 module whose types the engine loads without running Python.How it works
dagger generatebuilds the module's container, imports the module inside it (python -m dagger.mod entrypoint) and renders what its decorators registered intosdk/entrypoint/types.dang. The types are the ones the runtime would register, from the same code path (Module.describe()feeds bothto_typedefand the renderer).sdk/entrypoint/main.dangimplementsModuleEntrypoint:types()returns the literalTypeDeflist;call()runspython -m dagger.mod callin the module's container, which dispatches through the existing registry. A per-file content digest of every source file that can change the types guards against a stale entrypoint: a call after an edit is refused with a message to rundagger generate.sdk/entrypoint/build.dangis a copy of the newruntime/build.dang(the container build, split out ofruntime/main.dangbehindtype PythonModuleBuild) with the image pins inlined, so the entrypoint needs nothing but itself.dagger module init python --path <module> --static-entrypoint[=false]anddagger generate.What version 2 cannot carry is refused at
dagger generateinstead of dropped: module clients,cache=on a function, thelegacytemplate,include, asourceother than.,disableDefaultFunctionCaching, another runtime, and thecodegen,clientsordependenciestables.Design and history in
future/static-module-entrypoint.md, including why the types come from importing the module rather than a static analyzer (dagger/dagger#11803 and its removal in #13251).Testing
sdk/tests/mod:test_describe.py(parity with the previousto_typedefdecisions),test_entrypoint.py(goldens for the rendered Dang, digests in the engine'sFile.digest(excludeMetadata: true)format, theentrypointsubcommand),test_dispatch.py(thecallsubcommand).static-scope-init,static-scope-switch,static-scope-refusals,static-types-loadnext to the existing checks; all green locally on the CI engine.dagger functionson a generated module takes 1.2 s against 2.9 s on the dynamic path; calls run end to end; an edited or added source file, or a lock file added later, is refused naming the file; a.venv/or a permission-only change is not.CI's engine does not load version 2 yet, so the e2e checks prove generation and refusals; loading is covered by the dev-engine run above.
Follow-ups