Repository navigation
feat: an OpenCLI model, a clap → OpenCLI adapter and --mount for component CLIs (0.4.0) - #6
Merged
Merged
Conversation
A new library crate, versioned 0.1.0 on its own:
- Document and the types under it model the flat OpenCLI shape the chain
reads and writes. Every object keeps the keys it does not type in
`extensions`, so x-* keys round-trip; writing uses one canonical key
order (the root x-openapi right after info, as the converter writes it).
- Reading also accepts the upstream root-`command` shape and lifts it to
flat (lift_root_command); a document with both shapes is an error.
- Three generic extensions are typed: x-interactive {commands, keys},
x-environment [{name, description, values}] and x-passthrough {program}.
schema/opencli-extensions.schema.json is their contract, exported as
EXTENSIONS_SCHEMA_JSON beside the base schema (SCHEMA_JSON).
- The vendored base schema moves here from openapi2opencli/tests, byte for
byte; the converter's schema test reads it from the crate and also
round-trips every committed OpenCLI document through the typed model.
merge_fragment and its unit tests move from cli/src/opencli.rs into opencli_model byte for byte, so every document the binary merged before merges identically now (the 2.declare golden is unchanged). The binary keeps the command plumbing and calls the library.
…and x-* keys A fragment may now also set the root `examples` and `interactive` and any `x-*` key; each replaces what the document has, in place when present and otherwise where the converter would write it. `x-openapi` stays the converter's (a fragment setting it is an error), `x-passthrough` marks a command and never the root, and `x-interactive` / `x-environment` must have their typed shape. Documents merged before merge byte-identically (the 2.declare golden is unchanged).
…nt-drop `opensdk opencli ... --mount NAME=DOC [--mount-drop NAME:OPTION]...` runs the converter, then the --declare fragments, then each mount: a component CLI's OpenCLI document is placed at the command NAME (`parent/name` mounts under `parent`) by opencli_model::mount, under rules M1-M7: - M1: the component is an OpenCLI document (the root-`command` shape is lifted) whose conventions equal the host's; - M2: a command of that name may exist only as a passthrough stub (x-passthrough, no arguments, options or subcommands), which is filled and keeps its own name, aliases, hidden, marker and description; otherwise the command is appended under the spelling-collision rule; twice is an error; - M3: the component's root arguments, options, commands, exitCodes, examples, interactive, metadata and x-* keys move onto the command, its info.version becomes the metadata `componentVersion`, and an x-openapi binding anywhere is an error; - M4: the host's recursive options in scope are compared with every option in the mounted subtree by every spelling; a match is an error naming both unless the component's option is dropped, and a drop matching nothing is an error; - M5/M6: the mounted exit codes govern the subtree, nothing else is filtered, and the result is written in canonical key order (the host's own keys keep their order); - M7: one line per mount on stderr, `mounted <path> <version>: dropped --<option>`. The 3.mount fixture covers a filled stub with a drop and an appended component in the root-`command` shape; every rule has its error test. The schema test now validates every committed document against the extensions schema too, the mount goldens and components included.
`generate --spec <doc>` and `opencli --spec <doc>` lift a document whose root command sits under `command` (the upstream draft's shape) to the flat shape the backends and the merge take, through opencli_model::lift_root_command. A flat document is unchanged; one with both shapes at once is an error. Such a document generates the same tree as its flat twin.
A command marked `x-passthrough` hands every argument after it, unparsed, to another program. opencli2rust now emits exactly the hand-written shape for it: `.disable_help_flag(true)` and one `args` positional (`num_args(0..)`, `trailing_var_arg`, `allow_hyphen_values`, parsed to `OsString`), so `--help` after the command is the program's and a recursive root option is the host's only before the command or right after it. The command is bound through the generated `Commands` trait (`fn <name>(ctx, m)`), at any depth. Its documented arguments, options and subcommands are never read: no clap args, no handlers, and `root_options::check` skips them (it still checks the generated `args` positional). The marker on the root, or on a command that also binds an HTTP request, is a generation error. 13.passthrough is the golden (a top-level and a nested passthrough whose documented `--profile`/`-o` would collide with the recursive root options); runtime_smoke builds it and checks the argument hand-off. Every existing golden is byte-identical.
`try_opencli2go` returns an error naming the first command marked `x-passthrough` (or the document root); `opencli2go` panics with it, and `generate --lang go-cli` reports it. Walking a passthrough's documented children as generated commands would build a CLI that parses another program's flags, so the document is refused instead.
A new library crate, versioned 0.1.0 on its own. `from_command(&cmd)` reads a clap command tree (builder or derive) and `build()` returns an opencli_model::Document. There is no proc-macro, and clap is taken with `default-features = false` and `std`, so a CLI's clap features are left as they are; the opt-in `env` feature enables clap's `env` and maps `Arg::env`. The walk reads the tree after `Command::build` and compares it with the unbuilt tree: - clap's `help` subcommand and every help or version argument are skipped, matched by action, so an argument a CLI names `version` stays; - a global is `recursive: true` where it is declared, and the copies clap spreads to the subcommands are not written (a global declared on a subcommand stays there); - commands: name, visible aliases, `long_about` else `about`, hidden; the root's bin name, version, about and long_about fill `info`; - positionals in index order with required, arity (unless exactly one), the non-hidden possible values, the help heading as group; options with the long name else the short, the short and visible aliases, one argument per value name, no arguments and no default for a flag; - metadata `default` (unless hidden), `env`, `repeatable` (Append, Count), `conflictsWith` (groups unrolled), `exclusive`, `last`; - hidden commands and arguments are marked `hidden: true` (Hidden::Mark, the default) or left out (Hidden::Omit). What clap cannot know is declared beside it: exit codes, examples, x-environment entries, `interactive`, x-interactive commands and keys, x- extension keys, title/version/summary/description overrides, and per-command extras through `at(&path, |c| ...)` (an unknown path is an error; `passthrough` marks a command x-passthrough). `interactive_from_command` turns a REPL parsed with clap into x-interactive commands, and `snapshot::assert_file` keeps a committed document current in a test (`OPENCLI_UPDATE=1` rewrites it). Twelve goldens under __fixtures__ pin the mapping, built from builder and derive CLIs; each is read back through the typed model and validated against the base and the extensions schemas.
…nd mount The README's example runs as a doctest.
cli/opencli.json describes the `opensdk` binary: clap_opencli builds it from `opensdk::command::command()`, with the exit codes (and `diff`'s own 0/1/2 table) and examples declared in cli/tests/reference.rs. The test fails while the committed file differs from the clap tree; `OPENCLI_UPDATE=1 cargo test -p opensdk --test reference` rewrites it. The document is validated against the base and the extensions schemas and round-tripped through the typed model with every other committed OpenCLI document. scripts/release.sh bumps its `info.version` with the manifest's and commits it, so a release commit keeps the test green.
The rust job also lints and tests clap_opencli with its `env` feature, which a workspace build leaves off. The new clap-floor job locks clap to 4.5.58 (`cargo update -p clap --precise`), checks the lock took, and runs the adapter's suite with and without `env`, goldens included: the lock holds the newest 4.x, so nothing else builds the floor the crate claims.
…t and x-passthrough The changelog's unreleased 0.4.0 section lists opencli_model and clap_opencli (each versioned on its own from 0.1.0, which the header now says), --mount/--mount-drop, the root-command shape, x-passthrough in both generators and the dogfood document. The README's layout gains the two crates, the schema's new home, the mount and passthrough paragraphs and the widened --declare rules.
…re under Omit The walk records a global as declared before it decides to leave a hidden one out, so its copies further down are always recognized as copies.
…racter long alias is left out
The chain names an option by its long name and reads a one-character
alias as a short, so a one-character name is a long (`--x`). A
short-only clap option written as {"name": "x"} therefore read as
`--x`, which clap rejects, in every reader (the model's spellings, the
generators, a docs site), and a one-character long alias (`--y`) read
as the short `-y`.
build() and interactive_from_command() now return an error naming the
command and the option when an option has only a short (a hidden one
left out under Hidden::Omit is no error), and one-character long
aliases are left out. interactive_from_command returns a Result for it.
Golden 1 pins a one-character long (`--x` with the short `-y`); golden
6 leaves out a one-character visible alias.
A positional whose action is Append keeps collecting every value that follows (clap's usage shows it as `<name>...`), but with the default num_args (1..=1) it was written with no arity, which says exactly one value. Only derive's Vec<T> positionals, which set num_args(1..), came out right. A positional with Append now has no maximum in its arity. Golden 2 gains a builder case, checked against clap's own parse.
An ArgGroup that is not `multiple` (clap's default) makes its members exclude each other, and clap rejects `--csv --tsv` for such a group, but conflictsWith only read the conflicts declared on the argument itself, so the members were documented with no conflict. conflictsWith now adds the other members of every group the argument belongs to that is not `multiple`. A group's own conflicts_with has no public getter in clap; it joins the 'Not mapped' list in the crate docs, the README and the changelog. Golden 7 is re-blessed and gains a `multiple` group, whose members stay free; both are checked against clap's own parse.
The walk compares the built tree with the one it is given, taken as unbuilt, to tell clap's additions from the CLI's own. Given a command already built (by Command::build, a get_matches*_mut call or a completion generator), it documented clap's help subcommand tree and repeated every global as recursive on each subcommand, and returned Ok. build() and interactive_from_command() now return an error for a root that holds clap's own help flag, version flag or help subcommand while the switch that disables it is off: an unbuilt command cannot hold them then, since clap asserts ids and subcommand names unique. The docs of from_command, the crate, the README and the changelog say the command must be unbuilt. A test covers build(), a parse through _mut, the flags-only cases, a REPL, and a CLI's own help flag and help subcommand, which stay valid.
…::assert_file The diff walked str::lines, which ignores a final newline, so a committed file that differed only by a missing or extra newline at its end failed the check and then reported both sides as "(end of file)", or an empty line against the end, contrary to the promise that the message names the first line that differs. A difference only in the newlines at the end is now named as such, with the count on each side, and a difference in line endings that str::lines hides (a stray carriage return) too; an empty line shows as "(an empty line)". Unit tests cover the stripped final newline, an extra blank line under CRLF, a stray carriage return, a line added at the end and a blank line inside.
… mount fills At::passthrough only set the marker and kept what the walk mapped from the clap command. A real passthrough command in clap holds one trailing positional that collects the arguments (the hand-written shape, which opencli2rust generates too), so the document carried that `args` positional, the command was no passthrough stub, and mount refused it: a hand-written host described by clap_opencli could not be mounted into, and its docs showed the host's plumbing as the program's argument. The walk now leaves out the arguments, options and subcommands of a command marked passthrough, keeping its name, aliases, description and hidden flag. Golden 13 is built from the exact hand-written shape, and a test mounts golden 11's agent CLI into it.
mount() applied each drop in turn: the first removed the option, and a repeat (`profile` twice, or `profile` and `--profile`, which trim to one name) then matched nothing and failed with "matches no option of the component", although the option exists. The dedupe beside it could never run. A drop already applied is now skipped before the count is checked, so only a name that matches nothing in the subtree is an error. A unit test and a CLI test cover the repeats; a name matching nothing after a repeat is still an error.
…the host The model, the extensions schema and the README described x-passthrough as "the host hands every argument after this command, unparsed, to another program", which becomes a public contract docs converters render. The generated shape (and the hand-written one it copies) is narrower: the host parses its own recursive options given right after the command, and a `--` right after the command or after those options only ends them and is not handed on. The model docs, the schema description, both READMEs, the changelog and At::passthrough now state both exceptions; the generated shape is unchanged. The runtime smoke pins them: `agent --` hands nothing over, `agent --profile prod -- -p` hands `-p` with the profile parsed, and a later `--` is the program's.
interactive_from_command walked the REPL root as a CLI root, kept only its subcommands and dropped the root's own options. The root's globals were recorded as declared there, so the copies clap spreads to each subcommand were skipped too, and a global every session command accepts (`model --json`) appeared on none. The session commands are now walked with no inherited globals, so the copies are written on each one, recursive, and not again below it. The root's other arguments stay out, as documented. A test covers a REPL parsed with no binary name (a multicall root cannot hold arguments).
clap_opencli requires clap "4.5", which admits 4.5.0 to 4.5.57 too, but the floor job ran 4.5.58, the newest 4.5.x: a getter added anywhere in 4.5.1 to 4.5.58 would pass CI and break a consumer locked on an older 4.5 release. The job now runs `cargo update -p clap --precise 4.5.0` (it resolves, and the suite, goldens included, passes there with and without the env feature). A test in clap_opencli keeps the pin equal to the oldest release the manifest's requirement admits.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A host CLI that runs a component CLI as one of its commands (
acme agent ...handing every argument to an agent CLI) cannot document that component in its
OpenCLI reference today:
CLI has no document at all;
generators and docs converters cannot tell it apart from an ordinary command.
This release adds the pieces for that. There is a typed model with three
generic extensions. A library describes a clap CLI as an OpenCLI document. A
mount places a component's document at one command of a host's document. The
Rust generator gets the passthrough shape.
What
opencli_model0.1.0 (new crate, no clap):x-*keys round-trip, and output uses one canonical key order.
commandshape is read too, and lifted to flat.x-interactive {commands, keys},x-environment [{name, description, values}]andx-passthrough {program}.schema/opencli-extensions.schema.json. It is exportedas
EXTENSIONS_SCHEMA_JSON, beside the base schema,SCHEMA_JSON.merge_fragment, the--declaremerge, moves here unchanged. It then alsotakes root
examples,interactiveandx-*keys.mountfollows rules M1–M7: equal conventions, the stub fill or append,what moves, recursive-option collisions with drops, exit codes, no
filtering, and a one-line report.
clap_opencli0.1.0 (new crate):from_command(&cmd)...build()maps a builder or derive tree into anopencli_model::Document. It covers commands, positionals with arity,options with their visible aliases, and globals marked
recursiveonlywhere they are declared. Metadata:
default,env(theenvfeature),repeatable,conflictsWith,exclusiveandlast. Hidden items aremarked or left out.
environment variables,
interactive, a REPL's commands(
interactive_from_command),x-*keys and per-command extras(
at(&path, ...)).snapshot::assert_filekeeps a committed document current from a test.Supported: clap 4.5.0 and every later 4.x.
opensdk opencli ... --mount NAME=DOC [--mount-drop NAME:OPTION]...: afterthe converter and the
--declarefragments, each mount places a componentdocument and prints one report line on stderr.
x-passthrough:rust-cligenerates the hand-written shape: the help flag is disabled, onetrailing
OsStringpositional takes the arguments, and the handler isbound through the
Commandstrait. It never reads the command's children.go-clirefuses the marker for now.cli/opencli.jsonis built byclap_opencli, and adrift test checks it.
clap_opencli, plus lint and test runs ofits
envfeature.Review fixes in this round
{"name": "x"}, which everyreader takes as
--x. It is now an error that names the command and theoption. The chain keeps its rule that a one-character name is a long.
One-character long aliases are left out, because a one-character alias
reads as a short.
interactive_from_commandnow returns aResult.Appendand the defaultnum_argstakes anynumber of values. Its arity now has no maximum.
ArgGroupthat is notmultiple(clap's default)exclude each other, and they now get
conflictsWith. A group's ownconflicts_withhas no public getter in clap, so it is listed under "Notmapped".
Command::build,get_matches*_mutor a completion generator) is refused. Before, itsilently gave clap's help subcommand tree and a copy of every global.
snapshot::assert_filenow says so, instead of reporting "(end of file)" onboth sides.
passthroughnow comes out as a stub that a mount canfill. What clap parses for it (the trailing positional) is the host's
plumbing, so it is left out. A new golden is built from the exact
hand-written shape, and a test mounts a component into it.
--mount-dropgiven twice, or given once with dashes and once without,is applied once. Before, it failed with "matches no option".
x-passthroughcontract now states two exceptions. The host parses itsown recursive options given right after the command. A
--placed rightafter the command, or after those options, only ends them. The model docs,
the schema description, both READMEs and the changelog say this, and the
runtime smoke pins it.
admits. Before, it used 4.5.58. A test keeps the pin and the manifest in
step.
Each fix is its own commit, with a test.
How it was checked
cargo fmt --all -- --check: clean.cargo clippy --workspace --all-targets -- -D warnings: clean.cargo clippy -p clap_opencli --all-targets --features env -- -D warnings:clean.
XYD_SMOKE_RUST=1 cargo test --workspace: 644 passed, 0 failed, across 130test binaries. This includes the
13.passthroughruntime smoke with the new--cases. The other languages' compile smokes were skipped locally and runin CI.
cargo test -p clap_opencli --features env: 30 passed, 0 failed.cargo update -p clap --precise 4.5.0): 29 passed withoutfeatures and 30 with
--features env. The goldens are byte-identical at thefloor.
scripts/check-standalone.sh: OK.cion this branch, run 43659373307): all four jobs green: rust (fmt +clippy + goldens), cli smokes, e2e, and the clap 4.5 floor (
clap v4.5.0).clap_opencligoldens validates against the base schemaand the extensions schema, and reads back through the typed model. The
2.declaregolden and theopencli2rust/opencli2gogoldens areunchanged.
What is next / merge order
master. Nothing else has to land first.v0.4.0withscripts/release.sh, which builds the binaries.opencli_modelandclap_opencliship at 0.1.0 with that tag.git = "https://github.com/livesession/opensdk", tag = "v0.4.0".Publishing to crates.io is an optional later step: a first
cargo publishof both crates, then Trusted Publishing from
release.yml.opencli-extensions.schema.json. The producer lands first.