Skip to content

feat: an OpenCLI model, a clap → OpenCLI adapter and --mount for component CLIs (0.4.0) - #6

Merged
zdunecki merged 23 commits into
masterfrom
feat/clap-opencli
Nov 15, 2026
Merged

zdunecki merged 23 commits into
masterfrom
feat/clap-opencli

Conversation

@zdunecki

Copy link
Copy Markdown
Member

Why

A host CLI that runs a component CLI as one of its commands (acme agent ...
handing every argument to an agent CLI) cannot document that component in its
OpenCLI reference today:

  • the chain builds OpenCLI documents only from OpenAPI, so a hand-written clap
    CLI has no document at all;
  • nothing places one CLI's document inside another's;
  • the passthrough command a host uses for this has no meaning in the format, so
    generators and docs converters cannot tell it apart from an ordinary command.

This release adds the pieces for that. There is a typed model with three
generic extensions. A library describes a clap CLI as an OpenCLI document. A
mount places a component's document at one command of a host's document. The
Rust generator gets the passthrough shape.

What

  • opencli_model 0.1.0 (new crate, no clap):
    • A typed model of the flat OpenCLI shape the chain reads and writes. x-*
      keys round-trip, and output uses one canonical key order.
    • The upstream root-command shape is read too, and lifted to flat.
    • Three typed generic extensions: x-interactive {commands, keys},
      x-environment [{name, description, values}] and
      x-passthrough {program}.
    • Their contract is schema/opencli-extensions.schema.json. It is exported
      as EXTENSIONS_SCHEMA_JSON, beside the base schema, SCHEMA_JSON.
    • merge_fragment, the --declare merge, moves here unchanged. It then also
      takes root examples, interactive and x-* keys.
    • mount follows rules M1–M7: equal conventions, the stub fill or append,
      what moves, recursive-option collisions with drops, exit codes, no
      filtering, and a one-line report.
  • clap_opencli 0.1.0 (new crate):
    • from_command(&cmd)...build() maps a builder or derive tree into an
      opencli_model::Document. It covers commands, positionals with arity,
      options with their visible aliases, and globals marked recursive only
      where they are declared. Metadata: default, env (the env feature),
      repeatable, conflictsWith, exclusive and last. Hidden items are
      marked or left out.
    • What clap does not know is declared beside the tree: exit codes, examples,
      environment variables, interactive, a REPL's commands
      (interactive_from_command), x-* keys and per-command extras
      (at(&path, ...)).
    • snapshot::assert_file keeps a committed document current from a test.
    • There is no proc-macro, and clap is taken without default features.
      Supported: clap 4.5.0 and every later 4.x.
  • opensdk opencli ... --mount NAME=DOC [--mount-drop NAME:OPTION]...: after
    the converter and the --declare fragments, each mount places a component
    document and prints one report line on stderr.
  • x-passthrough:
    • rust-cli generates the hand-written shape: the help flag is disabled, one
      trailing OsString positional takes the arguments, and the handler is
      bound through the Commands trait. It never reads the command's children.
    • go-cli refuses the marker for now.
  • The binary's own cli/opencli.json is built by clap_opencli, and a
    drift test checks it.
  • CI adds a clap floor job for clap_opencli, plus lint and test runs of
    its env feature.

Review fixes in this round

  1. A short-only option used to be written as {"name": "x"}, which every
    reader takes as --x. It is now an error that names the command and the
    option. The chain keeps its rule that a one-character name is a long.
    One-character long aliases are left out, because a one-character alias
    reads as a short. interactive_from_command now returns a Result.
  2. A builder positional with Append and the default num_args takes any
    number of values. Its arity now has no maximum.
  3. The members of an ArgGroup that is not multiple (clap's default)
    exclude each other, and they now get conflictsWith. A group's own
    conflicts_with has no public getter in clap, so it is listed under "Not
    mapped".
  4. A command that clap has already built (Command::build,
    get_matches*_mut or a completion generator) is refused. Before, it
    silently gave clap's help subcommand tree and a copy of every global.
  5. When the only difference is the newlines at the end of the file,
    snapshot::assert_file now says so, instead of reporting "(end of file)" on
    both sides.
  6. A command marked passthrough now comes out as a stub that a mount can
    fill. What clap parses for it (the trailing positional) is the host's
    plumbing, so it is left out. A new golden is built from the exact
    hand-written shape, and a test mounts a component into it.
  7. A --mount-drop given twice, or given once with dashes and once without,
    is applied once. Before, it failed with "matches no option".
  8. The x-passthrough contract now states two exceptions. The host parses its
    own recursive options given right after the command. A -- placed right
    after the command, or after those options, only ends them. The model docs,
    the schema description, both READMEs and the changelog say this, and the
    runtime smoke pins it.
  9. A REPL's global options now appear on every session command.
  10. The clap floor job holds clap at 4.5.0, the oldest release the manifest
    admits. Before, it used 4.5.58. A test keeps the pin and the manifest in
    step.

Each fix is its own commit, with a test.

How it was checked

  • cargo fmt --all -- --check: clean.
  • cargo clippy --workspace --all-targets -- -D warnings: clean.
  • cargo clippy -p clap_opencli --all-targets --features env -- -D warnings:
    clean.
  • XYD_SMOKE_RUST=1 cargo test --workspace: 644 passed, 0 failed, across 130
    test binaries. This includes the 13.passthrough runtime smoke with the new
    -- cases. The other languages' compile smokes were skipped locally and run
    in CI.
  • cargo test -p clap_opencli --features env: 30 passed, 0 failed.
  • The clap floor (cargo update -p clap --precise 4.5.0): 29 passed without
    features and 30 with --features env. The goldens are byte-identical at the
    floor.
  • scripts/check-standalone.sh: OK.
  • CI (ci on this branch, run 43659373307): all four jobs green: rust (fmt +
    clippy + goldens), cli smokes, e2e, and the clap 4.5 floor (clap v4.5.0).
  • Every one of the 13 clap_opencli goldens validates against the base schema
    and the extensions schema, and reads back through the typed model. The
    2.declare golden and the opencli2rust / opencli2go goldens are
    unchanged.

What is next / merge order

  1. Merge this PR into master. Nothing else has to land first.
  2. Tag v0.4.0 with scripts/release.sh, which builds the binaries.
    opencli_model and clap_opencli ship at 0.1.0 with that tag.
  3. Until the two crates are on crates.io, consumers pin
    git = "https://github.com/livesession/opensdk", tag = "v0.4.0".
    Publishing to crates.io is an optional later step: a first cargo publish
    of both crates, then Trusted Publishing from release.yml.
  4. Docs converters that render OpenCLI can then vendor
    opencli-extensions.schema.json. The producer lands first.

A new library crate, versioned 0.1.0 on its own:

- Document and the types under it model the flat OpenCLI shape the chain
  reads and writes. Every object keeps the keys it does not type in
  `extensions`, so x-* keys round-trip; writing uses one canonical key
  order (the root x-openapi right after info, as the converter writes it).
- Reading also accepts the upstream root-`command` shape and lifts it to
  flat (lift_root_command); a document with both shapes is an error.
- Three generic extensions are typed: x-interactive {commands, keys},
  x-environment [{name, description, values}] and x-passthrough {program}.
  schema/opencli-extensions.schema.json is their contract, exported as
  EXTENSIONS_SCHEMA_JSON beside the base schema (SCHEMA_JSON).
- The vendored base schema moves here from openapi2opencli/tests, byte for
  byte; the converter's schema test reads it from the crate and also
  round-trips every committed OpenCLI document through the typed model.
merge_fragment and its unit tests move from cli/src/opencli.rs into
opencli_model byte for byte, so every document the binary merged before
merges identically now (the 2.declare golden is unchanged). The binary
keeps the command plumbing and calls the library.
…and x-* keys

A fragment may now also set the root `examples` and `interactive` and any
`x-*` key; each replaces what the document has, in place when present and
otherwise where the converter would write it. `x-openapi` stays the
converter's (a fragment setting it is an error), `x-passthrough` marks a
command and never the root, and `x-interactive` / `x-environment` must have
their typed shape. Documents merged before merge byte-identically (the
2.declare golden is unchanged).
…nt-drop

`opensdk opencli ... --mount NAME=DOC [--mount-drop NAME:OPTION]...` runs the
converter, then the --declare fragments, then each mount: a component CLI's
OpenCLI document is placed at the command NAME (`parent/name` mounts under
`parent`) by opencli_model::mount, under rules M1-M7:

- M1: the component is an OpenCLI document (the root-`command` shape is
  lifted) whose conventions equal the host's;
- M2: a command of that name may exist only as a passthrough stub
  (x-passthrough, no arguments, options or subcommands), which is filled and
  keeps its own name, aliases, hidden, marker and description; otherwise the
  command is appended under the spelling-collision rule; twice is an error;
- M3: the component's root arguments, options, commands, exitCodes,
  examples, interactive, metadata and x-* keys move onto the command, its
  info.version becomes the metadata `componentVersion`, and an x-openapi
  binding anywhere is an error;
- M4: the host's recursive options in scope are compared with every option
  in the mounted subtree by every spelling; a match is an error naming both
  unless the component's option is dropped, and a drop matching nothing is
  an error;
- M5/M6: the mounted exit codes govern the subtree, nothing else is
  filtered, and the result is written in canonical key order (the host's own
  keys keep their order);
- M7: one line per mount on stderr, `mounted <path> <version>: dropped
  --<option>`.

The 3.mount fixture covers a filled stub with a drop and an appended
component in the root-`command` shape; every rule has its error test. The
schema test now validates every committed document against the extensions
schema too, the mount goldens and components included.
`generate --spec <doc>` and `opencli --spec <doc>` lift a document whose
root command sits under `command` (the upstream draft's shape) to the flat
shape the backends and the merge take, through
opencli_model::lift_root_command. A flat document is unchanged; one with both
shapes at once is an error. Such a document generates the same tree as its
flat twin.
A command marked `x-passthrough` hands every argument after it, unparsed, to
another program. opencli2rust now emits exactly the hand-written shape for
it: `.disable_help_flag(true)` and one `args` positional (`num_args(0..)`,
`trailing_var_arg`, `allow_hyphen_values`, parsed to `OsString`), so
`--help` after the command is the program's and a recursive root option is
the host's only before the command or right after it. The command is bound
through the generated `Commands` trait (`fn <name>(ctx, m)`), at any depth.

Its documented arguments, options and subcommands are never read: no clap
args, no handlers, and `root_options::check` skips them (it still checks the
generated `args` positional). The marker on the root, or on a command that
also binds an HTTP request, is a generation error.

13.passthrough is the golden (a top-level and a nested passthrough whose
documented `--profile`/`-o` would collide with the recursive root options);
runtime_smoke builds it and checks the argument hand-off. Every existing
golden is byte-identical.
`try_opencli2go` returns an error naming the first command marked
`x-passthrough` (or the document root); `opencli2go` panics with it, and
`generate --lang go-cli` reports it. Walking a passthrough's documented
children as generated commands would build a CLI that parses another
program's flags, so the document is refused instead.
A new library crate, versioned 0.1.0 on its own. `from_command(&cmd)`
reads a clap command tree (builder or derive) and `build()` returns an
opencli_model::Document. There is no proc-macro, and clap is taken with
`default-features = false` and `std`, so a CLI's clap features are left as
they are; the opt-in `env` feature enables clap's `env` and maps `Arg::env`.

The walk reads the tree after `Command::build` and compares it with the
unbuilt tree:

- clap's `help` subcommand and every help or version argument are skipped,
  matched by action, so an argument a CLI names `version` stays;
- a global is `recursive: true` where it is declared, and the copies clap
  spreads to the subcommands are not written (a global declared on a
  subcommand stays there);
- commands: name, visible aliases, `long_about` else `about`, hidden; the
  root's bin name, version, about and long_about fill `info`;
- positionals in index order with required, arity (unless exactly one),
  the non-hidden possible values, the help heading as group; options with
  the long name else the short, the short and visible aliases, one argument
  per value name, no arguments and no default for a flag;
- metadata `default` (unless hidden), `env`, `repeatable` (Append, Count),
  `conflictsWith` (groups unrolled), `exclusive`, `last`;
- hidden commands and arguments are marked `hidden: true` (Hidden::Mark,
  the default) or left out (Hidden::Omit).

What clap cannot know is declared beside it: exit codes, examples,
x-environment entries, `interactive`, x-interactive commands and keys,
x- extension keys, title/version/summary/description overrides, and
per-command extras through `at(&path, |c| ...)` (an unknown path is an
error; `passthrough` marks a command x-passthrough).
`interactive_from_command` turns a REPL parsed with clap into
x-interactive commands, and `snapshot::assert_file` keeps a committed
document current in a test (`OPENCLI_UPDATE=1` rewrites it).

Twelve goldens under __fixtures__ pin the mapping, built from builder and
derive CLIs; each is read back through the typed model and validated
against the base and the extensions schemas.
…nd mount

The README's example runs as a doctest.
cli/opencli.json describes the `opensdk` binary: clap_opencli builds it
from `opensdk::command::command()`, with the exit codes (and `diff`'s own
0/1/2 table) and examples declared in cli/tests/reference.rs. The test
fails while the committed file differs from the clap tree;
`OPENCLI_UPDATE=1 cargo test -p opensdk --test reference` rewrites it.

The document is validated against the base and the extensions schemas and
round-tripped through the typed model with every other committed OpenCLI
document. scripts/release.sh bumps its `info.version` with the manifest's
and commits it, so a release commit keeps the test green.
The rust job also lints and tests clap_opencli with its `env` feature,
which a workspace build leaves off. The new clap-floor job locks clap to
4.5.58 (`cargo update -p clap --precise`), checks the lock took, and runs
the adapter's suite with and without `env`, goldens included: the lock
holds the newest 4.x, so nothing else builds the floor the crate claims.
…t and x-passthrough

The changelog's unreleased 0.4.0 section lists opencli_model and
clap_opencli (each versioned on its own from 0.1.0, which the header now
says), --mount/--mount-drop, the root-command shape, x-passthrough in both
generators and the dogfood document. The README's layout gains the two
crates, the schema's new home, the mount and passthrough paragraphs and the
widened --declare rules.
…re under Omit

The walk records a global as declared before it decides to leave a hidden
one out, so its copies further down are always recognized as copies.
…racter long alias is left out

The chain names an option by its long name and reads a one-character
alias as a short, so a one-character name is a long (`--x`). A
short-only clap option written as {"name": "x"} therefore read as
`--x`, which clap rejects, in every reader (the model's spellings, the
generators, a docs site), and a one-character long alias (`--y`) read
as the short `-y`.

build() and interactive_from_command() now return an error naming the
command and the option when an option has only a short (a hidden one
left out under Hidden::Omit is no error), and one-character long
aliases are left out. interactive_from_command returns a Result for it.
Golden 1 pins a one-character long (`--x` with the short `-y`); golden
6 leaves out a one-character visible alias.
A positional whose action is Append keeps collecting every value that
follows (clap's usage shows it as `<name>...`), but with the default
num_args (1..=1) it was written with no arity, which says exactly one
value. Only derive's Vec<T> positionals, which set num_args(1..), came
out right.

A positional with Append now has no maximum in its arity. Golden 2
gains a builder case, checked against clap's own parse.
An ArgGroup that is not `multiple` (clap's default) makes its members
exclude each other, and clap rejects `--csv --tsv` for such a group,
but conflictsWith only read the conflicts declared on the argument
itself, so the members were documented with no conflict.

conflictsWith now adds the other members of every group the argument
belongs to that is not `multiple`. A group's own conflicts_with has no
public getter in clap; it joins the 'Not mapped' list in the crate docs,
the README and the changelog. Golden 7 is re-blessed and gains a
`multiple` group, whose members stay free; both are checked against
clap's own parse.
The walk compares the built tree with the one it is given, taken as
unbuilt, to tell clap's additions from the CLI's own. Given a command
already built (by Command::build, a get_matches*_mut call or a
completion generator), it documented clap's help subcommand tree and
repeated every global as recursive on each subcommand, and returned Ok.

build() and interactive_from_command() now return an error for a root
that holds clap's own help flag, version flag or help subcommand while
the switch that disables it is off: an unbuilt command cannot hold
them then, since clap asserts ids and subcommand names unique. The
docs of from_command, the crate, the README and the changelog say the
command must be unbuilt. A test covers build(), a parse through _mut,
the flags-only cases, a REPL, and a CLI's own help flag and help
subcommand, which stay valid.
…::assert_file

The diff walked str::lines, which ignores a final newline, so a
committed file that differed only by a missing or extra newline at its
end failed the check and then reported both sides as "(end of file)",
or an empty line against the end, contrary to the promise that the
message names the first line that differs.

A difference only in the newlines at the end is now named as such,
with the count on each side, and a difference in line endings that
str::lines hides (a stray carriage return) too; an empty line shows as
"(an empty line)". Unit tests cover the stripped final newline, an
extra blank line under CRLF, a stray carriage return, a line added at
the end and a blank line inside.
… mount fills

At::passthrough only set the marker and kept what the walk mapped from
the clap command. A real passthrough command in clap holds one trailing
positional that collects the arguments (the hand-written shape, which
opencli2rust generates too), so the document carried that `args`
positional, the command was no passthrough stub, and mount refused it:
a hand-written host described by clap_opencli could not be mounted
into, and its docs showed the host's plumbing as the program's
argument.

The walk now leaves out the arguments, options and subcommands of a
command marked passthrough, keeping its name, aliases, description and
hidden flag. Golden 13 is built from the exact hand-written shape, and
a test mounts golden 11's agent CLI into it.
mount() applied each drop in turn: the first removed the option, and a
repeat (`profile` twice, or `profile` and `--profile`, which trim to
one name) then matched nothing and failed with "matches no option of
the component", although the option exists. The dedupe beside it could
never run.

A drop already applied is now skipped before the count is checked, so
only a name that matches nothing in the subtree is an error. A unit
test and a CLI test cover the repeats; a name matching nothing after a
repeat is still an error.
…the host

The model, the extensions schema and the README described x-passthrough
as "the host hands every argument after this command, unparsed, to
another program", which becomes a public contract docs converters
render. The generated shape (and the hand-written one it copies) is
narrower: the host parses its own recursive options given right after
the command, and a `--` right after the command or after those options
only ends them and is not handed on.

The model docs, the schema description, both READMEs, the changelog and
At::passthrough now state both exceptions; the generated shape is
unchanged. The runtime smoke pins them: `agent --` hands nothing over,
`agent --profile prod -- -p` hands `-p` with the profile parsed, and a
later `--` is the program's.
interactive_from_command walked the REPL root as a CLI root, kept only
its subcommands and dropped the root's own options. The root's globals
were recorded as declared there, so the copies clap spreads to each
subcommand were skipped too, and a global every session command accepts
(`model --json`) appeared on none.

The session commands are now walked with no inherited globals, so the
copies are written on each one, recursive, and not again below it. The
root's other arguments stay out, as documented. A test covers a REPL
parsed with no binary name (a multicall root cannot hold arguments).
clap_opencli requires clap "4.5", which admits 4.5.0 to 4.5.57 too,
but the floor job ran 4.5.58, the newest 4.5.x: a getter added anywhere
in 4.5.1 to 4.5.58 would pass CI and break a consumer locked on an
older 4.5 release.

The job now runs `cargo update -p clap --precise 4.5.0` (it resolves,
and the suite, goldens included, passes there with and without the env
feature). A test in clap_opencli keeps the pin equal to the oldest
release the manifest's requirement admits.
@zdunecki
zdunecki merged commit 86eeab4 into master Nov 15, 2026
8 checks passed
@zdunecki zdunecki mentioned this pull request Nov 15, 2026
@zdunecki
zdunecki deleted the feat/clap-opencli branch November 15, 2026 02:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant