Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .agents/skills/testing-pilot-corpora-gate/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,8 +183,8 @@ gate's own helpers are package-private but reusable (`pilotCorporaGate.files(t)`

`actionlint`, `shellcheck`, `python3 scripts/check-doc-links.py`, `gofmt`, `go vet`,
`go run -C tools ./cmd/pilot-diff` (validators pre-downloaded; ~4min, prints e.g.
the headline the committed baseline holds — `380 file(s), 345 fully agreeing; 38 agreed
diagnostic(s), 40 only ours, 1616 only the pilot's` at the `2026-08` pin, so read it from
the headline the committed baseline holds — `381 file(s), 345 fully agreeing; 38 agreed
diagnostic(s), 41 only ours, 1629 only the pilot's` at the `2026-08` pin, so read it from
`docs/project/pilot-differential-baseline.json` rather than from this line)
and `make lint` (staticcheck+gosec, ~2min) all work. There is **no** `yamllint` and **no**
`circleci` CLI, so `.circleci/config.yml` can only be parsed as YAML, not schema-validated — say so
Expand Down
8 changes: 4 additions & 4 deletions .agents/skills/testing-pilot-differential/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ GNU-format diagnostics **relative to `--root`**. Consequences for testing:
- `-validator /nonexistent` now says `run ./scripts/download-pilot-sysml-validator.sh`.
- Measured at the `2026-08` pin after bare parameters took their effective range `[0..*]`, removing
the adjudicated `Behaviors.kerml:14` multiplicity warning (the `[1]` `RocketEquation` inputs keep
its warning at `delta-v-budget.sysml:93`): `380 file(s), 345 fully agreeing; 38 agreed, 40 only
ours, 1616 only the pilot's`, JSON totals `openSysMLDiagnostics 80 / pilotDiagnostics 1656 /
its warning at `delta-v-budget.sysml:93`): `381 file(s), 345 fully agreeing; 38 agreed, 41 only
ours, 1629 only the pilot's`, JSON totals `openSysMLDiagnostics 81 / pilotDiagnostics 1669 /
severityMismatch 2`; ~2 min wall, byte-identical across runs *and* after a from-scratch rebuild of
`build/pilot-validator`. The six `kerml-examples` pilot-only rows the `2026-07` run carried (`The
opposite features 'owningType' … do not refer to each other`) are gone: the pilot fixed its
Expand Down Expand Up @@ -142,8 +142,8 @@ committed result of the *last refreshed* run, so **the harness is testable by re
but only while the baseline is current. Check that first. The latest rebaseline, after bare
parameters took their effective range `[0..*]` and removed the adjudicated `Behaviors.kerml:14`
warning (the `[1]` `RocketEquation` inputs still produce the warning at
`delta-v-budget.sysml:93`), is current: a live run gives `380 file(s), 345 fully agreeing; 38
agreed, 40 only ours, 1616 only the pilot's`, byte-identical to the committed baseline, and
`delta-v-budget.sysml:93`), is current: a live run gives `381 file(s), 345 fully agreeing; 38
agreed, 41 only ours, 1629 only the pilot's`, byte-identical to the committed baseline, and
`docs/project/pilot-differential.md`'s "Results" table matches. The prior rebaseline, when the
Legend of the Red Dragon example left for its own repository, gave
<!-- doc-count:historical -->`380 file(s), 344 fully agreeing; 38 agreed, 42 only ours, 1614 only the pilot's`.
Expand Down
4 changes: 2 additions & 2 deletions .agents/skills/testing-pilot-execution-referee/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,8 +148,8 @@ pilot answers the representation's own. See
`pilot-exec-diff: <file>:<line>: model no/such/model.sysml: stat <abs>: no
such file or directory`.
- **Additivity.** `go run -C tools ./cmd/pilot-diff` must still print the headline the
committed baseline holds (`380 file(s), 345 fully agreeing; 38 agreed
diagnostic(s), 40 only ours, 1616 only the pilot's` at the `2026-08` pin — read it from the baseline JSON, not from this line, since each
committed baseline holds (`381 file(s), 345 fully agreeing; 38 agreed
diagnostic(s), 41 only ours, 1629 only the pilot's` at the `2026-08` pin — read it from the baseline JSON, not from this line, since each
fix round moves it) and `jq -S` diff clean against
`docs/project/pilot-differential-baseline.json`; `git status --porcelain`
empty at the end.
Expand Down
4 changes: 2 additions & 2 deletions .agents/skills/testing-pilot-xpect/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -423,8 +423,8 @@ census in `w5c_census_test.go` is live two ways: perturb one pinned triple (e.g.
## Regression neighbour

`go run -C tools ./cmd/pilot-diff` (~1m12s) must still print the headline the *committed* baseline holds —
at the `2026-08` pin that is `380 file(s), 345 fully agreeing; 38 agreed diagnostic(s), 40
only ours, 1616 only the pilot's`. Read the number out of
at the `2026-08` pin that is `381 file(s), 345 fully agreeing; 38 agreed diagnostic(s), 41
only ours, 1629 only the pilot's`. Read the number out of
`docs/project/pilot-differential-baseline.json` rather than trusting this line, since a landing fix
round moves it. When the baseline is itself stale (it was at `19a3ce03`, holding 273 / 281 / 317), a
failing `cmp` against it is *not* evidence of an Xpect regression — compare the summary line, and see
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -312,11 +312,11 @@ The project is under active development, with the core infrastructure operationa
<!-- doc-counts:begin refereed-figures -->
**Measured against the pinned reference** (`PILOT_TAG=2026-08`, artifact `0.62.0`). Every number below is generated by `make docs-counts` from the committed baselines and gated; none of them is typed in by hand.

- **Corpus agreement:** 345 of 380 files agree diagnostic-by-diagnostic; 40 diagnostics are ours alone and 1616 the reference's alone, and the first number must be read by root: our diagnostics against the reference's own corpora fell while our non-standard-notation warnings on our own example models rose ([differential](docs/project/pilot-differential.md), `go run -C tools ./cmd/pilot-diff`).
- **Corpus agreement:** 345 of 381 files agree diagnostic-by-diagnostic; 41 diagnostics are ours alone and 1629 the reference's alone, and the first number must be read by root: our diagnostics against the reference's own corpora fell while our non-standard-notation warnings on our own example models rose ([differential](docs/project/pilot-differential.md), `go run -C tools ./cmd/pilot-diff`).
- **Declared-diagnostic silence:** of the 512 declared `errors` rows in the reference's own Xpect suites, we report nothing for 0. 245 we report word-for-word; 248 wording-only and 7 location-only differences are agreement in substance and are not counted as gaps; 0 more we report as a warning and 2 elsewhere in the file ([Xpect oracle](docs/project/pilot-xpect.md), `go run -C tools ./cmd/pilot-xpect`).
- **Scope agreement:** 230 of 230 declared scope assertions match exactly (same source).
- **Permissiveness gaps:** of 311 invalid models we wrote ourselves, the reference rejects 2 that we accept by default, and 300 both reject; 2 further cases agree only when we are asked strictly. We authored every one of these cases ourselves, so the denominator measures the reach of our own corpus and not our conformance; agreement reached only under an opt-in strict mode is weaker evidence than agreement by default ([rejection oracle](docs/project/pilot-rejection.md), `go run -C tools ./cmd/pilot-reject`).
- **Declared errata:** the registry declares 12 defect(s) in the published reference material — 4 with a specification-derived correction, 8 documented without one, since no intended reading can be inferred ([OMG issues](docs/project/omg-issues.md), `tools/oracle/errata`). Every figure above is as published and stays the conformance statement; running the same oracles over the corrected text instead reports 346 of 380 files agreeing, 39 diagnostics ours alone and 1616 the reference's alone, 0 declared rows we are silent on, and 0 of 311 authored cases the reference alone rejects. The corrected figures are diagnostic only: an erratum never reclassifies a divergence category, and the published corpus is never edited.
- **Declared errata:** the registry declares 12 defect(s) in the published reference material — 4 with a specification-derived correction, 8 documented without one, since no intended reading can be inferred ([OMG issues](docs/project/omg-issues.md), `tools/oracle/errata`). Every figure above is as published and stays the conformance statement; running the same oracles over the corrected text instead reports 346 of 381 files agreeing, 40 diagnostics ours alone and 1629 the reference's alone, 0 declared rows we are silent on, and 0 of 311 authored cases the reference alone rejects. The corrected figures are diagnostic only: an erratum never reclassifies a divergence category, and the published corpus is never edited.
- **Self-assessed surface:** the action, state-machine and classifier-behavior rows have no external referee at all — the four refereed figures above cannot see them, because the pinned artifact evaluates expressions but executes neither actions nor state machines. [Spec compliance](docs/project/spec-compliance.md) counts them.

What these numbers cannot show: the OMG corpora are demonstrations rather than an official conformance suite; the differential is one-directional, comparing the diagnostics the two implementations report on the same files; the Xpect suites are the pilot authors' test intent rather than a certification oracle; and none of these is a percentage of the specification — no global compliance figure is claimed anywhere.
Expand All @@ -328,7 +328,7 @@ What these numbers cannot show: the OMG corpora are demonstrations rather than a
**Test coverage:** top-level `Test` functions (counted from the `_test.go` files, as `go test ./...` runs them) covering parsers, semantics, runtime (actions, states, instances, operators, validation), behind golden ASTs, negatives, execution conformance cases, golden traces, runtime robustness cases and gRPC conformance and robustness cases. The figures are counted from the tree when the documentation site is built into the test inventory of [spec compliance](docs/project/spec-compliance.md), never committed, so a branch adding a test does not rewrite this page. A test skips only for want of something the run did not provide, and says what: the held-image round trip declines a conformance case that creates no instance, a few gate on a PDF or Mermaid toolchain, a pinned pilot artifact, the PSSM suite, a locale, a case-insensitive filesystem or a live Flexo stack, and the OMG corpus gates skip until the corpora are downloaded unless asked to fail.
**Parser coverage:** 105/105 bundled library files parse cleanly — the 94 official SysML v2 standard library files and the non-normative `OpenSysML Libraries/OpenSysMLMathFunctions.kerml`, `OpenSysML Libraries/DocumentQueries.sysml`, `OpenSysML Libraries/IdentityMetadata.sysml`, `OpenSysML Libraries/DiagramLayout.sysml`, `OpenSysML Libraries/OOSEM.sysml`, `OpenSysML Libraries/MOSA.sysml`, `OpenSysML Libraries/StateSpaceIntegration.sysml`, `OpenSysML Libraries/Stochastic.sysml`, `OpenSysML Libraries/RandomFunctions.kerml`, `OpenSysML Libraries/Simulation.sysml` and `OpenSysML Libraries/MigrationMetadata.sysml` extensions. Conformance verified by [stdlib_conformance_test.go](internal/workspace/libs/stdlib_conformance_test.go). Grammar reference: [OMG Xtext grammar](https://github.com/Systems-Modeling/SysML-v2-Pilot-Implementation/tree/master/org.omg.kerml.xtext/src/org/omg/kerml/xtext).
**Behavioral execution:** Calc/constraint/requirement/satisfy functional. Action/state executors handle nested invocation, control flow keywords, loop and conditional statements and the send statement (<!-- doc-counts:begin conformance-passing -->every conformance case passing<!-- doc-counts:end conformance-passing -->). Coverage is self-assessed against the specification text and the normative library: the pinned OMG pilot implementation evaluates expressions but does not execute actions or state machines headlessly, so no external implementation currently adjudicates these rows. See [spec compliance](docs/project/spec-compliance.md).
**Reference differential:** 380 files compared diagnostic-by-diagnostic against the pinned OMG pilot implementation (`2026-08`), 345 in full agreement; every divergence is enumerated and adjudicated in [the differential](docs/project/pilot-differential.md), reproducible with `go run -C tools ./cmd/pilot-diff`.
**Reference differential:** 381 files compared diagnostic-by-diagnostic against the pinned OMG pilot implementation (`2026-08`), 345 in full agreement; every divergence is enumerated and adjudicated in [the differential](docs/project/pilot-differential.md), reproducible with `go run -C tools ./cmd/pilot-diff`.
**Rejection oracle:** the reverse direction — do we reject what the reference rejects? 311 hand-written invalid models validated by both implementations, 302 rejected by both, 0 the pinned pilot rejects and we accept; the remainder only we reject — the control-node succession rules the pinned pilot leaves unimplemented and a non-Boolean succession guard it accepts once the standard library types it — and every permissiveness gap is enumerated with a reproducer and likely root cause in [the rejection oracle](docs/project/pilot-rejection.md), reproducible with `go run -C tools ./cmd/pilot-reject`. We wrote every case, so the count measures our coverage of the rejection surface, not our conformance — a sample, not a proof.
**Training examples:** 100/100 files clean, gated by `tests/corpus/testdata/training_examples_expected.txt`. Download with `./scripts/download-training-examples.sh` (from the [OMG training directory](https://github.com/Systems-Modeling/SysML-v2-Pilot-Implementation/tree/master/sysml/src/training)). See [training examples](docs/project/training-examples.md) for analysis.
**Semantic layer:** a complete implementation of runtime operators, feature chains and validation rules. See [examples/semantic-layer/](examples/semantic-layer/) for a full demonstration.
Expand Down
2 changes: 2 additions & 0 deletions changes/unreleased/deferred-keeper-marker.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
- **`MigrationMetadata::DeferredKeeper` marks the keeping accept of a deferred signal.** The SysML v1 migrator and the PSSM referee write the accept of each keeping loop of the standard deferred-signal encoding as `#MigrationMetadata::DeferredKeeper action receive accept kept : Sig;`, so the accept that keeps the signal for the state is declared rather than recognized by its shape. An accept node takes prefix metadata like any other usage (`#M action a accept e : E;`), as the grammar's `PrefixMetadataMember` allows.
- **The `deferred-keeper-unmarked` lint reports a keeping loop written without the marker.** An accept of a deferred signal at the root of the do action of a state annotated `MigrationMetadata::DeferredEvent` where no accept of that signal carries `MigrationMetadata::DeferredKeeper` — the shape the migrator wrote before the marker existed, not an ordinary consumer written beside a marked keeper — is reported as a warning in every mode, since the runtime now runs it as an ordinary accept that consumes the occurrence rather than keeping it; the model still analyses and runs, and re-migrating it or writing the marker on the accept clears the warning. The annotations are known by their resolved type, however the model spells them. The lint is switched off like the others, by its code.
1 change: 1 addition & 0 deletions changes/unreleased/deferred-keeper-marker.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- **An accept of a deferred signal written beside the keeping loop takes the occurrence first.** The runtime identified the keeping accept of a deferring state's do action by its position and signal, so an ordinary accept of the same signal declared at the do action's own level counted as a second keeper, and an occurrence arriving while both were parked could be kept instead of taken, depending on the schedule. The keeping accept is now the one the `MigrationMetadata::DeferredKeeper` annotation marks, lowered by resolved type into the action graph; every other accept is an ordinary consumer the keeper yields to whenever it can take the occurrence that step.
2 changes: 1 addition & 1 deletion cmd/sysml/usage.go
Original file line number Diff line number Diff line change
Expand Up @@ -596,7 +596,7 @@ func registerFlags(fs *flag.FlagSet) {
fs.StringVar(&queryText, "query", "", "Evaluate this OSLC Query text against the model and exit")

fs.Var(&modelChecks.validate, "validate", "Report the model's diagnostics and exit, nonzero on an error; -validate=<object> checks instead every assertion about that object (repeatable)")
fs.Var(&disabledLints, "disable-lint", "Leave this lint out of the model's diagnostics: undeclared-signal or port-type-mismatch, comma-separated or repeated")
fs.Var(&disabledLints, "disable-lint", "Leave this lint out of the model's diagnostics: undeclared-signal, port-type-mismatch or deferred-keeper-unmarked, comma-separated or repeated")
fs.BoolVar(&noRecordCache, "no-record-cache", false, "Parse every file loaded and hold it loaded, reading no interface record from the record cache and writing none; default off, or OPENSYSML_RECORD_CACHE=0")
fs.BoolVar(&strictMode, "strict", false, "Judge the model as conforming SysML v2: notation no pinned production admits is an error, not a warning; a SysML v1 migration writes none of it")
fs.Var(&modelChecks.constraints, "constraint", "Evaluate this constraint and exit (repeatable)")
Expand Down
9 changes: 5 additions & 4 deletions docs/guide/03-command-line.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,10 +261,11 @@ each extension is measured against. The same setting is available as `%strict` a

## Lints

Two further warnings, `undeclared-signal` and `port-type-mismatch`, are *lints*: the model
is valid SysML v2, but a `when <name>` that no declaration or `send`
accounts for, or a connection between ports whose definitions are unrelated, is almost
always a slip. `-strict` leaves them warnings, since they are not about notation.
Three further warnings, `undeclared-signal`, `port-type-mismatch` and
`deferred-keeper-unmarked`, are *lints*: the model is valid SysML v2, but a `when <name>` that
no declaration or `send` accounts for, a connection between ports whose definitions are
unrelated, or a deferred signal's accept loop written without the `DeferredKeeper` marker that
names it as the keeper, is almost always a slip. `-strict` leaves them warnings, since they are not about notation.
`-disable-lint <code>` switches one off (`%lint <code> off` at the prompt,
`disabledLints` in an editor); [the diagnostics reference](../reference/diagnostics.md)
states exactly what each reports.
Expand Down
4 changes: 3 additions & 1 deletion docs/guide/11-migrating-from-sysml-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -339,7 +339,9 @@ action fills from an accept loop while the state is active, substates included,
action sends the kept occurrences back to the object once the state is left, so the state
entered next takes them as if they had just arrived. The state is annotated
`@MigrationMetadata::DeferredEvent { ref :>> signal : Sig; }` as well, so a reader sees what
was deferred without reading the encoding; the encoding and its rules are described under
was deferred without reading the encoding, and the loop's accept is marked
`#MigrationMetadata::DeferredKeeper`, which names it as the one keeping the signal rather than
consuming it; the encoding and its rules are described under
[Deferred signals](../reference/sysml-v1-migration.md#deferred-signals). A transition out of
the deferring state into a `choice` accepts the signal in both modes, since the pseudostate
metadata spelling is written either way.
Expand Down
Loading
Loading