feat(trace-verify-ts): a TypeScript verifier for TRACE v0.2, scored against the Python one - #376
chernistry wants to merge 16 commits into
Conversation
A specification with one implementation says whatever that implementation
does. This is a second one, written from spec/trace-v0.2.md and schema/,
and it runs on WebCrypto alone: no network, no filesystem, no platform
module and no runtime dependency, so a record can be checked where there
is no Python runtime.
What it implements, with the rule each check comes from:
- the profile cutover of section 2, with the superseded v0.1 identifier
rejected under its own code rather than as an unknown profile;
- RFC 8785 canonicalization, including section 3.2.2's requirement to
reject an integer outside the JCS safe range and the I-JSON rule that
a lone surrogate has no canonical form;
- Ed25519 over the pre-image of section 3.2.2, the record with its
signature member absent;
- canonical unpadded base64url for the signature and for JWK members
(RFC 4648 sections 5 and 3.5), so one byte string has one spelling;
- RFC 7638 thumbprints, and the confirmation-key binding of section
3.2.2;
- freshness and nonce, with the bounds section 3.2.2 states;
- the three revocation outcomes of section 3.2.3 against a bundle the
caller passes in, with a bundle defect reported as an outcome rather
than thrown;
- the chain digest of section 3.1.3, over the complete parent record
with its signature present.
The schemas are compiled ahead of time with ajv, so nothing is evaluated
or fetched at run time, `pattern` is read as ECMA-262 as JSON Schema
2020-12 requires, and `format: uri` is asserted rather than annotated.
The generator records the SHA-256 of every schema file it read.
The order of the checks is the Python implementation's, so both report
the same first failure for the same record.
Signed-off-by: chernistry <sanderchernitsky@gmail.com>
… the build Three files. The unit tests pin the canonical forms and the refusals that have a published answer: RFC 8785 member ordering by UTF-16 code unit, the number and escape forms, the values that have no canonical form, the base64url alphabet and pad-bit rules, and the RFC 7638 section 3.1 example thumbprint. The vector tests run the canonicalization-boundary and revocation-bundle corpora through the verifier and check the outcome, the cause and every evidence member each vector states, plus the chain digest of every link in the delegation corpus. The third file tests the build rather than the logic: the compiled output imports nothing, so a platform module or a network call cannot arrive unnoticed; the schema digests the validators were generated from still match schema/; and every exported function refuses junk with the package's own error type rather than a TypeError, which is what a caller written against the documented contract catches. Signed-off-by: chernistry <sanderchernitsky@gmail.com>
A second implementation is only evidence if something compares them. The harness generates a corpus, runs both sides over it and reports every case where they disagree. The corpus holds each record as text, not as a parsed object, because several cases exist to probe a difference that only survives in text: an iat written 1785000000.0, a lone surrogate in a string, a signature spelled with padding. Both runners parse the same bytes with their own JSON parser, which is part of what is compared. It covers a signed record under 24 verifier configurations with 59 mutations each, an RFC 8785 corpus, RFC 7638 thumbprints, the chain digest of every record in the delegation corpus, this repository's own vectors, and the published conformance vectors from a pinned checkout whose files are recorded by SHA-256 rather than vendored. The Python side reports a rejection as an exception, and two distinct checks raise the same message from the same helper called twice, so the classifier reads the traceback to ask which call site it came from, and looks the line numbers up in the installed source rather than writing them down. An exception it does not recognise is reported, not guessed. 1644 cases: 1280 agree exactly, including every published vector, every thumbprint and every chain digest. The other 364 are 13 classes argued in differential/known-divergences.json, which matches on both the case and the pair of reported reasons, so a new disagreement inside a listed family is reported rather than absorbed. Nothing was adjusted to match the other side. Signed-off-by: chernistry <sanderchernitsky@gmail.com>
Two jobs. The unit job builds and tests on Node 20, 22 and 24: the floor is what the package declares and the ceiling is current, so a runtime leaving that matrix is a supported-versions decision rather than a CI adjustment. The differential job installs the Python implementation from this tree, checks out the published conformance vectors at a pinned commit, runs both sides over the generated corpus and fails on any disagreement that is not in the ledger. The vector checkout is allowed to fail: the corpus without it is this repository's own vectors plus the generated matrix, which is the larger part, and a verifier's CI should not go red because another repository is unreachable. Actions are pinned by commit, as the other workflows here are. Signed-off-by: chernistry <sanderchernitsky@gmail.com>
…tial run found Signed-off-by: chernistry <sanderchernitsky@gmail.com>
|
Please open focused issues for the reported Python bugs, each with a small reproducer and the rule it tests. Link the number-handling and signature-encoding questions to #247, keeping any proposed rule changes separate from implementation fixes. As agreed in #247, matching Python alone does not establish correctness. Where the specification leaves a question open, we need to settle it explicitly. |
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
|
Done, as asked. The five Python findings are four focused issues, each with a reproducer on the public API of 0.10.0 and the rule it tests: #379 ( On the verifier itself: if a separate repository is the better home for a second implementation, say so and I will move the package and its differential harness there, and this PR shrinks to the CI hook and the changelog line. Agreed on the #247 rule: agreement with the reference is not correctness. The run's value is the thirteen classes where the two disagree, and the ledger says which side each one stands on. |
lywinged
left a comment
There was a problem hiding this comment.
Reviewed at da8b9f0, which is @imran-siddique's uri.ts fix on your branch rather than one of
yours. No test workflow has run on that commit: trace-verify-ts.yml has run exactly once, at
5df7fd5 one commit back, where all three Node legs passed; the only workflow that has run on
the head is the approval gate. So everything below is my own runs, on Node 22.
npm test is 74 of 74. The differential at the pinned corpus is 1644 / 1280 / 364 / 0, which
reproduces your table row for row. All four issues reproduce on the published agentrust-trace
0.10.0 wheel, on the v0.10.0 tag and on main at ccbd5b9, each with a control, and every
control fired. @imran-siddique's ask is met: four issues, each naming the rule it tests. One
caveat on his "a reproducer on the public API of 0.10.0": only #379 is self-contained. The
others take BASE, KEY and NOW from it, and #382 also needs
examples/revocation-bundle/01-…json, which is not in the wheel.
I looked for a defect in the TypeScript and did not find one. The denominator: the RFC 3986
grammar rule by rule, the base64url decoder including both unused-bit masks, the chain digest,
the revocation path, and 1700 generated JCS cases: random doubles from raw bit patterns,
safe-range integers, exponent spellings, strings over an alphabet carrying U+0000, U+001F,
U+007F, U+2028, U+2029 and three astral planes, and objects with random key sets to drive the
member sort. 152 disagreements, all 152 of them |value| >= 2^53, which is entry 13's declared
class. Zero on any string and zero on any object.
Both chain digests in your #247 comment reproduce in full. Your own sentence there gives the
class: four unused bits is sixteen spellings, and all sixteen decode to the same 64 bytes, all
sixteen verify, and all sixteen give a distinct chain digest.
Four findings. Two are about what CI can see, and two are about text that has already shipped.
1. The published-vector group has never run, and CI cannot tell
README.md:82 carries --external. The "Reviewing it" recipe does not, so a reviewer following
it gets 1632 cases and 1268 identical rather than the 1644 and 1280 above it. And the workflow
does not reach it either, for a reason that is one path:
92: if [ -d .vectors/tests/vectors ]; then
93: python differential/generate_cases.py --external "$GITHUB_WORKSPACE/.vectors/tests/vectors"
97: working-directory: trace-verify-ts
Line 93 knows where the checkout is. Line 92 tests the same location relative to
working-directory, so it asks about $GITHUB_WORKSPACE/trace-verify-ts/.vectors/tests/vectors,
which never exists. The else arm always wins.
That is not a contingency. In the one run of this workflow, the differential against the Python implementation job logs the checkout succeeding into .../trace-spec/.vectors at 3af2b53, and
the very next step writes 1632 cases, then cases: 1632 / identical verdicts: 1268 / known divergences: 364 / unexpected: 0. The checkout step concluded success, so the
continue-on-error beside it has never been exercised either. The pin is fine; the -d test in
front of it is what missed. "Every published vector agrees" is true today, because I ran it; what
is absent is anything that would notice if it stopped being true.
The count was in the log, so the evidence was there. What is missing is an assertion on it.
external_vector_cases returns [] for a missing root and compare.py has no floor, so nothing
in the output distinguishes twelve agreeing published vectors from no published vectors at all:
both print unexpected: 0 and exit 0.
The pin's own comment is inert for the same reason, twice over. It says generate_cases.py
"writes the SHA-256 of every file it read into build/manifest.json", and the docstring says a
later run against a different checkout "is detected rather than absorbed". That write is at
generate_cases.py:421, inside external_vector_cases and after the return [] at 388, so in
the arm CI actually takes the manifest is never written at all. When it is written, nothing reads
it: build/ is gitignored and the artifact upload takes report.json only. Recorded, not
detected, and on the path CI takes, not even recorded. (The docstring also says "beside this
file"; BUILD = HERE / "build".)
2. The ledger ratchets in one direction only
The differential does discriminate, and I should say so before the finding: replace jcs.ts:128's
sort with compareCodePoints and it is 8 unexpected and exit 1; bump the corpus to current
trace-tests and it is 1 unexpected and exit 1. The undeclared direction, which is what the
description claims, holds.
The resolved direction does not. src/base64url.ts:41 is the RFC 4648 §3.5 check that four ledger
entries and your own proposed rule 2 on #247 rest on: entry 3 (22 cases), entry 4 (22), entry 12
(16) and 16 of entry 2's 26. Set unusedBits to 0 and rebuild:
npm test 73 pass, 1 fail
differential 1644 cases, 1356 identical, 288 known, 0 unexpected, exit 0
One unit test catches it, so CI would go red, and that is worth saying plainly. But the number
the submission leads with moves the wrong way and nothing says so: 76 more agreements, 76 fewer
divergences, exit 0, because a declared entry that stops being reached is not an error here. The
upstream parity test has this half in test_no_declared_divergence_has_quietly_been_resolved. I
went looking for the test that would prove me wrong: known-divergences.json is read by
compare.py --ledger and by nothing else in the tree. Measured from the other direction too, of
entry 1's nine pairs, removing one gives 6, 5, 2, 1 or 0 unexpected depending which, and two of
the nine can be removed with nothing becoming unexpected at all.
Which is the same shape as §1. In both, the number that would give it away is printed and nothing
asserts it: 1632 where 1644 was expected, 288 where 364 was. A human reading the log sees it; no
run fails.
3. "Five of the 13 … accepts a record", on two surfaces, and the second one is wider
Four, and the five items are not five of the thirteen. Item 5 is two cases where the reference
accepts nothing: the non-ASCII nonce raises, and the lone-surrogate bundle makes it reject.
Item 2's signature clause reads the other way on your own ledger:
m05-signature-trailing-newline is rejected / signature_malformed on the Python side in all
24 configurations, and entry 5 says "Both implementations reject" in as many words. I can read "a
trailing newline in signature passes" as a claim about the pattern, which is true and is what
entry 5 describes, but the sentence above it says "accepts a record" and that is what fixes the
reading.
The classes where Python verifies and this implementation rejects are entries 3, 8, 9 and 10.
Entries 1 and 7 go the other way, this implementation verifying where the reference refuses.
Your own filing has half of this already: #379 takes m27, m28, m29 and m31 as "one cause and one
fix", which is items 2 and 3 merged and the signature clause dropped, and the description did
not follow. So: #379 carries entries 8 and 9, #380 carries 10, entry 3 is on #247 as proposed
rule 2 where you said it lives, and #381 and #382 are entries 11 and 1, which are not acceptances
and are worth filing on their own terms.
CHANGELOG.md:16 carries the same sentence and needs more than a 5. Its clause is "a
trailing newline or a carriage return in a patterned field passes", and signature is a patterned
field: #/properties/signature is ^[A-Za-z0-9_-]+$. So the category makes the same false
acceptance claim the description made, over a wider set. Two more, while you are in there:
appraisal.verifier is format: uri and carries no pattern, so entry 9, one of the four real
acceptances, is named nowhere in the changelog's five; and entry 8 is three characters, U+000A,
U+000D and U+2028, where the changelog names two. The changelog is the copy that ships, and the
description is the one that does not.
One sentence more, in both places. "A test pins the SHA-256 of every schema file the
validators were built from against schema/, so a schema edit that is not rebuilt fails rather
than passing quietly." The test is live: build clean, edit a description in
schema/trace-claim.json, run node --test tests/hygiene.test.mjs, and test 2 fails. But every
invocation in the repository builds first, so the antecedent never arises. What it does hold is
that the validators in a given build came from those exact bytes, which is what catches
TRACE_SCHEMA_DIR pointed at a released schema set, and that is worth stating as itself.
README.md:77 carries the same trailing clause. CHANGELOG.md:16 states the pinning without the
conclusion, so that one is already right.
4. Smaller
- The line numbers in #381 and #382 are not 0.10.0's. Both issues say they reproduce on
agentrust-trace0.10.0, and on the published wheel both do. But in that wheel the
compare_digestnonce line issign.py:683and_canonical_bytes(bundle)is
revocation.py:159; this pull request's base has them at712and167, andmainat799
and200. #381 cites710-713and712, #382 cites166-172: the base's numbering. The two
blocks the issues quote are identical in all three trees, so no finding changes; the files
around them are not. Worth a minute because it is #247 item 3 in miniature, the surface
@imran-siddique called the one that keeps producing reports: someone who installs the version
you named lands on 683.revocation.py:16is 16 in all three. - When you bump the pin, the current corpus has one unexpected divergence and it is the
harness's. 1660 / 1295 / 364 / 1 attrace-testscf30e9b, on
external-invalid_canonical_plain_trace: Pythonschema_invalidatiat, TypeScript
invalid_argument.generate_cases.py:409setsnowfrom the record's owniatwhenever
that is anint, and this vector exists to carryiat: 100000000000000000000, so
verifyRecordrejects the caller's argument before it reads the record. Given a boundednow
the TypeScript saysschema_invalidatiattoo. One line, and the corpus becomes
1660 / 1296 / 364 / 0. oracle.pydies on inputrun.mjsreports on.run_verifyparsesrecord_jsonand
revocation_bundle_jsonoutside any try, so a case whose text this parser refuses takes the
process down:{not jsongives aJSONDecodeError, a 20000-deep nested array an uncaught
RecursionError, both exit 1 with the output file however far it got.run.mjsreturns
parse_errorandrecord_not_objectand exits 0.run_jcsat 190 has the guard; the same
unprotectedjson.loadsis inrun_thumbprintand in the chain-digest runner, so it is three
exceptarms rather than one. Latent today, since all 1644record_jsonvalues parse, and the
corpus that makes it live is the external one, because those are arbitrary JSON files from
another repository.- "The one canonicalization divergence, on
1.0e+21" is one in this corpus.
Number.isIntegeris true for every finite double at or above 2^53 and for their negatives, so
1e22,1e308,1.7976931348623157e308and-1e308all diverge the same way: a canonical
form on the Python side againstcanonicalization_failedhere. That is the 152 above. CHANGELOG.mdconflicts withmain. The failinggatecheck is
require-maintainer-approval.yml, not a broken build.
5. The reading questions, and one thing the instruments cannot see
Four of the thirteen are stricter-here, and three are one question: entries 2, 4 and 12 are the
RFC 4648 canonical form of a base64url member, which is your proposed rule 2. The fourth is not a
reading question. Entry 6 has both implementations rejecting a signature member present as 0,
false, [] or {}, and the difference is the reference's failure code. It is the only one of
the thirteen with nowhere filed to go.
The denominator above has a hole worth naming, because it is the same hole this review is about.
Rewriting codePointLength as value.length leaves npm test at 74 of 74 and the differential
at 1280 / 364 / 0, and so does rewriting timingSafeEqual as a === b. Neither is a defect: the
schema carries no maxLength and every minLength is 1, so the two length functions cannot
disagree about anything it constrains. But a constant-time property is not observable in a
verdict at all, so "the differential agrees" is not evidence about timingSafeEqual in either
direction, and a reader of the 1644 should know which of its silences are which.
On where the package should live: if a separate repository is chosen, pinning it from here is the
mechanism, with the -d test fixed first.
…n both directions The workflow tested for the trace-tests checkout at a path relative to trace-verify-ts, so the published-vector group never ran in CI: the one run logged 1632 cases where 1644 were expected, and nothing failed. The checkout is now required, generate_cases.py refuses a root it cannot read or that yields no record, and compare.py --expect-external N fails unless exactly N cases came from it. The manifest of vector digests is always written, printed with the count, and uploaded with the report. The ledger failed in one direction only: a declared divergence that stopped being reached lowered the number the run leads with and exited 0. Each entry now declares how many cases land on it, and an entry, a pair or a count the run does not bear out fails the run. Six pairs that attributed a signature or schema divergence to the bundle or trusted-key context move to the entries whose fix alone resolves them; the 364 is unchanged, its split is now 20/14/23/23/24/96/30/69/23/23/2/16/1. Setting the RFC 4648 unused-bits mask to zero, which one unit test already caught, now also fails the differential: entries 2, 3, 4 and 12 unreached, one pair of entry 1 never observed. Signed-off-by: chernistry <sanderchernitsky@gmail.com>
generate_cases.py set now from the vector's own iat whenever that was an int, so a vector whose point is an iat outside the safe-integer range made the TypeScript verifier refuse the caller's argument before it read the record. now is taken from iat only inside [0, 2^53-1]. Against trace-tests cf30e9b the corpus goes from 1660/1295/364/1 to 1660/1296/364/0; the pinned corpus is unchanged. Signed-off-by: chernistry <sanderchernitsky@gmail.com>
…rash oracle.py parsed record_json, revocation_bundle_json and value_json outside any try, so a text json.loads refuses, or one nested past the interpreter's stack, took the process down with the output file however far it got, where run.mjs reports parse_error and exits 0. Every runner on the Python side now reports parse_error for such a text, a RecursionError inside rfc8785 classifies as canonicalization_failed, which is what the other side reports for the same input, and run.mjs gains the same parse arm on its thumbprint and chain-digest runners. Latent in the committed corpus, since all 1644 texts parse; live for the external one, which is arbitrary JSON from another repository. Signed-off-by: chernistry <sanderchernitsky@gmail.com>
Since agentrust-io#116 the reference refuses a record whose eat_profile is outside the verifier's accepted set with a new message. The oracle mapped only the previous wording, so after merging main the 25 m16-profile-unknown cases reported unclassified against the TypeScript profile_unsupported. Signed-off-by: chernistry <sanderchernitsky@gmail.com>
…o sentences hold The changelog entry and the description said five of the 13 divergence classes were records the reference accepts. Four are (entries 3, 8, 9, 10); the other two are rejections of a different shape; and the "patterned field" clause covered signature, where the reference rejects. The clause now names subject (U+000A, U+000D, U+2028) and appraisal.verifier (format: uri, U+000A). The schema-digest test is described by what it holds, that a build came from exactly the schema bytes it names, rather than by a rebuild case no invocation in the repository reaches. The canonicalization divergence is named as its class, magnitude 2^53 and above, of which 1.0e+21 is the corpus's instance. A README paragraph says what the differential cannot observe: a constant-time property, and a length function the schema does not exercise. Signed-off-by: chernistry <sanderchernitsky@gmail.com>
|
Every finding reproduced here before anything changed: the
On the pushed tree: Location: wherever the maintainers want the package, the |
|
Alex, keep the TypeScript reference verifier and its comparison harness in Shared, language-neutral conformance cases belong in Before publishing, let's choose a package name that clearly distinguishes this Trust Record verifier from the registry's existing |
|
Understood on location: the verifier and its harness stay here, the pinned Name. The registry's |
|
Alex, use |
…ust-record-verify The package, its directory, the workflow and its path filters, the README and the changelog entry take the new name. The corpus seed in differential/generate_cases.py keeps its original spelling, as the named constant CORPUS_SEED_DOMAIN: every key in the 1644-case corpus derives from it, so changing it reissues the corpus and moves outcomes that known-divergences.json pins by case and by count. Differential against agentrust-trace 0.10.0 after the rename: 1644 cases, 1280 identical, 364 known, 0 unexpected; ledger 13 entries, 71 pairs, 0 unreached; 12 published vectors, manifest sha256:869550bf03073ba3. Unit tests 74/74 on Node 25. Signed-off-by: chernistry <sanderchernitsky@gmail.com>
…er has agentrust-io#383, agentrust-io#386, agentrust-io#387 and agentrust-io#388 fixed the four defects this harness reported, so the cases that recorded them stopped diverging and the ledger failed closed on five entries no case reached: the lone-surrogate revocation bundle (agentrust-io#382), `subject` and `appraisal.verifier` under ECMA-262 pattern semantics (agentrust-io#379), the IPv4-in-IPv6 literal with a leading zero (agentrust-io#380), and the non-ASCII nonce (agentrust-io#381). They leave the ledger rather than stay as claims nothing checks. Four counts move with them. The `c23-bundle-lone-surrogate` context used to diverge on every mutation and matched the first entry; now that it verifies like any other context, its cases fall through to the mutation families: m03 23 to 24, m04 23 to 24, m20/m21 30 to 32, m46/m47 16 to 17. The ledger's `reference` and the README's table no longer say 0.10.0 without qualification, because the behaviour under test is the tree's and the four fixes are not in that release. 1644 cases, 1412 identical, 232 known, 0 unexpected; ledger 8 entries, 33 pairs, 0 unreached; 12 published vectors, manifest sha256:869550bf03073ba3. Signed-off-by: chernistry <sanderchernitsky@gmail.com>
|
Renamed to One string keeps the old spelling, as the named constant Merging Current run: 1644 cases, 1412 identical, 232 known, 0 unexpected; ledger 8 entries, 33 pairs, 0 unreached; 12 published vectors, manifest |
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
|
Merged current main and aligned the TypeScript signature check with #390. Absent, null and empty signatures now report the same failures as Python; the resolved divergence is removed. Before the fix: 48 unexpected differences. After: 1,644 cases, 1,508 identical, 136 documented differences, zero unexpected; all 12 pinned published vectors agree. All 82 unit tests pass, including three new cases that failed before the fix. This follow-up needs independent review. |
|
@lywinged @rajnisht7 could one of you review the latest signature-handling fix and approve if it looks right? It brings absent, null and empty signatures into line with Python #390. All 82 unit tests passed, and the 1,644-case comparison has zero unexpected differences; all 12 pinned published vectors agree. @lywinged, this follows your earlier review. One independent review is enough, and I can finish the maintainer checks afterward. |
lywinged
left a comment
There was a problem hiding this comment.
Checked the follow-up at 01ca3a5 against main at 1adbe20, its second parent. Seven files
differ from the branch head; three came in verbatim from main, src/agentrust_trace/sign.py
among them, and git diff-tree --cc --name-only 01ca3a5 shows the merge itself touched only the
README, the ledger, verify.ts and units.test.mjs.
The workflow's own recipe with trace-tests pinned at 3af2b53: 1644 cases, 1508 identical, 136
known, 0 unexpected, compare.py exit 0, 12 cases from 12 files, manifest
sha256:869550bf03073ba3, 82 of 82 tests. Every figure in your comment reproduces, the 48 before
the fix included, and so do @chernistry's at 7e94708: 1412 identical, 232 known, 0 unexpected,
8 entries and 33 pairs, and 74 of 74 tests there.
The guard is right and nothing below blocks it. One disclosure, since the merge brings all
of main with it: #400 is mine, and I have not re-reviewed it here.
The 48 were created by #390, not by this branch
Before #390, absent, null and the empty string all reported signature_missing, on both sides.
That agreement was the blind spot rather than a clean bill: two of the three were wrong on both
sides at once, null by the #247 ruling and the empty string by the schema, and a differential
cannot see a defect the two implementations share. The sign.py that arrives with this merge is
what breaks the tie, which is why the fix rides on the merge commit.
Four states, each run end to end by swapping sign.py and verify.ts and rebuilding, all four
scored against the ledger this commit ships:
| Python | TypeScript guard | identical | unexpected |
|---|---|---|---|
| before #390 | before this commit | 1412 | 96 |
| before #390 | this commit | 1364 | 144 |
| after #390 | before this commit | 1460 | 48 |
| after #390 | this commit | 1508 | 0 |
identical does not depend on which ledger is in the tree and unexpected does: row one against
the eight-entry ledger @chernistry ran is his published 232 known and 0 unexpected, and 232 known
minus 136 known is exactly the 96 printed here.
Row one's 96 are m08, m09, m11 and m12 under 24 configurations, which left the ledger
because #390 made the reference call them malformed too. Row three's 48 are m06-signature-empty
and m10-signature-null under the same 24, and those are new: the reference now answers
schema_invalid at signature and signature_malformed, where the old guard answered
signature_missing to both. Row two's 144 is the 96 plus the 48, so the two effects are
independent. So #390 moved 96 into agreement and broke 48, and this commit closes those 48:
1412 + 96 - 48 + 48 = 1508, and it owns the last term. The old guard put back gives row three from
the other side: 1460, 48 unexpected, exit 1, all 48 rows printed.
One test is the only thing that separates your guard from the obvious one
I would have written encodedSignature === undefined, mirroring the value rather than the
membership. Built and re-scored, it gives the same 1508 and the same 0 unexpected. The 1644 cases
cannot separate the two: a present undefined member has no JSON spelling, the TypeScript runner
parses every case with a bare JSON.parse and no reviver, and generate_cases.py builds every
record as JSON text and never writes the token undefined.
It fails exactly one of the 82 tests, a present undefined signature is malformed, one of the
eight this commit adds. Object.hasOwn is the version that mirrors the reference's new
if "signature" not in record, and own() at text.ts:97 returns undefined for both cases,
which is why the near-miss conflates them. isPlainObject at verify.ts:229 refuses a non-object
before hasOwn is reached at 249, so the ordering holds.
Worth having on the record, since it is easy to assume otherwise: the three tests that fail before
the fix are null, present-undefined and the empty string, not absent, null and empty. The old
guard against the new tests gives 3 failures and 79 passes, and absent was already passing on both
sides.
Three small things, none of them blocking
- README lines 119 to 125, whose numbers this commit rewrote from
232and8to136and7.
"in the adversarial matrix" is false for entry 7, sincejcs-10is in the canonicalization
corpus, the row at line 112 that prints34 | 33; "once per verifier configuration" is false
for four of the seven entries; and "Six classes left the ledger" introduces a list of five. The
wording predates the commit, which read "Five classes" over a list of four. - The pull request description, which is @chernistry's and which
git grepcannot reach, carries
the same clause, next totrace-verify-ts/, 1280 identical and 13 classes. - One line on #247 would settle the empty string. The ruling there answers "present with the
wrong type", and an empty string is present and the right type, so it does not reach the 24
cases that turn on it. What decides them is the schema's own^[A-Za-z0-9_-]+$, which carries
nominLengthand does not need one.
What I did not check
Node 20 and 24, Python 3.12, and the hosted job's own logs: I ran Node 22 and Python 3.11.15
against the same pinned corpus, and nine of the ten check runs on 01ca3a5 pass. The tenth is
gate, the maintainer-approval workflow waiting on this review. Also Windows, the published
agentrust-trace wheel, @chernistry's figures from before the rename, and the rest of the pull
request beyond item 2.
What this changes
Adds
trace-verify-ts/: a second, independent implementation of TRACE v0.2 record verification, in TypeScript, plus a differential harness that scores it against the Python implementation in this repository.It runs on WebCrypto alone: no network, no filesystem, no platform module, no runtime dependency. That is what a browser, an edge worker or a service mesh sidecar needs to check a record where there is no Python runtime, which today is every consumer that is not a Python process.
ROADMAP.mdlists multi-language verification libraries; this is the TypeScript one.Why a second implementation rather than a binding
A specification with one implementation says whatever that implementation does. The value is not the TypeScript code, it is the 1644-case differential run: two implementations reading the same words and reaching the same verdict is evidence the words are unambiguous, and the cases where they do not are the places the specification is not yet saying what it means.
What it implements
The profile cutover of section 2, with the superseded v0.1 identifier rejected under its own failure code rather than as an unknown profile. RFC 8785 canonicalization, including section 3.2.2's requirement to reject an integer outside the JCS safe range, and the I-JSON rule that a lone surrogate has no canonical form. Ed25519 over the section 3.2.2 pre-image, the record with its
signaturemember absent. Canonical unpadded base64url for the signature and for JWK members (RFC 4648 sections 5 and 3.5). RFC 7638 thumbprints and the confirmation-key binding of section 3.2.2. Freshness and nonce. The three revocation outcomes of section 3.2.3 against a caller-supplied bundle, with a bundle defect reported as an outcome rather than thrown. The chain digest of section 3.1.3, over the complete parent record with its signature present.Schemas are compiled ahead of time with ajv, so nothing is evaluated or fetched at run time,
patternis read as ECMA-262 as JSON Schema 2020-12 requires, andformat: uriis asserted rather than annotated. A test pins the SHA-256 of every schema file the validators were built from againstschema/, so the validators in a build are known to come from exactly those bytes; a build made against another schema directory fails that test.The order of the checks is the Python implementation's, so both report the same first failure for the same record.
What the differential run found
Against
agentrust-trace0.10.0 and against this tree, with identical results:Every published vector agrees, every thumbprint agrees, every chain digest agrees. The remaining 364 are 13 classes, each appearing once per verifier configuration, all argued in
trace-verify-ts/differential/known-divergences.json. The ledger matches on both the case and the pair of reported reasons, so a new disagreement inside a listed family is reported rather than absorbed; each entry also declares how many cases land on it, and an entry, a pair or a count the run does not bear out fails it, so a divergence that stops happening on either side has to leave the ledger. CI reads the published vectors from a pinned checkout and asserts their case count. Nothing was adjusted to match the other side.What was filed from it
Four of the 13 are cases where the Python implementation accepts a record this specification's own rules exclude (ledger entries 3, 8, 9, 10); two more are rejections of a different shape (entries 11, 1); two are the JSON number model (entries 7, 13). Filed at the maintainer's request, one issue per rule:
format: urianchoring (entries 8, 9). Asubjectending in U+000A, U+000D or U+2028, or anappraisal.verifierending in U+000A, passes, because Python'sreanchors$before a final newline and matches.against characters ECMA-262 excludes. verify: schemapatternandformat: uriare evaluated with Pythonresemantics, so a trailing U+000A, U+000D or U+2028 insubjectandappraisal.verifierverifies #379.format: uriagainst RFC 3986 (entry 10).https://[::1.2.3.04]/is accepted althoughdec-octetexcludes a leading zero. verify:format: uriacceptshttps://[::1.2.3.04]/, which RFC 3986dec-octetexcludes #380.runtime.nonceleavesverify_recordas aTypeErrorfromhmac.compare_digest. verify:verify_recordleaves asTypeErrorwhenruntime.nonceis non-ASCII andexpected_nonceis set #381.unverified_for_revocation. verify: a revocation bundle with a lone surrogate rejects the record instead of reportingunverified_for_revocation#382.signaturemember present, so two records that differ only in that spelling both verify and have different chain digests. A specification question rather than a defect: Four surfaces state the rules and nothing says which wins: schema, reference model, normative spec and docs disagree #247, proposed rule 2.iatwritten1785000000.0is an integer afterJSON.parseand a float afterjson.loads, and a number of magnitude 2^53 or above has an RFC 8785 form on one side and none on the other (1.0e+21is this corpus's instance). Four surfaces state the rules and nothing says which wins: schema, reference model, normative spec and docs disagree #247, proposed rule 1.The remaining five (entries 2, 4, 5, 6, 12) are cases where both implementations reject and the stated reason differs because this one refuses a non-canonical encoding earlier; entry 6 (a
signaturemember present as0,false,[]or{}) is on #247 as a reading question.Scope
No signing: a verifier that can sign is a verifier that can be made to sign. No chain walk: the
delegationblock is normative in v0.2 and what a verifier does with a chain of them is not, so this ships one link and leaves the policy to its caller. No key or bundle fetching, which is what keeps the package offline. No ES256 or ES384 bundle signatures: the bundle schema admits them, a signature nobody checked grounds nothing, so they are reported asbundle_signature_unsupported.Reviewing it
The commits are separable and land in this order: the package, its tests, the differential harness, CI, the changelog entry.
Type of change
No normative text, schema or wire format changes. The addition is a new implementation and its tests.
Spec section
Implements sections 2, 3.1.3, 3.2.2 and 3.2.3 of
spec/trace-v0.2.md. Changes none of them.Checklist
git commit -s)CHANGELOG.mdupdated (for any normative change)<!-- CHANGED: #NNN: description -->in spec text