Skip to content

Throw on an unresolvable $ref instead of passing silently - #944

Open
Amoifr wants to merge 2 commits into
jsonrainbow:mainfrom
Amoifr:fix-916-unresolvable-ref
Open

Amoifr wants to merge 2 commits into
jsonrainbow:mainfrom
Amoifr:fix-916-unresolvable-ref

Conversation

@Amoifr

@Amoifr Amoifr commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Closes #916.

This takes option 1 from the issue, a validation error, and there is an argument for it beyond it being the least disruptive: CHECK_MODE_STRICT is currently more permissive than the default mode. Same schema, same data, on main:

default mode        : UnresolvableJsonPointerException thrown
CHECK_MODE_STRICT   : isValid = true, 0 errors

The legacy path already refuses to let a broken $ref through; it is only the draft-specific constraints that swallow it. So this is not just a missing feature, it is the strict mode being laxer than the mode it is supposed to tighten. After the change, strict mode reports one error and the default mode is untouched.

On the error constant

The issue suggests MISSING_ERROR or a new one. MISSING_ERROR is not available: its message is commented out in ConstraintError with Used for tests; this error is deliberately commented out, so it is reserved. This adds UNRESOLVABLE_REF with the message The $ref "%s" could not be resolved, which also names the pointer that failed rather than leaving the reader to find it.

On the offline case

The issue rightly raises unreachable remote schemas. Nothing changes there for anyone on the default mode, which already throws. For strict mode, a remote $ref that cannot be fetched now produces a validation error rather than a silent pass, which I think is the honest answer: the validator cannot claim a document conforms to a schema it never read. Option 3, gating on CHECK_MODE_EXCEPTIONS, remains available on top of this if you would rather make it configurable.

Verification

Two tests per draft: an unresolvable $ref is reported with the new constraint name, and a resolvable one still validates. Each fails without the change.

Full suite green at 3166 tests, including the official JSON-Schema-Test-Suite, which is what I would have expected to break had the previous behaviour been load-bearing. PHPStan clean.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical unresolved-reference errors can still be silently discarded by schema applicators.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 3 High severity

Open (3)
What changed in this PR

Reports unresolved $ref values as validation errors in strict draft-specific validation.

Changes:

  • Adds UNRESOLVABLE_REF.
  • Updates Draft 06, 07, and 2019 reference constraints.
  • Adds resolved and unresolved reference tests.
File Review
tests/​Constraints/​Draft2019/​RefConstraintTest.php Adds Draft 2019 reference tests.
tests/​Constraints/​Draft07/​RefConstraintTest.php Adds Draft 07 reference tests.
tests/​Constraints/​Draft06/​RefConstraintTest.php Adds Draft 06 reference tests.
src/​JsonSchema/​Constraints/​Drafts/​Draft2019/​RefConstraint.php Unresolved-reference errors can be erased by applicators; chained references may report the wrong pointer.
src/​JsonSchema/​Constraints/​Drafts/​Draft07/​RefConstraint.php Unresolved-reference errors can be erased by applicators; chained references may report the wrong pointer.
src/​JsonSchema/​Constraints/​Drafts/​Draft06/​RefConstraint.php Unresolved-reference errors can be erased by applicators; chained references may report the wrong pointer.
src/​JsonSchema/​ConstraintError.php Defines the new constraint error and message.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/JsonSchema/Constraints/Drafts/Draft06/RefConstraint.php Outdated
} catch (\Exception $e) {
// A $ref that cannot be resolved is a broken schema, not an absent keyword:
// returning here would let the value pass as if the $ref had not been written.
$this->addError(ConstraintError::UNRESOLVABLE_REF(), $path, ['ref' => $schema->{'$ref'}]);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, measured on all three drafts in strict mode: anyOf [$ref, {}], oneOf [$ref, {}] and not $ref all come out valid with an unresolvable $ref. I laid out the options in the main thread, since the fix changes the approach the PR was approved with.

Comment thread src/JsonSchema/Constraints/Drafts/Draft2019/RefConstraint.php Outdated
@DannyvdSluijs
DannyvdSluijs force-pushed the fix-916-unresolvable-ref branch from ed87c22 to ea53d6d Compare September 22, 2026 18:50
@Amoifr

Amoifr commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

@DannyvdSluijs thanks for the approval! Before you merge: CI went red after I rebased onto main, and digging into it turned up two things I'd rather you decide on.

  1. 44 official suite cases fail, all "is expected to be valid", all $refs the library cannot resolve yet ($anchor, URNs, $recursiveRef, location-independent $id). They used to pass by accident. Their 47 "is expected to be invalid" twins are already in the skip list, and with this PR they would pass, also by accident. So nothing regresses in what is actually supported, but the skip list needs the other half.
  2. Copilot is right about applicators, measured on all three drafts in strict mode: anyOf [$ref, {}], oneOf [$ref, {}] and not $ref all come out valid, because the error is an ordinary one that the branch logic discards. On main, oneOf and not were invalid by accident.

Two ways I see: keep the validation error, skip the 44 and handle the applicators in a follow-up; or switch to option 3 from #916 (throw in strict mode, honouring CHECK_MODE_EXCEPTIONS, like the default mode already does), which no applicator can swallow. I'd lean towards the second, but it changes what you approved, so your call. 🙂

@DannyvdSluijs

Copy link
Copy Markdown
Collaborator

@Amoifr Thanks for your efforts and your persistence. In all honestly the report #916 was created by Claude while I was working on something else and it asked confirmation to do so. I agreed and forgot about it.

Now reading the original report I believe that option 2 throwing the error is the best option (I've should have researched this earlier, my apologies.).
If a ref cannot be resolved there is an issue with the schema not with the data. The data is being validated not the schema itself. I would rather have the issue front and center and block users from using a broken schema for validation as the broken schema might be hiding many more validation errors.

The Strict mode was designed to validate the schema per its draft. Currently the default behaviour is a historical mix and mash of Draft 3 and Draft 4. Which I no longer wanted to support / Could not hold longer due to the incompatibilities between drafts.

Would you mind updating the PR to throw an exception? Or actually no longer capture the exception. Curious to see what the fallout would be.

@Amoifr
Amoifr force-pushed the fix-916-unresolvable-ref branch from ea53d6d to 04380ae Compare October 20, 2026 14:15
@Amoifr Amoifr changed the title Report an unresolvable $ref instead of passing silently Throw on an unresolvable $ref instead of passing silently Oct 20, 2026
@Amoifr

Amoifr commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

No need to apologise, it was a fun one to dig into, and thanks for making the call! 🙂 Done: the three draft RefConstraints no longer catch, so an unresolvable $ref surfaces as UnresolvableJsonPointerException (or ResourceNotFoundException for a remote one), and no applicator can swallow it anymore (anyOf, oneOf and not included, there is a test for anyOf). The unresolvableRef error from the first version is gone, it never shipped. I also rebased onto main.

The fallout you were curious about:

  • Default mode is unchanged: it already threw on all drafts, so only strict mode on Draft 6, 7 and 2019-09 behaves differently.
  • 44 official suite cases now throw, all "is expected to be valid" and all refs the library cannot resolve yet: $anchor and location-independent $id, URN base URIs, $recursiveRef/$recursiveAnchor, and a few relative refs under a changed base URI ("refs with relative uris and defs", "order of evaluation", "ref with absolute-path-reference", "remote ref with ref to defs"). I added them to the skip list next to their "invalid" twins, which were already skipped.

That last group is the real user-facing impact: a schema using one of those patterns in strict mode used to pass (partly by accident) and now throws. Fine by me given your reasoning, just flagging it so it does not come as a surprise in the release notes.

I also retitled the PR to match.

@DannyvdSluijs

Copy link
Copy Markdown
Collaborator

@Amoifr Code looks good but the pipeline still reports some issues. Could you take a look?

@Amoifr

Amoifr commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review! The two red tests are CurlTest, and they are not related to this PR: the runner image moved from 20261012 (last green runs) to 20261015, and its newer libcurl rejects a bare file path ("URL rejected: No host part in the URL"). The same two tests fail locally on a clean main with curl 8.18. I opened #989 to pass file:// URLs in the test, so this one should go green once it's in. 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RefConstraint silently passes validation when $ref cannot be resolved

3 participants