Repository navigation
Recover from invalid values while deserializing - #178
Conversation
* Introduce YamlInvalidValueHandling with FailFast as the default, so a converter rejection keeps throwing immediately unless a caller opts into FailAtEnd or Skip * Model each skipped value as YamlInvalidValue and aggregate them in YamlInvalidValuesException, whose location is that of the first value and whose inner exception names a later fatal error * Let the reporting session hold the invalid value delegates and collect values under FailAtEnd, and add YamlReader.CreateException so converters can reject a value at the current token before consuming it Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Wrap each member and constructor argument read in YamlObjectConverter with a filtered catch, so a YamlException thrown before the value was consumed skips the value under FailAtEnd and Skip instead of ending the read * Compare the token type together with the start offset in the filter, since Mark has no equality and a mapping shares its offset with its first key * Capture the skipped value as a YamlElement only when a callback or finalizer consumes it, keeping the FailAtEnd collection cheap Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Run the invalid value finalizer and throw YamlInvalidValuesException from the session's Complete, so a read that recovered values still fails under FailAtEnd while Skip lets it succeed * Fold an error that ends the read into the aggregate as its inner exception in the four DeserializeCore overloads, so the values recovered before the failure are not lost with it * Keep NotSupportedException and IOException outside the aggregation, as they report a configuration or source error rather than invalid content Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Wrap the matched branches of each generated ReadMember in one try/catch and set the target type per branch, so the catch covers value reads alone without emitting one try per member * Expose CanRecoverInvalidValue and RecoverInvalidValue as protected static members of YamlSerializerContext, since generated code lives in the user's assembly and cannot reach the reader's internal helpers * Reject SyntaxErrorException and SemanticErrorException in the filter, as a document the parser cannot read is not an invalid value and skipping the node would only throw again inside the handler Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Keep the message passed to each constructor and surface it as Reason, since Message bakes in the source name and marks and a caller that reports the location in its own format printed it twice * Return Reason from YamlInvalidValue.Message, so a callback or finalizer gets the converter's wording alone while Exception still locates the value Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: CodeAlta <noreply@codealta.io>
|
Thanks @StefanGreve, this is a very solid contribution! Merged via squash. Review notes:
I pushed one small commit (c8faa6b) with a regression test for the documented guarantee that a skipped key still counts for duplicate detection, for both a member and a constructor parameter. The test is reflection-only. While checking it I found that generated contexts don't detect duplicate member keys at all, even with valid values and the default The follow-ups you listed (recovering collection elements and dictionary values, This comment was written by CodeAlta. |
|
@xoofx Sounds good to me, I will do the follow-up tasks listed in this PR sometime next week. |
Why
When a converter rejects a value, deserialization stops at that value. For a document that a person edits by hand,
such as a configuration file or the front matter of a Markdown page, that is often the wrong trade: one malformed
date:costs the whole file, and the author learns about the second mistake only after fixing the first.The cases I care about:
date: Sep 13, 2026should lose its date and produce a warningwith file, line and column, not fail the build.
UnmappedMemberHandling.Disallowstopping at the first unknown key, which Add global unmapped member callback and finalizer #165 addressed for keys. System.Text.Json has open issues asking for the
same (Developers should be able to pass state to custom converters. dotnet/runtime#63795, System.Text.Json Add a feature that allows gathering all exceptions encountered when parsing json dotnet/runtime#38049).
What
One option and two opt-in delegates on
YamlSerializerOptions, working with reflection and source-generated contexts:InvalidValueHandlingselects what happens when a converter rejects a value:FailFast(default): the first invalid value throws, exactly as today.FailAtEnd: every recoverable value is skipped and the read continues. At the end the call throws oneYamlInvalidValuesExceptionlisting all of them, located at the first.Skip: every recoverable value is skipped and the call succeeds.InvalidValueCallbackruns once per skipped value, in document order.InvalidValuesFinalizerruns once perDeserialize/TryDeserializecall with every skipped value, only when there is at least one. UnderFailAtEnditruns before the exception is thrown and can throw to replace it. Neither runs under
FailFast.YamlInvalidValueis shaped likeYamlUnmappedMember:SourceName,Path($.servers[2].port),KeyStart,ValueStart,Name,TargetType,DeclaringType,Instance(nullfor constructor-bound types),Value(
YamlElement?, captured only when a callback or finalizer is set),Messageand theYamlExceptionthe converterthrew.
JsonObjectCreationHandling.Populate.A skipped constructor parameter keeps its default. The key still counts as present for required members and for
duplicate detection.
YamlReader.CreateException(message)gives a custom converter aYamlExceptionlocated at the current token. Thrownbefore the value is consumed, it is recoverable.
YamlException.Reasonis the message without thesource: (start) - (end):prefix.YamlInvalidValue.Messagereturns it, so a callback that formats the location itself does not print it twice.
Options stay immutable and shareable; all per-call state lives in the reader, in the same session that #165 added
for unmapped members.
Semantics
The mechanism follows Tomlyn, which wraps each member read in a filtered catch and skips the value when the reader
has not moved. Here:
reflection path and in generated code. Collection elements, dictionary values, the root value and extension data are
not recovered.
YamlExceptionthat is not a parser error (SyntaxErrorException,SemanticErrorException), and only while the reader is still on the token where the read began, compared by tokentype and start offset. Everything else stays fatal: other exception types, which Wrap exceptions thrown while reading a value in YamlException #177 still wraps in a
YamlExceptionat the entry points, and every read that already consumed part of the value. A converter bug istherefore never reported as invalid content.
FailAtEndthe call throwsYamlInvalidValuesExceptionwith the valuesrecovered so far and the fatal error as
InnerException. UnderSkipthe fatal error propagates unchanged. Thefinalizer does not run in either case.
NotSupportedExceptionandIOExceptionkeep propagating unchanged, as inWrap exceptions thrown while reading a value in YamlException #177.
UnmappedMemberHandling.Disallow,DuplicateKeyHandling.Error, a missing required member andpolymorphic discriminator errors, because none of them is thrown by a member value read.
TryDeserializereturnsfalseunderFailAtEndandtrueunderSkip.YamlUnmappedMemberalready documents.bad input.
One observable change outside
FailFast: for a malformed value such astitle: [Ten, the string converter's"expected scalar" exception is recovered, the skip then hits the unterminated sequence, and the parser's
SemanticErrorExceptionsurfaces instead of a plainYamlException.FailFastis unchanged.Performance
Under the default
FailFastnothing is created: no session, no path tracker, and the catch filters return at theoption check. The
tryregions in generated code exist in every mode, so I measured the default path againstmain(8fbc666) with the existing benchmarks. Allocations are identical to the byte and every time difference lies inside
one standard deviation.
SharpYaml_Deserialize_PocoSharpYaml_Deserialize_Poco_UnknownKeysSharpYaml_SourceGenerated_DeserializeSharpYaml_SourceGenerated_Deserialize_MergeKeysWith reporting enabled, the session and the path tracker from #165 are reused, and the value is captured as a
YamlElementonly when a callback or finalizer consumes it.API surface
YamlInvalidValueHandling(FailFast,FailAtEnd,Skip) and three properties onYamlSerializerOptions.YamlInvalidValue(sealed) andYamlInvalidValuesException : YamlException, socatch (YamlException)andTryDeserializekeep working.YamlReader.CreateException(string)for converters.YamlException.Reason.protected staticmembers onYamlSerializerContext,CanRecoverInvalidValueandRecoverInvalidValue,because generated code lives in the user's assembly. They stay off the reader's public surface; the alternative
would have been public reader members as in Add global unmapped member callback and finalizer #165.
Internally,
YamlUnmappedMemberSessionbecameYamlReportingSession, since it now carries both kinds of report.The names follow Maven's
--fail-fastand--fail-at-end;SkipmatchesUnmappedMemberHandling.Skipon the sameoptions object.
Follow-ups, not in this PR
abcinRetries: [1, abc, 3], or the value of a dictionary entry still fails the call in every mode, because the listand dictionary converters read their items in a loop with no catch around each one. A member of an object inside
a list is recovered, since that read goes through the object converter. I left elements out because skipping one
has no obvious meaning: a member keeps its default, but dropping an element shifts the indices after it, a
placeholder is impossible for a value type, and a dictionary entry could lose the entry or keep the key with a
default. Tomlyn stops at members as well. If wanted, it is the same filtered catch around each item read in the
collection converters and their generated equivalents, with "drop the item" as the rule.
YamlThrowHelper.ThrowNotSupported, a plainYamlException, before consuming anything, so underFailAtEndandSkipthat configuration error is reported asan invalid value. The generator throws
NotSupportedExceptiondirectly for its other configuration errors; aligningthe helper would be a small separate change.
DateOnlyconverter callsDateOnly.TryParsewith the invariant culture, so it accepts09/13/2026,while System.Text.Json accepts only
yyyy-MM-dd. Worth its own issue.Written with Claude Code (Claude Fable 5.1).