Add a build/write-prefetch unparse path driven by a dedicated Builder tree - #1735
Closed
olabusayoT wants to merge 6 commits into
Closed
olabusayoT wants to merge 6 commits into
olabusayoT wants to merge 6 commits into
Conversation
… tree Adds an off-by-default (useBuildWritePrefetch) two-pass unparse path: build races ahead of write over the same infoset tree via a coroutine handoff (BuildCoroutine/WriteCoroutine, UnparseSharedContext), resolving OVCs directly against the tree instead of via Suspensions where possible. Build is driven by a new Builder tree paralleling Unparser, wired through the grammar combinators, with unparseBeginForBuild/ unparseEndForBuild entry points on ElementUnparserBase that only the Builder tree calls. Write's coroutine thread spawns only if build's lead or pending-suspension count crosses a threshold; otherwise write runs inline on build's own thread. hasAnyPrefetchBeneficialOVC is scoped per compiling root, gated on the tunable, and requires a non-constant OVC expression via the new isPrefetchBeneficial. DAFFODIL-3065
builder was only gated on the useBuildWritePrefetch tunable, while the runtime flag that actually decides whether prefetch gets used additionally required hasAnyPrefetchBeneficialOVC. A schema with no prefetch-beneficial OVC never reaches unparseViaBuildThenWrite, so building the parallel Builder tree for it was pure waste even with the tunable globally on. Gate builder on the same combined condition. TestBuildState, TestLeadCounter, TestBoundedPrefetch, and TestBuildWriteArrayChoice's standalone-build test each call dp.ssrd.builder.get directly, bypassing that selection; their schemas had no OVC at all, so builder is now correctly Nope for them. Give each a dfdl:defineVariable-backed marker element (a variable reference has no element references and isn't a compile-time constant, unlike a literal, which the compiler folds to isConstant=true) so their schemas are actually prefetch-beneficial, and update the resulting element counts/output assertions. DAFFODIL-3065
ElementUnparserBase's writeContent and unparse independently ran the same before-content, content-dispatch, after-content, and setVariables sequence. Extract runElementContent(state, dispatch), taking only the content-dispatch step as a closure: writeContent's WriteUnparser-bypass check (needed to reach a nested group's own writeContent) versus unparse's plain, event-driven dispatch wrapped in unparse's own TermRuntimeData push/pop. WriteUnparser.writeWithPushPop already gave writeContent implementations a shared setup/dispatch/teardown skeleton; each of those same classes' unparse() hand-rolled the identical sequence separately. Extract withPushPop(state, setup, dispatch, teardown); writeWithPushPop becomes a thin specialization that builds its own dispatch closure (writeContent if bodyUnparser is a WriteUnparser, else unparse1). HiddenGroupCombinatorUnparser, DelimiterStackUnparser, DynamicEscapeSchemeUnparser, and the two SpecifiedLength unparsers' unparse() now call withPushPop directly instead of duplicating the try/finally by hand. ComplexNilOrContentUnparser's unparse() and writeContent() separately re-derived the same isNilled branch decision; extract chooseBodyUnparser(node) so both read it once. No behavior change, except SpecifiedLengthPrefixedUnparser's teardown (resolving the prefix length) now also runs if eUnparser.unparse1 throws during unparse(), matching writeContent's own teardown, which already ran unconditionally for the same reason. DAFFODIL-3065
…allers Each unparse()/writeContent() call runs once per matching element in the infoset, so a setup/dispatch/teardown argument built from an eta-expanded instance method or field (e.g. bodyUnparser.unparse1, pushDelimiterScope) closes over this and allocates a fresh closure on every single call, unless the JIT's escape analysis eliminates it across the trait/virtual- dispatch boundary into withPushPop. Hoist each such closure that only captures this/instance fields into a private val computed once per unparser instance (named funcXXX), reused by both writeContent and unparse where the same closure applies to both. A closure that also captures a per-call parameter (containerNode) is left as-is: it cannot be hoisted since containerNode genuinely differs on every call. Affects DelimiterStackUnparser, DynamicEscapeSchemeUnparser, HiddenGroupCombinatorUnparser, SpecifiedLengthExplicitImplicitUnparser, SpecifiedLengthPrefixedUnparser, and ElementUnparserBase's unparse() dispatch closure. No behavior change. DAFFODIL-3065
DaffodilTunables.withTunable generated one match over every tunable name in a single method; enough tunables now exist that this crossed the JVM's 64KB-per-method bytecode limit, failing compilation with "Method too large". Generate withTunablePart0..N instead, each holding tunablesPerPart (8) tunables' worth of cases, with a fallthrough to the next part on no match and the final part throwing the same "unknown tunable" error the single match used to. withTunable itself keeps its existing public signature, delegating to withTunablePart0. DAFFODIL-3065
olabusayoT
force-pushed
the
daf-3065-build-walker-builder
branch
from
September 29, 2026 14:43
846c1fa to
b244541
Compare
Contributor
Author
|
Superceded by #1736 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an off-by-default (useBuildWritePrefetch) two-pass unparse path: build races ahead of write over the same infoset tree via a coroutine handoff (BuildCoroutine/WriteCoroutine, UnparseSharedContext), resolving OVCs directly against the tree instead of via Suspensions where possible. Build is driven by a new Builder tree paralleling Unparser, wired through the grammar combinators, with unparseBeginForBuild/ unparseEndForBuild entry points on ElementUnparserBase that only the Builder tree calls. Write's coroutine thread spawns only if build's lead or pending-suspension count crosses a threshold; otherwise write runs inline on build's own thread. hasAnyPrefetchBeneficialOVC is scoped per compiling root, gated on the tunable, and requires a non-constant OVC expression via the new isPrefetchBeneficial. ElementUnparserBase and several WriteUnparser implementations' duplicated writeContent/unparse setup-dispatch-teardown logic is deduplicated via shared runElementContent/withPushPop helpers.
DAFFODIL-3065