Repository navigation
feat: zero-copy views, part 2: oneofs, repeated fields and wrapper types - #1007
jasperpotts wants to merge 20 commits into
Conversation
Extends pbj.views (docs/fast-codegen-plan.md section 6.5) so block items and the messages they contain get views: - oneofs: the view records the alternative seen last; xKind() returns the model's kind enum, and the other alternatives' accessors return defaults; - repeated fields: wrapping validates and counts the elements and records the first one; xCount() and a reusable cursor (for (var it = view.x(); it.next(); ) ...) read unpacked elements and packed runs alike, with child views for messages; - google.protobuf wrapper types: hasX() and a boxed x(), null when absent, as in the model. Messages with maps or repeated wrapper types still have no view. Generated tests check every oneof alternative and repeated element against the model. BlockItemViewBench, in a jmhMax source set built only with -PpbjPerformance=max, routes block items through a view and through parsing. Fixes #1006 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Integration Test Report 639 files 639 suites 58m 21s ⏱️ Results for commit 10acfd3. ♻️ This comment has been updated with latest results. |
|
Merged in the review follow-ups from below in the stack: exact-size bulk varint writes (#994), unique writer local names (#995), dead-code removal (#996), map entries without try/finally plus tests (#997), and strict string decoding without per-call allocations (#1010). On #999 the array parser's |
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com> # Conflicts: # pbj-core/pbj-compiler/src/main/java/com/hedera/pbj/compiler/impl/generators/ViewGenerator.java
…se child views - wrap(bytes, strictMode, maxDepth, maxSize) and wrap(in, offset, length, strictMode, maxDepth, maxSize) validate as parse does with the same arguments, and pass them to child views and repeated field cursors. Strict mode rejects unknown fields and repeated singular messages. - Repeated string elements are checked to be valid UTF-8 while wrapping. - A nested accessor returns its child view without indexing it again if it is still wrapped over the field since the parent was wrapped. - LookupHelper.hasView is a greatest fixed point, so a cycle of messages through one without a view gives none of them a view, whichever is looked at first. The build says why a view is denied to a message that asks for one in its comments. - A message with views that declares its own nested type named View is a clear compiler error instead of a duplicate class. - Removes the unused, unchecked static ArrayCursor helpers. - Documents views, and the reuse of child views, in the usage guide. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
|
Review follow-ups (second review, sections 1.3, 1.4 and 4), on this branch:
Code size (4.7): with every option on, the generated code grows by about 17%. That is expected and fine: the 15% cap in the plan applies to tier A only, and views are tier C (opt-in), so it doesn't count against it. Perf (4.2): not profiled yet. Child re-indexing is fixed; profiling |
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
…th limits Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
… the wrapped array Strict mode is fail fast, but a strict wrap only validated the top-level message: unknown fields, depth and UTF-8 inside nested messages, repeated message elements and wrapper values were only checked if they were read, so a strict wrap accepted input parseStrict rejects. In strict mode wrap now validates all of them, at any depth, before returning. Non-strict views stay lazy, and the docs now say plainly that a non-strict wrap does not validate untrusted input. Bytes accessors returned Bytes.wrap over the wrapped array, and xxxArray() returned the array itself, so a view over a plain array handed out bytes that changed when the caller reused it, and a view over Bytes exposed the Bytes' mutable backing array. Views now remember the Bytes they wrap: bytes fields, wrapper bytes and toModel() share it, or copy from a plain array, as the array parser does. xxxArray(), xxxOffset() and valueOffset() are replaced by xxxCopyTo() and valueCopyTo(), which copy without allocating. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
What and why
This PR is part 2 of zero-copy views (
docs/fast-codegen-plan.mdsection 6.5,pbj.views). It adds what block items need: oneofs, repeated fields andgoogle.protobufwrapper types. With-PpbjPerformance=max, every HAPI message without maps now has a view,BlockItemincluded.Performance (AMD EPYC 9275F): routing a block item through a view is 7.4× faster than parsing it with the array parser (300.9 → 40.8 ns), with 0 B instead of 1,565 B. Tier B, opt-in.
xKind()returns the model's existing kind enum (for exampleBlockItem.ItemOneOfType). Each alternative's accessors return its default unless it is the current one.xCount()gives the count, andx()returns a reusable cursor, as in the plan's sketch:next(); strings and bytes elements have offset, length and equals accessors that don't allocate.google.protobufwrapper types:hasX(), plusx()returning the boxed value, null when absent and the default when the wrapper is empty, as in the model. The value is decoded with the view's own cursor.BlockItemViewBench(newsrc/jmhMaxsource set, built only with-PpbjPerformance=max, where HAPI messages have views) routes each block item: its kind, the consensus time of state changes, the id and operation of every state change, and the length of signed transactions. It does this through a view and through parsing.Not in this PR: maps and repeated wrapper types. Messages containing them, directly or through a nested message, have no view.
Tests
-PpbjPerformance=maxthat is 488,619 integration tests, all passing, with views on 574 messages.pbj-core:spotlessApply qualityGate buildpasses.pbj-integration-tests:./gradlew build(options off) passes, 326,416 tests.Performance: AMD EPYC 9275F, 3 forks,
-prof gcViewBench(from feat: zero-copy views over encoded messages, part 1 #1005): reading every field of a flat message is 1.6× faster through a view, again with 0 B/op.Generated code size (full HAPI build,
-PpbjPerformance=max)mainclasses: 53,275,718 bytes, against 45,502,478 onmain(+17%), with every option on. Views are opt-in (tier C), and with options off generated main code is unchanged.Stacked on #1005.
Fixes #1006
🤖 Generated with Claude Code