[jasperpotts]

What and why

This PR adds part 1 of the opt-in byte-array parser from docs/fast-codegen-plan.md section 6.3 (pbj.fast_parse, #987), following the techniques in sections 5.2 and 5.3.

Performance (AMD EPYC 9275F): with fast_parse, Timestamp parses 31% faster (12.1 → 8.4 ns) and signed transactions 5% faster; block items are unchanged until part 2 (#1002), which covers the messages they contain. Tier B, opt-in.

• Runtime: ArrayCursor (io.buffer), a read position and limit over a byte[]:
  • at(tag), for one- and two-byte tags, used by the in-order fast path;
  • varint(), PBJ's XOR-chain reader from ByteArrayBufferedData with limit checks and the one-byte case inline;
  • fixed32/fixed64, and bool, which rejects values other than 0 and 1;
  • string, with strict UTF-8 decoding and an ASCII fast path;
  • bytes, a copy;
  • length, giving the same maxSize error messages, plus pushLimit/popLimit for nested messages;
  • parseWith(codec, …), for nested messages that have no array parser;
  • unknownField(…), with exactly the behaviour of parseImpl's defaultCase.
• Routing in the final Codec.parse methods: when the codec has an array parser (hasArrayParser()), array-backed BufferedData goes to parseArrayImpl, and the input is advanced by the bytes consumed. parse(Bytes) and parseStrict(Bytes) go there directly. Streams and direct buffers keep parseImpl.
• Generated code, for fast_parse messages: public static T parseArray(ArrayCursor, …).
  • It first tests for each field's tag in field order (one compare per field).
  • Then it runs a general loop with the same cases and errors as parseImpl: out-of-order and repeated values, unknown fields, and strict-mode duplicate checks.
  • Nested messages with an array parser are called directly; others go through input.parseWith(X.PROTOBUF, …).
• Mixed settings: whether a message from another file has an array parser is resolved from that message's own comments, its file's options and the project setting, so mixed settings work in both directions.

Part 1 scope. Array parsers are generated only for messages whose fields are all singular scalars, strings, bytes, enums or messages, and whose fast_parse nested messages also qualify. The second condition is needed because a nested message parsed with parseWith allocates a BufferedData wrapper, which made AllocationTest fail for SignedTransaction → SignatureMap. Part 2 (#next PR) supports every field kind, and the rule goes away.

Scope adjustment. The plan puts unknown fields in part 2, but every message needs them to be correct, so they are in part 1.

Known gap, fixed in part 2. For parse(Bytes), parseImpl returns bytes fields as views of the input, because Bytes is immutable. In this part ArrayCursor always copies, so results are equal but allocation is higher for bytes-heavy messages. Part 2 makes the cursor return views when it reads a Bytes.

Generated code for com.hedera.hapi.node.base.TransactionID, before and after

Before: only parseImpl. After, added:

public static @NonNull TransactionID parseArray(@NonNull final ArrayCursor input, final boolean strictMode,
        final boolean parseUnknownFields, final int maxDepth, final int maxSize) throws ParseException {
    ...
    // In-order fast path: writers emit fields in field-number order, so test for each tag in turn
    if (input.at((byte) 0x0A)) {
        final int messageLength = input.length(maxSize, "transactionValidStart");
        if (messageLength == 0) {
            temp_transactionValidStart = Timestamp.DEFAULT;
        } else {
            final int limitBefore = input.pushLimit(messageLength);
            temp_transactionValidStart = com.hedera.hapi.node.base.codec.TimestampProtoCodec.parseArray(input, strictMode, parseUnknownFields, maxDepth - 1, maxSize);
            input.popLimit(limitBefore);
        }
    }
    ...
    if (input.at((byte) 0x20)) {
        temp_nonce = (int) input.varint();
    }
    // Everything else, as parseImpl does: fields out of order or repeated, and unknown fields
    while (input.hasRemaining()) {
        final int tag = input.tag();
        switch (tag) {
            case 10 /* type=2 [MESSAGE] field=1 [transactionValidStart] */ -> {
                if (strictMode && temp_transactionValidStart != null) {
                    throw new ParseException("A non-repeated transactionValidStart encountered more than once");
                }
                ...
            }
            ...
            default -> $unknownFields = input.unknownField(tag, TransactionIDSchema.getField(tag >>> TAG_FIELD_OFFSET) != null, strictMode, parseUnknownFields, $unknownFields, maxSize);
        }
    }

Tests

• ArrayCursorTest: 20,000 random varints checked against BufferedData (fast and near-limit paths); truncation and limit checks; fixed values; strings (UTF-8, malformed, maxSize); bytes; bools; tags; nested limits; unknown fields (skip, collect, strict, bad tags); and cursors over BufferedData and Bytes slices.
• PerformanceOptionsTest: fast_parse adds parseArray only where enabled. A message comment opts out of the project setting, and models are unchanged.
• fastParseMixed.proto (new): flagged messages nesting unflagged ones and the reverse, plus a two-byte tag. Its generated tests run in both configurations.
• pbj-core: spotlessApply qualityGate build passes.
• pbj-integration-tests:
  • ./gradlew build passes, 326,405 tests;
  • ./gradlew build -PpbjPerformance=max passes, 326,405 tests, with fast_parse on for every message (233 HAPI and test messages get an array parser in this part). That covers the protoc comparisons, the non-canonical-order tests (the fallback loop), MaxDepthTest, MaxSizeTest, UnknownFieldsTest, MalformedMessageTest, TruncatedDataTests, ParserNeverWrapsTest, ParseLimitRestoreTest, the fuzz tests and AllocationTest.

Performance: AMD EPYC 9275F, before = #997 (no fast_parse), after = this PR with fast_parse on

3 forks, 3 × 1 s warmup, 5 × 1 s measurement, -prof gc.

Benchmark	Before ns/op	After ns/op	Change	Before B/op	After B/op
BlockItemBench.itemParseBytes	547.1 ± 19.8	548.2 ± 17.3	+0.2%	1,719	1,719
BlockItemBench.transactionParseBytes	76.1 ± 4.3	72.4 ± 1.1	-4.9%	395	395
ProtobufObjectBench.AccountDetailsBench.parsePbjByteArray	1,049 ± 23.9	1,041 ± 28.8	-0.8%	4,280	4,280
ProtobufObjectBench.EverythingBench.parsePbjByteArray	27,154 ± 669.7	27,118 ± 616.3	-0.1%	49,752	49,752
ProtobufObjectBench.TimeStampBench.parsePbjByteArray	12.1 ± 0.3	8.4 ± 0.2	-30.9%	40	72
SampleBlockBench.parsePbjByteArray	46,282,917 ± 2,136,796	46,775,220 ± 2,376,244	+1.1%	127,042,063	127,042,067
SampleBlockBench.parsePbjByteBufferDirect	65,295,601 ± 2,264,980	66,054,305 ± 3,157,919	+1.2%	127,042,197	127,042,199
SampleBlockBench.parsePbjInputStream	50,913,371 ± 2,126,821	50,403,576 ± 2,749,839	-1.0%	127,156,297	127,156,290

• Timestamp parses 31% faster (12.1 → 8.4 ns), and signed transactions 5% faster. Block items and whole blocks are unchanged. In this part most HAPI messages (BlockItem, StateChanges, AccountID …) contain oneofs or repeated fields, so they keep parseImpl. Part 2 measures the full effect.
• +32 B per top-level parse, for the ArrayCursor itself (TimeStampBench: 40 → 72 B/op). It is passed through the overridable parseArrayImpl, so escape analysis does not remove it.

Generated code size (full HAPI build)

	main	#997	This PR, options off	This PR, -PpbjPerformance=max
All classes	57,723,772	58,373,007	58,704,484	59,089,804

With options off, generated HAPI code is unchanged; the difference is the new test proto. With options on, codec classes grow to 8,181,412 bytes, from 7,742,386 (+5.7%), for the 233 messages with array parsers.

Stacked on #997.

Fixes #998

🤖 Generated with Claude Code

[github-actions]

JUnit Test Report

   554 files  + 2     554 suites  +2   1m 9s ⏱️ +2s
 2 073 tests +17   2 069 ✅ +17   4 💤 ±0  0 ❌ ±0 
11 605 runs  +17  11 577 ✅ +17  28 💤 ±0  0 ❌ ±0 

Results for commit aa60849. ± Comparison against base commit a086f6c.

♻️ This comment has been updated with latest results.

[github-actions]

Integration Test Report

    630 files      630 suites   43m 27s ⏱️
163 932 tests 163 932 ✅ 0 💤 0 ❌
326 559 runs  326 559 ✅ 0 💤 0 ❌

Results for commit aa60849.

♻️ This comment has been updated with latest results.

[jasperpotts]

Merged in the review follow-ups from below in the stack: exact-size bulk varint writes (#994), unique writer local names (#995), dead-code removal (#996), map entries without try/finally plus tests (#997), and strict string decoding without per-call allocations (#1010). On #999 the array parser's ArrayCursor.string now uses the shared Utf8Tools.decodeUtf8Strict. The plan was updated too (#991). Re-verified: pbj-core spotless, quality gate and build, and the integration build with options off and with -PpbjPerformance=max, all passing. The benchmark tables in this PR's description predate these changes; writes and string parsing are now faster than shown (see #994 and #1010).

[jasperpotts]

Review follow-ups (second review, section 5), on this branch:

• Tests (5.1): ArrayCursorTest covers a varint of ten bytes that all have the continuation bit (DataEncodingException, as BufferedData), with exactly ten bytes left and with more, and unknown fields with wire types 6 and 7 in strict mode.
• Comment (5.2): the generator now says why tags of three or more bytes (field numbers above 2047) are left out of the in-order fast path: the general loop that follows reads any field in any order.
• Pre-existing behaviour (5.3): a repeated occurrence of a singular embedded message overwrites the earlier one (and strict mode rejects it) instead of merging as the protobuf spec says. The array parser matches parseImpl, which is right for this series. Spec merging would be a separate issue if the team wants it.

Two pre-existing bugs turned up while extending the test protos: #1011 (hashCode() repeats the code of the fields before a map; fixed in #1015) and #1012 (JSON indents a map inside a nested message differently from protoc).