[jasperpotts]

Closes #1027. Stacked on #1023 (1018-flyweight-writers).

Summary

A synthetic HAPI block with the structure and value sizes of the external block-stream workload, so its results can be reproduced and decided in this repo. Plus benchmarks, read-every-field visitors and a hand-written reference codec, the in-repo target. All of it is test and benchmark code (src/testMax, src/jmhMax); nothing ships.

Performance (AMD EPYC 9275F): this PR adds the workload and changes no library code. On it, the whole stack against 0.16.4 (final run, ns per transaction): block decode 3,002 → 1,353 ns (2.2×), block encode 3,389 → 1,410 ns (2.4×), transaction decode 1,202 → 148 ns (8.1×); writers encode with 0 B at 1.5× the reference codec, and views read every field with 0 B at about 3×. Full table: #1023 (comment)

• SyntheticBlock: a deterministic generator (fixed seed) of 20,000 transactions, using only HAPI messages.
  • Per transaction: a signed transaction (~53 B opaque), its result, 2.94 balance-change results, order book results (1.16 added, 0.51 trades, 0.96 deleted), an anonymous result with a 3-entry transfer list, and a state-changes item.
  • State changes: 12.6 per transaction. Token relations (including fee state with automatic association), account balances, full and private NFTs, NFT deletes, the entity-id singleton, and the receipt queue's push and pop.
  • Batches: 12% of transactions carry 5× the changes and balance changes.
  • Value sizes: accounts 2 or 3 varint bytes (50 hot), tokens 1 or 2, NFT serials 5 (sequential from 7×10⁹), balances 5-8 bytes, quantities 2-5, prices 3-6, fees 4-5.
  • Block framing: a header, footer and proof.
  • Generated as a flat value stream, which model objects (build), the generated writers (write) and the reference codec all read in the same order.
  • transactions(): 20,000 SignedTransactions with one signature pair each (80% ECDSA secp256k1, 20% ed25519), crypto-transfer bodies, and 12% atomic batches of 2-20 inner transactions.
• ScanObjects / ScanViews: read every field of the subset's ~25 message types, set or not, through objects or through views, and return the same sum. Numbers, booleans, enum ordinals and oneof field numbers are added; bytes contribute their length and first and last byte.
• ReferenceCodec: a benchmark-only, hand-written, zero-allocation codec for the subset, as fast as I could make it, so it is a real target.
  • Encode from values: back to front into its own buffer, as the external codec does, so no length is ever guessed or moved. Varints of up to 8 bytes are one 8-byte store; repeated sub-messages are gathered into primitive arrays first, then written last to first.
  • Read every field: in order, on byte[], with an in-order fast path per message and a switch for anything else, and a branch-light varint reader.
  • Decode: into light records.
  • It skips strict-mode validation.
• SyntheticBlockTest, ReferenceCodecTest:
  • The generator's shape is asserted, and the block is the same on every run.
  • Objects, writeReverse, item writers, the whole-block writer and the reference encoder write the same bytes for every item and for the whole block.
  • Objects, views, the reference scan and reference decode return the same sums for every item, the block and every transaction.
• SyntheticBlockBench: per transaction, with -prof gc; listed in the table below.

The shape, and why bytes are higher than the external 680 B

	Synthetic	External
Items per transaction	9.58	9.7 (group nonce, 0.11, skipped)
State changes per transaction	12.67	12.6
Bytes per state-changes item	434	316
Bytes per transaction	815	680

The extra bytes come from the same fields and values: HAPI wraps every id in a message. AccountID, TokenID and NftID cost about 3 more bytes each than a bare varint, and a nested Timestamp with varint nanos costs about 4.5 more than a fixed64. A transaction's state changes carry about 36 ids and 4.5 timestamps, about 127 bytes. The test asserts 680 + 127 and 316 + 127.

EPYC results

AMD EPYC 9275F, JDK 25.0.4, pinned to one CCD (taskset -c 0-2,24-26) of an idle machine, -Xms16g -Xmn12g. 3 forks × 5 iterations of 2 s, -prof gc. Every value is per transaction (block operations ÷ 20,000). PBJ is this PR's head with every option on (-PpbjPerformance=max, writers on).

Benchmark	ns per transaction	B per transaction
encodeFresh: fresh objects, write(byte[])	1,397 ± 23	8,105 *
encodeFreshToBytes	1,417 ± 12	8,919 *
encodeReverseFresh	979 ± 9	8,105 *
produceObjectsToBytes: values → objects → toBytes	1,886 ± 47	9,347
produceObjectsReverse: values → objects → writeReverse	1,416 ± 33	8,066
produceWriters: values → item writers	1,359 ± 3	0
produceWritersBlockHashed: Block.Writer, SHA-384 of each item	3,331 ± 94	0
referenceProduce	758 ± 5	0
decode: Block.PROTOBUF.parse(Bytes)	1,457 ± 19	8,565
referenceDecode	997 ± 36	3,750
scanObjects: parse, then read every field	2,095 ± 51	8,565
scanViews: read every field through views	2,400 ± 3	22
referenceScan	682 ± 14	0
txDecode: SignedTransaction + TransactionBody	151 ± 6	1,169
txScanObjects	173 ± 2	1,169
txScanViews	244 ± 3	0
referenceTxDecode	110 ± 16	693
referenceTxScan	56 ± 10	0

* The encode…Fresh allocation includes building the objects in @Setup(Level.Invocation), which isn't timed.

Validation against the external ratios (±20%)

PBJ ÷ reference	Synthetic	External	Difference
Writers ÷ reference produce	1.79×	1.7×	+5%	✓
Block decode (decode ÷ referenceDecode)	1.46×	1.3×	+12%	✓
Writers ÷ objects + writeReverse	0.96×	0.9×	+7%	✓
Views read every field (scanViews ÷ referenceScan)	3.52×	2.7×	+30%	✗
Transaction views read every field (not in the list)	4.36×	2.4×	+82%

Three of the four validate. With the reference encoder now writing back to front, writers against it validate; against the earlier forward encoder they were 1.28×. The views ratio now misses. The reference reader got faster, and HAPI has more small nested messages per byte than the external format, because every id and timestamp is a message: that per-message cost is exactly what views pay. I didn't change the composition to make it fit, so read the view ratios here as an upper bound on the external gap.

What the numbers already say

• Views read every field: 3.5× the reference, and slower than parsing into objects (2,400 against 2,095 ns per transaction). This is the biggest gap, and it's where the view fixes in the next PRs go. scanViews also allocates 22 B per transaction, all from the boxed Long that a wrapper view getter (SingletonUpdateChange.View.entityNumberValue()) returns.
• Transaction read every field through views: 4.4×. Part of this is that a view can't be wrapped around another view's bytes field, so bodyBytes is copied to a scratch array before the body's view reads it.
• Writers: 0 B, 4% faster than objects + writeReverse, and 1.79× the reference encoder. Decode is 1.46×, with more than twice the reference's allocation.

Running it

cd pbj-integration-tests
./gradlew test -PpbjPerformance=max --tests '*synthetic*'
./gradlew jmhJar -PpbjPerformance=max
java -jar build/libs/pbj-integration-tests-0.0.0.1-test-jmh.jar SyntheticBlockBench -f 3 -prof gc

🤖 Generated with Claude Code

[github-actions]

JUnit Test Report

   559 files  ±0     559 suites  ±0   1m 12s ⏱️ ±0s
 2 111 tests ±0   2 107 ✅ ±0   4 💤 ±0  0 ❌ ±0 
11 643 runs  ±0  11 615 ✅ ±0  28 💤 ±0  0 ❌ ±0 

Results for commit 2fe51fe. ± Comparison against base commit b42070f.

♻️ This comment has been updated with latest results.

[github-actions]

Integration Test Report

    659 files  + 2      659 suites  +2   50m 23s ⏱️ - 8m 43s
164 373 tests +11  164 373 ✅ +11  0 💤 ±0  0 ❌ ±0 
652 336 runs  +11  652 336 ✅ +11  0 💤 ±0  0 ❌ ±0 

Results for commit 2fe51fe. ± Comparison against base commit b42070f.

♻️ This comment has been updated with latest results.