Skip to content

perf: 1029: writers pay less per nested message, and nothing for small ones - #1033

Open
jasperpotts wants to merge 14 commits into
1027-synthetic-block-benchfrom
1029-writer-open-close
Open

jasperpotts wants to merge 14 commits into
1027-synthetic-block-benchfrom
1029-writer-open-close

Conversation

@jasperpotts

@jasperpotts jasperpotts commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Closes #1029. Stacked on #1028 (1027-synthetic-block-bench).

Summary

The flyweight writers (#1023) paid most of their time per nested message: opening a child, checking every field, and patching its length when it ended. This PR cuts that bookkeeping, and for small messages removes it entirely. Everything here is in the writers, so it only applies with pbj.writers on (tier B); models, codecs and the default configuration are unchanged.

Performance (AMD EPYC 9275F): on the synthetic HAPI block, writing through the writers is 18.9% faster (1,381 → 1,121 ns per transaction) and still allocates 0 B; block writes with SHA-384 per item are 6.5% faster. Tier B, opt-in with pbj.writers.

  • The writer's state is one guard field (531cbf59, 698ba8e6, a96d0161):
    • Sentinels mean "not open" (Integer.MAX_VALUE), "a child is open" (MAX_VALUE - 1) and "written by writeFrom" (MAX_VALUE - 2), so checking a field stays one compare, and ending an open child moves onto that compare's slow path.
    • The separate state and offset fields are gone, and closing a nested message stores less.
    • endChild() is small enough for the JIT to inline into the field checks.
    • Two- and three-byte varints are written inline.
  • Small messages are written in one call (5493cdac, 56f153f3, 987f5f07): a message with up to six number, bool or enum fields, and no repeated fields, maps or other messages, gets a setter on its parent's writer that takes the fields as arguments. It computes the length and writes the tag, length and fields directly, with no child writer and nothing to close:
    amount.accountID(0, 0, accountNum);              // was: amount.accountID().accountNum(accountNum) + an implicit close
    result.consensusTimestamp(seconds, nanos);
    associate.addTokens(0, 0, tokenNum);             // elements of repeated small messages too
    Enums are passed as their proto ordinal. This covers ids, timestamps, durations, fractions, key ids and token relations; 439 such setters across HAPI.
  • Messages under 128 bytes encoded are opened without a guess (openSmallChild): their length is always one byte, whatever the buffer's length reservation.

Tried and reverted (recorded in docs/fast-codegen-plan.md §9)

  • The buffer tracking the innermost open writer, so field checks needn't look for an open child: 12% slower, an extra load and compare per field.
  • Four- to eight-byte varints inlined (a 316-byte copy of the bulk writer, which is over the JIT's 325-byte inlining limit): 2% slower here, and 5-9% slower on a second, external block workload.
  • Nine-byte varints in one store: neutral on both workloads.

EPYC results

AMD EPYC 9275F, JDK 25.0.4, one CCD (taskset -c 0-2,24-26) of an idle machine, 3 forks × 5 iterations of 2 s, -prof gc. ns per transaction of the synthetic HAPI block (#1028), every option on.

Step produceWriters produceWritersBlockHashed Allocation
Base (#1028) 1,381 ± 22 3,246 ± 14 0 B
Inlinable auto-close, 2-3 byte varints 1,383 ± 10 (+0.1%) 3,276 ± 63 (+0.9%) 0 B
State folded into the guard 1,362 ± 35 (−1.4%) 3,253 ± 19 (+0.2%) 0 B
Smaller close 1,348 ± 20 (−2.4%) 3,233 ± 29 (−0.4%) 0 B
One-call small messages (head) 1,121 ± 6 (−18.9%) 3,033 ± 51 (−6.5%) 0 B

The one-call setters are the whole win; the state changes before them are neutral, and kept because the writer is simpler with them. In the full benchmark run at each PR head, produceWriters went from 1,359 to 1,146 ns (−15.7%), which is 1.52× the hand-written reference encoder (756 ns in the same run), down from 1.79×. produceWritersBlockHashed includes SHA-384 of every item.

API

The one-call setters add public methods to generated writers (439 in HAPI). They are overloads next to the existing child accessors, so nothing is removed or renamed, but it is new surface to review.

Tests

  • InlineSetterTest: ids (with their oneof's one number alternative), timestamps and repeated elements written in one call give the same bytes as the child writer and as the model, for zero, small, large and negative values, and the field order is still checked.
  • WriterGenerationTest: which messages get one-call setters, enums included: six fields qualify, while seven, a string field or two number alternatives of one oneof don't; and small children are opened with openSmallChild.
  • The whole integration suite with every option on writes every test object through its writer and compares it with the model's encoding.

🤖 Generated with Claude Code

jasperpotts and others added 13 commits October 30, 2026 23:02
A CHILD guard value marks an open nested message, so ending it on the
parent's next field is a small non-recursive-in-practice call the JIT
inlines, instead of checkSlow (89 bytes, not inlined) plus closeInside.
varintWide writes 2- and 3-byte varints inline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Open, ended and written-by-writeFrom are guard values, so opening and
closing a nested message store one field less each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
The offset is the start, updated when the bytes move, and a one-byte
length that fits leaves the adaptive guess as it is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
…guess

A message field whose type has at most four integer or bool fields (ids,
timestamps; a oneof may have one integer alternative) gets an overload on
the parent writer that writes it from its fields' values in one call:
tag, length computed from the values, fields. No child writer is opened.
Children whose largest encoding is under 128 bytes open with
openSmallChild, a one-byte length without consulting the reservation.
The synthetic workload's writer code uses the new overloads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Enums are passed as their ordinals. On HAPI this adds 4 overloads to the
413 of the four-field limit, with no measurable change in class size.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
A repeated field whose element type qualifies gets addXxx(values...) too,
writing one element per call. On HAPI: 22 more overloads, 439 in all.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Values from 2^56 to 2^63, such as int64 nanosecond times, went through a
call to ProtoArrayWriterTools that the JIT did not inline, which an EPYC
profile of an external block workload put at 8.5% of encoding. They are now
written inline: the low 56 bits as eight continued groups of seven in one
little-endian store, then the last byte. Other sizes are unchanged, as one
8-byte store for three to eight bytes measured slower on EPYC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
ProtoArrayWriterTools.writeVarIntWide, which writers called for varints of
four bytes and more, is 437 bytes of bytecode, over the 325 the JIT inlines
for hot methods, so every such value was a call. The four- to eight-byte
cases are now in ProtoWriteBuffer.varint4To8, the same exact-size stores in
316 bytes, which the JIT inlines; negative values, ten bytes, still call it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
This reverts commit f2c68bb. On AMD EPYC, with an external block workload,
it made block encoding 5-9% slower and transaction encoding 9% slower,
reproduced in two runs: inlining a 316-byte writer at every varint site of
the large generated write methods cost more than the call it saved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
This reverts commit e134322. On AMD EPYC it was neutral: writers +1.6% and
-1.5% hashed on the HAPI workload, within noise, and neutral on an external
block workload full of nine-byte nanosecond times. The plan's lessons record it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

JUnit Test Report

   559 files     559 suites   1m 18s ⏱️
 2 112 tests  2 108 ✅  4 💤 0 ❌
11 644 runs  11 616 ✅ 28 💤 0 ❌

Results for commit de84353.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Integration Test Report

    660 files      660 suites   44m 41s ⏱️
164 377 tests 164 377 ✅ 0 💤 0 ❌
652 340 runs  652 340 ✅ 0 💤 0 ❌

Results for commit de84353.

♻️ This comment has been updated with latest results.

Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
@jasperpotts
jasperpotts force-pushed the 1029-writer-open-close branch from e0fb5c6 to de84353 Compare November 2, 2026 20:33
@jasperpotts
jasperpotts force-pushed the 1027-synthetic-block-bench branch from a0a4673 to 2fe51fe Compare November 2, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant