Repository navigation
perf: parse string fields without per-call decoder and buffer allocations - #1010
Open
jasperpotts wants to merge 9 commits into
Open
jasperpotts wants to merge 9 commits into
jasperpotts wants to merge 9 commits into
Conversation
…ions ProtoParserTools.readString allocated a ByteBuffer, a CharsetDecoder and a CharBuffer for every string field. It now decodes straight from the input's backing array when there is one (array-backed BufferedData, or Bytes), reached through an unexported com.hedera.pbj.runtime.internal bridge that io.buffer installs, so no public API widens; other inputs read the bytes into one array. The shared Utf8Tools.decodeUtf8Strict returns "" for zero length, a compact Latin-1 String for ASCII, and otherwise validates the bytes exactly as the JDK decoder does in REPORT mode before decoding them, still rejecting malformed UTF-8 with MalformedProtobufException. Utf8StrictDecodeTest compares it with the JDK decoder over every sequence of up to three bytes, four-byte sequences around their limits and random text. StringParseBench parses a token create body with ASCII and non-ASCII strings. Fixes #1008 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Refs #1008 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
This was referenced Oct 30, 2026
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
ProtoParserTools.readStringused to do four things for every string field it parsed:ByteBufferand copy the bytes into it;CharsetDecoder;CharBuffer;String.Performance (AMD EPYC 9275F): parsing string fields is 28-33% faster for ASCII and 12-16% for non-ASCII, and allocates 41-64% less (1,168 → 416 B for a token-create body). Tier A, on by default.
This PR removes those allocations and copies. It is a new tier A item from the review of the first PRs in the series, carried forward from old draft PR #21's idea, and it's always on.
BufferedData, orBytesthroughparse(Bytes). The array is reached through a new, unexported package,com.hedera.pbj.runtime.internal(ArrayInputs), with a bridge thatio.bufferinstalls when its input classes load. So no public API is widened; the accessors onByteArrayBufferedData,BytesandRandomAccessSequenceAdapterstay package-private. Other inputs read the bytes into one array.Utf8Tools.decodeUtf8Strict(byte[], offset, length):""for zero length;new String(…, ISO_8859_1), a compact Latin-1 string;isWellFormedUtf8), thennew String(…, UTF_8). That constructor is only safe after validation, because it replaces malformed input instead of failing.MalformedProtobufException, exactly as before.Tests
Utf8StrictDecodeTestcompares the validator and decoder with the JDK decoder in REPORT mode over:ExtendedUtf8MessageWithStringTest,MalformedMessageTest), the fuzz tests and the protoc comparisons pass.pbj-core:spotlessCheck qualityGate buildpasses.pbj-integration-tests:./gradlew buildand-PpbjPerformance=maxpass, 326,369 tests each.Performance: AMD EPYC 9275F, before = #997, after = this PR
StringParseBench: a HAPITokenCreateTransactionBodywith name, symbol and memo, with ASCII or non-ASCII strings, parsed fromBytesand from aBufferedData. 3 forks,-prof gc.BufferedData.Generated code size
Unchanged; this is a runtime change.
Stacked on #997. PR #999 (array parser, part 1) is now stacked on this one.
Fixes #1008
🤖 Generated with Claude Code