Skip to content

feat(listen): forward binary request bodies byte-exact - #463

Open
leggetter wants to merge 3 commits into
mainfrom
feat/listen-binary-bodies
Open

leggetter wants to merge 3 commits into
mainfrom
feat/listen-binary-bodies

Conversation

@leggetter

@leggetter leggetter commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

hookdeck listen now forwards binary request bodies (non-UTF-8 payloads, and multipart file uploads) byte-exact.

  • Handshake: sends X-Hookdeck-CLI-Capabilities: binary on every websocket connect.
  • Forwarding: decodes request.data_base64 and forwards the original bytes with the original Content-Type. The CLI does not parse multipart.
  • Compatibility: text attempts and older servers keep using data_string.
  • Server side: hookdeck/core#5709. Until it deploys, this PR changes nothing at runtime.
Other changes
  • A body that fails to decode fails the attempt immediately instead of waiting for the server timeout.
  • The TUI summarises binary bodies by size instead of printing them.
Tests

End to end (local stack, hookdeck/core#5709):

  • New CLI receives raw PNG, JPEG, PDF and octet-stream bodies byte-exact.
  • New CLI receives a multipart upload with image and audio parts byte-exact (needs hookdeck/http-ingestion#547).
  • A retried binary event is redelivered byte-exact.
  • Old CLI (the v3.0.3 release): raw binary fails with CLI_BINARY_UNSUPPORTED.
  • Old CLI: multipart is still delivered, as lossy text.
  • Old and new CLI listening at once each get the right outcome.

Acceptance tests (test/acceptance/listen_binary_test.go, listen tag):

  • A local app saves each request to disk; every file must match the original SHA-256 and still decode.
  • The old CLI is the v3.0.3 release, downloaded and checked against a pinned SHA-256.
  • Gated until hookdeck/core#5709 deploys (multipart until hookdeck/http-ingestion#547 ships). A follow-up PR removes the gates.

Unit:

  • binary_body_test.go: byte-exact forwarding (raw and multipart), text path unchanged, invalid base64 rejected.
  • client_test.go: the capability header is sent on connect.

Size, one-off local run:

  • A body just under the 10 MB cap goes through the proxy and CLI byte-exact, as a ~13.3 MB websocket message. This run posted to the proxy's /deliver directly: local ingestion can't store bodies over 2.5 MB without GCS.
  • Bodies just under 2.5 MB pass end to end, raw and multipart.

Not tested:

  • Staging or production, including their load balancers with a ~13 MB websocket message.

🤖 Generated with Claude Code

leggetter and others added 3 commits September 27, 2026 16:59
`hookdeck listen` now advertises X-Hookdeck-CLI-Capabilities: binary on
every websocket connect. Servers that support it send binary bodies as
request.data_base64 with body_format: binary; the CLI decodes them and
forwards the original bytes with the original Content-Type (multipart
boundary included) without parsing multipart. Text attempts and servers
that predate data_base64 keep using data_string.

- A body that fails to decode fails the attempt immediately instead of
  waiting for the server timeout.
- The TUI summarises binary bodies by size instead of printing them.
- Acceptance test TestListenForwardsBinaryBodiesByteExact (listen tag) is
  gated on HOOKDECK_CLI_TESTING_BINARY_DELIVERY (and
  HOOKDECK_CLI_TESTING_MULTIPART_BINARY_DELIVERY for multipart) until the
  server side is deployed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…inary bodies

- TestListenForwardsBinaryFilesThatTheAppCanSave: a local app saves each
  request to disk (raw body by Content-Type, multipart via
  ParseMultipartForm) and every file must match the original SHA-256 and
  still decode: PNG, JPEG, PDF, every byte value, and a multipart upload
  with a PNG and a WAV.
- TestListenRetriesBinaryBodyByteExact: the first delivery returns 500,
  `gateway event retry` redelivers, and attempt 2 carries the same bytes.
- TestListenBinaryWithOldAndNewCLIListening: this CLI and one built with
  no advertised capabilities listen on the same source as separate CLI
  clients. This CLI gets exact bytes; the other's event fails with
  CLI_BINARY_UNSUPPORTED, and multipart reaches it as lossy text.

The advertised capabilities are now a variable so the test can build a
CLI that advertises nothing with -ldflags -X, instead of skipping when no
old binary is available.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The mixed-version test now downloads the v3.0.3 release for the current
platform, verifies it against a pinned SHA-256, and runs it alongside
this CLI. That is the real last release before binary delivery, so the
ldflags hook that built a CLI advertising no capabilities is removed and
the capabilities header is a constant again.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@leggetter
leggetter marked this pull request as ready for review September 28, 2026 10:57
@leggetter
leggetter requested a balanced review from Copilot September 28, 2026 11:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants