Skip to content

feat: add Cloudflare Queues destination - #907

Open
alexluong wants to merge 8 commits into
mainfrom
feat/cloudflare-queues-destination
Open

alexluong wants to merge 8 commits into
mainfrom
feat/cloudflare-queues-destination

Conversation

@alexluong

@alexluong alexluong commented May 20, 2026 •

Copy link
Copy Markdown
Collaborator

Adds Cloudflare Queues as a destination type. Each event is pushed with one request to Cloudflare's POST /accounts/{account_id}/queues/{queue_id}/messages as {"body": {"data": …, "metadata": …}, "content_type": "json"}, authenticated with an API token that has Account > Queues > Edit (Queues Write).

Continues #657 by @Swahjak, whose commit is kept as the first commit. The commits after it rebase onto current main (shared HTTP client from #1021, format errors as failed attempts from #957, DestinationUpdate discriminator), fix the API payload and response handling, and correct the docs against the live API. Closes #655. Supersedes #657.

Behavior

  • account_id and queue_id must be 32-character lowercase hex, which is how Cloudflare shows them. Cloudflare rejects queue names, dashed UUIDs and uppercase IDs on every push, so Outpost rejects them with a 422 when the destination is created or updated.
  • Any non-2xx response, or a 2xx with success: false, is a failed attempt that keeps Cloudflare's error code and message. A token without Queues Write gets the same 401 "Authentication error" as an invalid token. Cloudflare doesn't return a 403 for it.
  • Cloudflare's 128,000-byte message limit includes Outpost's data/metadata wrapper. Larger events fail with 413 and aren't checked before sending, same as SQS, Kinesis and Pub/Sub.
  • The OpenAPI DestinationType enum also gains kafka, which was missing. This changes the generated SDK enums.

Tests

The fixtures in destcfqueues_publish_test.go are real Cloudflare responses (success, 401, 400, 404, 413, 500) recorded against a live queue, with IDs scrubbed. TestCloudflareQueuesDestination_Validate covers the ID format.

Tested manually against a real queue: publish and read back through a pull consumer, bad token, token without Queues Write, bad account and queue IDs, partial PATCH, and an oversize event.

SDK regen and the spec-sdk-tests contract test come in a follow-up after merge.

🤖 Generated with Claude Code

Swahjak and others added 8 commits September 28, 2026 23:24
Add support for publishing events to Cloudflare Queues via their HTTP API.

Implements #655

- Add destcfqueues Go provider with Publish, Validate, and ComputeTarget
- Add provider metadata (metadata.json and instructions.md)
- Update OpenAPI spec with CloudflareQueues schemas
- Add user documentation (cloudflare-queues.mdx)
- Register provider in default.go
- Add validation and publish unit tests
- Add SDK test factory and integration tests
@alexluong
alexluong force-pushed the feat/cloudflare-queues-destination branch from 8e4a350 to 74c972b Compare September 28, 2026 16:51
@alexluong
alexluong marked this pull request as ready for review September 28, 2026 16:51

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Destination - Cloudflare Queues

2 participants