Repository navigation
feat(query,storage,http): download queries and ranged storage reads - #944
Conversation
…e URL) and ranged storage reads
`query({ input, policy, download: { expiresInMs?, handle } })`: GET /_x/query/<name> without
`_token` runs the policy and mints `{ url, expiresAt }`; with `_token` it streams the file
(content-disposition, no-store, nosniff, CSP sandbox, one Range window → 206, 416). The token is a
core seal() of the query name, the input's fingerprint, the expiry and the minting actor: forged,
re-pointed or expired tokens are 403 / 410. The typed client exposes `.url(input)` for a download.
`disk.get(key, { range: [first, last] })` on the memory, local and s3 drivers (inclusive, clamped,
X_STORAGE_RANGE_UNSATISFIABLE past the end; maxGetBytes bounds the window). The byte-range parser is
one helper in @ultimat3/http, shared by /_storage and site assets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Fix failing CI on #944
Queued — waiting for a free box. 🤖 developerz.ai, an automated agent, not a human — developerz-ai[bot] |
…ries Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
✅ Reviewed — nothing blocking0 actionable comment(s) · grounded on your code ⏱ 49s wall clock · 🤖 developerz.ai — automated review, running on your model and your box. |
There was a problem hiding this comment.
Review summary — 59 file(s), 14 finding(s).
Critical 0 · Major 2 · Minor 12 · Nit 0
completed — reviewer[bot]: Adds download queries and ranged-read contracts. Found a declaration compatibility break, inaccurate OpenAPI file responses, and misleading repair/privacy guidance. completed [bot] — Adds ranged storage reads and shared HTTP range handling for downloads.
| File | Findings |
|---|---|
packages/query/README.md |
1 minor |
packages/query/src/download-errors.ts |
1 minor |
packages/query/src/download-http.test.ts |
1 minor |
packages/query/src/download-http.ts |
1 major, 1 minor |
packages/query/src/openapi.ts |
2 minor |
packages/query/src/query.ts |
1 minor |
packages/storage/src/driver-s3.ts |
1 major |
packages/storage/src/errors.ts |
1 minor |
packages/storage/src/range.test.ts |
2 minor |
wiki/Queries-And-Live-Queries.md |
2 minor |
🤖 developerz.ai review — automated, running on your model and your box. What is this? · Reviewed by router/gpt-6.1-sol on box hv-dedicated-dev-marshall — 953.1k in / 22.5k out over 19 calls; cost unknown: no price on record for router/gpt-6.1-sol.
…o-referrer, s3 ranged read re-checked against a second HEAD, coded refusals and runnable fixes, OpenAPI file bodies and 401 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Review summary — 24 file(s), 6 finding(s).
Critical 0 · Major 2 · Minor 4 · Nit 0
[bot] completed: Hardens download headers and range-error contracts and retries raced S3 reads. Provider 416 responses still bypass race recovery, and the EOF repair can exceed the read ceiling. completed [bot] — Tightens download security, OpenAPI/type contracts and ranged-read diagnostics.
| File | Findings |
|---|---|
packages/query/src/download-http.test.ts |
1 minor |
packages/query/src/download-http.ts |
1 minor |
packages/storage/src/driver-s3.ts |
2 major |
packages/storage/src/errors-range.ts |
1 minor |
packages/storage/src/range.test.ts |
1 minor |
4 non-blocking finding(s) below major (no thread on a re-review — only blocking findings open threads)
packages/query/src/download-http.test.ts:223minor · test [bot] This isolation test still passes withimg-src *, which permits its hostile image to load evil.example despite the claimed restriction. Assert the complete expected handler CSP as a literal, or add a browser test proving that the external resource is blocked.packages/query/src/download-http.ts:105minor · docs [bot]img-src 'self'andmedia-src 'self'allow any same-origin URL, not just this file, so inline HTML can request other app paths despite the stated guarantee. Describe the same-origin allowance here and in fileResponse's comment instead of claiming only the file itself can load.packages/storage/src/errors-range.ts:26minor · bug [bot] With a 10-byte object and maxGetBytes: 6, this EOF fix retries [0,9] and immediately fails with X_STORAGE_TOO_LARGE. Pass the read ceiling to rangeUnsatisfiable and clamp the suggested end so the repair succeeds (CLAUDE.md: errors are instructions).packages/storage/src/range.test.ts:190minor · test [bot] This shrink-past-start test receives an empty slice from FakeS3Client, whereas S3 rejects bytes=6-9 on a four-byte object with 416, so the provider-failure path is never exercised. Make the fake emit S3Error InvalidRange/416 for this request and retain the X_STORAGE_RANGE_UNSATISFIABLE assert…
🤖 developerz.ai review — automated, running on your model and your box. What is this? · Reviewed by router/gpt-6.1-sol on box hv-dedicated-dev-din — 499.8k in / 9.67k out over 11 calls; cost unknown: no price on record for router/gpt-6.1-sol.
…ze and etag from the same response; the provider's 416 is X_STORAGE_RANGE_UNSATISFIABLE Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…anged-storage # Conflicts: # CHANGELOG.md # framework.manifest.json # packages/cli/src/serve-graph.test.ts
There was a problem hiding this comment.
Review summary — 14 file(s), 4 finding(s).
Critical 0 · Major 1 · Minor 3 · Nit 0
completed: Reviewed ranged storage reads and their published contracts. Found uncoded S3 body-read failures, acceptance of incomplete ranges, and non-executable invariant fixes. Completed reviewed the supplied ranged-storage diff.
| File | Findings |
|---|---|
packages/storage/src/driver-s3-range.ts |
1 major, 1 minor |
packages/storage/src/range.test.ts |
1 minor |
packages/storage/src/range.ts |
1 minor |
3 non-blocking finding(s) below major (no thread on a re-review — only blocking findings open threads)
packages/storage/src/driver-s3-range.ts:57minor · bug A GET for [2,5] answered with Content-Range: bytes 2-3/10 and two bytes passes every check and returns an incomplete window as success. Require the served end to equal Math.min(capped, size - 1) and validate the total size before accepting the response.packages/storage/src/range.test.ts:114minor · test Moving readS3Range's arrayBuffer() before its ceiling check leaves this test green, so it does not enforce the promised refusal before buffering. Inject a response with an observable body-read seam and assert it remains unread when X_STORAGE_TOO_LARGE is raised.packages/storage/src/range.ts:46minor · bug Both malformed-range fixes append uncommented em-dash prose to a JavaScript call, so pasting either X_INVARIANT repair produces a syntax error. Put the explanation inside a JavaScript comment and supply concrete valid offsets, as CLAUDE.md's executable-fix rule requires.
🤖 developerz.ai review — automated, running on your model and your box. What is this? · Reviewed by router/gpt-6.1-sol on box hv-dedicated-5 — 505.9k in / 2.4k out over 8 calls; cost unknown: no price on record for router/gpt-6.1-sol.
…_FAILED Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Review summary — 3 file(s), 0 finding(s).
completed: S3 ranged reads now translate response-body failures into coded storage errors, with a regression test for a broken 206 stream. No defects found in this increment.
🤖 developerz.ai review — automated, running on your model and your box. What is this? · Reviewed by router/gpt-6.1-sol on box hv-dedicated-dev-marshall — 59.5k in / 219 out over 2 calls; cost unknown: no price on record for router/gpt-6.1-sol.
Why
Gaps from a production app (tesote/bank-integrations
docs/upstream-issues.md§23–24):What
query(), not a ninth primitive, so they keep the query's input, policy, name, route, manifest row and typed client.GET /_x/query/<kebab>without_tokenruns the policy, rate limit and audit exactly as a read does, and answers{ url, expiresAt }._tokenit serves the file without a session. Headers arecontent-disposition(ASCIIfilenameplusfilename*),content-length,cache-control: no-store,nosniffandcontent-security-policy: sandbox. OneRange:window gives a 206; past the end, a 416.seal()(AES-256-GCM, purposeultimate.query.download:<name>) over the version, name, input fingerprint, expiry and minting actor (nofacts). The seal is checked before the input is parsed. A forged or re-pointed token is 403 (X_QUERY_DOWNLOAD_TOKEN_INVALID); an expired one is 410..url(input)for a download query (type-pinned). On the server,q.url(input, { ctx }).sql,live,single,rows,subscribes,cacheandmcpbesidedownloadare type errors andX_QUERY_DOWNLOAD_INVALID..live(),.page(),.as()and MCP areX_QUERY_DOWNLOAD_ONLY.diff-operationsreports rows ↔ download as breaking.disk.get(key, { range: [first, last] })on memory, local and s3:X_STORAGE_RANGE_UNSATISFIABLE(416) at or past the end;maxGetBytesbounds the window, not the object;@ultimat3/httpbyte-range helper:parseByteRange,byteWindow,contentRangeOf. The two cli copies (/_storage, site assets) now use it; both now ignore an emptybytes=-.Tests
size, token checked before input).bun run verifyis green. Pins raised with measured numbers:examples/dummy/feed167.5 → 167.6 kB (+128 B for the client'surl()).The stale
dummy/social-media-clonex_job_workers.sqldump is regenerated (draining_at). The app-gate e2e needs Chrome, which wasn't on the authoring box, so CI runs it.🤖 Generated with Claude Code