Skip to content

Data Access: refuse dataset downloads with no time window - #849

Merged
AlexGodbehere merged 3 commits into
devfrom
ag/bounded-download
Nov 3, 2026
Merged

AlexGodbehere merged 3 commits into
devfrom
ag/bounded-download

Conversation

@AlexGodbehere

Copy link
Copy Markdown
Contributor

POST v1/data/:uuid downloads every resolved source over its own time range. A Sparkplug source, or a union of them, has no range, so downloading a machine dataset read every device's whole history. In testing, one machine dataset reached 660 MB before the client was stopped. This PR refuses that download and says how to bound it.

What changes

  • The data download takes an optional window. from and to (ISO 8601) go in the request body. They narrow each source's resolved range and never widen it.
  • A download with no end is refused. If any source is still open at either end, Data Access returns 422 with {"error": "This dataset has no time window, so downloading it would return its whole history. Add from and to to the request, or download a session dataset over the time you need."}. A window outside the dataset's own, an unparseable date, or from not before to is refused the same way, with its own reason.
  • Sessions are unchanged. A dataset bounded by Session limits downloads with no window in the body, exactly as before.
  • Admin UI: the Data tab's download sends the time range in view for a dataset with no window, so it keeps working. The header, list and add-on downloads show the service's reason when refused, instead of a bare status. The Use tab's example request includes a window when the dataset needs one.
  • Docs: docs/services/data-access.md describes the window and the refusal, and the release notes say what clients need to change.

Who this affects

Any client that downloads a device or union dataset without a window now gets 422. That includes DataAccess.download_data in the JS service client, which sends no window. The fix is to download a session over the time needed, or to pass from and to.

How to test

  1. cd acs-data-access && npm test: 364 pass. The window tests are in tests/current/download_filter.test.js.
  2. cd acs-admin && npm test: 844 pass.
  3. On a test cluster, POST /v1/data/<a machine union> with {} returns 422 with the reason above, and Influx sees no query.
  4. The same request with {"from": "<an hour ago>", "to": "<now>"} returns a CSV covering only that hour.
  5. POST /v1/data/<a session> with {} returns the session's CSV, as before.
  6. In the Admin UI, open a machine dataset's Data tab, pin a metric and download. The file covers the range on screen. Download CSV in the page header shows the "no time window" message.

Release notes

Data Access no longer downloads a dataset with no time window of its own, such as a machine's device or union dataset, because that read the device's whole history. It returns 422 with a reason. Download a session dataset over the time you need, or pass from and to in the request body. Sessions download as before.

🤖 Generated with Claude Code

POST v1/data/:uuid read every resolved source over its own range, and a
Sparkplug source or a union of them has none, so the request read the
device's whole history. On production a machine dataset ran to hundreds
of megabytes before the client gave up.

The request body now takes an optional from and to, which narrow each
source's range and never widen it. If any source is still open at
either end, the service returns 422 with a reason that says how to
bound it: pass from and to, or download a session dataset. A window
outside the dataset's own, or with from not before to, is refused too.
Sessions download as before.
The Data tab's download sends the time range in view for a dataset with
no window of its own, so it keeps working now that Data Access refuses
unbounded downloads. The other download buttons show the service's
reason when it refuses, rather than a bare HTTP status. The Use tab's
example request includes a window when the dataset needs one.
@AlexGodbehere
AlexGodbehere merged commit ac9887b into dev Nov 3, 2026
2 checks passed
@AlexGodbehere
AlexGodbehere deleted the ag/bounded-download branch November 3, 2026 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant