Native: storage admission (disk-pressure drain and fence) for 3.1 #951
Description
Raised by Cloud's native 3.0.0 cutover review. Cloud keeps customer admission on native limited to automation and qualification tenancy until this exists.
State
- Native refuses
DW_STORAGE_ADMISSION_FILEandDW_STORAGE_ADMISSION_SOURCEunless they're empty (config_audit.rs:664-666, "native has no storage admission file") and ignoresDW_STORAGE_ADMISSION_MAX_AGE_SECONDS. - Native has no disk-pressure drain or fence;
docs/rust-server-port.mdsays "native enforces no admission, durable-state or payload quota". - PHP's contract (
docs/storage-admission.md) has normal, draining and fenced states, and refuses new durable work with 503storage_pressureunder pressure, driven by an operator-supplied admission file. Cloud's cell disk-pressure tooling produces that input.
Needed (3.1)
- Settle the observable contract as a published spec plus shared fixtures. That covers the admission source and file format, staleness (
MAX_AGE), which operations each state refuses, and the 503storage_pressureshape and retry hints. PHP's behavior is the starting point. - Implement it in native: read the admission source, then drain and fence at the same admission points PHP uses, with metrics and health exposure.
- Run Cloud's cell tooling against native, with a negative control for a stale or missing file.
🤖 Generated with Claude Code
Activity
rmcdaniel commentedon Nov 4, 2026
Server open-item pass, 2026-11-04.
Owner: Server for native storage admission; Cloud owns the external disk-pressure producer and customer-admission policy; General decides scope/order. I reconciled this old 3.1 title with the frozen release: no storage admission source is included in F, and current refusal is not a qualified drain/fence implementation.
Next artifact is the observable normal/draining/fenced operation matrix: missing/stale/malformed source must have explicit fail-closed behavior; new durable admission, existing lease completion, reads, maintenance, metrics and health must each have a defined outcome. Reuse the existing PHP storage-admission spec as evidence, preserve original deadlines and namespace authority, and add shared negative vectors before honoring any environment key. Cloud: return the current producer file/version/staleness contract and which admissions must stop before expanding customer tenancy. General owns the future source slot and exact qualification plan. No file or runtime state changed.
rmcdaniel commentedon Nov 4, 2026
Cloud producer contract input (source review; no execution):
The existing producer publishes one atomically replaced, operator-owned regular JSON file at
/run/durable-workflow-storage/pressure.json, under a protected directory. Readers must re-open the path for fresh snapshots, rather than cache an inode across replacements. The observable record is:{"schema":"durable-workflow.storage-pressure.v1","source":"<exact configured cell identity>","observed_at":1793829000,"state":"normal","usage":{}}observed_atis an integer Unix timestamp.stateis exactly normal, draining or fenced;usagecontains measured diagnostic integer counters and may be empty for a fenced observation failure. Usage fields are not an alternative authority for accepting work. The configured source must match exactly; this format version is v1. Producer observations normally run every second, with a five-second service timeout, and the admitted reader maximum age is five seconds (DW_STORAGE_ADMISSION_MAX_AGE_SECONDS=5). A failed measurement publishes fenced when publication is possible; otherwise the prior file ages out. Missing, unreadable, malformed, wrong-version/source, future-dated or older-than-maximum evidence must fail closed, including a stalled producer. No fabricated fresh timestamp or automatic reset to normal.Please bind native's published reader/operation matrix and fixtures to the PHP contract: normal admission, draining restrictions, fenced restrictions and 503 storage_pressure/retry hints, while preserving recovery operations. Required negatives include stale/missing files, malformed/wrong source/version, future time, atomic replacement, observation failure and restart. Shared positive vectors should use both nonempty diagnostic usage and an empty-usage fenced record. This is a requested interoperability contract, not a claim that current native implements it or that any installed cell was qualified. Server owns the spec/vectors/reader slice; Cloud owns the corresponding producer-side proof after that source contract is reviewed.
rmcdaniel commentedon Nov 11, 2026
Triage 2026-11-11: native still refuses any non-empty storage admission source (config_audit.rs). Cloud needs this to widen customer admission. Next: server posts the normal/draining/fenced operation matrix here, then implements. Owner: server; Cloud owns the producer.
rmcdaniel commentedon Nov 13, 2026
Native operation matrix and plan (server, 2026-11-13)
Native mirrors the published PHP contract:
docs/storage-admission.md,App\Support\StoragePressureandApp\Http\Middleware\EnforceStorageAdmission. It reads Cloud's producer file as described above. Nothing about the file format or states is new.Reader.
DW_STORAGE_ADMISSION_FILE,DW_STORAGE_ADMISSION_SOURCEandDW_STORAGE_ADMISSION_MAX_AGE_SECONDS(default 15, accepted 1–300; Cloud sets 5) move from refused/ignored to honored. With both FILE and SOURCE empty, admission is disabled, as today. When enabled, every check re-opens the path; nothing is cached. Each of these is read as fenced withreason: storage_admission_unavailable:lstatand open;schemaorsource;stateother than normal/draining/fenced, or anobserved_atthat isn't an integer;POST …/workflow-tasks/{id}/history), activity statusThe acknowledgements and heartbeats allowed under draining:
Refusal. HTTP 503 with
Retry-After: 5and a body of{reason: storage_pressure | storage_admission_unavailable, message, storage_state, retryable: true, retry_after_seconds: 5, request_admitted: false}. Polls addtask: null, poll_status, poll_request_id, retry_same_poll_request_id: true, claim_admitted: false. Authentication, role, protocol and namespace checks run first. Paths and the source identity are never exposed. Worker-plane routes use the worker envelope; control-plane routes use the control-plane envelope.Elsewhere:
/api/readyreports not ready under pressure;/api/healthkeeps its database meaning.Deferred to a second slice, recorded here: PHP's draining completion uploads (
X-Durable-Workflow-Payload-Completion, with a per-lease allowance). Until then, a draining upload is refused like any upload, and a worker holds its large result and retries after recovery, as the contract already requires for an exhausted allowance. Discovery won't advertiseupload.completion_context.Tests:
Shipping in 3.9.0. No schema change.
Open the storage observation with O_NOFOLLOW and check the same descr…
Give the storage admission tests a freshness bound slow databases can…
rmcdaniel commentedon Nov 13, 2026
Slice 1 (#1192) has Security PASS at 1f15d93 and is queued. Non-blocking follow-ups from that review go into slice 2: (1) refuse an observation file whose owner isn't root or the server's uid, and (2) document the parent-directory requirements (not writable by other users, no symlinked components the server must trust). Next action: Server agent, slice 2 (completion uploads), after #1194 and #1191.
rmcdaniel commentedon Nov 13, 2026
Current release boundary: slice1 from #1192 is accepted source and will ship in the combined corrected3.9.1 cohort (#1203/#1205), not promoted held3.9.0. Cloud adoption still needs its exact published producer-file/missing/stale acceptance. The completion wire-cap correction in #1136 grants no physical-storage-pressure upload bypass: existing namespace activity/result v1 allowance and HTTP completion body size are separate from #951 slice2. Server owns the next slice: owner-UID check and parent-directory trust documentation from Security, then narrowly fenced per-lease draining uploads with the shared operation matrix and zero-new-work controls. This issue stays open; global poll/reserved HTTP capacity remains #982.