Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 3 additions & 62 deletions docs/constraints/structural-limits.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,69 +261,10 @@ The full contract is available in the `structural_limits` section of the backend

Waterline surfaces structural-limit failures in the exceptions table with the `structural_limit` failure category. The timeline failure details include the limit kind, current value, and configured ceiling.

## Server request-boundary limits
## Service Mode limits {#server-request-boundary-limits}

When using the standalone server, a separate set of caps is enforced at the HTTP request boundary before a workflow task, signal, update, or query ever reaches the control plane. These limits fail fast with a `422 validation_failed` (or `413 payload_too_large` for whole-body checks) response so clients learn the rejection reason without the server writing any row to the database.

| Limit | Default | Config key | What it caps |
|---|---|---|---|
| Body size | 2 MiB | `server.limits.max_payload_bytes` | Total HTTP request body bytes, except a workflow-task completion on native Server 3.6.0 or later (next row) |
| Workflow-task completion size | 32 MiB (16 × body size) | derived from `server.limits.max_payload_bytes` | Native Server 3.6.0 or later: a request that completes a workflow task with commands. Each command's arguments still meet the per-payload limit |
| Memo size | 256 KiB | `server.limits.max_memo_bytes` | Serialized memo on `POST /workflows` and `POST /schedules` |
| Search-attribute count | 100 | `server.limits.max_search_attributes` | Registered custom search attributes per namespace |
| Search-attribute key length | 128 bytes | `server.limits.max_search_attribute_key_length` | Length of a single SA key on start |
| Search-attribute value size | 2 KiB | `server.limits.max_search_attribute_value_bytes` | Each string value (and each element of an array value) on start |
| Signal / update / query name | 256 bytes | `server.limits.max_operation_name_length` | URL path segment for signal/update/query names |
| `workflow_id` length | 128 chars | controller validator | Workflow ID on `POST /workflows` |
| `workflow_type` / `task_queue` / `business_key` | 255 chars | controller validator | String fields on `POST /workflows` |
| `request_id` | 255 chars | controller validator | Deduplication token on signals/updates/cancel/terminate |
| `reason` | 1,000 chars | controller validator | Reason text on cancel/terminate/archive |

Each limit is individually configurable via `DW_*` environment variables (see `config/dw-contract.php` for the full contract). Setting a value of `0` disables the check for that specific limit, but leaves the others in force.

The currently-configured values are published under `limits` on the `GET /api/cluster/info` response, so clients can discover them at runtime:

```json
{
"limits": {
"max_payload_bytes": 2097152,
"max_memo_bytes": 262144,
"max_search_attributes": 100,
"max_search_attribute_key_length": 128,
"max_search_attribute_value_bytes": 2048,
"max_operation_name_length": 256,
"max_pending_activities": 2000,
"max_pending_children": 2000
}
}
```

Validation errors are returned in the standard control-plane error envelope with `reason: "validation_failed"` and a `validation_errors` map keyed by the offending field (`signal_name`, `update_name`, `query_name`, `search_attributes`, `memo`, and so on). Payload-size rejections use `reason: "payload_too_large"` and the `413` status code.
Server and Cloud HTTP request limits are separate from these embedded Laravel controls. See the [Server request reference](../polyglot/server-config-reference.md#server-request-boundary-limits).

### Workflow-task completion size {#workflow-task-completion-size}

On native Server 3.6.0 and later, a request that completes a workflow task
with commands (`POST /api/worker/workflow-tasks/{id}/complete` and
`POST /api/worker/activity-tasks/{id}/continuation/complete`) may be up to 16
times `max_payload_bytes`: 32 MiB with the default 2 MiB. It follows
`DW_MAX_PAYLOAD_BYTES` and has no setting of its own. Each command's arguments
still meet the per-payload limit inside the completion. PHP Server 2.x and
earlier native releases cap a completion at one payload limit, like any other
request.

A larger completion is refused with `413 payload_too_large` and
`limit_kind: workflow_task_completion_bytes` before anything is recorded.
Replaying the task produces the same commands, so sending the completion again
gets the same refusal. PHP SDK 3.17.0 and later never resend it: the worker
fails the run once with a `fail_workflow` command whose message starts
`Structural limit exceeded: workflow_task_completion_bytes:` and names both
sizes, and the Server records the failure with
`failure_category = structural_limit`. The Python and Rust SDKs will do the
same in their next releases.

To keep a completion within the limit:

- Split a large fan-out into smaller batches across workflow tasks.
- Move work into child workflows, or continue as new.
- Store large arguments with external payload storage, so the completion
carries references instead of the payloads.
Service Mode completion caps and SDK refusal behavior are covered in the [Server reference](../polyglot/server-config-reference.md#workflow-task-completion-size).
23 changes: 23 additions & 0 deletions docs/polyglot/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,7 @@ runtime environment and not in the profile file.
| `dw workflow:signal <workflow-id> <signal-name>` | Send a signal. | `--run-id`, input options, `--json` |
| `dw workflow:query <workflow-id> <query-name>` | Execute a read-only workflow query. | `--run-id`, input options, `--json` |
| `dw workflow:update <workflow-id> <update-name>` | Submit or execute a workflow update. | `--wait=accepted\|completed`, `--run-id`, input options, `--json` |
| `dw workflow:update-outcome <workflow-id> <update-id>` | Read a retained update outcome without submitting it again. | `--wait=accepted\|completed`, `--wait-timeout`, `--json` |
| `dw workflow:cancel [workflow-id]` | Close one workflow or a selected batch as cancelled; no workflow-code cleanup runs. | `--reason`, `--run-id`, `--all-matching`, `--type`, `--status`, `--limit`, `--yes`, `--json` |
| `dw workflow:terminate <workflow-id>` | Force terminate a workflow. | `--reason`, `--run-id`, `--json` |
| `dw workflow:repair <workflow-id>` | Ask the server to repair a stuck or retryable run. | `--json` |
Expand All @@ -262,6 +263,28 @@ dw workflow:cancel --all-matching='WorkflowType = "ImportJob"' --limit=25 --yes
validate canonical request fields against the server-published request
contract. Non-canonical legacy aliases are rejected before the request is sent.

### Read an update outcome {#read-update-outcome}

CLI 2.8 and later can read an accepted update by its `update_id` on a Server
that advertises `capabilities.update_outcome_reads` (native 3.6.0 or later).
Use the ID from the accepted receipt:

```bash
dw workflow:update-outcome order-123 "$UPDATE_ID"
dw workflow:update-outcome order-123 "$UPDATE_ID" --wait=completed --wait-timeout=30
```

The default `--wait=accepted` answers immediately. An accepted update that has
not finished stays pending, including when a completion wait expires. A
completed update can have a null result. `--json` returns the checked receipt,
so use its `update_status` to distinguish these cases. Each invocation makes
one outcome read and never resubmits the update.

`--wait-timeout` requires `--wait=completed` and an integer of at least one
second. A failed update exits 2. An unknown or pruned update ID exits 5, which
does not prove the update was never admitted. A Server without the capability,
including PHP Server 2.x, exits 8 before an outcome request is sent.

## Service Call Inspection

Use `dw service-call:describe <endpoint> <service> <operation> <call-id>`
Expand Down
5 changes: 5 additions & 0 deletions docs/polyglot/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,11 @@ Every command that accepts caller payloads uses the same input shape:
and `--input-encoding=json|raw|base64`. JSON is the default; raw and base64
inputs are passed as one positional workflow argument.

Use `dw workflow:update-outcome` to read an accepted update later without
submitting it again. Pending is distinct from a completed null result. See
[read an update outcome](./cli-reference.md#read-update-outcome) for the Server
capability requirement and wait options.

### Bridge Adapters

```bash
Expand Down
20 changes: 19 additions & 1 deletion docs/polyglot/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,17 @@ Python SDK 2.15 and 2.16 add:
See the [2.16.0 changelog](https://github.com/durable-workflow/sdk-python/blob/2.16.0/CHANGELOG.md)
for the complete list.

Python SDK 2.17 adds `Client.update_outcome` and
`Client.update_outcome_receipt` to read an accepted update later without
submitting it again. These methods require
`capabilities.update_outcome_reads` (native Server 3.6.0 or later).
`update_outcome` waits for completion by default and raises `UpdateStillPending`
if the update is unfinished. A returned `None` is a completed null result.
`update_outcome_receipt` defaults to the accepted stage and preserves the
explicit `update_status`. See the
[Python SDK update-outcome guide](https://python.durable-workflow.com/reference/client/#read-an-update-outcome)
for language-specific examples.

## Quickstart

Here is a complete Python program that defines a workflow with one activity, starts it against a local Durable Workflow server, and waits for the result:
Expand Down Expand Up @@ -452,13 +463,20 @@ end-to-end rollout walkthrough; the CLI mirrors of these methods are
| `await client.export_history(workflow_id, run_id)` | decoded archival history payload | `WorkflowNotFound`, auth/server errors |
| `await client.signal_workflow(workflow_id, signal_name, args=None)` | `None` | `WorkflowNotFound`, `InvalidArgument`, auth/server errors |
| `await client.query_workflow(workflow_id, query_name, args=None)` | decoded query result | `QueryFailed`, `WorkflowNotFound`, auth/server errors |
| `await client.update_workflow(workflow_id, update_name, args=None, ...)` | decoded update result | `UpdateRejected`, `WorkflowNotFound`, auth/server errors |
| `await client.update_workflow(workflow_id, update_name, args=None, ...)` | update receipt (`dict`) | `UpdateRejected`, `UpdateValidationFailed`, `UpdateFailed`, `WorkflowNotFound`, auth/server errors |
| `await client.cancel_workflow(workflow_id, reason=None)` | `None` | `WorkflowNotFound`, auth/server errors |
| `await client.terminate_workflow(workflow_id, reason=None)` | `None` | `WorkflowNotFound`, auth/server errors |
| `await client.repair_workflow(workflow_id)` | `WorkflowCommandResult` | `WorkflowNotFound`, auth/server errors |
| `await client.archive_workflow(workflow_id, reason=None)` | `WorkflowCommandResult` | `WorkflowNotFound`, auth/server errors |
| `await client.get_result(handle, poll_interval=0.5, timeout=30.0)` | decoded workflow output | `WorkflowFailed`, `WorkflowCancelled`, `WorkflowTerminated`, `TimeoutError` |

`update_workflow` defaults to `wait_for="accepted"`. Its receipt can have
`update_status="accepted"` and `result=None`. With `wait_for="completed"`,
`wait_timeout_seconds` bounds the wait. On expiry, the receipt stays accepted
and sets `wait_timed_out=True`. Only `update_status="completed"` identifies a
finished result, including `None`. Keep `update_id` for
[later outcome reads](https://python.durable-workflow.com/reference/client/#read-an-update-outcome).

`start_workflow` accepts `workflow_type`, `task_queue`, optional
`workflow_id`, optional `input`, `duplicate_policy`, `memo`,
`search_attributes`, `business_key`, `execution_timeout_seconds`, and
Expand Down
15 changes: 15 additions & 0 deletions docs/polyglot/rust.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,21 @@ protocol manifests determine compatibility.

See the [4.13.0 changelog](https://github.com/durable-workflow/sdk-rust/blob/4.13.0/CHANGELOG.md).

## Read retained update outcomes {#read-update-outcome}

Rust SDK 4.15 adds `Client::update_outcome` and
`Client::update_outcome_receipt` to read an accepted update by ID without
submitting it again. These methods require `capabilities.update_outcome_reads`
(native Server 3.6.0 or later). `update_outcome` returns
`Error::UpdateStillPending` for an unfinished update. A completed null result
is `Ok(Value::Null)`. Receipt reads preserve the explicit `update_status`.
See the [Client API](https://rust.durable-workflow.com/durable_workflow/struct.Client.html#method.update_outcome)
for the Rust interface.

Rust SDK 4.15.0 still applies the ordinary payload ceiling to the whole
command batch. See the [known planner limit](./server-config-reference.md#rust-completion-planning)
before running a large fan-out without external payload storage.

## Condition wait deadlines and worker upgrades {#condition-wait-deadlines}

Rust SDK 4.11.2 measures an ungrouped `wait_condition` timeout from its first
Expand Down
89 changes: 89 additions & 0 deletions docs/polyglot/server-config-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -342,6 +342,95 @@ default.
| `DW_MAX_NEXUS_OPERATIONS_PER_CALLER` | `200` | Maximum Nexus operations returned per caller from the operations history surface before clients must paginate. | `WORKFLOW_MAX_NEXUS_OPERATIONS_PER_CALLER` |
| `DW_COMPRESSION_ENABLED` | `true` | Enable JSON response compression above the minimum size threshold. | `WORKFLOW_SERVER_COMPRESSION_ENABLED` |

## Server request-boundary limits {#server-request-boundary-limits}

When using the standalone server, a separate set of caps is enforced at the HTTP request boundary before a workflow task, signal, update, or query ever reaches the control plane. These limits fail fast with a `422 validation_failed` (or `413 payload_too_large` for whole-body checks) response so clients learn the rejection reason without the server writing any row to the database.

| Limit | Default | Config key | What it caps |
|---|---|---|---|
| Body size | 2 MiB | `server.limits.max_payload_bytes` | Total HTTP request body bytes, except a workflow-task completion on native Server 3.6.0 or later (next row) |
| Workflow-task completion size | 32 MiB (16 × body size) | derived from `server.limits.max_payload_bytes` | Native Server 3.6.0 or later: a request that completes a workflow task with commands. Each command's arguments still meet the per-payload limit |
| Memo size | 256 KiB | `server.limits.max_memo_bytes` | Serialized memo on `POST /workflows` and `POST /schedules` |
| Search-attribute count | 100 | `server.limits.max_search_attributes` | Registered custom search attributes per namespace |
| Search-attribute key length | 128 bytes | `server.limits.max_search_attribute_key_length` | Length of a single SA key on start |
| Search-attribute value size | 2 KiB | `server.limits.max_search_attribute_value_bytes` | Each string value (and each element of an array value) on start |
| Signal / update / query name | 256 bytes | `server.limits.max_operation_name_length` | URL path segment for signal/update/query names |
| `workflow_id` length | 128 chars | controller validator | Workflow ID on `POST /workflows` |
| `workflow_type` / `task_queue` / `business_key` | 255 chars | controller validator | String fields on `POST /workflows` |
| `request_id` | 255 chars | controller validator | Deduplication token on signals/updates/cancel/terminate |
| `reason` | 1,000 chars | controller validator | Reason text on cancel/terminate/archive |

Use the documented `DW_*` settings for configurable PHP Server 2.x limits.
Controller-validator bounds are fixed. Native Server 3.x supports the settings
listed in its [implementation notes](#native-server-3x). On native 3.8.0,
`DW_MAX_PAYLOAD_BYTES=0` sets a zero-byte ceiling for ordinary requests and
workflow-task completions, rejecting nonempty bodies. Its operation-name
ceiling is fixed at 256 bytes.

The currently-configured values are published under `limits` on the `GET /api/cluster/info` response, so clients can discover them at runtime:

```json
{
"limits": {
"max_payload_bytes": 2097152,
"max_memo_bytes": 262144,
"max_search_attributes": 100,
"max_search_attribute_key_length": 128,
"max_search_attribute_value_bytes": 2048,
"max_operation_name_length": 256,
"max_pending_activities": 2000,
"max_pending_children": 2000
}
}
```

Validation errors are returned in the standard control-plane error envelope with `reason: "validation_failed"` and a `validation_errors` map keyed by the offending field (`signal_name`, `update_name`, `query_name`, `search_attributes`, `memo`, and so on). Payload-size rejections use `reason: "payload_too_large"` and the `413` status code.

### Workflow-task completion size {#workflow-task-completion-size}

On native Server 3.6.0 and later, a request that completes a workflow task
with commands (`POST /api/worker/workflow-tasks/{id}/complete` and
`POST /api/worker/activity-tasks/{id}/continuation/complete`) may be up to 16
times `max_payload_bytes`: 32 MiB with the default 2 MiB. It follows
`DW_MAX_PAYLOAD_BYTES` and has no setting of its own. Each command's arguments
still meet the per-payload limit inside the completion. PHP Server 2.x and
earlier native releases cap a completion at one payload limit, like any other
request.

A larger completion is refused with `413 payload_too_large` and
`limit_kind: workflow_task_completion_bytes` before anything is recorded.
Replaying the task produces the same commands, so sending the completion again
gets the same refusal. When PHP SDK 3.17.0 or later receives this typed refusal, it does not resend
the completion: the worker fails the run once with a `fail_workflow` command whose message starts
`Structural limit exceeded: workflow_task_completion_bytes:` and names both
sizes, and the Server records the failure with
`failure_category = structural_limit`.

Python SDK 2.17 and Rust SDK 4.15 include handling for the typed refusal.
On Server 3.8.0, an oversized completion can fail at the transport before the
worker receives HTTP 413. In the published Python test, `httpx.ReadError` left
the workflow pending instead of failing structurally. Published-Server
over-limit acceptance remains open under the
[Server response fix](https://github.com/durable-workflow/.github/issues/228#issuecomment-6848969678).
Reduce completion sizes until that path is qualified.

To keep a completion within the limit:

- Split a large fan-out into smaller batches across workflow tasks.
- Move work into child workflows, or continue as new.
- Store large arguments with external payload storage, so the completion
carries references instead of the payloads.

#### Rust command planning {#rust-completion-planning}

Rust SDK 4.15.0 still uses the ordinary per-payload limit as the aggregate
command-planning ceiling (2 MiB by default), before the Server's larger
completion allowance. Without namespace external payload storage, the
published 300 × 10 KB fan-out is refused with `external_payload_unavailable`;
the worker exits and the run remains pending. Use smaller command batches
while the [SDK aggregate planner fix](https://github.com/durable-workflow/sdk-rust/issues/254)
is pending. This SDK path does not yet implement the larger Server allowance.

## Docker Bootstrap And Provenance

| Variable | Default | Purpose | Legacy alias |
Expand Down
Loading
Loading