Repository navigation
feat: store and report WarmThroughput on tables and global secondary indexes - #474
LeeroyHannigan wants to merge 1 commit into
Conversation
…indexes Since Terraform AWS provider 6.13.0 the aws_dynamodb_table resource waits for WarmThroughput.Status to read ACTIVE after every table and index create or update and treats a description without the member as not found, so an apply against ExtendDB polled until it timed out (#462). ExtendDB now accepts WarmThroughput on CreateTable, UpdateTable and GlobalSecondaryIndexUpdates (Create and Update), stores what the client sent, and reports the member with the shape, defaults, lifecycle and validation measured on Amazon DynamoDB on 2026-10-23: * Absent while the table or index is CREATING unless the request named it (then echoed as sent with Status UPDATING); 12000 read and 4000 write units on an on-demand table and its indexes, the provisioned units on a provisioned table and index; present with Status on every CreateTable, DescribeTable, UpdateTable and DeleteTable description. * An update reports the previous values with Status UPDATING for the control plane delay, then the new values ACTIVE; a decrease is validated against the reported values. A switch to PAY_PER_REQUEST raises a stored value to the on-demand minimum. A capacity change on a never-set provisioned table moves the reported default with the units. * Refusals carry the service's sentences: below the on-demand minimum, below the provisioned units, below the current value, each naming "table" or "index <name>"; one action per GlobalSecondaryIndexUpdate element; one update per index; on an on-demand table an index Update may only carry OnDemandThroughput or WarmThroughput. * UpdateTable with a table-level WarmThroughput refuses companions in the measured order: Server-Side Encryption, TableClass, DeletionProtection (each with its own sentence), OnDemandThroughput (needs BillingMode), a GlobalSecondaryIndexUpdates Delete, then BillingMode, ProvisionedThroughput, StreamSpecification or vector index updates ("WarmThroughput must be the only operation in the request"). GlobalSecondaryIndexUpdates Update beside it is accepted. A request carrying only WarmThroughput is accepted. SSESpecification is now read on UpdateTableInput so it can be refused rather than dropped. * A restored table starts with the defaults; a GlobalSecondaryIndexOverride carrying WarmThroughput is refused with the service's sentence. Nothing acts on the values. There is no pre-warming and no scaling at the database level, and the stored numbers have no effect on throughput, throttling or storage; they exist so clients that read the member work. The account maximum the service enforces is not enforced here. Storage: one nullable warm_throughput column on tables and indexes on SQLite (schema plus shape convergence), PostgreSQL (001_schema.sql plus shape steps for existing catalogs) and Cassandra (migration V006, catalog version 0.0.5 so extenddb migrate applies it); a document field on MongoDB. Each backend stores one JSON record (the client's sides, the previous sides, and the UPDATING deadline); the wire member is derived in one place, TableDescription::prepare_for_response. On PostgreSQL, UpdateTable builds a new GSI before it answers (#401), which takes longer than the 0.25 s UPDATING window the index row is inserted with. The window of each GSI the request created with WarmThroughput starts again once the build has run, so the response reports it UPDATING as the service does. Tests: unit tests for the record, defaults and every rule; three dual-target wire files (lifecycle and validation, update shapes and billing changes, backup and restore) covering every measured row; a Terraform apply of the configuration from the provider issue plus a table with a GSI on every backend, with Terraform 1.17.0 and provider 6.70.0 pinned and checksum-asserted in CI; a Cassandra test for a catalog that has not run V006. Measured on live DynamoDB: the three wire files pass against the service (12, 13 and 1 tests) and against SQLite, PostgreSQL, MongoDB and Cassandra. Cassandra lib (61) and direct_integration (151) suites pass. Negative controls: with the member removed the Terraform apply fails with the provider's "couldn't find resource (21 retries)" after about three and a half minutes, which is the hang from the issue; with the validation removed the three refusal rows fail; five further built mutants (table member, index member, UPDATING deadline, only-operation rule, swapped defaults) each fail the wire file. Documented in docs/differences-from-dynamodb.md (Capacity and Throttling) and docs/getting-started.md. Closes #462.
3b40060 to
0af920c
Compare
|
Rebased on main at abf8082; conflicts in crates/storage-postgres/src/update_table.rs, crates/storage-sqlite/src/update_table.rs and crates/storage/src/util/mod.rs, resolved by keeping #401's GSI build flow (the new index is inserted CREATING) with this branch's warm_throughput column and bind, and both sides' imports and util exports. The branch's commit also returned six files from #214 to their state before it: it deleted docs/rfcs/0003-backend-acceptance-criteria.md and removed the links to it from CONTRIBUTING.md, README.md, docs/design/04-component-storage.md, docs/rfcs/0002-backend-plugin-policy.md and docs/rfcs/README.md. Each of those files was byte-identical to its version before #214, and neither the commit message nor this PR mentions them, so they now match main and this PR no longer touches them. On PostgreSQL, UpdateTable now builds the new GSI before it answers (#401), which takes longer than the 0.25 s UPDATING window, so test_gsi_create_with_warm_throughput_in_update_table failed with Status ACTIVE in the response. restart_created_gsi_warm_throughput in crates/storage-postgres/src/warm_throughput.rs starts that window again after the build. A DescribeTable during the build reads IndexStatus CREATING with WarmThroughput UPDATING; the response to UpdateTable then starts the window again, so the ACTIVE flip of the index and the UPDATING window of its WarmThroughput are observed in that order. tests/python/test_warm_throughput*.py (26) and tests/test_gsi_online_backfill.py pass on PostgreSQL. |
Closes #462.
Summary
Since Terraform AWS provider 6.13.0 (September 2025),
aws_dynamodb_tablewaits forWarmThroughput.Statusto readACTIVEafter every table and index create or update, and treats aDescribeTablewithout the member as not found. ExtendDB omitted the member, so everyterraform applyagainst it polled untilcouldn't find resource (21 retries). The same hang against DynamoDB Local is hashicorp/terraform-provider-aws#44344, closed by the provider team as an emulator-side fix; the current provider is 6.70.0 and the only workaround was pinning 6.12.0.ExtendDB now stores and reports
WarmThroughputon tables and global secondary indexes on all four backends, with the shape, defaults, lifecycle and validation measured on Amazon DynamoDB.Nothing acts on the values. There is no pre-warming and no scaling at the database level, and the stored numbers have no effect on throughput, throttling or storage. They exist so clients that read the member work. This is stated in
docs/differences-from-dynamodb.md(Capacity and Throttling), indocs/getting-started.md, and at the storage field.What the service does, and what ExtendDB now does
Every rule below was measured on live DynamoDB (us-east-1, 2026-10-23) before it was written; the raw observations are in the dual-target wire tests' docstrings as row ids.
{12000, 4000, ACTIVE}on an on-demand table and its GSIs, the provisioned units on a provisioned table and its GSIs; LSIs carry noneStatus: UPDATINGwhile CREATING; filled once ACTIVE (a read-only{20000}becomes{20000, 4000})UPDATING, then the new valuesACTIVE; a decrease is validated against the reported valuesValidationExceptionwith the service's sentence, namingtableorindex <name>WarmThroughput must be the only operation in the request); a GSI Update beside it is acceptedExtendDB differs in timing only: it returns to
ACTIVEaftercontrol_plane_delay_secondswhere the service takes minutes, and it does not enforce the account maximum (LimitExceededExceptionon the service).Storage
One nullable
warm_throughputcolumn ontablesandindexes: SQLite (schema plus shape convergence on existing catalogs), PostgreSQL (001_schema.sqlplus two shape steps applied at serve time), Cassandra (migration V006; catalog version 0.0.5 soextenddb migrateapplies it and the backup gate stops refusing). MongoDB uses a document field. Each backend stores one JSON record; the wire member is derived in one place,TableDescription::prepare_for_response, on every response path.UpdateTableInputnow readsSSESpecificationso the member can be refused instead of silently dropped.Tests
tests/python/: lifecycle and validation (12), update shapes and billing changes (13), backup and restore (1). They pass against the service and against SQLite, PostgreSQL, MongoDB and Cassandra.tests/terraform/warm-throughput/: the configuration from the provider issue plus a table with a GSI, applied with Terraform 1.17.0 and provider 6.70.0 (both pinned, the Terraform zip checksum-asserted against the published sums) on every backend in CI viadevtools/run-tests --terraform; asserts a clean apply, a second plan with no changes, and destroys.run_catalog_migrationsapplies it in place).Negative controls: with the member removed the Terraform apply fails with the provider's
couldn't find resource (21 retries)after about three and a half minutes, which is the hang from the issue; with the validation removed the refusal rows fail; five further built mutants (table member, index member, UPDATING deadline, only-operation rule, swapped defaults) each fail the wire file.Also run on the final tree: Cassandra lib (61) and
direct_integration(151, including the V006 test) suites; MongoDB and Cassandra comprehensive runs (69 each); fmt, clippy 1.97-D warnings, MSRV 1.88; no dependency change.Out of scope, filed separately
Measured on
mainwhile working this issue, not touched here: the service's per-member "must be the only operation" rules for TableClass, DeletionProtection and Server-Side Encryption when WarmThroughput is absent (ExtendDB accepts those pairs); a GSI reportingIndexStatus: ACTIVEwhile its table is CREATING;TableStatus: ACTIVEin the response to a capacity change where the service says UPDATING;UpdateTablewith an emptyWarmThroughput: {}or an Update naming a missing index, where the service answers its ownInternalFailure.