From 0af920cd86a0eaf697765212ecce61636fc5f20e Mon Sep 17 00:00:00 2001 From: Lee Hannigan Date: Fri, 23 Oct 2026 21:14:17 +0000 Subject: [PATCH] feat: store and report WarmThroughput on tables and global secondary indexes Since Terraform AWS provider 6.13.0 the aws_dynamodb_table resource waits for WarmThroughput.Status to read ACTIVE after every table and index create or update and treats a description without the member as not found, so an apply against ExtendDB polled until it timed out (#462). ExtendDB now accepts WarmThroughput on CreateTable, UpdateTable and GlobalSecondaryIndexUpdates (Create and Update), stores what the client sent, and reports the member with the shape, defaults, lifecycle and validation measured on Amazon DynamoDB on 2026-10-23: * Absent while the table or index is CREATING unless the request named it (then echoed as sent with Status UPDATING); 12000 read and 4000 write units on an on-demand table and its indexes, the provisioned units on a provisioned table and index; present with Status on every CreateTable, DescribeTable, UpdateTable and DeleteTable description. * An update reports the previous values with Status UPDATING for the control plane delay, then the new values ACTIVE; a decrease is validated against the reported values. A switch to PAY_PER_REQUEST raises a stored value to the on-demand minimum. A capacity change on a never-set provisioned table moves the reported default with the units. * Refusals carry the service's sentences: below the on-demand minimum, below the provisioned units, below the current value, each naming "table" or "index "; one action per GlobalSecondaryIndexUpdate element; one update per index; on an on-demand table an index Update may only carry OnDemandThroughput or WarmThroughput. * UpdateTable with a table-level WarmThroughput refuses companions in the measured order: Server-Side Encryption, TableClass, DeletionProtection (each with its own sentence), OnDemandThroughput (needs BillingMode), a GlobalSecondaryIndexUpdates Delete, then BillingMode, ProvisionedThroughput, StreamSpecification or vector index updates ("WarmThroughput must be the only operation in the request"). GlobalSecondaryIndexUpdates Update beside it is accepted. A request carrying only WarmThroughput is accepted. SSESpecification is now read on UpdateTableInput so it can be refused rather than dropped. * A restored table starts with the defaults; a GlobalSecondaryIndexOverride carrying WarmThroughput is refused with the service's sentence. Nothing acts on the values. There is no pre-warming and no scaling at the database level, and the stored numbers have no effect on throughput, throttling or storage; they exist so clients that read the member work. The account maximum the service enforces is not enforced here. Storage: one nullable warm_throughput column on tables and indexes on SQLite (schema plus shape convergence), PostgreSQL (001_schema.sql plus shape steps for existing catalogs) and Cassandra (migration V006, catalog version 0.0.5 so extenddb migrate applies it); a document field on MongoDB. Each backend stores one JSON record (the client's sides, the previous sides, and the UPDATING deadline); the wire member is derived in one place, TableDescription::prepare_for_response. On PostgreSQL, UpdateTable builds a new GSI before it answers (#401), which takes longer than the 0.25 s UPDATING window the index row is inserted with. The window of each GSI the request created with WarmThroughput starts again once the build has run, so the response reports it UPDATING as the service does. Tests: unit tests for the record, defaults and every rule; three dual-target wire files (lifecycle and validation, update shapes and billing changes, backup and restore) covering every measured row; a Terraform apply of the configuration from the provider issue plus a table with a GSI on every backend, with Terraform 1.17.0 and provider 6.70.0 pinned and checksum-asserted in CI; a Cassandra test for a catalog that has not run V006. Measured on live DynamoDB: the three wire files pass against the service (12, 13 and 1 tests) and against SQLite, PostgreSQL, MongoDB and Cassandra. Cassandra lib (61) and direct_integration (151) suites pass. Negative controls: with the member removed the Terraform apply fails with the provider's "couldn't find resource (21 retries)" after about three and a half minutes, which is the hang from the issue; with the validation removed the three refusal rows fail; five further built mutants (table member, index member, UPDATING deadline, only-operation rule, swapped defaults) each fail the wire file. Documented in docs/differences-from-dynamodb.md (Capacity and Throttling) and docs/getting-started.md. Closes #462. --- .github/workflows/integration-cassandra.yml | 4 +- .github/workflows/integration-mongodb.yml | 4 +- .github/workflows/integration.yml | 10 +- crates/core/src/types/mod.rs | 5 + crates/core/src/types/table.rs | 82 ++- crates/core/src/types/warm_throughput.rs | 345 ++++++++++++ .../core/src/types/warm_throughput/tests.rs | 338 ++++++++++++ crates/core/src/validation/mod.rs | 21 + crates/core/src/validation/warm_throughput.rs | 381 ++++++++++++++ .../src/validation/warm_throughput/tests.rs | 493 ++++++++++++++++++ crates/engine/src/backup/overrides.rs | 18 +- crates/engine/src/backup/tests.rs | 9 +- crates/engine/src/gsi_updates.rs | 333 ++++++++++++ crates/engine/src/import_export.rs | 1 + crates/engine/src/lib.rs | 2 + crates/engine/src/update_table.rs | 166 +++--- crates/engine/src/warm_throughput.rs | 452 ++++++++++++++++ .../catalog/V006__warm_throughput.cql | 18 + crates/storage-cassandra/src/backup_engine.rs | 1 + crates/storage-cassandra/src/create_table.rs | 64 ++- crates/storage-cassandra/src/lib.rs | 3 +- crates/storage-cassandra/src/migrations.rs | 6 +- crates/storage-cassandra/src/table_helpers.rs | 10 +- crates/storage-cassandra/src/update_table.rs | 91 +++- .../storage-cassandra/src/warm_throughput.rs | 107 ++++ crates/storage-cassandra/tests/common/mod.rs | 6 + .../tests/direct/backup_engine.rs | 67 ++- .../tests/direct/cassandra_engine.rs | 1 + .../storage-cassandra/tests/direct/streams.rs | 1 + .../tests/direct/table_engine.rs | 1 + .../tests/direct/warm_throughput_catalog.rs | 158 ++++++ .../tests/direct_integration.rs | 2 + .../tests/ttl_integration.rs | 4 + crates/storage-mongodb/src/backup_engine.rs | 2 + .../storage-mongodb/src/backup_live_tests.rs | 4 + crates/storage-mongodb/src/lib.rs | 1 + crates/storage-mongodb/src/table_engine.rs | 31 ++ crates/storage-mongodb/src/warm_throughput.rs | 169 ++++++ .../migrations/001_schema.sql | 8 + crates/storage-postgres/src/backup_engine.rs | 1 + crates/storage-postgres/src/create_table.rs | 42 +- crates/storage-postgres/src/delete_table.rs | 5 +- crates/storage-postgres/src/lib.rs | 1 + crates/storage-postgres/src/migrations.rs | 28 + crates/storage-postgres/src/table_helpers.rs | 18 +- crates/storage-postgres/src/update_table.rs | 26 +- .../storage-postgres/src/warm_throughput.rs | 168 ++++++ .../storage-postgres/tests/key_collation.rs | 1 + crates/storage-postgres/tests/twi_conflict.rs | 1 + .../tests/vector_control_plane.rs | 4 + crates/storage-sqlite/src/backup.rs | 10 +- crates/storage-sqlite/src/bootstrapper.rs | 7 +- crates/storage-sqlite/src/create_table.rs | 41 +- crates/storage-sqlite/src/lib.rs | 1 + crates/storage-sqlite/src/schema.rs | 39 ++ crates/storage-sqlite/src/table_helpers.rs | 13 +- crates/storage-sqlite/src/update_table.rs | 21 +- crates/storage-sqlite/src/warm_throughput.rs | 308 +++++++++++ crates/storage/src/backup_definition.rs | 1 + crates/storage/src/util/mod.rs | 5 + crates/storage/src/util/warm_throughput.rs | 148 ++++++ devtools/run-tests | 28 +- docs/differences-from-dynamodb.md | 1 + docs/getting-started.md | 4 + docs/manuals/01-architecture-guide.md | 2 +- docs/manuals/07-upgrade-manual.md | 15 +- tests/python/test_warm_throughput.py | 446 ++++++++++++++++ tests/python/test_warm_throughput_backup.py | 121 +++++ tests/python/test_warm_throughput_updates.py | 420 +++++++++++++++ tests/python/warm_throughput_helpers.py | 124 +++++ tests/terraform/install-terraform.sh | 47 ++ tests/terraform/warm-throughput/main.tf | 95 ++++ tests/terraform/warm-throughput/run.sh | 89 ++++ 73 files changed, 5501 insertions(+), 199 deletions(-) create mode 100644 crates/core/src/types/warm_throughput.rs create mode 100644 crates/core/src/types/warm_throughput/tests.rs create mode 100644 crates/core/src/validation/warm_throughput.rs create mode 100644 crates/core/src/validation/warm_throughput/tests.rs create mode 100644 crates/engine/src/gsi_updates.rs create mode 100644 crates/engine/src/warm_throughput.rs create mode 100644 crates/storage-cassandra/migrations/catalog/V006__warm_throughput.cql create mode 100644 crates/storage-cassandra/src/warm_throughput.rs create mode 100644 crates/storage-cassandra/tests/direct/warm_throughput_catalog.rs create mode 100644 crates/storage-mongodb/src/warm_throughput.rs create mode 100644 crates/storage-postgres/src/warm_throughput.rs create mode 100644 crates/storage-sqlite/src/warm_throughput.rs create mode 100644 crates/storage/src/util/warm_throughput.rs create mode 100644 tests/python/test_warm_throughput.py create mode 100644 tests/python/test_warm_throughput_backup.py create mode 100644 tests/python/test_warm_throughput_updates.py create mode 100644 tests/python/warm_throughput_helpers.py create mode 100755 tests/terraform/install-terraform.sh create mode 100644 tests/terraform/warm-throughput/main.tf create mode 100755 tests/terraform/warm-throughput/run.sh diff --git a/.github/workflows/integration-cassandra.yml b/.github/workflows/integration-cassandra.yml index 08ad150a8..439088d5e 100644 --- a/.github/workflows/integration-cassandra.yml +++ b/.github/workflows/integration-cassandra.yml @@ -145,8 +145,10 @@ jobs: run: pip install -r requirements.txt - name: Build release (cassandra backend) run: cargo build --release --no-default-features --features cassandra + - name: Install Terraform (pinned version, checksum asserted) + run: echo "TERRAFORM=$(tests/terraform/install-terraform.sh "$RUNNER_TEMP/terraform")" >> "$GITHUB_ENV" - name: Run Cassandra pytest suite - run: devtools/run-cassandra-tests -- --pytest --comprehensive --parallel + run: devtools/run-cassandra-tests -- --pytest --comprehensive --terraform --parallel cassandra-rust: runs-on: extenddb_ubuntu-2404_4-core diff --git a/.github/workflows/integration-mongodb.yml b/.github/workflows/integration-mongodb.yml index 3df405a75..8eb7ac6fc 100644 --- a/.github/workflows/integration-mongodb.yml +++ b/.github/workflows/integration-mongodb.yml @@ -48,8 +48,10 @@ jobs: run: pip install -r requirements.txt - name: Build release (mongodb backend) run: cargo build --release --no-default-features --features mongodb-test-hooks + - name: Install Terraform (pinned version, checksum asserted) + run: echo "TERRAFORM=$(tests/terraform/install-terraform.sh "$RUNNER_TEMP/terraform")" >> "$GITHUB_ENV" - name: Run MongoDB pytest suite - run: devtools/run-mongodb-tests -- --pytest --comprehensive --parallel + run: devtools/run-mongodb-tests -- --pytest --comprehensive --terraform --parallel mongodb-rust: runs-on: extenddb_ubuntu-2404_4-core diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml index 8a4818f75..58fa92517 100644 --- a/.github/workflows/integration.yml +++ b/.github/workflows/integration.yml @@ -89,13 +89,16 @@ jobs: - name: Install Python dependencies run: pip install -r requirements.txt + - name: Install Terraform (pinned version, checksum asserted) + run: echo "TERRAFORM=$(tests/terraform/install-terraform.sh "$RUNNER_TEMP/terraform")" >> "$GITHUB_ENV" + - name: Run integration tests env: EXTENDDB_TEST_ENDPOINT: https://127.0.0.1:18443 EXTENDDB_ADMIN_USER: admin EXTENDDB_ADMIN_PASSWORD: ${{ steps.init.outputs.admin_password }} EXTENDDB_TEST_PG_ADMIN_CONNECTION_STRING: postgresql://postgres:devpass@127.0.0.1:5432 - run: devtools/run-tests --extenddb --pytest --comprehensive --parallel --filter "not import_export" + run: devtools/run-tests --extenddb --pytest --comprehensive --terraform --parallel --filter "not import_export" # run-tests restarts the server daemonized to apply the import/export # config, and a daemonized server logs to syslog, not the job log. Without @@ -150,12 +153,15 @@ jobs: - name: Install Python dependencies run: pip install -r requirements.txt + - name: Install Terraform (pinned version, checksum asserted) + run: echo "TERRAFORM=$(tests/terraform/install-terraform.sh "$RUNNER_TEMP/terraform")" >> "$GITHUB_ENV" + - name: Run integration tests env: EXTENDDB_TEST_ENDPOINT: https://127.0.0.1:18443 EXTENDDB_ADMIN_USER: admin EXTENDDB_ADMIN_PASSWORD: ${{ steps.init.outputs.admin_password }} - run: devtools/run-tests --extenddb --pytest --comprehensive --parallel --filter "not import_export" + run: devtools/run-tests --extenddb --pytest --comprehensive --terraform --parallel --filter "not import_export" # Same rationale as the postgres job: the daemonized server logs to # syslog, so this dump is the only server-side evidence on failure. diff --git a/crates/core/src/types/mod.rs b/crates/core/src/types/mod.rs index e21f75110..c8c0544ea 100755 --- a/crates/core/src/types/mod.rs +++ b/crates/core/src/types/mod.rs @@ -11,6 +11,7 @@ mod query; mod stream; mod table; mod transaction; +mod warm_throughput; pub use attribute_value::AttributeValue; pub use backup::{ @@ -76,3 +77,7 @@ pub use transaction::{ TransactGetItem, TransactGetItemsInput, TransactGetItemsOutput, TransactPut, TransactUpdate, TransactWriteItem, TransactWriteItemsInput, TransactWriteItemsOutput, }; +pub use warm_throughput::{ + ON_DEMAND_WARM_READ_UNITS, ON_DEMAND_WARM_WRITE_UNITS, WarmThroughput, WarmThroughputDefaults, + WarmThroughputDescription, WarmThroughputRecord, WarmThroughputStatus, epoch_seconds_now, +}; diff --git a/crates/core/src/types/table.rs b/crates/core/src/types/table.rs index fdebcab30..2982af087 100755 --- a/crates/core/src/types/table.rs +++ b/crates/core/src/types/table.rs @@ -303,6 +303,14 @@ pub struct GsiDescription { pub item_count: i64, #[serde(rename = "IndexArn")] pub index_arn: String, + /// Derived by `TableDescription::prepare_for_response` from + /// `warm_throughput_record`; backends leave this `None`. + #[serde(rename = "WarmThroughput", skip_serializing_if = "Option::is_none")] + pub warm_throughput: Option, + /// What the backend stored for this index, if a client ever set it. Not a + /// wire member; see [`super::WarmThroughputRecord`]. Nothing acts on it. + #[serde(skip)] + pub warm_throughput_record: Option, } /// Local secondary index description returned in responses. @@ -336,6 +344,12 @@ pub struct GsiInput { skip_serializing_if = "Option::is_none" )] pub provisioned_throughput: Option, + #[serde( + rename = "WarmThroughput", + default, + skip_serializing_if = "Option::is_none" + )] + pub warm_throughput: Option, } /// Local secondary index definition for `CreateTable` requests. @@ -822,6 +836,46 @@ impl TableDescription { } else if let Some(summary) = self.restore_summary.as_mut() { summary.restore_in_progress = true; } + self.populate_warm_throughput(super::epoch_seconds_now()); + } + + /// Derive the `WarmThroughput` member of the table and of each global + /// secondary index from the stored records (see + /// [`super::WarmThroughputRecord::describe_for`] for the measured rules). + /// Local secondary indexes carry none. Reached through + /// [`Self::prepare_for_response`]; `now` is epoch seconds. + pub fn populate_warm_throughput(&mut self, now: f64) { + let pay_per_request = self + .billing_mode_summary + .as_ref() + .is_some_and(|s| s.billing_mode == BillingMode::PayPerRequest); + let table_defaults = super::WarmThroughputDefaults::for_billing( + pay_per_request, + self.provisioned_throughput.read_capacity_units, + self.provisioned_throughput.write_capacity_units, + ); + self.warm_throughput = super::WarmThroughputRecord::describe_for( + self.warm_throughput_record.as_ref(), + self.table_status == TableStatus::Creating, + table_defaults, + now, + ); + for gsi in self.global_secondary_indexes.iter_mut().flatten() { + let (rcu, wcu) = gsi.provisioned_throughput.as_ref().map_or((0, 0), |pt| { + (pt.read_capacity_units, pt.write_capacity_units) + }); + let defaults = super::WarmThroughputDefaults::for_billing(pay_per_request, rcu, wcu); + // An index created with its table is CREATING while the table is; + // one added by UpdateTable is CREATING while it backfills. + let creating = + self.table_status == TableStatus::Creating || gsi.index_status == "CREATING"; + gsi.warm_throughput = super::WarmThroughputRecord::describe_for( + gsi.warm_throughput_record.as_ref(), + creating, + defaults, + now, + ); + } } /// Populate `TableThroughputModeSummary` as an exact mirror of @@ -901,6 +955,14 @@ pub struct TableDescription { pub on_demand_throughput: Option, #[serde(rename = "RestoreSummary", skip_serializing_if = "Option::is_none")] pub restore_summary: Option, + /// Derived by [`Self::prepare_for_response`] from `warm_throughput_record`, + /// the billing mode and the provisioned units; backends leave this `None`. + #[serde(rename = "WarmThroughput", skip_serializing_if = "Option::is_none")] + pub warm_throughput: Option, + /// What the backend stored for the table, if a client ever set it. Not a + /// wire member; see [`super::WarmThroughputRecord`]. Nothing acts on it. + #[serde(skip)] + pub warm_throughput_record: Option, } /// Details of a restore reported on the restored table's description. @@ -963,6 +1025,8 @@ pub struct CreateTableInput { pub table_class: Option, #[serde(rename = "OnDemandThroughput")] pub on_demand_throughput: Option, + #[serde(rename = "WarmThroughput", default)] + pub warm_throughput: Option, } /// `CreateTable` response body. @@ -1063,6 +1127,8 @@ pub struct CreateGsiAction { skip_serializing_if = "Option::is_none" )] pub provisioned_throughput: Option, + #[serde(rename = "WarmThroughput", default)] + pub warm_throughput: Option, } /// Delete an existing GSI from a table. @@ -1072,14 +1138,17 @@ pub struct DeleteGsiAction { pub index_name: String, } -/// Update provisioned throughput on an existing GSI. +/// Update an existing GSI. /// -/// Recognized by the deserializer but not yet implemented — the engine -/// returns a clear "not yet supported" error. +/// `WarmThroughput` is stored and reported (see [`super::WarmThroughputRecord`]). +/// A provisioned-throughput update is recognized by the deserializer but not +/// yet implemented: the engine returns a clear "not yet supported" error. #[derive(Debug, Clone, PartialEq, Deserialize)] pub struct UpdateGsiAction { #[serde(rename = "IndexName")] pub index_name: String, + #[serde(rename = "WarmThroughput", default)] + pub warm_throughput: Option, } /// `UpdateTable` request body. @@ -1110,6 +1179,13 @@ pub struct UpdateTableInput { /// unaffected. #[serde(rename = "VectorIndexUpdates", default)] pub vector_index_updates: Option>, + #[serde(rename = "WarmThroughput", default)] + pub warm_throughput: Option, + /// Accepted so that a request carrying it is recognised: `UpdateTable` + /// does not act on it (as before), but a table-level `WarmThroughput` + /// beside it is refused with the service's sentence. + #[serde(rename = "SSESpecification", default)] + pub sse_specification: Option, } /// `UpdateTable` response body. diff --git a/crates/core/src/types/warm_throughput.rs b/crates/core/src/types/warm_throughput.rs new file mode 100644 index 000000000..fadbef9c4 --- /dev/null +++ b/crates/core/src/types/warm_throughput.rs @@ -0,0 +1,345 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! `WarmThroughput` on tables and global secondary indexes. +//! +//! Measured against Amazon DynamoDB (us-east-1, 2026-10-23). The service +//! reports `WarmThroughput` as `{ReadUnitsPerSecond, WriteUnitsPerSecond, +//! Status}` on an ACTIVE table and on each of its ACTIVE global secondary +//! indexes; local secondary indexes carry none. A client that sends none gets +//! 12000 read and 4000 write units on an on-demand table and its indexes, and +//! the provisioned units on a provisioned table and each provisioned index. +//! +//! ExtendDB stores the values a client sends and reports them with the same +//! shape, defaults and validation. Nothing acts on them: there is no +//! pre-warming and no scaling at the database level, and the stored numbers +//! have no effect on throughput, throttling or storage. They exist so clients +//! that read the member (the Terraform AWS provider since 6.13.0 waits for +//! `Status: ACTIVE` after every table and index change) work. + +use serde::{Deserialize, Serialize}; + +/// Read units an on-demand table or index reports when the client sent none. +pub const ON_DEMAND_WARM_READ_UNITS: i64 = 12_000; +/// Write units an on-demand table or index reports when the client sent none. +pub const ON_DEMAND_WARM_WRITE_UNITS: i64 = 4_000; + +/// `WarmThroughput` as a client sends it on `CreateTable`, `UpdateTable`, a +/// `GlobalSecondaryIndexes` element and a `GlobalSecondaryIndexUpdates` +/// `Create` or `Update` action. Either member may be omitted. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)] +pub struct WarmThroughput { + #[serde( + rename = "ReadUnitsPerSecond", + default, + skip_serializing_if = "Option::is_none" + )] + pub read_units_per_second: Option, + #[serde( + rename = "WriteUnitsPerSecond", + default, + skip_serializing_if = "Option::is_none" + )] + pub write_units_per_second: Option, +} + +/// The `Status` member of a reported `WarmThroughput`. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +pub enum WarmThroughputStatus { + #[serde(rename = "ACTIVE")] + Active, + #[serde(rename = "UPDATING")] + Updating, +} + +/// `WarmThroughput` as the service reports it on `TableDescription` and +/// `GlobalSecondaryIndexDescription`. +/// +/// Both unit members are optional because the `CreateTable` response echoes a +/// partial request as sent: a request of `{ReadUnitsPerSecond: 20000}` was +/// answered with `{ReadUnitsPerSecond: 20000, Status: UPDATING}` and no write +/// member, and `DescribeTable` once ACTIVE then read `{20000, 4000, ACTIVE}`. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +pub struct WarmThroughputDescription { + #[serde( + rename = "ReadUnitsPerSecond", + default, + skip_serializing_if = "Option::is_none" + )] + pub read_units_per_second: Option, + #[serde( + rename = "WriteUnitsPerSecond", + default, + skip_serializing_if = "Option::is_none" + )] + pub write_units_per_second: Option, + #[serde(rename = "Status")] + pub status: WarmThroughputStatus, +} + +/// What a backend persists for a table or index whose client set +/// `WarmThroughput`: the units the client sent (a side the client never sent +/// stays `None` and is reported from the defaults), the units the resource +/// reported before the last `UpdateTable` (reported again while `UPDATING`, +/// as the service does), and the epoch second until which the record reports +/// `UPDATING`. +/// +/// Stored as one JSON value in the catalog. Nothing reads these numbers except +/// the description builder: they do not pre-warm anything, scale anything or +/// change how a request is throttled or stored. +#[derive(Debug, Clone, Copy, PartialEq, Serialize, Deserialize, Default)] +pub struct WarmThroughputRecord { + #[serde(rename = "ReadUnitsPerSecond", default)] + pub read_units_per_second: Option, + #[serde(rename = "WriteUnitsPerSecond", default)] + pub write_units_per_second: Option, + /// The side the resource reported when the last `UpdateTable` was + /// validated (`None` means the default of the moment). Reported, and + /// decreases are validated against it, while the record is `UPDATING`. + #[serde(rename = "PreviousReadUnitsPerSecond", default)] + pub previous_read_units_per_second: Option, + #[serde(rename = "PreviousWriteUnitsPerSecond", default)] + pub previous_write_units_per_second: Option, + /// Epoch seconds. `Status` is `UPDATING` while `now < updating_until`. + #[serde(rename = "UpdatingUntil", default)] + pub updating_until: f64, +} + +/// The defaults a table or index reports when the client never set warm +/// throughput, and the values that fill a side the client left out. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct WarmThroughputDefaults { + pub read_units_per_second: i64, + pub write_units_per_second: i64, + /// On an on-demand resource a stored side below the default is reported + /// as the default: switching a provisioned table to PAY_PER_REQUEST + /// resets a below-minimum warm throughput to 12000/4000 (measured, rows + /// T7c and T7d). + pub pay_per_request: bool, +} + +impl WarmThroughputDefaults { + /// On-demand: 12000 read, 4000 write. Provisioned: the resource's own + /// `ReadCapacityUnits` and `WriteCapacityUnits`. Measured on a + /// PAY_PER_REQUEST table and on a 1/1 provisioned table with a 1/1 index. + #[must_use] + pub fn for_billing( + pay_per_request: bool, + read_capacity_units: i64, + write_capacity_units: i64, + ) -> Self { + if pay_per_request { + Self { + read_units_per_second: ON_DEMAND_WARM_READ_UNITS, + write_units_per_second: ON_DEMAND_WARM_WRITE_UNITS, + pay_per_request, + } + } else { + Self { + read_units_per_second: read_capacity_units, + write_units_per_second: write_capacity_units, + pay_per_request, + } + } + } + + /// The pair `(read, write)` of stored sides resolved against these + /// defaults: an unset side is the default; on an on-demand resource a + /// side below the default is the default. + #[must_use] + pub fn resolve(self, read: Option, write: Option) -> (i64, i64) { + let side = |stored: Option, default: i64| match stored { + Some(v) if self.pay_per_request => v.max(default), + Some(v) => v, + None => default, + }; + ( + side(read, self.read_units_per_second), + side(write, self.write_units_per_second), + ) + } + + /// The description an untouched resource reports once it is ACTIVE. + #[must_use] + pub fn describe(self) -> WarmThroughputDescription { + WarmThroughputDescription { + read_units_per_second: Some(self.read_units_per_second), + write_units_per_second: Some(self.write_units_per_second), + status: WarmThroughputStatus::Active, + } + } +} + +impl WarmThroughputRecord { + /// The record a `CreateTable` (or a GSI `Create`) writes: the request as + /// sent, reporting `UPDATING` until `updating_until`. + /// + /// The previous pair is the request itself: a created resource that is + /// past CREATING inside its window (an index added by `UpdateTable` is + /// `ACTIVE` here at once) reports the requested values with `UPDATING` + /// and then the same values with `ACTIVE`, as the service does for an + /// index being created (rows S8a, V5). A side the request leaves out + /// reports the default. + #[must_use] + pub fn from_request(spec: &WarmThroughput, updating_until: f64) -> Self { + Self { + read_units_per_second: spec.read_units_per_second, + write_units_per_second: spec.write_units_per_second, + previous_read_units_per_second: spec.read_units_per_second, + previous_write_units_per_second: spec.write_units_per_second, + updating_until, + } + } + + /// The record an `UpdateTable` writes over `existing` at `now`: each side + /// the request names replaces the stored side, a side it leaves out keeps + /// the stored value (or stays unset and keeps reporting the default), the + /// `UPDATING` deadline restarts, and the previous pair becomes what + /// `existing` reported at `now` (its own previous pair while it was still + /// `UPDATING`, its stored sides otherwise), so a request inside the window + /// keeps reporting the values the window started from. + /// + /// A `spec` with neither side (`WarmThroughput: {}`) changes no value and + /// restarts the window. The service answers `InternalFailure` with an empty + /// message to that request on `UpdateTable` (row S9b); ExtendDB accepts it + /// and does not imitate a service fault. + #[must_use] + pub fn apply_update( + existing: Option, + spec: &WarmThroughput, + updating_until: f64, + now: f64, + ) -> Self { + let existing = existing.unwrap_or_default(); + let (previous_read, previous_write) = if existing.is_updating_at(now) { + ( + existing.previous_read_units_per_second, + existing.previous_write_units_per_second, + ) + } else { + ( + existing.read_units_per_second, + existing.write_units_per_second, + ) + }; + Self { + read_units_per_second: spec + .read_units_per_second + .or(existing.read_units_per_second), + write_units_per_second: spec + .write_units_per_second + .or(existing.write_units_per_second), + previous_read_units_per_second: previous_read, + previous_write_units_per_second: previous_write, + updating_until, + } + } + + /// The effective units once the window has passed: the stored side where + /// set, the default otherwise (clamped on an on-demand resource). + #[must_use] + pub fn effective(&self, defaults: WarmThroughputDefaults) -> (i64, i64) { + defaults.resolve(self.read_units_per_second, self.write_units_per_second) + } + + /// The pair the resource reports at `now`: the previous pair while + /// `UPDATING` (measured: the service keeps reporting the values the + /// window started from, rows U2, S2a, S2c), the effective pair afterwards. + /// Decreases are validated against this pair (row U3: inside the window a + /// request carrying the previous values is accepted; row S2b: on a + /// settled table the same request is a decrease). + #[must_use] + pub fn reported(&self, defaults: WarmThroughputDefaults, now: f64) -> (i64, i64) { + if self.is_updating_at(now) { + defaults.resolve( + self.previous_read_units_per_second, + self.previous_write_units_per_second, + ) + } else { + self.effective(defaults) + } + } + + /// Whether the record still reports `UPDATING` at `now` (epoch seconds). + #[must_use] + pub fn is_updating_at(&self, now: f64) -> bool { + now < self.updating_until + } + + /// The description once the owning table or index is past CREATING: both + /// sides filled, the previous pair with `UPDATING` until the deadline, + /// then the new pair with `ACTIVE`. + #[must_use] + pub fn describe( + &self, + defaults: WarmThroughputDefaults, + now: f64, + ) -> WarmThroughputDescription { + let (read, write) = self.reported(defaults, now); + WarmThroughputDescription { + read_units_per_second: Some(read), + write_units_per_second: Some(write), + status: if self.is_updating_at(now) { + WarmThroughputStatus::Updating + } else { + WarmThroughputStatus::Active + }, + } + } + + /// The `CreateTable` response shape while the resource is still CREATING: + /// the request echoed as sent (a side the client left out is absent) with + /// `Status: UPDATING`. Measured on a `{ReadUnitsPerSecond: 20000}` request. + #[must_use] + pub fn echo_while_creating(&self) -> WarmThroughputDescription { + WarmThroughputDescription { + read_units_per_second: self.read_units_per_second, + write_units_per_second: self.write_units_per_second, + status: WarmThroughputStatus::Updating, + } + } + + /// The member a response carries for a resource in `status`, or `None` + /// when the service omits it. + /// + /// - No record and the resource is CREATING: absent (measured on the + /// `CreateTable` response and the first `DescribeTable`). A record with + /// neither side (a `WarmThroughput: {}` request) is the same: the + /// service answers no member while CREATING (row T1). + /// - No record otherwise: the defaults, `ACTIVE`. + /// - A record while CREATING: the request echoed, `UPDATING`. + /// - A record otherwise: both sides filled, the previous pair with + /// `UPDATING` until the deadline, then the new pair with `ACTIVE`. + #[must_use] + pub fn describe_for( + record: Option<&Self>, + creating: bool, + defaults: WarmThroughputDefaults, + now: f64, + ) -> Option { + match (record, creating) { + (None, true) => None, + (Some(record), true) + if record.read_units_per_second.is_none() + && record.write_units_per_second.is_none() => + { + None + } + (None, false) => Some(defaults.describe()), + (Some(record), true) => Some(record.echo_while_creating()), + (Some(record), false) => Some(record.describe(defaults, now)), + } + } +} + +/// Seconds since the Unix epoch, for the `UPDATING` deadline and its check. +#[must_use] +#[allow(clippy::cast_precision_loss)] +pub fn epoch_seconds_now() -> f64 { + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map_or(0.0, |d| d.as_secs_f64()) +} + +#[cfg(test)] +mod tests; diff --git a/crates/core/src/types/warm_throughput/tests.rs b/crates/core/src/types/warm_throughput/tests.rs new file mode 100644 index 000000000..62cf714aa --- /dev/null +++ b/crates/core/src/types/warm_throughput/tests.rs @@ -0,0 +1,338 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! Tests for the `WarmThroughput` shapes and the stored record. + +use super::*; + +fn on_demand() -> WarmThroughputDefaults { + WarmThroughputDefaults::for_billing(true, 0, 0) +} + +#[test] +fn on_demand_defaults_are_the_measured_12000_and_4000() { + assert_eq!( + on_demand(), + WarmThroughputDefaults { + read_units_per_second: 12_000, + write_units_per_second: 4_000, + pay_per_request: true, + } + ); +} + +#[test] +fn provisioned_defaults_are_the_provisioned_units() { + assert_eq!( + WarmThroughputDefaults::for_billing(false, 1, 1), + WarmThroughputDefaults { + read_units_per_second: 1, + write_units_per_second: 1, + pay_per_request: false, + } + ); +} + +/// Rows T7a to T7d: a read-only record `{100}` on a provisioned 5/5 table +/// reads `{100, 5}`, follows a capacity change to `{100, 10}`, and resets to +/// `{12000, 4000}` once the table is on-demand (a side below the on-demand +/// default is reported as the default), after which `12000/4000` is not a +/// decrease. +#[test] +fn an_on_demand_resource_reports_at_least_the_on_demand_defaults() { + let record = WarmThroughputRecord::from_request( + &WarmThroughput { + read_units_per_second: Some(100), + write_units_per_second: None, + }, + 0.0, + ); + assert_eq!( + record.effective(WarmThroughputDefaults::for_billing(false, 5, 5)), + (100, 5) + ); + assert_eq!( + record.effective(WarmThroughputDefaults::for_billing(false, 10, 10)), + (100, 10) + ); + assert_eq!(record.effective(on_demand()), (12_000, 4_000)); + // A stored side above the default is kept on an on-demand resource. + let high = WarmThroughputRecord::from_request( + &WarmThroughput { + read_units_per_second: Some(20_000), + write_units_per_second: Some(50), + }, + 0.0, + ); + assert_eq!(high.effective(on_demand()), (20_000, 4_000)); +} + +/// Row T1: `WarmThroughput: {}` on CreateTable reports no member while the +/// table is CREATING, and the defaults once ACTIVE. +#[test] +fn an_empty_request_reports_no_member_while_creating() { + let record = WarmThroughputRecord::from_request(&WarmThroughput::default(), 10.0); + assert_eq!( + WarmThroughputRecord::describe_for(Some(&record), true, on_demand(), 1.0), + None + ); + assert_eq!( + WarmThroughputRecord::describe_for(Some(&record), false, on_demand(), 11.0), + Some(on_demand().describe()) + ); +} + +/// Rows U2, S2a, S2c, U3 and S2b: while UPDATING the record reports the pair +/// the window started from, and that pair is what a decrease is measured +/// against; once settled the new pair is reported and compared. +#[test] +fn the_previous_pair_is_reported_and_compared_while_updating() { + // Never set (defaults 12000/4000), raised to 24000/8000 at t=0, window to 10. + let raised = WarmThroughputRecord::apply_update( + None, + &WarmThroughput { + read_units_per_second: Some(24_000), + write_units_per_second: Some(8_000), + }, + 10.0, + 0.0, + ); + assert_eq!(raised.reported(on_demand(), 5.0), (12_000, 4_000)); + assert_eq!( + raised.describe(on_demand(), 5.0), + WarmThroughputDescription { + read_units_per_second: Some(12_000), + write_units_per_second: Some(4_000), + status: WarmThroughputStatus::Updating, + } + ); + assert_eq!(raised.reported(on_demand(), 10.0), (24_000, 8_000)); + // U3: inside the window a request carrying the previous values validates + // against 12000/4000 and, applied, keeps reporting 12000/4000. + let inside = WarmThroughputRecord::apply_update( + Some(raised), + &WarmThroughput { + read_units_per_second: Some(12_000), + write_units_per_second: Some(4_000), + }, + 15.0, + 5.0, + ); + assert_eq!(inside.reported(on_demand(), 6.0), (12_000, 4_000)); + assert_eq!(inside.reported(on_demand(), 15.0), (12_000, 4_000)); + // S2c: the same values re-sent on a settled table report themselves UPDATING. + let same = WarmThroughputRecord::apply_update( + Some(raised), + &WarmThroughput { + read_units_per_second: Some(24_000), + write_units_per_second: Some(8_000), + }, + 20.0, + 11.0, + ); + assert_eq!(same.reported(on_demand(), 12.0), (24_000, 8_000)); + assert_eq!( + same.describe(on_demand(), 12.0).status, + WarmThroughputStatus::Updating + ); +} + +/// The service omits the member on a CREATING table that was created +/// without one, and reports the defaults ACTIVE once the table is. +#[test] +fn no_record_is_absent_while_creating_and_defaults_once_active() { + assert_eq!( + WarmThroughputRecord::describe_for(None, true, on_demand(), 0.0), + None + ); + let json = serde_json::to_value( + WarmThroughputRecord::describe_for(None, false, on_demand(), 0.0).expect("present"), + ) + .expect("serialises"); + assert_eq!( + json, + serde_json::json!({ + "ReadUnitsPerSecond": 12000, + "WriteUnitsPerSecond": 4000, + "Status": "ACTIVE" + }) + ); +} + +/// Measured: `{ReadUnitsPerSecond: 20000}` on CreateTable was echoed as +/// `{ReadUnitsPerSecond: 20000, Status: UPDATING}` with no write member, +/// and read back as `{20000, 4000, ACTIVE}` once the table was ACTIVE. +#[test] +fn a_partial_request_is_echoed_while_creating_and_filled_once_active() { + let spec = WarmThroughput { + read_units_per_second: Some(20_000), + write_units_per_second: None, + }; + let record = WarmThroughputRecord::from_request(&spec, 10.0); + let creating = serde_json::to_value( + WarmThroughputRecord::describe_for(Some(&record), true, on_demand(), 1.0).expect("present"), + ) + .expect("serialises"); + assert_eq!( + creating, + serde_json::json!({ "ReadUnitsPerSecond": 20000, "Status": "UPDATING" }) + ); + let active = WarmThroughputRecord::describe_for(Some(&record), false, on_demand(), 11.0) + .expect("present"); + assert_eq!( + active, + WarmThroughputDescription { + read_units_per_second: Some(20_000), + write_units_per_second: Some(4_000), + status: WarmThroughputStatus::Active, + } + ); +} + +#[test] +fn the_status_is_updating_until_the_deadline_then_active() { + let record = WarmThroughputRecord::from_request(&WarmThroughput::default(), 100.0); + assert_eq!( + record.describe(on_demand(), 99.9).status, + WarmThroughputStatus::Updating + ); + assert_eq!( + record.describe(on_demand(), 100.0).status, + WarmThroughputStatus::Active + ); +} + +/// An update names one side: that side changes, the other keeps its stored +/// value, and the deadline restarts. +#[test] +fn an_update_replaces_only_the_sides_it_names() { + let existing = WarmThroughputRecord { + read_units_per_second: Some(100), + write_units_per_second: Some(50), + previous_read_units_per_second: None, + previous_write_units_per_second: None, + updating_until: 5.0, + }; + let updated = WarmThroughputRecord::apply_update( + Some(existing), + &WarmThroughput { + read_units_per_second: Some(200), + write_units_per_second: None, + }, + 9.0, + 6.0, + ); + assert_eq!( + updated, + WarmThroughputRecord { + read_units_per_second: Some(200), + write_units_per_second: Some(50), + previous_read_units_per_second: Some(100), + previous_write_units_per_second: Some(50), + updating_until: 9.0, + } + ); + // No stored record: the unnamed side stays unset and reports the default. + let fresh = WarmThroughputRecord::apply_update( + None, + &WarmThroughput { + read_units_per_second: None, + write_units_per_second: Some(8_000), + }, + 9.0, + 6.0, + ); + assert_eq!(fresh.effective(on_demand()), (12_000, 8_000)); +} + +/// The persisted shape round-trips, and a record written without a +/// deadline (or a side) reads back. +#[test] +fn the_record_round_trips_through_json() { + let record = WarmThroughputRecord { + read_units_per_second: Some(1), + write_units_per_second: None, + previous_read_units_per_second: Some(3), + previous_write_units_per_second: None, + updating_until: 1.5, + }; + let text = serde_json::to_string(&record).expect("serialises"); + assert_eq!( + serde_json::from_str::(&text).expect("parses"), + record + ); + let sparse: WarmThroughputRecord = + serde_json::from_str(r#"{"WriteUnitsPerSecond": 7}"#).expect("parses"); + assert_eq!(sparse.write_units_per_second, Some(7)); + assert_eq!(sparse.updating_until, 0.0); +} + +#[test] +fn the_request_shape_accepts_either_side_alone() { + let read_only: WarmThroughput = + serde_json::from_str(r#"{"ReadUnitsPerSecond": 20000}"#).expect("parses"); + assert_eq!(read_only.read_units_per_second, Some(20_000)); + assert_eq!(read_only.write_units_per_second, None); + let empty: WarmThroughput = serde_json::from_str("{}").expect("parses"); + assert_eq!(empty, WarmThroughput::default()); +} + +/// Rows S8a and V5: an index created by `UpdateTable` is past CREATING at +/// once here, and inside its window reports what was requested with +/// `UPDATING`, then the same values `ACTIVE`; a side left out reports the +/// default. +#[test] +fn a_created_resource_reports_the_request_while_updating() { + let record = WarmThroughputRecord::from_request( + &WarmThroughput { + read_units_per_second: Some(30_000), + write_units_per_second: Some(8_000), + }, + 10.0, + ); + assert_eq!( + WarmThroughputRecord::describe_for(Some(&record), false, on_demand(), 5.0), + Some(WarmThroughputDescription { + read_units_per_second: Some(30_000), + write_units_per_second: Some(8_000), + status: WarmThroughputStatus::Updating, + }) + ); + assert_eq!( + WarmThroughputRecord::describe_for(Some(&record), false, on_demand(), 10.0), + Some(WarmThroughputDescription { + read_units_per_second: Some(30_000), + write_units_per_second: Some(8_000), + status: WarmThroughputStatus::Active, + }) + ); + let partial = WarmThroughputRecord::from_request( + &WarmThroughput { + read_units_per_second: Some(20_000), + write_units_per_second: None, + }, + 10.0, + ); + assert_eq!(partial.reported(on_demand(), 5.0), (20_000, 4_000)); +} + +/// Row S9b: the service refuses `UpdateTable` with `WarmThroughput: {}` with +/// an empty `InternalFailure`; ExtendDB accepts it, keeps both values and +/// restarts the window. +#[test] +fn an_empty_update_keeps_the_values_and_restarts_the_window() { + let settled = WarmThroughputRecord::apply_update( + None, + &WarmThroughput { + read_units_per_second: Some(24_000), + write_units_per_second: Some(8_000), + }, + 1.0, + 0.0, + ); + let empty = + WarmThroughputRecord::apply_update(Some(settled), &WarmThroughput::default(), 12.0, 2.0); + assert_eq!(empty.effective(on_demand()), (24_000, 8_000)); + assert_eq!(empty.reported(on_demand(), 3.0), (24_000, 8_000)); + assert!(empty.is_updating_at(3.0)); +} diff --git a/crates/core/src/validation/mod.rs b/crates/core/src/validation/mod.rs index 3decb4666..f6a382975 100755 --- a/crates/core/src/validation/mod.rs +++ b/crates/core/src/validation/mod.rs @@ -2,11 +2,21 @@ // SPDX-License-Identifier: Apache-2.0 pub mod number; pub mod vector_item; +pub mod warm_throughput; pub use vector_item::{ MAX_HASH_KEY_SIZE, MAX_INLINE_FILTER_SIZE, validate_vector_write, validate_vector_write_changed, vector_components, vector_norm, }; +pub use warm_throughput::{ + DELETE_GSI_WITH_WARM_THROUGHPUT, DELETION_PROTECTION_MUST_BE_ONLY_OPERATION, + INDEX_CHANGE_WITH_STREAM_CHANGE, ON_DEMAND_THROUGHPUT_NEEDS_BILLING_MODE, + SSE_MUST_BE_ONLY_OPERATION, TABLE_CLASS_MUST_BE_ONLY_OPERATION, + WARM_THROUGHPUT_MUST_BE_ONLY_OPERATION, WarmThroughputSubject, + validate_create_table_warm_throughput, validate_created_gsi_warm_throughput, + validate_warm_throughput_is_only_operation, validate_warm_throughput_on_create, + validate_warm_throughput_on_update, +}; use crate::error::{DynamoDbError, ErrorMessageKey, error_message}; use crate::limits::LimitsConfig; @@ -126,6 +136,7 @@ pub fn validate_create_table( ))?; validate_vector_indexes(input)?; validate_stream_specification(input)?; + validate_create_table_warm_throughput(input)?; Ok(()) } @@ -2289,6 +2300,7 @@ mod tests { deletion_protection_enabled: None, table_class: None, on_demand_throughput: None, + warm_throughput: None, } } @@ -2532,6 +2544,7 @@ mod tests { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }]); assert!(validate_create_table(&input, &limits).is_ok()); } @@ -2564,6 +2577,7 @@ mod tests { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }]); assert!(validate_create_table(&input, &limits).is_err()); } @@ -2592,6 +2606,7 @@ mod tests { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }]); assert!(validate_create_table(&input, &limits).is_err()); } @@ -2626,6 +2641,7 @@ mod tests { read_capacity_units: 5, write_capacity_units: 5, }), + warm_throughput: None, }]); let err = validate_create_table(&input, &limits).unwrap_err(); assert!( @@ -2654,6 +2670,7 @@ mod tests { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }]); assert!(validate_create_table(&input, &limits).is_ok()); } @@ -2699,6 +2716,7 @@ mod tests { write_capacity_units: write, } }), + warm_throughput: None, }]); input } @@ -2748,6 +2766,7 @@ mod tests { key_schema: vec![make_ks("gsi_pk", KeyType::Hash)], projection: include(0..gsi_end), provisioned_throughput: None, + warm_throughput: None, }]); input.local_secondary_indexes = Some(vec![crate::types::LsiInput { index_name: "my-lsi".to_owned(), @@ -3575,6 +3594,7 @@ mod tests { non_key_attributes: non_key, }, provisioned_throughput: None, + warm_throughput: None, }; input.global_secondary_indexes = Some(vec![gsi(None)]); @@ -3614,6 +3634,7 @@ mod tests { non_key_attributes: non_key, }, provisioned_throughput: None, + warm_throughput: None, }]); input }; diff --git a/crates/core/src/validation/warm_throughput.rs b/crates/core/src/validation/warm_throughput.rs new file mode 100644 index 000000000..75502195e --- /dev/null +++ b/crates/core/src/validation/warm_throughput.rs @@ -0,0 +1,381 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! The measured `WarmThroughput` request rules (Amazon DynamoDB, us-east-1, +//! 2026-10-23). Every refusal is a `ValidationException` whose text is the +//! service's own. +//! +//! - On-demand, `CreateTable`: a side below the on-demand default (12000 read, +//! 4000 write) is refused with the "initial throughput for OnDemand" +//! sentence. Read is checked before write (a 0/0 request reports the read +//! message). +//! - Provisioned, `CreateTable`: a side below the resource's provisioned units +//! is refused with the "lower than ReadCapacityUnits of +//! ProvisionedThroughput" sentence (both member forms measured; a value +//! equal to the units is accepted). +//! - `UpdateTable`, either billing mode: a side below the CURRENT warm +//! throughput is refused with the "decreasing WarmThroughput is not +//! supported" sentence. The same values as current are accepted. +//! - Every sentence names its subject: `for table` on the table, `for index +//! ` on a global secondary index (measured on create and update, +//! both members). +//! - `UpdateTable`: a table-level `WarmThroughput` is an operation that must +//! stand alone, checked before the value rules. Measured per companion: +//! `SSESpecification`, `TableClass` and `DeletionProtectionEnabled` each +//! answer their own member's "must be the only operation" sentence (those +//! members have the same rule on their own); `OnDemandThroughput`, with or +//! without `BillingMode`, answers its own "can only be combined with +//! BillingMode" sentence; a `GlobalSecondaryIndexUpdates` `Delete` answers +//! its own sentence; `StreamSpecification`, `BillingMode` and +//! `ProvisionedThroughput` answer the WarmThroughput sentence; +//! a `GlobalSecondaryIndexUpdates` `Update` is accepted beside it (table and +//! index both go `UPDATING`). See [`validate_warm_throughput_is_only_operation`]. + +use crate::error::DynamoDbError; +use crate::types::{ + BillingMode, CreateGsiAction, CreateTableInput, GsiInput, ON_DEMAND_WARM_READ_UNITS, + ON_DEMAND_WARM_WRITE_UNITS, ProvisionedThroughput, UpdateTableInput, WarmThroughput, +}; + +const PREFIX: &str = "One or more parameter values were invalid: Requested "; + +const ON_DEMAND_MINIMUM_SUFFIX: &str = " is lower than initial throughput for OnDemand. See: \ + https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/\ + on-demand-capacity-mode.html#on-demand-capacity-mode-initial"; + +const DECREASE_SUFFIX: &str = " is lower than current WarmThroughput, decreasing \ + WarmThroughput is not supported"; + +/// The measured refusal for a table-level `WarmThroughput` sent beside a +/// `StreamSpecification`, `BillingMode` or `ProvisionedThroughput` change. +pub const WARM_THROUGHPUT_MUST_BE_ONLY_OPERATION: &str = "One or more parameter values were \ + invalid: WarmThroughput must be the only operation in the request"; + +/// The measured refusal for `TableClass` beside another operation. +pub const TABLE_CLASS_MUST_BE_ONLY_OPERATION: &str = "One or more parameter values were \ + invalid: TableClass modification must be the only operation in the request"; + +/// The measured refusal for `DeletionProtectionEnabled` beside another +/// operation. +pub const DELETION_PROTECTION_MUST_BE_ONLY_OPERATION: &str = "One or more parameter values \ + were invalid: DeletionProtection modification must be the only operation in the request"; + +/// The measured refusal for `OnDemandThroughput` beside a table +/// `WarmThroughput`, with or without `BillingMode` (rows T4a, T4d). +pub const ON_DEMAND_THROUGHPUT_NEEDS_BILLING_MODE: &str = "One or more parameter values were \ + invalid: OnDemandThroughput can only be combined with BillingMode in an UpdateTable request"; + +/// The measured refusal for `SSESpecification` beside another operation +/// (rows S1a to S1c). +pub const SSE_MUST_BE_ONLY_OPERATION: &str = "One or more parameter values were invalid: \ + Server-Side Encryption modification must be the only operation in the request"; + +/// The measured refusal for a `GlobalSecondaryIndexUpdates` `Delete` beside a +/// table `WarmThroughput` (row S9a; the index stays). +pub const DELETE_GSI_WITH_WARM_THROUGHPUT: &str = "One or more parameter values were invalid: \ + Delete global secondary index cannot be specified when updating WarmThroughput"; + +/// The measured refusal for a `GlobalSecondaryIndexUpdates` `Delete` beside a +/// table `WarmThroughput` and a `StreamSpecification` (row V3a). +pub const INDEX_CHANGE_WITH_STREAM_CHANGE: &str = "One or more parameter values were invalid: \ + You cannot create or delete index while changing stream status"; + +/// The resource a `WarmThroughput` sentence is about: the service writes +/// `for table` on the table and `for index ` on a global secondary +/// index (measured on create and update, both members). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum WarmThroughputSubject<'a> { + Table, + Index(&'a str), +} + +impl std::fmt::Display for WarmThroughputSubject<'_> { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::Table => f.write_str("table"), + Self::Index(name) => write!(f, "index {name}"), + } + } +} + +/// The on-demand minimum refusal for `member` (`ReadUnitsPerSecond` or +/// `WriteUnitsPerSecond`) on `subject`. Measured byte for byte on both +/// members, on a table and on an index. +#[must_use] +pub fn warm_throughput_below_on_demand_minimum( + member: &str, + subject: WarmThroughputSubject<'_>, +) -> String { + format!("{PREFIX}{member} for WarmThroughput for {subject}{ON_DEMAND_MINIMUM_SUFFIX}") +} + +/// The provisioned-units refusal for `member` and its provisioned +/// counterpart (`ReadCapacityUnits` or `WriteCapacityUnits`) on `subject`. +/// Measured on both members on a table and on the read member on an index. +#[must_use] +pub fn warm_throughput_below_provisioned( + member: &str, + capacity_member: &str, + subject: WarmThroughputSubject<'_>, +) -> String { + format!( + "{PREFIX}{member} for WarmThroughput for {subject} is lower than {capacity_member} of \ + ProvisionedThroughput" + ) +} + +/// The decrease refusal for `member` on `subject`. Measured on both members +/// on a table (either billing mode) and on an index. +#[must_use] +pub fn warm_throughput_decrease(member: &str, subject: WarmThroughputSubject<'_>) -> String { + format!("{PREFIX}{member} for WarmThroughput for {subject}{DECREASE_SUFFIX}") +} + +/// The `CreateTable` rule for one resource (the table or one index) against +/// the billing mode it is created under and, when provisioned, its units. +/// +/// # Errors +/// +/// `ValidationException` with the measured sentence for the first side that +/// is too low, read before write. +pub fn validate_warm_throughput_on_create( + spec: &WarmThroughput, + subject: WarmThroughputSubject<'_>, + billing_mode: BillingMode, + provisioned: Option<&ProvisionedThroughput>, +) -> Result<(), DynamoDbError> { + match billing_mode { + BillingMode::PayPerRequest => { + if spec + .read_units_per_second + .is_some_and(|r| r < ON_DEMAND_WARM_READ_UNITS) + { + return Err(DynamoDbError::ValidationException( + warm_throughput_below_on_demand_minimum("ReadUnitsPerSecond", subject), + )); + } + if spec + .write_units_per_second + .is_some_and(|w| w < ON_DEMAND_WARM_WRITE_UNITS) + { + return Err(DynamoDbError::ValidationException( + warm_throughput_below_on_demand_minimum("WriteUnitsPerSecond", subject), + )); + } + } + BillingMode::Provisioned => { + // A provisioned resource without units is refused by the throughput + // rules before this one runs; nothing to compare against here. + let Some(units) = provisioned else { + return Ok(()); + }; + if spec + .read_units_per_second + .is_some_and(|r| r < units.read_capacity_units) + { + return Err(DynamoDbError::ValidationException( + warm_throughput_below_provisioned( + "ReadUnitsPerSecond", + "ReadCapacityUnits", + subject, + ), + )); + } + if spec + .write_units_per_second + .is_some_and(|w| w < units.write_capacity_units) + { + return Err(DynamoDbError::ValidationException( + warm_throughput_below_provisioned( + "WriteUnitsPerSecond", + "WriteCapacityUnits", + subject, + ), + )); + } + } + } + Ok(()) +} + +/// The `UpdateTable` rule for one resource against its current effective warm +/// throughput: no side may go below the current value. Equal is accepted. +/// +/// # Errors +/// +/// `ValidationException` with the measured decrease sentence, read before write. +pub fn validate_warm_throughput_on_update( + spec: &WarmThroughput, + subject: WarmThroughputSubject<'_>, + current: (i64, i64), +) -> Result<(), DynamoDbError> { + if spec.read_units_per_second.is_some_and(|r| r < current.0) { + return Err(DynamoDbError::ValidationException( + warm_throughput_decrease("ReadUnitsPerSecond", subject), + )); + } + if spec.write_units_per_second.is_some_and(|w| w < current.1) { + return Err(DynamoDbError::ValidationException( + warm_throughput_decrease("WriteUnitsPerSecond", subject), + )); + } + Ok(()) +} + +/// The `UpdateTable` rules for a request carrying a table-level +/// `WarmThroughput`, checked before the value rules. Measured per companion +/// on an on-demand table with one GSI, all values otherwise valid +/// (2026-10-23; row labels are those of the live runs): +/// +/// | beside `WarmThroughput` | refusal | rows | +/// |-------------------------------------------|----------------------------------|------| +/// | `SSESpecification` | Server-Side Encryption must be the only operation | S1a, S1b, S1c | +/// | `TableClass` | TableClass must be the only operation | V3b, V4, S10b | +/// | `DeletionProtectionEnabled` | DeletionProtection must be the only operation | S1d, T4c, T5d | +/// | `OnDemandThroughput`, with or without `BillingMode` | OnDemandThroughput can only be combined with BillingMode | T4a, T4d | +/// | `GlobalSecondaryIndexUpdates` with a `Delete` | Delete global secondary index cannot be specified when updating WarmThroughput | S9a | +/// | `GlobalSecondaryIndexUpdates` with a `Delete` and a `StreamSpecification` | You cannot create or delete index while changing stream status | V3a | +/// | `StreamSpecification` | WarmThroughput must be the only operation | T5b | +/// | `ProvisionedThroughput`, no `BillingMode` | WarmThroughput must be the only operation | V2 | +/// | `BillingMode` and `ProvisionedThroughput` | WarmThroughput must be the only operation | UT2 | +/// | `GlobalSecondaryIndexUpdates` with an `Update` | accepted; table and index go UPDATING | | +/// +/// Order, as measured: `SSESpecification`, then `TableClass`, then +/// `DeletionProtectionEnabled`, then `OnDemandThroughput`, then the index +/// `Delete`, then the WarmThroughput rule. SSE outranks TableClass, +/// DeletionProtection and Streams (S1a, S1b, S1c); TableClass outranks +/// DeletionProtection and Streams, and outranks a `Delete` (V3b); the triple +/// TableClass, DeletionProtection and Streams answers the TableClass sentence +/// (V4); DeletionProtection outranks Streams (S1d); TableClass and +/// DeletionProtection outrank `OnDemandThroughput` (T4b, T4c). The position of +/// the index `Delete` relative to `SSESpecification`, `DeletionProtectionEnabled` +/// and `OnDemandThroughput` was not measured; it sits after them, where the +/// measured pairs allow. A missing table with a companion answers the +/// companion's sentence (S10b TableClass, T5b Streams, T5d DeletionProtection) +/// because this function runs before the table is read; a bad value alone, or +/// an index `Update` alone, reaches the table lookup and answers +/// `ResourceNotFoundException` (S10, T5a, T5c). +/// +/// TableClass, DeletionProtection and SSE have the same "only operation" rule +/// on their own, without WarmThroughput (measured), but ExtendDB does not apply +/// it elsewhere yet; here the companion's own sentence is used only when +/// WarmThroughput is present, so a request without WarmThroughput is never +/// refused by this function. `VectorIndexUpdates` is an ExtendDB extension and +/// is grouped with the WarmThroughput sentence, unmeasured. `AttributeDefinitions` +/// is not an operation. +/// +/// # Errors +/// +/// `ValidationException` with the measured sentence for the companion. +pub fn validate_warm_throughput_is_only_operation( + input: &UpdateTableInput, +) -> Result<(), DynamoDbError> { + if input.warm_throughput.is_none() { + return Ok(()); + } + let refuse = |sentence: &str| Err(DynamoDbError::ValidationException(sentence.to_owned())); + if input.sse_specification.is_some() { + return refuse(SSE_MUST_BE_ONLY_OPERATION); + } + if input.table_class.is_some() { + return refuse(TABLE_CLASS_MUST_BE_ONLY_OPERATION); + } + if input.deletion_protection_enabled.is_some() { + return refuse(DELETION_PROTECTION_MUST_BE_ONLY_OPERATION); + } + if input.on_demand_throughput.is_some() { + return refuse(ON_DEMAND_THROUGHPUT_NEEDS_BILLING_MODE); + } + if input + .global_secondary_index_updates + .iter() + .flatten() + .any(|u| u.delete.is_some()) + { + return refuse(if input.stream_specification.is_some() { + INDEX_CHANGE_WITH_STREAM_CHANGE + } else { + DELETE_GSI_WITH_WARM_THROUGHPUT + }); + } + let warm_rule = input.billing_mode.is_some() + || input.table_throughput_mode.is_some() + || input.provisioned_throughput.is_some() + || input.stream_specification.is_some() + || input + .vector_index_updates + .as_ref() + .is_some_and(|u| !u.is_empty()); + if warm_rule { + return refuse(WARM_THROUGHPUT_MUST_BE_ONLY_OPERATION); + } + Ok(()) +} + +/// `validate_warm_throughput_on_create` for the table and each of its global +/// secondary indexes in a `CreateTable` request. +/// +/// # Errors +/// +/// The first refusal, table first, then indexes in request order. +pub fn validate_create_table_warm_throughput( + input: &CreateTableInput, +) -> Result<(), DynamoDbError> { + let billing_mode = input.billing_mode.unwrap_or(BillingMode::Provisioned); + if let Some(spec) = &input.warm_throughput { + validate_warm_throughput_on_create( + spec, + WarmThroughputSubject::Table, + billing_mode, + input.provisioned_throughput.as_ref(), + )?; + } + for gsi in input.global_secondary_indexes.iter().flatten() { + validate_gsi_warm_throughput_on_create(gsi, billing_mode)?; + } + Ok(()) +} + +/// The create rule for one `GlobalSecondaryIndexes` element. +/// +/// # Errors +/// +/// See [`validate_warm_throughput_on_create`]. +pub fn validate_gsi_warm_throughput_on_create( + gsi: &GsiInput, + billing_mode: BillingMode, +) -> Result<(), DynamoDbError> { + match &gsi.warm_throughput { + Some(spec) => validate_warm_throughput_on_create( + spec, + WarmThroughputSubject::Index(&gsi.index_name), + billing_mode, + gsi.provisioned_throughput.as_ref(), + ), + None => Ok(()), + } +} + +/// The create rule for a `GlobalSecondaryIndexUpdates` `Create` action, under +/// the table's billing mode. +/// +/// # Errors +/// +/// See [`validate_warm_throughput_on_create`]. +pub fn validate_created_gsi_warm_throughput( + create: &CreateGsiAction, + billing_mode: BillingMode, +) -> Result<(), DynamoDbError> { + match &create.warm_throughput { + Some(spec) => validate_warm_throughput_on_create( + spec, + WarmThroughputSubject::Index(&create.index_name), + billing_mode, + create.provisioned_throughput.as_ref(), + ), + None => Ok(()), + } +} + +#[cfg(test)] +mod tests; diff --git a/crates/core/src/validation/warm_throughput/tests.rs b/crates/core/src/validation/warm_throughput/tests.rs new file mode 100644 index 000000000..e2df21df0 --- /dev/null +++ b/crates/core/src/validation/warm_throughput/tests.rs @@ -0,0 +1,493 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! Tests for the measured `WarmThroughput` request rules. Each test names the +//! rows of the live run it reproduces. + +use super::*; + +fn spec(read: Option, write: Option) -> WarmThroughput { + WarmThroughput { + read_units_per_second: read, + write_units_per_second: write, + } +} + +fn message(result: Result<(), DynamoDbError>) -> String { + match result.expect_err("refused") { + DynamoDbError::ValidationException(m) => m, + other => panic!("{other:?}"), + } +} + +const TABLE: WarmThroughputSubject<'static> = WarmThroughputSubject::Table; +const GSI: WarmThroughputSubject<'static> = WarmThroughputSubject::Index("gsi"); + +const ON_DEMAND_READ_TABLE: &str = "One or more parameter values were invalid: Requested \ + ReadUnitsPerSecond for WarmThroughput for table is lower than initial throughput for \ + OnDemand. See: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/\ + on-demand-capacity-mode.html#on-demand-capacity-mode-initial"; + +fn five_five() -> ProvisionedThroughput { + ProvisionedThroughput { + read_capacity_units: 5, + write_capacity_units: 5, + } +} + +/// Measured rows C2, C3 and C7: the whole strings. +#[test] +fn on_demand_create_below_the_minimum_reports_the_measured_sentence() { + let read = message(validate_warm_throughput_on_create( + &spec(Some(11_999), Some(4_000)), + TABLE, + BillingMode::PayPerRequest, + None, + )); + assert_eq!(read, ON_DEMAND_READ_TABLE); + let write = message(validate_warm_throughput_on_create( + &spec(Some(12_000), Some(3_999)), + TABLE, + BillingMode::PayPerRequest, + None, + )); + assert_eq!( + write, + ON_DEMAND_READ_TABLE.replace("ReadUnitsPerSecond", "WriteUnitsPerSecond") + ); + // 0/0 reports the read side: read is checked first. + let both = message(validate_warm_throughput_on_create( + &spec(Some(0), Some(0)), + TABLE, + BillingMode::PayPerRequest, + None, + )); + assert_eq!(both, read); +} + +/// Rows C1 and C4: the minimum itself and a partial request are accepted. +#[test] +fn on_demand_create_at_or_above_the_minimum_is_accepted() { + validate_warm_throughput_on_create( + &spec(Some(12_000), Some(4_000)), + TABLE, + BillingMode::PayPerRequest, + None, + ) + .expect("the minimum"); + validate_warm_throughput_on_create( + &spec(Some(20_000), None), + TABLE, + BillingMode::PayPerRequest, + None, + ) + .expect("read alone"); +} + +/// Rows C5, C6, P1 and P2: a provisioned 5/5 table refuses 1/1 (read form) +/// and 5/4 (write form), accepts 5/5 (equal) and 100/50. +#[test] +fn provisioned_create_below_the_units_reports_the_measured_sentence() { + let units = five_five(); + let read = message(validate_warm_throughput_on_create( + &spec(Some(1), Some(1)), + TABLE, + BillingMode::Provisioned, + Some(&units), + )); + assert_eq!( + read, + "One or more parameter values were invalid: Requested ReadUnitsPerSecond for \ + WarmThroughput for table is lower than ReadCapacityUnits of ProvisionedThroughput" + ); + let write = message(validate_warm_throughput_on_create( + &spec(Some(5), Some(4)), + TABLE, + BillingMode::Provisioned, + Some(&units), + )); + assert_eq!( + write, + "One or more parameter values were invalid: Requested WriteUnitsPerSecond for \ + WarmThroughput for table is lower than WriteCapacityUnits of ProvisionedThroughput" + ); + validate_warm_throughput_on_create( + &spec(Some(5), Some(5)), + TABLE, + BillingMode::Provisioned, + Some(&units), + ) + .expect("equal to the units"); + validate_warm_throughput_on_create( + &spec(Some(100), Some(50)), + TABLE, + BillingMode::Provisioned, + Some(&units), + ) + .expect("above the units"); +} + +/// Rows U1 to U5, U7, UT1 and P3: equal and higher are accepted, lower is +/// refused with the decrease sentence on either billing mode and member. +#[test] +fn update_below_current_reports_the_decrease_sentence() { + validate_warm_throughput_on_update(&spec(Some(12_000), Some(4_000)), TABLE, (12_000, 4_000)) + .expect("same values"); + validate_warm_throughput_on_update(&spec(Some(24_000), Some(8_000)), TABLE, (12_000, 4_000)) + .expect("higher"); + validate_warm_throughput_on_update(&spec(Some(30_000), None), TABLE, (12_000, 4_000)) + .expect("read alone"); + let read = message(validate_warm_throughput_on_update( + &spec(Some(11_999), Some(4_000)), + TABLE, + (12_000, 4_000), + )); + assert_eq!( + read, + "One or more parameter values were invalid: Requested ReadUnitsPerSecond for \ + WarmThroughput for table is lower than current WarmThroughput, decreasing \ + WarmThroughput is not supported" + ); + // U7: a provisioned table at 100/50 refuses 2/2 with the same sentence. + assert_eq!( + message(validate_warm_throughput_on_update( + &spec(Some(2), Some(2)), + TABLE, + (100, 50) + )), + read + ); + // UT1 and P3: the write form, whole. + let write = message(validate_warm_throughput_on_update( + &spec(Some(12_000), Some(3_999)), + TABLE, + (12_000, 4_000), + )); + assert_eq!( + write, + "One or more parameter values were invalid: Requested WriteUnitsPerSecond for \ + WarmThroughput for table is lower than current WarmThroughput, decreasing \ + WarmThroughput is not supported" + ); + assert_eq!( + message(validate_warm_throughput_on_update( + &spec(Some(5), Some(4)), + TABLE, + (5, 5) + )), + write + ); +} + +/// Rows G1, G2, G3, UG1 and UG2: the index form of every sentence, whole. +#[test] +fn index_sentences_name_the_index() { + assert_eq!( + message(validate_warm_throughput_on_create( + &spec(Some(11_999), Some(4_000)), + GSI, + BillingMode::PayPerRequest, + None, + )), + ON_DEMAND_READ_TABLE.replace("for table", "for index gsi") + ); + assert_eq!( + message(validate_warm_throughput_on_create( + &spec(Some(12_000), Some(3_999)), + GSI, + BillingMode::PayPerRequest, + None, + )), + "One or more parameter values were invalid: Requested WriteUnitsPerSecond for \ + WarmThroughput for index gsi is lower than initial throughput for OnDemand. See: \ + https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/\ + on-demand-capacity-mode.html#on-demand-capacity-mode-initial" + ); + let units = five_five(); + assert_eq!( + message(validate_warm_throughput_on_create( + &spec(Some(1), Some(1)), + GSI, + BillingMode::Provisioned, + Some(&units), + )), + "One or more parameter values were invalid: Requested ReadUnitsPerSecond for \ + WarmThroughput for index gsi is lower than ReadCapacityUnits of ProvisionedThroughput" + ); + assert_eq!( + message(validate_warm_throughput_on_update( + &spec(Some(11_999), Some(4_000)), + GSI, + (12_000, 4_000), + )), + "One or more parameter values were invalid: Requested ReadUnitsPerSecond for \ + WarmThroughput for index gsi is lower than current WarmThroughput, decreasing \ + WarmThroughput is not supported" + ); + assert_eq!( + message(validate_warm_throughput_on_update( + &spec(Some(12_000), Some(3_999)), + GSI, + (12_000, 4_000), + )), + "One or more parameter values were invalid: Requested WriteUnitsPerSecond for \ + WarmThroughput for index gsi is lower than current WarmThroughput, decreasing \ + WarmThroughput is not supported" + ); +} + +#[test] +fn create_table_checks_the_table_then_each_index_by_name() { + let input: CreateTableInput = serde_json::from_value(serde_json::json!({ + "TableName": "t", + "KeySchema": [{"AttributeName": "pk", "KeyType": "HASH"}], + "AttributeDefinitions": [{"AttributeName": "pk", "AttributeType": "S"}], + "BillingMode": "PAY_PER_REQUEST", + "WarmThroughput": {"ReadUnitsPerSecond": 12000, "WriteUnitsPerSecond": 4000}, + "GlobalSecondaryIndexes": [{ + "IndexName": "by-g", + "KeySchema": [{"AttributeName": "pk", "KeyType": "HASH"}], + "Projection": {"ProjectionType": "ALL"}, + "WarmThroughput": {"ReadUnitsPerSecond": 1, "WriteUnitsPerSecond": 1} + }] + })) + .expect("input"); + let msg = message(validate_create_table_warm_throughput(&input)); + assert_eq!( + msg, + ON_DEMAND_READ_TABLE.replace("for table", "for index by-g") + ); + let mut ok = input.clone(); + ok.global_secondary_indexes = None; + validate_create_table_warm_throughput(&ok).expect("table alone"); +} + +/// The per-companion rules, as measured (service-pairs run and the review +/// rounds of 2026-10-23): a table-level WarmThroughput alone is accepted; +/// SSE, TableClass and DeletionProtection companions answer their own +/// sentence, in that order; OnDemandThroughput answers its own, with or +/// without BillingMode; an index Delete answers its own; Streams, billing mode +/// and provisioned throughput answer the WarmThroughput sentence; an index +/// Update companion is accepted. +#[test] +fn a_table_warm_throughput_must_be_the_only_operation() { + let input = |extra: serde_json::Value| -> UpdateTableInput { + let mut v = serde_json::json!({ + "TableName": "t", + "WarmThroughput": {"ReadUnitsPerSecond": 12000, "WriteUnitsPerSecond": 4000} + }); + for (k, val) in extra.as_object().expect("object") { + v[k] = val.clone(); + } + serde_json::from_value(v).expect("input") + }; + let gsi_update = serde_json::json!({"GlobalSecondaryIndexUpdates": [{"Update": { + "IndexName": "gsi", "WarmThroughput": {"ReadUnitsPerSecond": 12000}}}]}); + validate_warm_throughput_is_only_operation(&input(serde_json::json!({}))).expect("alone"); + validate_warm_throughput_is_only_operation(&input(serde_json::json!({ + "AttributeDefinitions": [{"AttributeName": "pk", "AttributeType": "S"}] + }))) + .expect("attribute definitions are not an operation"); + validate_warm_throughput_is_only_operation(&input(gsi_update.clone())) + .expect("an index update beside a table WarmThroughput is accepted"); + + let sse = serde_json::json!({"Enabled": true}); + let stream_off = serde_json::json!({"StreamEnabled": false}); + let stream_on = serde_json::json!({"StreamEnabled": true, "StreamViewType": "NEW_IMAGE"}); + let delete = serde_json::json!([{"Delete": {"IndexName": "gsi"}}]); + let cases = [ + // S1a, S1b, S1c: SSE outranks TableClass, DeletionProtection and Streams. + ( + serde_json::json!({"SSESpecification": sse}), + SSE_MUST_BE_ONLY_OPERATION, + ), + ( + serde_json::json!({"SSESpecification": sse, "TableClass": "STANDARD"}), + SSE_MUST_BE_ONLY_OPERATION, + ), + ( + serde_json::json!({"SSESpecification": sse, "DeletionProtectionEnabled": false}), + SSE_MUST_BE_ONLY_OPERATION, + ), + ( + serde_json::json!({"SSESpecification": sse, "StreamSpecification": stream_on}), + SSE_MUST_BE_ONLY_OPERATION, + ), + ( + serde_json::json!({"TableClass": "STANDARD"}), + TABLE_CLASS_MUST_BE_ONLY_OPERATION, + ), + ( + serde_json::json!({"DeletionProtectionEnabled": true}), + DELETION_PROTECTION_MUST_BE_ONLY_OPERATION, + ), + // T4d: OnDemandThroughput without BillingMode. + ( + serde_json::json!({"OnDemandThroughput": {"MaxReadRequestUnits": 10}}), + ON_DEMAND_THROUGHPUT_NEEDS_BILLING_MODE, + ), + // T4a: with BillingMode the sentence is the same (the service names + // BillingMode in it, and still refuses). + ( + serde_json::json!({"BillingMode": "PAY_PER_REQUEST", + "OnDemandThroughput": {"MaxReadRequestUnits": 10}}), + ON_DEMAND_THROUGHPUT_NEEDS_BILLING_MODE, + ), + // T4b, T4c: TableClass and DeletionProtection outrank OnDemandThroughput. + ( + serde_json::json!({"TableClass": "STANDARD", + "OnDemandThroughput": {"MaxReadRequestUnits": 10}}), + TABLE_CLASS_MUST_BE_ONLY_OPERATION, + ), + ( + serde_json::json!({"DeletionProtectionEnabled": true, + "OnDemandThroughput": {"MaxReadRequestUnits": 10}}), + DELETION_PROTECTION_MUST_BE_ONLY_OPERATION, + ), + // S9a: an index Delete; V3b: TableClass outranks it; V3a: with Streams + // the service names the stream change. + ( + serde_json::json!({"GlobalSecondaryIndexUpdates": delete}), + DELETE_GSI_WITH_WARM_THROUGHPUT, + ), + ( + serde_json::json!({"GlobalSecondaryIndexUpdates": delete, "TableClass": "STANDARD"}), + TABLE_CLASS_MUST_BE_ONLY_OPERATION, + ), + ( + serde_json::json!({"GlobalSecondaryIndexUpdates": delete, + "StreamSpecification": stream_on}), + INDEX_CHANGE_WITH_STREAM_CHANGE, + ), + ( + serde_json::json!({"StreamSpecification": stream_off}), + WARM_THROUGHPUT_MUST_BE_ONLY_OPERATION, + ), + ( + serde_json::json!({"BillingMode": "PROVISIONED", + "ProvisionedThroughput": {"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}}), + WARM_THROUGHPUT_MUST_BE_ONLY_OPERATION, + ), + // V2: ProvisionedThroughput without BillingMode. + ( + serde_json::json!({"ProvisionedThroughput": + {"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}}), + WARM_THROUGHPUT_MUST_BE_ONLY_OPERATION, + ), + // S1d: DeletionProtection outranks Streams. + ( + serde_json::json!({"DeletionProtectionEnabled": true, + "StreamSpecification": stream_off}), + DELETION_PROTECTION_MUST_BE_ONLY_OPERATION, + ), + // V4: the triple answers the TableClass sentence. + ( + serde_json::json!({"TableClass": "STANDARD", "DeletionProtectionEnabled": true, + "StreamSpecification": stream_on}), + TABLE_CLASS_MUST_BE_ONLY_OPERATION, + ), + ]; + for (extra, expected) in cases { + assert_eq!( + message(validate_warm_throughput_is_only_operation(&input( + extra.clone() + ))), + expected, + "{extra}" + ); + } + // No table-level member: this function refuses nothing, whatever else is + // there (the companions' own rules are out of scope here). + let no_warm: UpdateTableInput = serde_json::from_value(serde_json::json!({ + "TableName": "t", + "DeletionProtectionEnabled": true, + "TableClass": "STANDARD", + "SSESpecification": sse, + "StreamSpecification": {"StreamEnabled": false}, + "GlobalSecondaryIndexUpdates": [{"Update": { + "IndexName": "gsi", "WarmThroughput": {"ReadUnitsPerSecond": 12000}}}] + })) + .expect("input"); + validate_warm_throughput_is_only_operation(&no_warm).expect("no table-level member"); +} + +/// The sentences, whole (rows S1a, S9a, V3a). +#[test] +fn the_companion_sentences_are_the_measured_strings() { + assert_eq!( + SSE_MUST_BE_ONLY_OPERATION, + "One or more parameter values were invalid: Server-Side Encryption modification must \ + be the only operation in the request" + ); + assert_eq!( + DELETE_GSI_WITH_WARM_THROUGHPUT, + "One or more parameter values were invalid: Delete global secondary index cannot be \ + specified when updating WarmThroughput" + ); + assert_eq!( + INDEX_CHANGE_WITH_STREAM_CHANGE, + "One or more parameter values were invalid: You cannot create or delete index while \ + changing stream status" + ); +} + +/// Rows S10, T5a, T5c (missing table): a bad value alone, and an index +/// `Update` alone, are not refused here, so the request reaches the table +/// lookup and the service's `ResourceNotFoundException`. Rows S10b, T5b, T5d: +/// with a companion this function answers first, before the table is read. +#[test] +fn a_missing_table_reaches_the_lookup_unless_a_companion_answers_first() { + let input = |v: serde_json::Value| -> UpdateTableInput { + let mut v = v; + v["TableName"] = serde_json::json!("missing"); + serde_json::from_value(v).expect("input") + }; + // S10, T5a: a value below any default, nothing else. + validate_warm_throughput_is_only_operation(&input(serde_json::json!({ + "WarmThroughput": {"ReadUnitsPerSecond": 11999, "WriteUnitsPerSecond": 4000} + }))) + .expect("S10: reaches the lookup"); + validate_warm_throughput_is_only_operation(&input(serde_json::json!({ + "WarmThroughput": {"ReadUnitsPerSecond": 1} + }))) + .expect("T5a: reaches the lookup"); + // T5c: an index Update alone. + validate_warm_throughput_is_only_operation(&input(serde_json::json!({ + "GlobalSecondaryIndexUpdates": [{"Update": { + "IndexName": "gsi", "WarmThroughput": {"ReadUnitsPerSecond": 12000}}}] + }))) + .expect("T5c: reaches the lookup"); + let with = |extra: serde_json::Value| { + let mut v = serde_json::json!({"WarmThroughput": {"ReadUnitsPerSecond": 12000}}); + v[extra + .as_object() + .expect("object") + .keys() + .next() + .expect("key") + .as_str()] = extra + .as_object() + .expect("object") + .values() + .next() + .expect("value") + .clone(); + message(validate_warm_throughput_is_only_operation(&input(v))) + }; + assert_eq!( + with(serde_json::json!({"TableClass": "STANDARD_INFREQUENT_ACCESS"})), + TABLE_CLASS_MUST_BE_ONLY_OPERATION, + "S10b" + ); + assert_eq!( + with(serde_json::json!({"StreamSpecification": + {"StreamEnabled": true, "StreamViewType": "NEW_IMAGE"}})), + WARM_THROUGHPUT_MUST_BE_ONLY_OPERATION, + "T5b" + ); + assert_eq!( + with(serde_json::json!({"DeletionProtectionEnabled": false})), + DELETION_PROTECTION_MUST_BE_ONLY_OPERATION, + "T5d" + ); +} diff --git a/crates/engine/src/backup/overrides.rs b/crates/engine/src/backup/overrides.rs index 5adf8c3c6..ac6b07eec 100644 --- a/crates/engine/src/backup/overrides.rs +++ b/crates/engine/src/backup/overrides.rs @@ -46,8 +46,12 @@ //! on-demand restore is accepted and not recorded (ExtendDB keeps no //! per-index on-demand maximum); on a PROVISIONED restore it contradicts the //! billing mode and is refused. A member of an entry outside the service -//! model's (`WarmThroughput`, for one) is refused, since ExtendDB cannot apply -//! it. An `OnDemandThroughputOverride` replaces the backup's on-demand +//! model's is refused, since ExtendDB cannot apply it. A GSI entry's +//! `WarmThroughput` is refused with the service's own sentence (measured +//! 2026-10-23, row S6d). A restored table and its indexes report the +//! on-demand warm throughput defaults, as the service's restore does (row +//! S6c); the backup records no warm throughput (row S6b). An +//! `OnDemandThroughputOverride` replaces the backup's on-demand //! throughput as a whole: a member it leaves out means no maximum, as on //! `CreateTable`. @@ -132,6 +136,12 @@ pub(super) fn gsi_throughput_required(index_name: &str) -> DynamoDbError { )) } +/// The service's refusal of a `WarmThroughput` member on a +/// `GlobalSecondaryIndexOverride` entry (measured, row S6d), after the +/// "One or more parameter values were invalid: " prefix. +const WARM_THROUGHPUT_OVERRIDE: &str = + "WarmThroughput is not supported on Global Secondary Indexes via RestoreTableFromBackup"; + fn invalid(detail: String) -> DynamoDbError { DynamoDbError::ValidationException(format!( "One or more parameter values were invalid: {detail}" @@ -357,6 +367,9 @@ fn apply_gsis( .map(|(entry, recorded)| { let given = &entry.index; let name = &given.index_name; + if entry.unknown.iter().any(|m| m == "WarmThroughput") { + return Err(invalid(WARM_THROUGHPUT_OVERRIDE.to_owned())); + } same_definition( name, &entry.unknown, @@ -383,6 +396,7 @@ fn apply_gsis( key_schema: recorded.key_schema.clone(), projection: recorded.projection.clone(), provisioned_throughput: given.provisioned_throughput.clone(), + warm_throughput: None, }) }) .collect() diff --git a/crates/engine/src/backup/tests.rs b/crates/engine/src/backup/tests.rs index f7bd86281..d42221b11 100644 --- a/crates/engine/src/backup/tests.rs +++ b/crates/engine/src/backup/tests.rs @@ -213,6 +213,8 @@ fn described(request: &RestoreRequest) -> TableDescription { index_size_bytes: 0, item_count: 0, index_arn: String::new(), + warm_throughput: None, + warm_throughput_record: None, }) .collect() }), @@ -258,6 +260,7 @@ fn gsi(name: &str, pk: &str, pt: Option) -> GsiInput { non_key_attributes: None, }, provisioned_throughput: pt, + warm_throughput: None, } } @@ -1350,11 +1353,13 @@ async fn index_overrides_the_service_refuses_are_refused() { BillingMode is PAY_PER_REQUEST", ), ( - "a member ExtendDB cannot apply", + "a warm throughput on a GSI entry", json!({ "GlobalSecondaryIndexOverride": [entry_with( "WarmThroughput", json!({ "ReadUnitsPerSecond": 12000 }), )] }), - "index gsi1: member WarmThroughput is not supported", + // Row S6d, whole: the service's own sentence. + "One or more parameter values were invalid: WarmThroughput is not supported on \ + Global Secondary Indexes via RestoreTableFromBackup", ), ( "an LSI member ExtendDB cannot apply", diff --git a/crates/engine/src/gsi_updates.rs b/crates/engine/src/gsi_updates.rs new file mode 100644 index 000000000..6ca3f7caf --- /dev/null +++ b/crates/engine/src/gsi_updates.rs @@ -0,0 +1,333 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! The per-element rules for `UpdateTable`'s `GlobalSecondaryIndexUpdates`, +//! checked before any storage read. +//! +//! Measured against Amazon DynamoDB on 2026-10-23 (rows V1a, V1b, T3b, V3c, +//! T3d, T3e): +//! +//! - An element with two or more of `Create`, `Update`, `Delete`: +//! "Only one global secondary index action is allowed per +//! GlobalSecondaryIndexUpdate object" (V1a Update plus Create, V1b Create +//! plus Delete). +//! - Two actions naming the same index, in one element or in two: "Only one +//! global secondary index update per index is allowed simultaneously. +//! Index: " (T3b one element, V3c two). It is checked before the +//! one-action rule, so an Update and a Delete of one index answer it. +//! - An `Update` on an on-demand table that carries anything other than +//! `WarmThroughput` (a `ProvisionedThroughput`, or nothing at all): "The +//! only Updates for index: when TableThroughputMode is +//! PAY_PER_REQUEST can be to OnDemandThroughput, WarmThroughput" (T3d, +//! T3e). The billing mode is known only once the table is read, so that +//! rule lives in `warm_throughput::validate_warm_throughput_updates`; here +//! an `Update` without `WarmThroughput` on a table whose mode is not yet +//! known is refused as not yet supported (the service's sentence for a +//! provisioned table was not measured). +//! +//! A `Create` has a valid name, a key schema of defined attributes, and (when +//! given) a throughput of at least 1 unit each; a `Delete` has a valid name. + +use extenddb_core::error::DynamoDbError; +use extenddb_core::types::UpdateTableInput; + +/// Row V1a and V1b, whole. +pub const ONE_ACTION_PER_ELEMENT: &str = "One or more parameter values were invalid: Only one \ + global secondary index action is allowed per GlobalSecondaryIndexUpdate object"; + +/// Rows T3b and V3c: the sentence for `name`. +#[must_use] +pub fn one_update_per_index(name: &str) -> String { + format!( + "One or more parameter values were invalid: Only one global secondary index update per \ + index is allowed simultaneously. Index: {name}" + ) +} + +/// Rows T3d and T3e: the sentence for an on-demand table's index `name`. +#[must_use] +pub fn on_demand_update_members(name: &str) -> String { + format!( + "One or more parameter values were invalid: The only Updates for index: {name} when \ + TableThroughputMode is PAY_PER_REQUEST can be to OnDemandThroughput, WarmThroughput" + ) +} + +/// Whether any `Update` element carries nothing but its index name (a +/// `ProvisionedThroughput` on it is not a member of ExtendDB's action, so it +/// reads the same). Such an element needs the table's billing mode for its +/// sentence, so the caller reads the table first. +#[must_use] +pub(crate) fn has_bare_update(input: &UpdateTableInput) -> bool { + input + .global_secondary_index_updates + .iter() + .flatten() + .any(|u| { + u.update + .as_ref() + .is_some_and(|a| a.warm_throughput.is_none()) + }) +} + +/// Refuse every `Update` element that carries nothing but its index name: +/// on an on-demand table with the measured sentence (rows T3d, T3e); on a +/// provisioned table, where the service's sentence was not measured, with +/// ExtendDB's own "not yet supported". +/// +/// # Errors +/// +/// `ValidationException`. +pub(crate) fn refuse_bare_updates( + input: &UpdateTableInput, + pay_per_request: bool, +) -> Result<(), DynamoDbError> { + if let Some(upd) = input + .global_secondary_index_updates + .iter() + .flatten() + .filter_map(|u| u.update.as_ref()) + .find(|a| a.warm_throughput.is_none()) + { + return Err(DynamoDbError::ValidationException(if pay_per_request { + on_demand_update_members(&upd.index_name) + } else { + "UpdateGlobalSecondaryIndex is not yet supported".to_owned() + })); + } + Ok(()) +} + +/// Apply the element rules to `input`. +/// +/// # Errors +/// +/// `ValidationException` with the measured sentence (or the existing +/// ExtendDB sentence where the service's was not measured). +pub(crate) fn validate_gsi_updates(input: &UpdateTableInput) -> Result<(), DynamoDbError> { + let Some(updates) = &input.global_secondary_index_updates else { + return Ok(()); + }; + // Rows T3b and V3c: the per-index rule is applied to every name in every + // element before any element is looked at on its own, so an element that + // names one index twice (an Update and a Delete of `gsi`, row T3b) answers + // it rather than the one-action rule; two actions on different indexes + // (rows V1a, V1b) pass this and reach the one-action rule below. + let mut seen: Vec<&str> = Vec::new(); + for update in updates { + for name in [ + update.create.as_ref().map(|c| c.index_name.as_str()), + update.update.as_ref().map(|u| u.index_name.as_str()), + update.delete.as_ref().map(|d| d.index_name.as_str()), + ] + .into_iter() + .flatten() + { + if seen.contains(&name) { + return Err(DynamoDbError::ValidationException(one_update_per_index( + name, + ))); + } + seen.push(name); + } + } + for update in updates { + let actions = usize::from(update.create.is_some()) + + usize::from(update.update.is_some()) + + usize::from(update.delete.is_some()); + if actions > 1 { + return Err(DynamoDbError::ValidationException( + ONE_ACTION_PER_ELEMENT.to_owned(), + )); + } + if actions == 0 { + return Err(DynamoDbError::ValidationException( + "One or more parameter values were invalid: GlobalSecondaryIndexUpdate must \ + contain Create, Update, or Delete" + .to_owned(), + )); + } + if let Some(upd) = &update.update { + // An Update carrying WarmThroughput is stored (checked against the + // index's current values once the table is read). One carrying + // nothing else is refused by [`refuse_bare_updates`] once the + // table's billing mode is known. + extenddb_core::validation::validate_index_name(&upd.index_name)?; + } + if let Some(create) = &update.create { + extenddb_core::validation::validate_index_name(&create.index_name)?; + extenddb_core::validation::validate_index_throughput_minimum( + &create.index_name, + create.provisioned_throughput.as_ref(), + )?; + if create.key_schema.is_empty() { + return Err(DynamoDbError::ValidationException( + "One or more parameter values were invalid: KeySchema must not be empty for \ + GSI creation" + .to_owned(), + )); + } + let attr_defs = input.attribute_definitions.as_deref().unwrap_or(&[]); + for ks in &create.key_schema { + if !attr_defs + .iter() + .any(|ad| ad.attribute_name == ks.attribute_name) + { + return Err(DynamoDbError::ValidationException(format!( + "One or more parameter values were invalid: Some index key attributes \ + are not defined in AttributeDefinitions. Keys: [{}], \ + AttributeDefinitions: [{}]", + ks.attribute_name, + attr_defs + .iter() + .map(|ad| ad.attribute_name.as_str()) + .collect::>() + .join(", ") + ))); + } + } + } + if let Some(delete) = &update.delete { + extenddb_core::validation::validate_index_name(&delete.index_name)?; + } + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::{Value, json}; + + fn input(updates: Value) -> UpdateTableInput { + serde_json::from_value(json!({ + "TableName": "t", + "AttributeDefinitions": [{ "AttributeName": "g", "AttributeType": "S" }], + "GlobalSecondaryIndexUpdates": updates, + })) + .unwrap() + } + + fn message(result: Result<(), DynamoDbError>) -> String { + match result.expect_err("refused") { + DynamoDbError::ValidationException(m) => m, + other => panic!("{other:?}"), + } + } + + fn create(name: &str) -> Value { + json!({ "Create": { + "IndexName": name, + "KeySchema": [{ "AttributeName": "g", "KeyType": "HASH" }], + "Projection": { "ProjectionType": "ALL" }, + }}) + } + + fn warm_update(name: &str) -> Value { + json!({ "Update": { "IndexName": name, "WarmThroughput": { "ReadUnitsPerSecond": 12000 } } }) + } + + /// Rows V1a and V1b: an element with two actions, whichever two, and the + /// Create or Delete in it is never reached (the Create here keys on an + /// undefined attribute and would otherwise be refused differently). + #[test] + fn an_element_with_two_actions_is_refused_before_its_parts_are_checked() { + let mut mixed = warm_update("gsi"); + mixed["Create"] = json!({ + "IndexName": "bad", + "KeySchema": [{ "AttributeName": "nope", "KeyType": "HASH" }], + "Projection": { "ProjectionType": "ALL" }, + }); + assert_eq!( + message(validate_gsi_updates(&input(json!([mixed])))), + ONE_ACTION_PER_ELEMENT + ); + let mut update_delete = warm_update("gsi"); + update_delete["Delete"] = json!({ "IndexName": "other" }); + assert_eq!( + message(validate_gsi_updates(&input(json!([update_delete])))), + ONE_ACTION_PER_ELEMENT + ); + let mut create_delete = create("gsi-two"); + create_delete["Delete"] = json!({ "IndexName": "gsi" }); + assert_eq!( + message(validate_gsi_updates(&input(json!([create_delete])))), + ONE_ACTION_PER_ELEMENT + ); + } + + /// Row T3b: one element carrying an Update and a Delete of the same index + /// answers the per-index sentence, not the one-action sentence. + #[test] + fn one_element_naming_an_index_twice_answers_the_per_index_sentence() { + let mut same = warm_update("gsi"); + same["Delete"] = json!({ "IndexName": "gsi" }); + assert_eq!( + message(validate_gsi_updates(&input(json!([same])))), + one_update_per_index("gsi") + ); + } + + /// Rows T3b and V3c: two elements on one index. + #[test] + fn two_elements_on_one_index_are_refused() { + let msg = message(validate_gsi_updates(&input(json!([ + { "Delete": { "IndexName": "gsi" } }, + warm_update("gsi"), + ])))); + assert_eq!( + msg, + "One or more parameter values were invalid: Only one global secondary index update \ + per index is allowed simultaneously. Index: gsi" + ); + validate_gsi_updates(&input(json!([ + { "Delete": { "IndexName": "gsi" } }, + warm_update("other"), + ]))) + .expect("different indexes"); + } + + /// Rows T3d and T3e: an Update with nothing but the index name (or a + /// member ExtendDB's action does not carry) on an on-demand table answers + /// the measured sentence; on a provisioned table ExtendDB's own. + #[test] + fn a_bare_update_is_refused_with_the_billing_mode_sentence() { + let bare = input(json!([{ "Update": { "IndexName": "gsi" } }])); + validate_gsi_updates(&bare).expect("the element rules pass; the mode decides"); + assert!(has_bare_update(&bare)); + assert!(!has_bare_update(&input(json!([warm_update("gsi")])))); + assert_eq!( + message(refuse_bare_updates(&bare, true)), + "One or more parameter values were invalid: The only Updates for index: gsi when \ + TableThroughputMode is PAY_PER_REQUEST can be to OnDemandThroughput, WarmThroughput" + ); + assert!(message(refuse_bare_updates(&bare, false)).contains("not yet supported")); + refuse_bare_updates(&input(json!([warm_update("gsi")])), true).expect("warm update"); + } + + #[test] + fn a_created_gsi_throughput_below_one_is_refused() { + for (read, write) in [(0, 0), (0, 1), (1, 0), (-1, 5)] { + let mut c = create("gix"); + c["Create"]["ProvisionedThroughput"] = + json!({ "ReadCapacityUnits": read, "WriteCapacityUnits": write }); + let m = message(validate_gsi_updates(&input(json!([c])))); + assert!(m.ends_with("for index: gix"), "({read}, {write}): {m}"); + } + let mut ok = create("gix"); + ok["Create"]["ProvisionedThroughput"] = + json!({ "ReadCapacityUnits": 1, "WriteCapacityUnits": 1 }); + validate_gsi_updates(&input(json!([ok]))).unwrap(); + } + + #[test] + fn an_empty_element_and_an_undefined_key_attribute_are_refused() { + assert!(message(validate_gsi_updates(&input(json!([{}])))).contains("must contain")); + let mut c = create("gix"); + c["Create"]["KeySchema"] = json!([{ "AttributeName": "nope", "KeyType": "HASH" }]); + assert!( + message(validate_gsi_updates(&input(json!([c])))) + .contains("Some index key attributes are not defined") + ); + } +} diff --git a/crates/engine/src/import_export.rs b/crates/engine/src/import_export.rs index a6ecba4b9..dd9ad359d 100755 --- a/crates/engine/src/import_export.rs +++ b/crates/engine/src/import_export.rs @@ -357,6 +357,7 @@ fn create_table_input_from_params(tcp: &TableCreationParameters) -> CreateTableI deletion_protection_enabled: None, table_class: None, on_demand_throughput: None, + warm_throughput: None, } } diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs index 74083fbfa..40592178f 100755 --- a/crates/engine/src/lib.rs +++ b/crates/engine/src/lib.rs @@ -20,6 +20,7 @@ mod describe_table; mod expected; mod expression_helpers; mod get_item; +mod gsi_updates; mod import_export; mod import_export_io; mod index_helpers; @@ -41,6 +42,7 @@ mod ttl; mod update_item; mod update_table; mod vector_gate; +mod warm_throughput; pub use batch_get_item::handle_batch_get_item; pub use batch_write_item::handle_batch_write_item; diff --git a/crates/engine/src/update_table.rs b/crates/engine/src/update_table.rs index 6db763f82..061fd29de 100755 --- a/crates/engine/src/update_table.rs +++ b/crates/engine/src/update_table.rs @@ -88,18 +88,20 @@ pub async fn handle_update_table( .as_ref() .is_some_and(|u| !u.is_empty()); - // Validate: at least one field must be specified. + // Validate: at least one field must be specified. An UpdateTable carrying + // only WarmThroughput is accepted (measured 2026-10-23). if input.billing_mode.is_none() && input.provisioned_throughput.is_none() && input.deletion_protection_enabled.is_none() && input.stream_specification.is_none() && input.table_class.is_none() && input.on_demand_throughput.is_none() + && input.warm_throughput.is_none() && !has_gsi_updates && !has_vector_updates { return Err(DynamoDbError::ValidationException( - "At least one of BillingMode, ProvisionedThroughput, DeletionProtectionEnabled, StreamSpecification, or GlobalSecondaryIndexUpdates must be specified".to_owned(), + "At least one of BillingMode, ProvisionedThroughput, DeletionProtectionEnabled, StreamSpecification, GlobalSecondaryIndexUpdates, or WarmThroughput must be specified".to_owned(), )); } @@ -140,21 +142,39 @@ pub async fn handle_update_table( )); } - validate_gsi_updates(&input)?; + // A table-level WarmThroughput stands alone in its request, with the + // measured per-companion sentences (measured 2026-10-23). Checked before + // the value rules, as the service does: it fired on requests whose values + // were otherwise valid. GlobalSecondaryIndexUpdates is accepted beside it. + extenddb_core::validation::validate_warm_throughput_is_only_operation(&input)?; + + crate::gsi_updates::validate_gsi_updates(&input)?; // A created GSI's projected attributes count toward the table's limit // together with the indexes it already has: the total CreateTable checks, // and so the total a restore of this table's backups checks. - if let Some(updates) = &input.global_secondary_index_updates - && updates.iter().any(|u| { - u.create.as_ref().is_some_and(|c| { - c.projection - .non_key_attributes - .as_ref() - .is_some_and(|a| !a.is_empty()) + let projects_attributes = input + .global_secondary_index_updates + .as_ref() + .is_some_and(|u| { + u.iter().any(|u| { + u.create.as_ref().is_some_and(|c| { + c.projection + .non_key_attributes + .as_ref() + .is_some_and(|a| !a.is_empty()) + }) }) - }) - { + }); + // WarmThroughput may not go below the current values, a created index's + // follows the table's billing mode, and an index Update carrying nothing + // answers a sentence that names the mode: all need the table as it is. + let touches_warm = crate::warm_throughput::touches_warm_throughput(&input); + let bare_update = crate::gsi_updates::has_bare_update(&input); + // A switch to PAY_PER_REQUEST stores the on-demand minimum as the warm + // throughput of a provisioned table that is below it, so it needs the table. + let switches_to_on_demand = matches!(input.billing_mode, Some(BillingMode::PayPerRequest)); + if projects_attributes || touches_warm || bare_update || switches_to_on_demand { let existing = ctx .storage .describe_table( @@ -165,9 +185,29 @@ pub async fn handle_update_table( ) .await .map_err(|e| update_table_err_to_dynamo(e, &input.table_name))?; - extenddb_core::validation::validate_projected_attribute_total( - projected_after_gsi_updates(&existing, updates), - )?; + if let Some(updates) = &input.global_secondary_index_updates + && projects_attributes + { + extenddb_core::validation::validate_projected_attribute_total( + projected_after_gsi_updates(&existing, updates), + )?; + } + if bare_update { + let pay_per_request = existing + .billing_mode_summary + .as_ref() + .is_some_and(|s| s.billing_mode == BillingMode::PayPerRequest); + crate::gsi_updates::refuse_bare_updates(&input, pay_per_request)?; + } + if touches_warm { + crate::warm_throughput::validate_warm_throughput_updates(&input, &existing)?; + } + if switches_to_on_demand { + // After every validation: the client cannot send WarmThroughput + // beside BillingMode, so this is the only way both reach a backend. + input.warm_throughput = + crate::warm_throughput::on_demand_switch_warm_throughput(&existing); + } } let table_name = input.table_name.clone(); @@ -203,70 +243,6 @@ pub async fn handle_update_table( serialize_output(&output) } -/// The per-entry rules for `GlobalSecondaryIndexUpdates`, checked before any -/// storage read: each entry has exactly one of Create, Update, or Delete, -/// and a Create has a valid name, a key schema of defined attributes, and -/// (when given) a throughput of at least 1 unit each. -fn validate_gsi_updates(input: &UpdateTableInput) -> Result<(), DynamoDbError> { - if let Some(updates) = &input.global_secondary_index_updates { - for update in updates { - if update.create.is_some() && update.delete.is_some() { - return Err(DynamoDbError::ValidationException( - "One or more parameter values were invalid: Only one of Create or Delete can be specified per GlobalSecondaryIndexUpdate".to_owned(), - )); - } - if let Some(ref upd) = update.update { - // S3: Acknowledge the Update action but reject it as unsupported. - let _ = upd; - return Err(DynamoDbError::ValidationException( - "UpdateGlobalSecondaryIndex is not yet supported".to_owned(), - )); - } - if update.create.is_none() && update.delete.is_none() { - return Err(DynamoDbError::ValidationException( - "One or more parameter values were invalid: GlobalSecondaryIndexUpdate must contain Create, Update, or Delete".to_owned(), - )); - } - // M3: Validate index names the same way CreateTable does. - if let Some(create) = &update.create { - extenddb_core::validation::validate_index_name(&create.index_name)?; - extenddb_core::validation::validate_index_throughput_minimum( - &create.index_name, - create.provisioned_throughput.as_ref(), - )?; - if create.key_schema.is_empty() { - return Err(DynamoDbError::ValidationException( - "One or more parameter values were invalid: KeySchema must not be empty for GSI creation".to_owned(), - )); - } - // Validate that all key attributes are defined in AttributeDefinitions - let attr_defs = input.attribute_definitions.as_deref().unwrap_or(&[]); - for ks in &create.key_schema { - if !attr_defs - .iter() - .any(|ad| ad.attribute_name == ks.attribute_name) - { - return Err(DynamoDbError::ValidationException(format!( - "One or more parameter values were invalid: Some index key attributes are not defined in AttributeDefinitions. \ - Keys: [{}], AttributeDefinitions: [{}]", - ks.attribute_name, - attr_defs - .iter() - .map(|ad| ad.attribute_name.as_str()) - .collect::>() - .join(", ") - ))); - } - } - } - if let Some(delete) = &update.delete { - extenddb_core::validation::validate_index_name(&delete.index_name)?; - } - } - } - Ok(()) -} - /// The table's projected `NonKeyAttributes` once `updates` apply: its LSIs, /// its GSIs that are not deleted, and the GSIs created. fn projected_after_gsi_updates( @@ -351,34 +327,6 @@ mod tests { use super::*; use extenddb_storage::error::StorageError; - #[test] - fn a_created_gsi_throughput_below_one_is_refused() { - let input = |pt: Value| -> UpdateTableInput { - serde_json::from_value(serde_json::json!({ - "TableName": "t", - "AttributeDefinitions": [{ "AttributeName": "g", "AttributeType": "S" }], - "GlobalSecondaryIndexUpdates": [{ "Create": { - "IndexName": "gix", - "KeySchema": [{ "AttributeName": "g", "KeyType": "HASH" }], - "Projection": { "ProjectionType": "ALL" }, - "ProvisionedThroughput": pt, - }}], - })) - .unwrap() - }; - for (read, write) in [(0, 0), (0, 1), (1, 0), (-1, 5)] { - let pt = serde_json::json!({ "ReadCapacityUnits": read, "WriteCapacityUnits": write }); - match validate_gsi_updates(&input(pt)).unwrap_err() { - DynamoDbError::ValidationException(m) => { - assert!(m.ends_with("for index: gix"), "{m}"); - } - other => panic!("({read}, {write}): {other:?}"), - } - } - let ok = serde_json::json!({ "ReadCapacityUnits": 1, "WriteCapacityUnits": 1 }); - validate_gsi_updates(&input(ok)).unwrap(); - } - #[test] fn projected_total_counts_kept_indexes_and_created_ones() { use extenddb_core::types::{ @@ -400,6 +348,8 @@ mod tests { index_size_bytes: 0, item_count: 0, index_arn: String::new(), + warm_throughput: None, + warm_throughput_record: None, }, GsiDescription { index_name: "dropped".to_owned(), @@ -410,6 +360,8 @@ mod tests { index_size_bytes: 0, item_count: 0, index_arn: String::new(), + warm_throughput: None, + warm_throughput_record: None, }, ]), local_secondary_indexes: Some(vec![LsiDescription { diff --git a/crates/engine/src/warm_throughput.rs b/crates/engine/src/warm_throughput.rs new file mode 100644 index 000000000..6434be79c --- /dev/null +++ b/crates/engine/src/warm_throughput.rs @@ -0,0 +1,452 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! `UpdateTable` checks for `WarmThroughput`, against the table's current +//! description. +//! +//! The create-time rules need only the request; these need what the table +//! reports now (an update may not lower a value, and a created index follows +//! the table's billing mode), so they run after a `DescribeTable` of the +//! target. Measured against Amazon DynamoDB on 2026-10-23; see +//! `extenddb_core::validation::warm_throughput` for the sentences. + +use extenddb_core::error::DynamoDbError; +use extenddb_core::types::{ + BillingMode, ON_DEMAND_WARM_READ_UNITS, ON_DEMAND_WARM_WRITE_UNITS, TableDescription, + UpdateTableInput, WarmThroughput, WarmThroughputDefaults, epoch_seconds_now, +}; +use extenddb_core::validation::{ + WarmThroughputSubject, validate_created_gsi_warm_throughput, validate_warm_throughput_on_update, +}; + +/// Whether `input` carries any `WarmThroughput`, on the table or in a +/// `GlobalSecondaryIndexUpdates` action. +#[must_use] +pub(crate) fn touches_warm_throughput(input: &UpdateTableInput) -> bool { + input.warm_throughput.is_some() + || input + .global_secondary_index_updates + .iter() + .flatten() + .any(|u| { + u.update + .as_ref() + .is_some_and(|a| a.warm_throughput.is_some()) + || u.create + .as_ref() + .is_some_and(|a| a.warm_throughput.is_some()) + }) +} + +/// The warm throughput the table reports now (the pair the window started +/// from while UPDATING, measured rows U3 and S2b) and the billing mode it is +/// under, from a description the backend returned. +fn current_table_values(existing: &TableDescription) -> (BillingMode, (i64, i64)) { + let now = epoch_seconds_now(); + let pay_per_request = existing + .billing_mode_summary + .as_ref() + .is_some_and(|s| s.billing_mode == BillingMode::PayPerRequest); + let defaults = WarmThroughputDefaults::for_billing( + pay_per_request, + existing.provisioned_throughput.read_capacity_units, + existing.provisioned_throughput.write_capacity_units, + ); + let current = existing.warm_throughput_record.as_ref().map_or_else( + || { + ( + defaults.read_units_per_second, + defaults.write_units_per_second, + ) + }, + |r| r.reported(defaults, now), + ); + let mode = if pay_per_request { + BillingMode::PayPerRequest + } else { + BillingMode::Provisioned + }; + (mode, current) +} + +/// The `WarmThroughput` an `UpdateTable` that switches a provisioned table to +/// PAY_PER_REQUEST stores with the switch, or `None` when the switch changes +/// nothing about it. +/// +/// Measured against Amazon DynamoDB on 2026-10-23: a provisioned table whose +/// warm throughput is below the on-demand minimum reads 12000 read and 4000 +/// write units once it is on-demand, whether the client set it (`{100}` on a +/// 5/5 table, rows T7a to T7d; `100/50` on an explicit table, row S7c) or +/// never did (a never-set 10/10 table, row S7a); a side at or above the +/// minimum is kept. The values stay when the table is switched back to +/// provisioned (row S7b: `12000/4000` after the flip back to 5/5), so they are +/// stored, not derived from the billing mode at read time. Each side becomes +/// the larger of what the table settles on (its stored side, else its +/// provisioned units) and the on-demand minimum, and both sides are stored. +/// +/// The service answers the switch with the previous values and `Status: +/// ACTIVE` while the table is `UPDATING`; ExtendDB stores the values through +/// the ordinary update path, so the member reads `UPDATING` for +/// `control_plane_delay_seconds`, with the previous values only when the +/// record held them (a never-set table reports the on-demand values at +/// once). The client cannot send `WarmThroughput` beside `BillingMode` (the +/// only-operation rule refuses it), so a request reaching a backend with both +/// carries this reset. +/// +/// Global secondary indexes are not stored here: a provisioned index never +/// set reports the on-demand minimum while the table is on-demand, and its +/// own units again after a switch back; the service's behaviour for an index +/// was not measured. +#[must_use] +pub(crate) fn on_demand_switch_warm_throughput( + existing: &TableDescription, +) -> Option { + let already_on_demand = existing + .billing_mode_summary + .as_ref() + .is_some_and(|s| s.billing_mode == BillingMode::PayPerRequest); + if already_on_demand { + return None; + } + let provisioned = WarmThroughputDefaults::for_billing( + false, + existing.provisioned_throughput.read_capacity_units, + existing.provisioned_throughput.write_capacity_units, + ); + let record = existing.warm_throughput_record.as_ref(); + let (read, write) = record.map_or( + ( + provisioned.read_units_per_second, + provisioned.write_units_per_second, + ), + |r| r.effective(provisioned), + ); + let raised = ( + read.max(ON_DEMAND_WARM_READ_UNITS), + write.max(ON_DEMAND_WARM_WRITE_UNITS), + ); + let both_stored = record + .is_some_and(|r| r.read_units_per_second.is_some() && r.write_units_per_second.is_some()); + if both_stored && raised == (read, write) { + return None; + } + Some(WarmThroughput { + read_units_per_second: Some(raised.0), + write_units_per_second: Some(raised.1), + }) +} + +/// Apply the measured update rules to `input` against `existing`: +/// +/// - the table's `WarmThroughput` may not go below its current values +/// (the "for table" sentence); +/// - an `Update` action's `WarmThroughput` may not go below the named +/// index's current values (the "for index " sentence), and the +/// index must exist; +/// - a `Create` action's `WarmThroughput` follows the create rule under the +/// table's current billing mode. +/// +/// # Errors +/// +/// `ValidationException` with the measured sentence; +/// `ResourceNotFoundException` for an `Update` naming an index the table +/// does not have. +pub(crate) fn validate_warm_throughput_updates( + input: &UpdateTableInput, + existing: &TableDescription, +) -> Result<(), DynamoDbError> { + let (billing_mode, current) = current_table_values(existing); + let now = epoch_seconds_now(); + if let Some(spec) = &input.warm_throughput { + validate_warm_throughput_on_update(spec, WarmThroughputSubject::Table, current)?; + } + let pay_per_request = billing_mode == BillingMode::PayPerRequest; + for action in input.global_secondary_index_updates.iter().flatten() { + if let Some(create) = &action.create { + validate_created_gsi_warm_throughput(create, billing_mode)?; + } + if let Some(update) = &action.update + && let Some(spec) = &update.warm_throughput + { + let gsi = existing + .global_secondary_indexes + .iter() + .flatten() + .find(|g| g.index_name == update.index_name) + .ok_or_else(|| { + DynamoDbError::ResourceNotFoundException(format!( + "Requested resource not found: Index {} for table {}", + update.index_name, input.table_name + )) + })?; + let (rcu, wcu) = gsi.provisioned_throughput.as_ref().map_or((0, 0), |pt| { + (pt.read_capacity_units, pt.write_capacity_units) + }); + let defaults = WarmThroughputDefaults::for_billing(pay_per_request, rcu, wcu); + let current = gsi.warm_throughput_record.as_ref().map_or_else( + || { + ( + defaults.read_units_per_second, + defaults.write_units_per_second, + ) + }, + |r| r.reported(defaults, now), + ); + validate_warm_throughput_on_update( + spec, + WarmThroughputSubject::Index(&update.index_name), + current, + )?; + } + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use extenddb_core::types::{ + BillingModeSummary, GsiDescription, Projection, ProjectionType, + ProvisionedThroughputDescription, WarmThroughputRecord, + }; + use serde_json::json; + + fn on_demand_table(record: Option) -> TableDescription { + TableDescription { + table_name: "t".to_owned(), + billing_mode_summary: Some(BillingModeSummary { + billing_mode: BillingMode::PayPerRequest, + last_update_to_pay_per_request_date_time: None, + }), + warm_throughput_record: record, + global_secondary_indexes: Some(vec![GsiDescription { + index_name: "gsi".to_owned(), + key_schema: Vec::new(), + projection: Projection { + projection_type: ProjectionType::All, + non_key_attributes: None, + }, + index_status: "ACTIVE".to_owned(), + provisioned_throughput: Some(ProvisionedThroughputDescription::default()), + index_size_bytes: 0, + item_count: 0, + index_arn: String::new(), + warm_throughput: None, + warm_throughput_record: None, + }]), + ..Default::default() + } + } + + fn input(v: serde_json::Value) -> UpdateTableInput { + let mut v = v; + v["TableName"] = json!("t"); + serde_json::from_value(v).expect("input") + } + + /// U4: below the on-demand default on a table that never set one. + #[test] + fn a_table_update_below_the_current_default_is_refused() { + let err = validate_warm_throughput_updates( + &input(json!({"WarmThroughput": {"ReadUnitsPerSecond": 11999}})), + &on_demand_table(None), + ) + .expect_err("refused"); + assert!(matches!(err, DynamoDbError::ValidationException(m) if m.contains("decreasing"))); + } + + /// U1 and U2: same and higher are accepted. + #[test] + fn a_table_update_at_or_above_current_is_accepted() { + let existing = on_demand_table(Some(WarmThroughputRecord { + read_units_per_second: Some(20_000), + write_units_per_second: None, + previous_read_units_per_second: None, + previous_write_units_per_second: None, + updating_until: 0.0, + })); + validate_warm_throughput_updates( + &input(json!({"WarmThroughput": {"ReadUnitsPerSecond": 20000, "WriteUnitsPerSecond": 4000}})), + &existing, + ) + .expect("same"); + validate_warm_throughput_updates( + &input(json!({"WarmThroughput": {"ReadUnitsPerSecond": 20001}})), + &existing, + ) + .expect("higher"); + // The stored read side is what "current" means. + validate_warm_throughput_updates( + &input(json!({"WarmThroughput": {"ReadUnitsPerSecond": 12000}})), + &existing, + ) + .expect_err("below the stored 20000"); + } + + /// U6: an index update follows the same rule against the index's values. + #[test] + fn an_index_update_is_checked_against_the_index_and_must_name_one() { + let existing = on_demand_table(None); + validate_warm_throughput_updates( + &input(json!({"GlobalSecondaryIndexUpdates": [{"Update": { + "IndexName": "gsi", + "WarmThroughput": {"ReadUnitsPerSecond": 12000, "WriteUnitsPerSecond": 4000} + }}]})), + &existing, + ) + .expect("same as the index default"); + let low = validate_warm_throughput_updates( + &input(json!({"GlobalSecondaryIndexUpdates": [{"Update": { + "IndexName": "gsi", + "WarmThroughput": {"WriteUnitsPerSecond": 3999} + }}]})), + &existing, + ) + .expect_err("below"); + assert!( + matches!(&low, DynamoDbError::ValidationException(m) + if m.contains("WriteUnitsPerSecond for WarmThroughput for index gsi is lower than current")), + "{low:?}" + ); + let missing = validate_warm_throughput_updates( + &input(json!({"GlobalSecondaryIndexUpdates": [{"Update": { + "IndexName": "nope", + "WarmThroughput": {"ReadUnitsPerSecond": 12000} + }}]})), + &existing, + ) + .expect_err("no such index"); + assert!(matches!( + missing, + DynamoDbError::ResourceNotFoundException(_) + )); + } + + /// A created index's WarmThroughput follows the create rule under the + /// table's billing mode. + #[test] + fn a_created_index_is_checked_under_the_table_billing_mode() { + let err = validate_warm_throughput_updates( + &input(json!({"GlobalSecondaryIndexUpdates": [{"Create": { + "IndexName": "new", + "KeySchema": [{"AttributeName": "g", "KeyType": "HASH"}], + "Projection": {"ProjectionType": "ALL"}, + "WarmThroughput": {"ReadUnitsPerSecond": 1} + }}]})), + &on_demand_table(None), + ) + .expect_err("below the on-demand minimum"); + assert!( + matches!(&err, DynamoDbError::ValidationException(m) + if m.contains("for index new is lower than initial throughput for OnDemand")), + "{err:?}" + ); + } + + fn provisioned_table(units: i64, record: Option) -> TableDescription { + TableDescription { + table_name: "t".to_owned(), + provisioned_throughput: ProvisionedThroughputDescription { + read_capacity_units: units, + write_capacity_units: units, + ..Default::default() + }, + warm_throughput_record: record, + ..Default::default() + } + } + + fn stored(read: Option, write: Option) -> Option { + Some(WarmThroughputRecord { + read_units_per_second: read, + write_units_per_second: write, + previous_read_units_per_second: None, + previous_write_units_per_second: None, + updating_until: 0.0, + }) + } + + fn pair(w: Option) -> Option<(i64, i64)> { + w.map(|w| { + ( + w.read_units_per_second.expect("read"), + w.write_units_per_second.expect("write"), + ) + }) + } + + /// Rows S7a, S7c, T7c: a switch to on-demand stores at least the on-demand + /// minimum on both sides, from a never-set table, a read-only record and + /// an explicit pair below it. + #[test] + fn a_switch_to_on_demand_stores_the_minimum_for_a_low_warm_throughput() { + assert_eq!( + pair(on_demand_switch_warm_throughput(&provisioned_table( + 10, None + ))), + Some((12_000, 4_000)), + "S7a: never set, 10/10" + ); + assert_eq!( + pair(on_demand_switch_warm_throughput(&provisioned_table( + 10, + stored(Some(100), None) + ))), + Some((12_000, 4_000)), + "T7c: {{100}} on a table now at 10/10" + ); + assert_eq!( + pair(on_demand_switch_warm_throughput(&provisioned_table( + 10, + stored(Some(100), Some(50)) + ))), + Some((12_000, 4_000)), + "S7c: explicit 100/50" + ); + } + + /// A side at or above the minimum is kept; nothing is stored when both + /// sides are stored and already at or above it, and none when the table is + /// on-demand already. + #[test] + fn a_switch_to_on_demand_keeps_sides_at_or_above_the_minimum() { + assert_eq!( + pair(on_demand_switch_warm_throughput(&provisioned_table( + 10, + stored(Some(20_000), None) + ))), + Some((20_000, 4_000)) + ); + assert_eq!( + pair(on_demand_switch_warm_throughput(&provisioned_table( + 10, + stored(Some(30_000), Some(9_000)) + ))), + None + ); + assert_eq!( + on_demand_switch_warm_throughput(&on_demand_table(None)), + None + ); + // Provisioned units above the minimum are what the table settles on. + assert_eq!( + pair(on_demand_switch_warm_throughput(&provisioned_table( + 50_000, None + ))), + Some((50_000, 50_000)) + ); + } + + #[test] + fn touches_is_true_for_any_warm_throughput_member() { + assert!(!touches_warm_throughput(&input( + json!({"DeletionProtectionEnabled": true}) + ))); + assert!(touches_warm_throughput(&input( + json!({"WarmThroughput": {}}) + ))); + assert!(touches_warm_throughput(&input(json!({ + "GlobalSecondaryIndexUpdates": [{"Update": {"IndexName": "g", "WarmThroughput": {}}}] + })))); + } +} diff --git a/crates/storage-cassandra/migrations/catalog/V006__warm_throughput.cql b/crates/storage-cassandra/migrations/catalog/V006__warm_throughput.cql new file mode 100644 index 000000000..9e1f19a30 --- /dev/null +++ b/crates/storage-cassandra/migrations/catalog/V006__warm_throughput.cql @@ -0,0 +1,18 @@ +-- Copyright 2026 ExtendDB contributors +-- SPDX-License-Identifier: Apache-2.0 +-- WarmThroughput on tables and global secondary indexes (catalog version 0.0.5). +-- +-- tables.warm_throughput and indexes.warm_throughput hold the WarmThroughput a +-- client set, as one JSON text (see extenddb_core::types::WarmThroughputRecord). +-- The value is stored and reported by DescribeTable so clients that read the +-- member work. Nothing acts on it: there is no pre-warming and no scaling at +-- the database level, and it has no effect on throughput, throttling or +-- storage. A row written before this migration has no value and reports the +-- defaults for its billing mode. +-- +-- NOTE: the migration runner splits files on the semicolon character, +-- including inside comments. Do not use one in a comment. +ALTER TABLE tables ADD IF NOT EXISTS warm_throughput text; +ALTER TABLE indexes ADD IF NOT EXISTS warm_throughput text; + +INSERT INTO settings (key, value) VALUES ('catalog_version', '0.0.5'); diff --git a/crates/storage-cassandra/src/backup_engine.rs b/crates/storage-cassandra/src/backup_engine.rs index 7c070934e..d9d30ac0d 100755 --- a/crates/storage-cassandra/src/backup_engine.rs +++ b/crates/storage-cassandra/src/backup_engine.rs @@ -574,6 +574,7 @@ impl CassandraEngine { write_capacity_units: pt.write_capacity_units, } }), + warm_throughput: None, }) .collect(); let local_secondary_indexes = table diff --git a/crates/storage-cassandra/src/create_table.rs b/crates/storage-cassandra/src/create_table.rs index 5628926b4..4aa409fd4 100644 --- a/crates/storage-cassandra/src/create_table.rs +++ b/crates/storage-cassandra/src/create_table.rs @@ -10,7 +10,10 @@ use extenddb_core::types::{ TableStatus, }; use extenddb_storage::error::StorageError; -use extenddb_storage::util::{index_arn, stream_arn, table_arn}; +use extenddb_storage::util::{ + index_arn, stream_arn, table_arn, warm_throughput_record_for_create, + warm_throughput_record_json, +}; use crate::CassandraEngine; use crate::backup_engine::RestoreHeartbeat; @@ -263,6 +266,17 @@ impl CassandraEngine { Value::from(deletion_protection), Value::from(status_transition_at), ]; + // WarmThroughput as sent, reported back and acted on by nothing (see + // catalog migration V006). UPDATING for the same delay the CREATING to + // ACTIVE flip takes. Named only when sent, for the same reason as the + // restore columns: a catalog that has not run V006 still serves an + // ordinary CreateTable. + let warm_record = + warm_throughput_record_for_create(input.warm_throughput.as_ref(), delay_seconds); + if let Some(record) = warm_record.as_ref() { + columns.push_str(", warm_throughput"); + values.push(Value::from(warm_throughput_record_json(record)?.as_str())); + } if let Some(r) = restore { columns.push_str( ", restore_source_backup_arn, restore_source_table_arn, restore_date_time, \ @@ -336,6 +350,7 @@ impl CassandraEngine { // Insert GSI metadata let mut gsi_index_ids: Vec = Vec::new(); + let mut gsi_warm_records = Vec::new(); if let Some(gsis) = &input.global_secondary_indexes { for gsi in gsis { let gsi_ks = serde_json::to_value(&gsi.key_schema) @@ -358,26 +373,39 @@ impl CassandraEngine { .map_err(|e| StorageError::Internal(e.to_string()))?; let index_id = uuid::Uuid::new_v4().to_string(); + let gsi_warm = + warm_throughput_record_for_create(gsi.warm_throughput.as_ref(), delay_seconds); + let mut index_values = vec![ + Value::from(table_id.as_str()), + Value::from(gsi.index_name.as_str()), + Value::from(index_id.as_str()), + Value::from(gsi_ks.to_string().as_str()), + Value::from(gsi_proj.to_string().as_str()), + match gsi_pt.as_ref() { + Some(v) => Value::from(v.to_string().as_str()), + None => Value::NotSet, + }, + ]; + // Named only when sent; see the table insert above. + let (warm_column, warm_placeholder) = match gsi_warm.as_ref() { + Some(record) => { + index_values + .push(Value::from(warm_throughput_record_json(record)?.as_str())); + (", warm_throughput", ", ?") + } + None => ("", ""), + }; + gsi_warm_records.push(gsi_warm); let insert_index_cql = format!( "INSERT INTO {catalog_keyspace}.indexes (table_id, index_name, index_id, index_type, \ - key_schema, projection, index_status, provisioned_throughput) \ - VALUES (?, ?, ?, 'GSI', ?, ?, 'ACTIVE', ?)" + key_schema, projection, index_status, provisioned_throughput{warm_column}) \ + VALUES (?, ?, ?, 'GSI', ?, ?, 'ACTIVE', ?{warm_placeholder})" ); self.session .query_with_values( &insert_index_cql, - cdrs_tokio::query::QueryValues::SimpleValues(vec![ - Value::from(table_id.as_str()), - Value::from(gsi.index_name.as_str()), - Value::from(index_id.as_str()), - Value::from(gsi_ks.to_string().as_str()), - Value::from(gsi_proj.to_string().as_str()), - match gsi_pt.as_ref() { - Some(v) => Value::from(v.to_string().as_str()), - None => Value::NotSet, - }, - ]), + cdrs_tokio::query::QueryValues::SimpleValues(index_values), ) .await .map_err(|e| { @@ -513,7 +541,8 @@ impl CassandraEngine { let gsis = input.global_secondary_indexes.as_ref().map(|gs| { gs.iter() - .map(|g| GsiDescription { + .zip(gsi_warm_records) + .map(|(g, warm)| GsiDescription { index_name: g.index_name.clone(), key_schema: g.key_schema.clone(), projection: g.projection.clone(), @@ -539,6 +568,8 @@ impl CassandraEngine { &input.table_name, &g.index_name, ), + warm_throughput: None, + warm_throughput_record: warm, }) .collect() }); @@ -644,6 +675,9 @@ impl CassandraEngine { restore_in_progress: true, }), vector_indexes: None, + // Derived by the engine from the record; see core. + warm_throughput: None, + warm_throughput_record: warm_record, }; Ok((description, heartbeat)) } diff --git a/crates/storage-cassandra/src/lib.rs b/crates/storage-cassandra/src/lib.rs index 8f49e9664..e4ba4c8f5 100644 --- a/crates/storage-cassandra/src/lib.rs +++ b/crates/storage-cassandra/src/lib.rs @@ -7,7 +7,7 @@ //! as the underlying database. It provides a DynamoDB-compatible API backed //! by Cassandra's distributed architecture. -pub const CATALOG_VERSION: &str = "0.0.4"; +pub const CATALOG_VERSION: &str = "0.0.5"; mod admin_store; mod authorization_store; @@ -33,6 +33,7 @@ pub mod table_engine; mod table_helpers; pub mod ttl_worker; mod update_table; +mod warm_throughput; mod worker_store; pub mod workers; diff --git a/crates/storage-cassandra/src/migrations.rs b/crates/storage-cassandra/src/migrations.rs index b1bc4fe20..4f70394fb 100644 --- a/crates/storage-cassandra/src/migrations.rs +++ b/crates/storage-cassandra/src/migrations.rs @@ -45,7 +45,7 @@ type CassandraSession = Session< >; /// Embedded catalog migration files, applied in order. -pub(crate) const CATALOG_MIGRATIONS: &[(&str, &str)] = &[ +pub const CATALOG_MIGRATIONS: &[(&str, &str)] = &[ ( "V001__initial_schema.cql", include_str!("../migrations/catalog/V001__initial_schema.cql"), @@ -66,6 +66,10 @@ pub(crate) const CATALOG_MIGRATIONS: &[(&str, &str)] = &[ "V005__backup_definitions_and_restore_registration.cql", include_str!("../migrations/catalog/V005__backup_definitions_and_restore_registration.cql"), ), + ( + "V006__warm_throughput.cql", + include_str!("../migrations/catalog/V006__warm_throughput.cql"), + ), ]; /// Embedded data migration files, applied in order. diff --git a/crates/storage-cassandra/src/table_helpers.rs b/crates/storage-cassandra/src/table_helpers.rs index 94b6eb0cf..b24531e2d 100644 --- a/crates/storage-cassandra/src/table_helpers.rs +++ b/crates/storage-cassandra/src/table_helpers.rs @@ -10,7 +10,7 @@ use extenddb_core::types::{ TableStatus, }; use extenddb_storage::error::StorageError; -use extenddb_storage::util::{index_arn, stream_arn}; +use extenddb_storage::util::{index_arn, parse_warm_throughput_record, stream_arn}; use crate::CassandraEngine; @@ -165,6 +165,9 @@ impl CassandraEngine { .ok() .flatten() .and_then(|s: String| serde_json::from_str(&s).ok()); + // Absent on a catalog that has not run V006 (SELECT * tolerates it). + let warm_json: Option = table_row.get_by_name("warm_throughput").ok().flatten(); + let warm_throughput_record = parse_warm_throughput_record(warm_json.as_deref())?; // A restore target reports its registration for as long as the row // carries it; the engine shows it only while the table is CREATING. @@ -218,6 +221,7 @@ impl CassandraEngine { if index_type == "GSI" { let pt_str: Option = row.get_r_by_name("provisioned_throughput").ok(); + let warm_json: Option = row.get_by_name("warm_throughput").ok().flatten(); gsis.push(GsiDescription { index_name: index_name.clone(), @@ -228,6 +232,8 @@ impl CassandraEngine { index_size_bytes: 0, item_count: 0, index_arn: index_arn(&self.region, account_id, &table_name, &index_name), + warm_throughput: None, + warm_throughput_record: parse_warm_throughput_record(warm_json.as_deref())?, }); } else { lsis.push(LsiDescription { @@ -301,6 +307,8 @@ impl CassandraEngine { on_demand_throughput, restore_summary, vector_indexes: None, + warm_throughput: None, + warm_throughput_record, }) } } diff --git a/crates/storage-cassandra/src/update_table.rs b/crates/storage-cassandra/src/update_table.rs index 4a20b8618..187c6fa36 100644 --- a/crates/storage-cassandra/src/update_table.rs +++ b/crates/storage-cassandra/src/update_table.rs @@ -12,7 +12,9 @@ use extenddb_core::types::{ AttributeDefinition, BillingMode, KeySchemaElement, TableDescription, UpdateTableInput, }; use extenddb_storage::error::StorageError; -use extenddb_storage::util::effective_attribute_definitions; +use extenddb_storage::util::{ + effective_attribute_definitions, warm_throughput_record_for_create, warm_throughput_record_json, +}; use crate::CassandraEngine; use crate::cassandra_util::query_optional; @@ -151,6 +153,59 @@ impl CassandraEngine { add_update!("on_demand_throughput", odt_json.as_str()); } + // WarmThroughput on the table and on updated indexes: stored as sent, + // nothing acts on it. Validated by the engine against the current values. + // + // A request with a BillingMode and a WarmThroughput is the one the + // engine builds for a switch to PAY_PER_REQUEST (a client cannot send + // both). It must not fail on a catalog that has not run V006, where + // the column is absent: the switch is skipped past the warm throughput + // and the reported value still follows the on-demand minimum at read + // time. A request that names WarmThroughput on its own fails there, as + // documented. + let warm_json = match &input.warm_throughput { + Some(spec) => { + match self + .merged_table_warm_throughput(account_id, &input.table_name, spec) + .await + { + Ok(json) => Some(json), + Err(e) if input.billing_mode.is_some() => { + tracing::warn!( + "update_table: warm throughput not stored with the billing mode \ + change (catalog migration V006 not applied?): {e}" + ); + None + } + Err(e) => return Err(e), + } + } + None => None, + }; + if let Some(json) = warm_json.as_deref() { + add_update!("warm_throughput", json); + } + for update in input.global_secondary_index_updates.iter().flatten() { + if let Some(action) = &update.update + && let Some(spec) = &action.warm_throughput + { + let json = self + .merged_index_warm_throughput(&table_id, &action.index_name, spec) + .await?; + add_index_op!( + format!( + "UPDATE {catalog_ks}.indexes SET warm_throughput = ? \ + WHERE table_id = ? AND index_name = ?" + ), + QueryValues::SimpleValues(vec![ + Value::from(json.as_str()), + Value::from(table_id.as_str()), + Value::from(action.index_name.as_str()), + ]) + ); + } + } + // Stream specification: add to batch, then handle shard init separately // (init_stream_shards writes to a different keyspace and does its own batch). let mut needs_shard_init = false; @@ -272,21 +327,35 @@ impl CassandraEngine { .map_err(|e| StorageError::Internal(e.to_string()))? .unwrap_or_default(); + let mut index_values = vec![ + Value::from(table_id.as_str()), + Value::from(create.index_name.as_str()), + Value::from(index_id.as_str()), + Value::from(idx_ks_json.as_str()), + Value::from(proj_json.as_str()), + Value::from(pt_json.as_str()), + ]; + // WarmThroughput named only when sent, as in create_table. + let gsi_warm = warm_throughput_record_for_create( + create.warm_throughput.as_ref(), + self.control_plane_delay_secs().await, + ); + let (warm_column, warm_placeholder) = match gsi_warm.as_ref() { + Some(record) => { + index_values + .push(Value::from(warm_throughput_record_json(record)?.as_str())); + (", warm_throughput", ", ?") + } + None => ("", ""), + }; add_index_op!( format!( "INSERT INTO {catalog_ks}.indexes \ (table_id, index_name, index_id, index_type, key_schema, \ - projection, index_status, provisioned_throughput) \ - VALUES (?, ?, ?, 'GSI', ?, ?, 'CREATING', ?)" + projection, index_status, provisioned_throughput{warm_column}) \ + VALUES (?, ?, ?, 'GSI', ?, ?, 'CREATING', ?{warm_placeholder})" ), - QueryValues::SimpleValues(vec![ - Value::from(table_id.as_str()), - Value::from(create.index_name.as_str()), - Value::from(index_id.as_str()), - Value::from(idx_ks_json.as_str()), - Value::from(proj_json.as_str()), - Value::from(pt_json.as_str()), - ]) + QueryValues::SimpleValues(index_values) ); gsi_creates.push((index_id, create.index_name.clone())); surviving_index_key_schemas.push(create.key_schema.clone()); diff --git a/crates/storage-cassandra/src/warm_throughput.rs b/crates/storage-cassandra/src/warm_throughput.rs new file mode 100644 index 000000000..7581b9d3d --- /dev/null +++ b/crates/storage-cassandra/src/warm_throughput.rs @@ -0,0 +1,107 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! Stored `WarmThroughput` for the Cassandra backend: the control-plane delay +//! the `UPDATING` window takes, and the `UpdateTable` merges. +//! +//! The record is written as sent (one JSON text per table and per index, see +//! catalog migration V006) and read back by DescribeTable; nothing pre-warms +//! or scales on it, and it has no effect on throughput, throttling or +//! storage. See `extenddb_core::types::WarmThroughputRecord`. + +use cdrs_tokio::types::IntoRustByName; +use extenddb_core::types::WarmThroughput; +use extenddb_storage::error::StorageError; +use extenddb_storage::util::{warm_throughput_record_for_update, warm_throughput_record_json}; + +use crate::CassandraEngine; +use crate::cassandra_util::query_optional; + +impl CassandraEngine { + /// `control_plane_delay_seconds` from the settings table (0.25 when + /// unset): the window a fresh record reports `UPDATING` for, the same + /// one the CREATING to ACTIVE flip uses. + pub(crate) async fn control_plane_delay_secs(&self) -> f64 { + let catalog_keyspace = self.catalog_keyspace(); + let query = format!("SELECT value FROM {catalog_keyspace}.settings WHERE key = ?"); + query_optional::( + &self.session, + &query, + cdrs_tokio::query_values!("control_plane_delay_seconds"), + "control_plane_delay_seconds", + ) + .await + .ok() + .flatten() + .and_then(|row| { + let value: String = row.get_r_by_name("value").ok()?; + value.parse::().ok() + }) + .unwrap_or(0.25) + } + + /// The JSON an `UpdateTable` stores in `tables.warm_throughput`: the + /// request merged over the current record. Read as its own statement so + /// a catalog that has not run V006 only fails a request that names the + /// member. + pub(crate) async fn merged_table_warm_throughput( + &self, + account_id: &str, + table_name: &str, + spec: &WarmThroughput, + ) -> Result { + let catalog_ks = self.catalog_keyspace(); + let row = query_optional::( + &self.session, + &format!( + "SELECT warm_throughput FROM {catalog_ks}.tables \ + WHERE account_id = ? AND table_name = ?" + ), + cdrs_tokio::query_values!(account_id, table_name), + "update_table warm throughput", + ) + .await? + .ok_or_else(|| StorageError::TableNotFound(table_name.to_owned()))?; + let existing: Option = row.get_by_name("warm_throughput").ok().flatten(); + let record = warm_throughput_record_for_update( + existing.as_deref(), + spec, + self.control_plane_delay_secs().await, + )?; + warm_throughput_record_json(&record) + } + + /// The JSON an `UpdateTable` stores in `indexes.warm_throughput` for a + /// `GlobalSecondaryIndexUpdates.Update`: the request merged over the + /// index's current record. `IndexNotFound` when the table has no such GSI. + pub(crate) async fn merged_index_warm_throughput( + &self, + table_id: &str, + index_name: &str, + spec: &WarmThroughput, + ) -> Result { + let catalog_ks = self.catalog_keyspace(); + let row = query_optional::( + &self.session, + &format!( + "SELECT index_type, warm_throughput FROM {catalog_ks}.indexes \ + WHERE table_id = ? AND index_name = ?" + ), + cdrs_tokio::query_values!(table_id, index_name), + "update_table gsi warm throughput", + ) + .await? + .ok_or_else(|| StorageError::IndexNotFound(index_name.to_owned()))?; + let index_type: Option = row.get_by_name("index_type").ok().flatten(); + if index_type.as_deref() != Some("GSI") { + return Err(StorageError::IndexNotFound(index_name.to_owned())); + } + let existing: Option = row.get_by_name("warm_throughput").ok().flatten(); + let record = warm_throughput_record_for_update( + existing.as_deref(), + spec, + self.control_plane_delay_secs().await, + )?; + warm_throughput_record_json(&record) + } +} diff --git a/crates/storage-cassandra/tests/common/mod.rs b/crates/storage-cassandra/tests/common/mod.rs index c1fd1e971..5a367bb51 100644 --- a/crates/storage-cassandra/tests/common/mod.rs +++ b/crates/storage-cassandra/tests/common/mod.rs @@ -262,6 +262,7 @@ impl TestTable { tags: None, table_class: None, deletion_protection_enabled: None, + warm_throughput: None, }; let table_desc = engine.create_table(account_id, input).await.unwrap(); @@ -326,6 +327,7 @@ impl TestTable { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }; let input = extenddb_core::types::CreateTableInput { @@ -344,6 +346,7 @@ impl TestTable { tags: None, table_class: None, deletion_protection_enabled: None, + warm_throughput: None, }; let table_desc = engine.create_table(&account_id, input).await.unwrap(); @@ -419,6 +422,7 @@ impl TestTable { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }; let input = extenddb_core::types::CreateTableInput { @@ -437,6 +441,7 @@ impl TestTable { tags: None, table_class: None, deletion_protection_enabled: None, + warm_throughput: None, }; let table_desc = engine.create_table(&account_id, input).await.unwrap(); @@ -534,6 +539,7 @@ impl TestTable { tags: None, table_class: None, deletion_protection_enabled: None, + warm_throughput: None, }; let table_desc = engine.create_table(&account_id, input).await.unwrap(); diff --git a/crates/storage-cassandra/tests/direct/backup_engine.rs b/crates/storage-cassandra/tests/direct/backup_engine.rs index 30fe919ac..ff5f8a663 100644 --- a/crates/storage-cassandra/tests/direct/backup_engine.rs +++ b/crates/storage-cassandra/tests/direct/backup_engine.rs @@ -48,21 +48,21 @@ fn key(id: &str, sort: Option<&str>) -> Item { key } -fn ks(name: &str, key_type: KeyType) -> KeySchemaElement { +pub(crate) fn ks(name: &str, key_type: KeyType) -> KeySchemaElement { KeySchemaElement { attribute_name: name.to_owned(), key_type, } } -fn attr(name: &str, attribute_type: ScalarAttributeType) -> AttributeDefinition { +pub(crate) fn attr(name: &str, attribute_type: ScalarAttributeType) -> AttributeDefinition { AttributeDefinition { attribute_name: name.to_owned(), attribute_type, } } -async fn activate_tables(engine: &CassandraEngine) { +pub(crate) async fn activate_tables(engine: &CassandraEngine) { tokio::time::sleep(Duration::from_millis(350)).await; engine .process_control_plane_transitions() @@ -113,7 +113,11 @@ async fn put_all(engine: &CassandraEngine, key_info: &TableKeyInfo, items: Vec TableDescription { +pub(crate) async fn describe( + engine: &CassandraEngine, + account_id: &str, + name: &str, +) -> TableDescription { engine .describe_table( account_id, @@ -657,6 +661,7 @@ async fn test_restore_reproduces_gsi_and_lsi_contents() { non_key_attributes: Some(vec!["value".to_owned()]), }, provisioned_throughput: None, + warm_throughput: None, }]), local_secondary_indexes: Some(vec![LsiInput { index_name: "ByLocal".to_owned(), @@ -1360,6 +1365,7 @@ async fn test_backup_without_definition_restores_as_legacy() { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }]), ..Default::default() }, @@ -1571,26 +1577,42 @@ async fn test_restore_of_deleted_backup_is_not_found() { // ── Isolated catalogs (own keyspace prefix) ──────────────────────────────── -/// Catalog migrations V001-V004: the catalog shape before V005 added -/// `backups_by_arn.definition/active_restores` and `tables.restore_*`. -const CATALOG_THROUGH_V004: &[&str] = &[ +/// The catalog migration files in order, so a test can build the catalog +/// shape of an earlier release by applying the first N statement by +/// statement, as the runner does. +pub(crate) const CATALOG_MIGRATION_FILES: &[&str] = &[ include_str!("../../migrations/catalog/V001__initial_schema.cql"), include_str!("../../migrations/catalog/V002__backup_restore.cql"), include_str!("../../migrations/catalog/V003__account_scoped_idempotency.cql"), include_str!("../../migrations/catalog/V004__ttl_backfill_cursor.cql"), + include_str!("../../migrations/catalog/V005__backup_definitions_and_restore_registration.cql"), ]; /// An engine on its own keyspace prefix (so its sweeps and workers touch no /// other test's tables) with one provisioned account. With `through_v004` /// the catalog has run only catalog migrations V001-V004, as a catalog not /// yet migrated by `extenddb migrate --yes`; otherwise every migration. -async fn isolated_engine( +pub(crate) async fn isolated_engine( tag: &str, through_v004: bool, ) -> ( std::sync::Arc, String, crate::helpers::DropKeyspacesOnDrop, +) { + isolated_engine_through(tag, if through_v004 { Some(4) } else { None }).await +} + +/// [`isolated_engine`] with the catalog at the shape after the first +/// `through` catalog migrations (recorded in `schema_history` as the runner +/// records them), or fully migrated when `None`. +pub(crate) async fn isolated_engine_through( + tag: &str, + through: Option, +) -> ( + std::sync::Arc, + String, + crate::helpers::DropKeyspacesOnDrop, ) { let mut config = crate::helpers::test_config(); config.keyspace_prefix = format!("f4{tag}_{}", &crate::helpers::unique_test_id()[..10]); @@ -1609,26 +1631,29 @@ async fn isolated_engine( .await .unwrap(); } - if through_v004 { + if let Some(through) = through { // Statement by statement, as the migration runner does, recording - // versions 1-4 in schema_history as the runner did for a 0.0.3 catalog. + // the applied versions in schema_history as the runner did. session.query(format!("USE {catalog_ks}")).await.unwrap(); - for file in CATALOG_THROUGH_V004 { + for file in &CATALOG_MIGRATION_FILES[..through] { for statement in file.split(';').map(str::trim).filter(|s| !s.is_empty()) { - session.query(statement).await.expect("V001-V004 statement"); + session + .query(statement) + .await + .expect("earlier catalog statement"); } } - for version in 1..=4_i32 { + for version in 1..=i32::try_from(through).expect("small") { session .query_with_values( format!( "INSERT INTO {catalog_ks}.schema_history (version, description, applied_at) \ - VALUES (?, 'through_v004', toTimestamp(now()))" + VALUES (?, 'earlier_release', toTimestamp(now()))" ), cdrs_tokio::query_values!(version), ) .await - .expect("record V001-V004"); + .expect("record earlier catalog versions"); } } else { extenddb_storage_cassandra::migrations::run_catalog_migrations(&session, &catalog_ks) @@ -1651,8 +1676,8 @@ async fn isolated_engine( (engine, account_id, drop_keyspaces) } -/// The columns catalog migration V005 adds are absent from `keyspace`. -async fn assert_catalog_lacks_v005(engine: &CassandraEngine, keyspace: &str) { +/// The column names of every table in `keyspace`. +pub(crate) async fn catalog_columns(engine: &CassandraEngine, keyspace: &str) -> Vec { let rows = extenddb_storage_cassandra::cassandra_util::query_rows::( &engine.session_arc(), "SELECT column_name FROM system_schema.columns WHERE keyspace_name = ?", @@ -1672,6 +1697,12 @@ async fn assert_catalog_lacks_v005(engine: &CassandraEngine, keyspace: &str) { .unwrap() }) .collect(); + columns +} + +/// The columns catalog migration V005 adds are absent from `keyspace`. +async fn assert_catalog_lacks_v005(engine: &CassandraEngine, keyspace: &str) { + let columns = catalog_columns(engine, keyspace).await; assert!(columns.iter().any(|c| c == "table_status"), "{columns:?}"); for v005 in [ "restore_source_backup_arn", @@ -1713,6 +1744,7 @@ async fn test_table_operations_work_on_a_catalog_without_v005() { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }]), ..Default::default() }, @@ -1979,6 +2011,7 @@ async fn test_restore_heartbeat_runs_during_target_ddl() { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }); lsis.push(LsiInput { index_name: format!("L{i}"), diff --git a/crates/storage-cassandra/tests/direct/cassandra_engine.rs b/crates/storage-cassandra/tests/direct/cassandra_engine.rs index 52a5306c6..abf431cc1 100644 --- a/crates/storage-cassandra/tests/direct/cassandra_engine.rs +++ b/crates/storage-cassandra/tests/direct/cassandra_engine.rs @@ -125,6 +125,7 @@ mod tests { deletion_protection_enabled: None, table_class: None, tags: None, + warm_throughput: None, }; // Create table diff --git a/crates/storage-cassandra/tests/direct/streams.rs b/crates/storage-cassandra/tests/direct/streams.rs index 865774d3a..496f338f4 100644 --- a/crates/storage-cassandra/tests/direct/streams.rs +++ b/crates/storage-cassandra/tests/direct/streams.rs @@ -65,6 +65,7 @@ async fn setup_stream_table( tags: None, table_class: None, deletion_protection_enabled: None, + warm_throughput: None, }; let desc = engine.create_table(&account_id, input).await.unwrap(); diff --git a/crates/storage-cassandra/tests/direct/table_engine.rs b/crates/storage-cassandra/tests/direct/table_engine.rs index 1005f113d..2f3fbdc17 100644 --- a/crates/storage-cassandra/tests/direct/table_engine.rs +++ b/crates/storage-cassandra/tests/direct/table_engine.rs @@ -61,6 +61,7 @@ mod tests { tags: None, deletion_protection_enabled: None, table_class: None, + warm_throughput: None, }; engine diff --git a/crates/storage-cassandra/tests/direct/warm_throughput_catalog.rs b/crates/storage-cassandra/tests/direct/warm_throughput_catalog.rs new file mode 100644 index 000000000..07fd629c9 --- /dev/null +++ b/crates/storage-cassandra/tests/direct/warm_throughput_catalog.rs @@ -0,0 +1,158 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! Catalog migration V006 (the stored `WarmThroughput` columns) on a catalog +//! that has not run it: the shape a deployment migrated by the previous +//! release has. +//! +//! Ordinary table and item operations work there, as they do without V005. +//! A request that names the member (`CreateTable` or `UpdateTable` with +//! `WarmThroughput`) needs the column and fails until `extenddb migrate +//! --yes` applies V006; `DescribeTable` reports the defaults. The catalog +//! version constant moves to 0.0.5 so `extenddb migrate` sees the catalog as +//! behind and applies V006 (without the bump the command reported nothing to +//! do while the backup paths listed V006 as pending and refused). + +use extenddb_core::types::{ + CreateTableInput, DeleteTableInput, TableStatus, UpdateTableInput, WarmThroughputStatus, +}; +use extenddb_storage::TableEngine; +use extenddb_storage_cassandra::CATALOG_VERSION; + +use crate::backup_engine::{activate_tables, catalog_columns, describe, isolated_engine_through}; + +/// The constant and the last migration file agree, the same tripwire the +/// PostgreSQL runner has: adding a migration without moving the version +/// leaves `extenddb migrate` with nothing to do. +#[test] +fn the_catalog_version_is_the_one_v006_writes() { + assert_eq!(CATALOG_VERSION, "0.0.5"); + let v006 = include_str!("../../migrations/catalog/V006__warm_throughput.cql"); + assert!( + v006.contains("('catalog_version', '0.0.5')"), + "V006 must stamp the catalog version the binary expects" + ); + assert_eq!( + extenddb_storage_cassandra::migrations::CATALOG_MIGRATIONS.len(), + 6, + "a catalog migration was added or removed; move CATALOG_VERSION and this count" + ); +} + +#[tokio::test] +async fn test_table_operations_work_on_a_catalog_without_v006() { + if crate::helpers::skip_without_cassandra() { + return; + } + let (engine, account_id, _drop_keyspaces) = isolated_engine_through("v5", Some(5)).await; + let columns = catalog_columns(&engine, &engine.catalog_keyspace()).await; + assert!( + columns.iter().any(|c| c == "restore_heartbeat"), + "V005 present" + ); + assert!( + !columns.iter().any(|c| c == "warm_throughput"), + "V006 must not have run: {columns:?}" + ); + + let input = |name: &str, warm: Option| -> CreateTableInput { + let mut v = serde_json::json!({ + "TableName": name, + "KeySchema": [{"AttributeName": "id", "KeyType": "HASH"}], + "AttributeDefinitions": [ + {"AttributeName": "id", "AttributeType": "S"}, + {"AttributeName": "g", "AttributeType": "S"} + ], + "BillingMode": "PAY_PER_REQUEST", + "GlobalSecondaryIndexes": [{ + "IndexName": "ByG", + "KeySchema": [{"AttributeName": "g", "KeyType": "HASH"}], + "Projection": {"ProjectionType": "ALL"} + }] + }); + if let Some(w) = warm { + v["WarmThroughput"] = w; + } + serde_json::from_value(v).expect("input") + }; + // A CreateTable without the member works and describes the defaults. + engine + .create_table(&account_id, input("Unmigrated", None)) + .await + .expect("CreateTable without WarmThroughput on a catalog without V006"); + activate_tables(&engine).await; + let mut described = describe(&engine, &account_id, "Unmigrated").await; + assert_eq!(described.table_status, TableStatus::Active); + described.prepare_for_response(); + let wt = described.warm_throughput.expect("defaults"); + assert_eq!( + ( + wt.read_units_per_second, + wt.write_units_per_second, + wt.status + ), + (Some(12_000), Some(4_000), WarmThroughputStatus::Active) + ); + + // Naming the member needs the column. + let refused = engine + .create_table( + &account_id, + input( + "NeedsV006", + Some(serde_json::json!({"ReadUnitsPerSecond": 20000})), + ), + ) + .await; + assert!( + refused.is_err(), + "CreateTable with WarmThroughput before V006" + ); + let update: UpdateTableInput = serde_json::from_value(serde_json::json!({ + "TableName": "Unmigrated", + "WarmThroughput": {"ReadUnitsPerSecond": 20000} + })) + .expect("update"); + assert!( + engine.update_table(&account_id, update).await.is_err(), + "UpdateTable with WarmThroughput before V006" + ); + + // `extenddb migrate --yes` applies V006 (the runner skips the recorded + // files); the same engine then stores the member without a restart. + extenddb_storage_cassandra::migrations::run_catalog_migrations( + &engine.session_arc(), + &engine.catalog_keyspace(), + ) + .await + .expect("apply V006"); + let columns = catalog_columns(&engine, &engine.catalog_keyspace()).await; + assert_eq!( + columns.iter().filter(|c| *c == "warm_throughput").count(), + 2, + "tables and indexes both gain the column" + ); + let update: UpdateTableInput = serde_json::from_value(serde_json::json!({ + "TableName": "Unmigrated", + "WarmThroughput": {"ReadUnitsPerSecond": 20000} + })) + .expect("update"); + let mut updated = engine + .update_table(&account_id, update) + .await + .expect("UpdateTable with WarmThroughput after V006"); + updated.prepare_for_response(); + assert_eq!( + updated.warm_throughput.expect("member").status, + WarmThroughputStatus::Updating + ); + engine + .delete_table( + &account_id, + DeleteTableInput { + table_name: "Unmigrated".to_owned(), + }, + ) + .await + .expect("DeleteTable"); +} diff --git a/crates/storage-cassandra/tests/direct_integration.rs b/crates/storage-cassandra/tests/direct_integration.rs index 43eb28b81..77b27f4d1 100644 --- a/crates/storage-cassandra/tests/direct_integration.rs +++ b/crates/storage-cassandra/tests/direct_integration.rs @@ -57,3 +57,5 @@ mod transact_write_items; mod transaction_ledger; #[path = "direct/users.rs"] mod users; +#[path = "direct/warm_throughput_catalog.rs"] +mod warm_throughput_catalog; diff --git a/crates/storage-cassandra/tests/ttl_integration.rs b/crates/storage-cassandra/tests/ttl_integration.rs index 279d79492..c9c106bb5 100644 --- a/crates/storage-cassandra/tests/ttl_integration.rs +++ b/crates/storage-cassandra/tests/ttl_integration.rs @@ -513,6 +513,7 @@ async fn test_ttl_sweep_emits_service_remove_stream_record() { tags: None, table_class: None, deletion_protection_enabled: None, + warm_throughput: None, }, ) .await @@ -3104,6 +3105,7 @@ async fn test_ttl_enabled_table_rejects_new_async_gsi() { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }), update: None, delete: None, @@ -3122,6 +3124,8 @@ async fn test_ttl_enabled_table_rejects_new_async_gsi() { table_class: None, on_demand_throughput: None, vector_index_updates: None, + warm_throughput: None, + sse_specification: None, }, ) .await; diff --git a/crates/storage-mongodb/src/backup_engine.rs b/crates/storage-mongodb/src/backup_engine.rs index beedee97c..17aee41fe 100644 --- a/crates/storage-mongodb/src/backup_engine.rs +++ b/crates/storage-mongodb/src/backup_engine.rs @@ -1826,6 +1826,7 @@ impl BackupEngine for MongoEngine { &index_doc, "provisioned_throughput", )?, + warm_throughput: None, }); } "LSI" => { @@ -2438,6 +2439,7 @@ mod tests { read_capacity_units: 7, write_capacity_units: 8, }), + warm_throughput: None, }; let kept = restore_gsi_throughput("PROVISIONED", Some(vec![gsi.clone()])).expect("gsis"); assert!(kept[0].provisioned_throughput.is_some()); diff --git a/crates/storage-mongodb/src/backup_live_tests.rs b/crates/storage-mongodb/src/backup_live_tests.rs index 75ed23761..8c69b381e 100644 --- a/crates/storage-mongodb/src/backup_live_tests.rs +++ b/crates/storage-mongodb/src/backup_live_tests.rs @@ -167,6 +167,7 @@ async fn source_table_for(engine: &MongoEngine, account: &str, name: &str, items non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }]), local_secondary_indexes: Some(vec![LsiInput { index_name: "l1".to_owned(), @@ -1688,6 +1689,8 @@ async fn live_update_table_on_a_creating_restore_target_is_refused() { table_class: None, on_demand_throughput: None, vector_index_updates: None, + warm_throughput: None, + sse_specification: None, }; let changes = vec![ UpdateTableInput { @@ -1720,6 +1723,7 @@ async fn live_update_table_on_a_creating_restore_target_is_refused() { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }), update: None, delete: None, diff --git a/crates/storage-mongodb/src/lib.rs b/crates/storage-mongodb/src/lib.rs index c1d28882f..459f97829 100644 --- a/crates/storage-mongodb/src/lib.rs +++ b/crates/storage-mongodb/src/lib.rs @@ -24,6 +24,7 @@ pub mod pushdown; mod stream_engine; mod table_engine; mod ttl_worker; +mod warm_throughput; mod worker_store; pub use bootstrapper::MongoBootstrapper; diff --git a/crates/storage-mongodb/src/table_engine.rs b/crates/storage-mongodb/src/table_engine.rs index aa8c5272c..9e8b31eca 100644 --- a/crates/storage-mongodb/src/table_engine.rs +++ b/crates/storage-mongodb/src/table_engine.rs @@ -218,6 +218,12 @@ impl MongoEngine { // table return ResourceNotFound, matching DynamoDB and the postgres // backend. let delay_secs = self.control_plane_delay_seconds().await; + // WarmThroughput as sent, reported back and acted on by nothing. + // UPDATING for the same delay the CREATING to ACTIVE flip takes. + let (warm_record, warm_bson) = crate::warm_throughput::warm_record_bson_for_create( + input.warm_throughput.as_ref(), + delay_secs, + )?; let (table_status, status_transition_at): (&str, bson::Bson) = if defer_active { // Restore path: enter CREATING with no scheduled transition. The // caller flips the table to ACTIVE (or schedules the transition) @@ -254,6 +260,7 @@ impl MongoEngine { "table_class": table_class_bson, "sse_specification": sse_spec_bson, "on_demand_throughput": on_demand_bson, + "warm_throughput": warm_bson, }; if let Some(provenance) = restore { provenance.insert_into(&mut table_doc); @@ -351,6 +358,11 @@ impl MongoEngine { let index_status = if defer_active { "CREATING" } else { "ACTIVE" }; let backfill_mode = if defer_active { Some("restore") } else { None }; + let (gsi_warm_record, gsi_warm_bson) = + crate::warm_throughput::warm_record_bson_for_create( + gsi.warm_throughput.as_ref(), + delay_secs, + )?; let mut index_doc = doc! { "_id": { "table_id": &table_id, "index_name": &gsi.index_name }, "index_id": &index_id, @@ -359,6 +371,7 @@ impl MongoEngine { "projection": projection_bson, "index_status": index_status, "provisioned_throughput": index_pt_bson.unwrap_or(bson::Bson::Null), + "warm_throughput": gsi_warm_bson, }; if let Some(mode) = backfill_mode { index_doc.insert("backfill_mode", mode); @@ -395,6 +408,8 @@ impl MongoEngine { index_size_bytes: 0, item_count: 0, index_arn: index_arn_val, + warm_throughput: None, + warm_throughput_record: gsi_warm_record, }); } Some(descs) @@ -557,6 +572,7 @@ impl MongoEngine { sse_description, table_class_summary, on_demand_throughput: input.on_demand_throughput, + warm_throughput_record: warm_record, // Fields for features this backend does not implement, vector // indexes today, take their defaults. Adding one to // TableDescription then does not break this build. @@ -879,6 +895,11 @@ impl MongoEngine { } } + // WarmThroughput on the table and on updated indexes: stored as sent, + // nothing acts on it. Validated by the engine against the current values. + self.store_warm_throughput_updates(account_id, &input) + .await?; + if clear_provisioned_throughput { let table_doc = tables_coll .find_one(doc! { @@ -977,6 +998,10 @@ impl MongoEngine { // Live writes during the backfill window sync through // sync_indexes (upserts) so the eventual state is // convergent regardless of interleaving. + let (_, gsi_warm_bson) = crate::warm_throughput::warm_record_bson_for_create( + create.warm_throughput.as_ref(), + self.control_plane_delay_seconds().await, + )?; let index_doc = doc! { "_id": { "table_id": &desc.table_id, "index_name": &create.index_name }, "index_id": &index_id, @@ -985,6 +1010,7 @@ impl MongoEngine { "projection": projection_bson, "index_status": "CREATING", "provisioned_throughput": pt_bson.unwrap_or(bson::Bson::Null), + "warm_throughput": gsi_warm_bson, }; self.catalog_db @@ -1548,6 +1574,10 @@ impl MongoEngine { index_size_bytes: 0, item_count: 0, index_arn: idx_arn, + warm_throughput: None, + warm_throughput_record: crate::warm_throughput::warm_record_from_doc( + idx_doc, + )?, }); } "LSI" => { @@ -1644,6 +1674,7 @@ impl MongoEngine { table_class_summary, on_demand_throughput, restore_summary, + warm_throughput_record: crate::warm_throughput::warm_record_from_doc(doc)?, // Fields for features this backend does not implement, vector // indexes today, take their defaults. Adding one to // TableDescription then does not break this build. diff --git a/crates/storage-mongodb/src/warm_throughput.rs b/crates/storage-mongodb/src/warm_throughput.rs new file mode 100644 index 000000000..be1605e92 --- /dev/null +++ b/crates/storage-mongodb/src/warm_throughput.rs @@ -0,0 +1,169 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! Stored `WarmThroughput` for `MongoEngine`: the BSON shape of the record on +//! table and index documents, and the `UpdateTable` merges. +//! +//! The record is written as sent and read back by DescribeTable; nothing +//! pre-warms or scales on it, and it has no effect on throughput, throttling +//! or storage. See `extenddb_core::types::WarmThroughputRecord`. + +use bson::{Bson, Document, doc}; +use extenddb_core::types::{ + UpdateTableInput, WarmThroughput, WarmThroughputRecord, epoch_seconds_now, +}; +use extenddb_storage::error::StorageError; +use extenddb_storage::util::warm_throughput_record_for_create; + +use crate::MongoEngine; + +/// The `warm_throughput` field value for a record: the record as a document, +/// or `Null` when the client never set one. +pub(crate) fn warm_record_bson( + record: Option<&WarmThroughputRecord>, +) -> Result { + match record { + Some(r) => bson::to_bson(r).map_err(|e| StorageError::Internal(e.to_string())), + None => Ok(Bson::Null), + } +} + +/// The record on a table or index document, if any. +/// +/// # Errors +/// +/// `Internal` when the stored field is present but not a record. +pub(crate) fn warm_record_from_doc( + doc: &Document, +) -> Result, StorageError> { + match doc.get("warm_throughput") { + None | Some(Bson::Null) => Ok(None), + Some(b) => bson::from_bson(b.clone()) + .map(Some) + .map_err(|e| StorageError::Internal(format!("warm throughput record: {e}"))), + } +} + +/// The field a create path writes for a request's `WarmThroughput`. +pub(crate) fn warm_record_bson_for_create( + spec: Option<&WarmThroughput>, + delay_secs: f64, +) -> Result<(Option, Bson), StorageError> { + let record = warm_throughput_record_for_create(spec, delay_secs); + let bson = warm_record_bson(record.as_ref())?; + Ok((record, bson)) +} + +impl MongoEngine { + /// Store the request's table-level `WarmThroughput` and each + /// `GlobalSecondaryIndexUpdates.Update.WarmThroughput`, merged over what + /// is stored. Read-modify-write without a transaction, like the rest of + /// this backend's UpdateTable. The engine has already validated the + /// values against the current ones. + pub(crate) async fn store_warm_throughput_updates( + &self, + account_id: &str, + input: &UpdateTableInput, + ) -> Result<(), StorageError> { + let gsi_updates: Vec<_> = input + .global_secondary_index_updates + .iter() + .flatten() + .filter_map(|u| u.update.as_ref()) + .filter_map(|u| u.warm_throughput.as_ref().map(|wt| (&u.index_name, wt))) + .collect(); + if input.warm_throughput.is_none() && gsi_updates.is_empty() { + return Ok(()); + } + let now = epoch_seconds_now(); + let updating_until = now + self.control_plane_delay_seconds().await.max(0.0); + let tables_coll = self.catalog_db.collection::("tables"); + let table_filter = + doc! { "_id": { "account_id": account_id, "table_name": &input.table_name } }; + let table_doc = tables_coll + .find_one(table_filter.clone()) + .await + .map_err(|e| StorageError::Internal(e.to_string()))? + .ok_or_else(|| StorageError::TableNotFound(input.table_name.clone()))?; + + if let Some(spec) = &input.warm_throughput { + let record = WarmThroughputRecord::apply_update( + warm_record_from_doc(&table_doc)?, + spec, + updating_until, + now, + ); + tables_coll + .update_one( + table_filter, + doc! { "$set": { "warm_throughput": warm_record_bson(Some(&record))? } }, + ) + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + } + + if gsi_updates.is_empty() { + return Ok(()); + } + let table_id = table_doc + .get_str("table_id") + .map_err(|_| StorageError::Internal("missing table_id".to_owned()))?; + let indexes_coll = self.catalog_db.collection::("indexes"); + for (index_name, spec) in gsi_updates { + let filter = doc! { + "_id": { "table_id": table_id, "index_name": index_name }, + "index_type": "GSI", + }; + let index_doc = indexes_coll + .find_one(filter.clone()) + .await + .map_err(|e| StorageError::Internal(e.to_string()))? + .ok_or_else(|| StorageError::IndexNotFound(index_name.clone()))?; + let record = WarmThroughputRecord::apply_update( + warm_record_from_doc(&index_doc)?, + spec, + updating_until, + now, + ); + indexes_coll + .update_one( + filter, + doc! { "$set": { "warm_throughput": warm_record_bson(Some(&record))? } }, + ) + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + } + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_record_round_trips_through_bson_and_null_reads_as_none() { + let record = WarmThroughputRecord { + read_units_per_second: Some(20_000), + write_units_per_second: None, + previous_read_units_per_second: Some(12_000), + previous_write_units_per_second: None, + updating_until: 1.5, + }; + let doc = doc! { "warm_throughput": warm_record_bson(Some(&record)).expect("bson") }; + assert_eq!(warm_record_from_doc(&doc).expect("parses"), Some(record)); + let null = doc! { "warm_throughput": Bson::Null }; + assert_eq!(warm_record_from_doc(&null).expect("parses"), None); + assert_eq!(warm_record_from_doc(&doc! {}).expect("parses"), None); + assert_eq!(warm_record_bson(None).expect("bson"), Bson::Null); + } + + #[test] + fn garbage_in_the_field_is_an_internal_error() { + let bad = doc! { "warm_throughput": "not a record" }; + assert!(matches!( + warm_record_from_doc(&bad), + Err(StorageError::Internal(m)) if m.starts_with("warm throughput record:") + )); + } +} diff --git a/crates/storage-postgres/migrations/001_schema.sql b/crates/storage-postgres/migrations/001_schema.sql index 3203651f4..5aaf2669f 100644 --- a/crates/storage-postgres/migrations/001_schema.sql +++ b/crates/storage-postgres/migrations/001_schema.sql @@ -35,6 +35,12 @@ CREATE TABLE IF NOT EXISTS tables ( table_class TEXT, sse_specification JSONB, on_demand_throughput JSONB, + -- WarmThroughput as the client set it (see + -- extenddb_core::types::WarmThroughputRecord). Stored and reported only: + -- nothing pre-warms or scales on it, and it has no effect on throughput, + -- throttling or storage. A catalog created before this column existed + -- gets it from the shape step in migrations.rs. + warm_throughput JSONB, PRIMARY KEY (account_id, table_name), CONSTRAINT tables_table_id_unique UNIQUE (table_id) ); @@ -54,6 +60,8 @@ CREATE TABLE IF NOT EXISTS indexes ( index_status TEXT NOT NULL DEFAULT 'ACTIVE', provisioned_throughput JSONB, propagation_delay_ms INTEGER, + -- Same stored-only WarmThroughput record as tables.warm_throughput. + warm_throughput JSONB, PRIMARY KEY (table_id, index_name), CONSTRAINT indexes_table_id_fkey FOREIGN KEY (table_id) REFERENCES tables(table_id) ON DELETE CASCADE, diff --git a/crates/storage-postgres/src/backup_engine.rs b/crates/storage-postgres/src/backup_engine.rs index 20baee942..378b8d9a1 100755 --- a/crates/storage-postgres/src/backup_engine.rs +++ b/crates/storage-postgres/src/backup_engine.rs @@ -413,6 +413,7 @@ impl PostgresEngine { key_schema, projection, provisioned_throughput: throughput_from_catalog(ipt.as_ref()), + warm_throughput: None, }); } } diff --git a/crates/storage-postgres/src/create_table.rs b/crates/storage-postgres/src/create_table.rs index fc41c8c0b..eb61889fb 100755 --- a/crates/storage-postgres/src/create_table.rs +++ b/crates/storage-postgres/src/create_table.rs @@ -8,7 +8,7 @@ use extenddb_core::types::{ ProvisionedThroughputDescription, SseDescription, SseType, TableDescription, TableStatus, }; use extenddb_storage::error::StorageError; -use extenddb_storage::util::{index_arn, stream_arn, table_arn}; +use extenddb_storage::util::{index_arn, stream_arn, table_arn, warm_throughput_record_for_create}; use crate::PostgresEngine; @@ -182,6 +182,18 @@ impl PostgresEngine { } } + // WarmThroughput as sent, reported back and acted on by nothing (see + // the shape step on `tables.warm_throughput`). UPDATING for the same + // delay the CREATING to ACTIVE flip takes. + let delay_secs = Self::control_plane_delay_secs(&mut tx).await?; + let warm_record = + warm_throughput_record_for_create(input.warm_throughput.as_ref(), delay_secs); + let warm_json = warm_record + .as_ref() + .map(serde_json::to_value) + .transpose() + .map_err(|e| StorageError::Internal(e.to_string()))?; + // Insert table metadata, returning creation timestamp and actual status. // Use PG error code 23505 for robust duplicate detection instead of string matching. // H-5: Insert as CREATING with a scheduled transition to ACTIVE, @@ -198,7 +210,8 @@ impl PostgresEngine { (account_id, table_name, key_schema, attribute_definitions, billing_mode, provisioned_throughput, stream_specification, table_status, creation_date_time, table_arn, table_id, deletion_protection_enabled, - status_transition_at, table_class, sse_specification, on_demand_throughput) + status_transition_at, table_class, sse_specification, on_demand_throughput, + warm_throughput) VALUES ($1, $2, $3, $4, $5, $6, $7, CASE WHEN $14 THEN 'CREATING' WHEN (SELECT secs FROM delay) = 0 @@ -209,7 +222,7 @@ impl PostgresEngine { THEN NULL ELSE NOW() + make_interval(secs => (SELECT secs FROM delay)) END, - $11, $12, $13) + $11, $12, $13, $15) RETURNING EXTRACT(EPOCH FROM creation_date_time)::FLOAT8, table_status", ) .bind(account_id) @@ -226,6 +239,7 @@ impl PostgresEngine { .bind(&sse_spec_json) .bind(&on_demand_json) .bind(defer_active) + .bind(&warm_json) .fetch_one(&mut *tx) .await; let (creation_epoch, actual_status): (f64, String) = match inserted { @@ -260,8 +274,17 @@ impl PostgresEngine { // ProvisionedThroughput) so DescribeTable can deserialize it without // failing on the missing NumberOfDecreasesToday field. let mut gsi_index_ids: Vec = Vec::new(); + let mut gsi_warm_records = Vec::new(); if let Some(gsis) = &input.global_secondary_indexes { for gsi in gsis { + let gsi_warm = + warm_throughput_record_for_create(gsi.warm_throughput.as_ref(), delay_secs); + let gsi_warm_json = gsi_warm + .as_ref() + .map(serde_json::to_value) + .transpose() + .map_err(|e| StorageError::Internal(e.to_string()))?; + gsi_warm_records.push(gsi_warm); let gsi_ks = serde_json::to_value(&gsi.key_schema) .map_err(|e| StorageError::Internal(e.to_string()))?; let gsi_proj = serde_json::to_value(&gsi.projection) @@ -285,8 +308,8 @@ impl PostgresEngine { sqlx::query( r"INSERT INTO indexes (table_id, index_name, index_id, index_type, key_schema, projection, - index_status, provisioned_throughput) - VALUES ($1, $2, $3, 'GSI', $4, $5, 'ACTIVE', $6)", + index_status, provisioned_throughput, warm_throughput) + VALUES ($1, $2, $3, 'GSI', $4, $5, 'ACTIVE', $6, $7)", ) .bind(&table_id) .bind(&gsi.index_name) @@ -294,6 +317,7 @@ impl PostgresEngine { .bind(&gsi_ks) .bind(&gsi_proj) .bind(&gsi_pt) + .bind(&gsi_warm_json) .execute(&mut *tx) .await .map_err(|e| StorageError::Internal(e.to_string()))?; @@ -552,7 +576,8 @@ impl PostgresEngine { let gsis = input.global_secondary_indexes.as_ref().map(|gs| { gs.iter() - .map(|g| GsiDescription { + .zip(gsi_warm_records) + .map(|(g, warm)| GsiDescription { index_name: g.index_name.clone(), key_schema: g.key_schema.clone(), projection: g.projection.clone(), @@ -578,6 +603,8 @@ impl PostgresEngine { &input.table_name, &g.index_name, ), + warm_throughput: None, + warm_throughput_record: warm, }) .collect() }); @@ -700,6 +727,9 @@ impl PostgresEngine { .as_ref() .map(|tc| serde_json::json!({ "TableClass": tc })), on_demand_throughput: input.on_demand_throughput, + // Derived by the engine from the record; see core. + warm_throughput: None, + warm_throughput_record: warm_record, // Every field is populated deliberately, with no // `..Default::default()` spread. This response is the complete // description of what was just created, so a new core field should diff --git a/crates/storage-postgres/src/delete_table.rs b/crates/storage-postgres/src/delete_table.rs index 7475eed6a..5bff85307 100755 --- a/crates/storage-postgres/src/delete_table.rs +++ b/crates/storage-postgres/src/delete_table.rs @@ -30,7 +30,8 @@ impl PostgresEngine { EXTRACT(EPOCH FROM creation_date_time)::FLOAT8 as creation_epoch, table_size_bytes, item_count, table_arn, table_id, deletion_protection_enabled, stream_label, - table_class, sse_specification, on_demand_throughput + table_class, sse_specification, on_demand_throughput, + warm_throughput FROM tables WHERE account_id = $1 AND table_name = $2 AND table_status IN ('ACTIVE', 'CREATING') FOR UPDATE", ) @@ -73,7 +74,7 @@ impl PostgresEngine { // Fetch indexes for the response description. let index_rows: Vec = sqlx::query_as( r"SELECT index_name, index_id, index_type, key_schema, projection, - index_status, provisioned_throughput + index_status, provisioned_throughput, warm_throughput FROM indexes WHERE table_id = $1", ) .bind(&row.table_id) diff --git a/crates/storage-postgres/src/lib.rs b/crates/storage-postgres/src/lib.rs index e9f1ae2e9..9e6d0ec1c 100755 --- a/crates/storage-postgres/src/lib.rs +++ b/crates/storage-postgres/src/lib.rs @@ -31,6 +31,7 @@ mod ttl_worker; mod update_table; mod vector; mod vector_search; +mod warm_throughput; mod worker_store; mod workers; diff --git a/crates/storage-postgres/src/migrations.rs b/crates/storage-postgres/src/migrations.rs index 2873bbc77..16cd4d8b8 100755 --- a/crates/storage-postgres/src/migrations.rs +++ b/crates/storage-postgres/src/migrations.rs @@ -136,6 +136,34 @@ const CATALOG_SHAPE_STEPS: &[ShapeStep] = &[ apply: "ALTER TABLE table_restores ALTER COLUMN restore_date_time DROP DEFAULT", at_serve: false, }, + // WarmThroughput as the client set it (one JSONB record per table and per + // index, see extenddb_core::types::WarmThroughputRecord). Stored and + // reported only: nothing pre-warms or scales on it, and it has no effect + // on throughput, throttling or storage. `001_schema.sql` creates the + // columns on a fresh catalog; these steps add them to a catalog that ran + // the earlier revision of that file. Nullable, so rows written before the + // column report the defaults for their billing mode. A shape step rather + // than a numbered migration, so no catalog version moves. + ShapeStep { + what: "add tables.warm_throughput", + needed: "SELECT EXISTS(SELECT 1 FROM information_schema.tables \ + WHERE table_schema = 'public' AND table_name = 'tables') \ + AND NOT EXISTS(SELECT 1 FROM information_schema.columns \ + WHERE table_schema = 'public' AND table_name = 'tables' \ + AND column_name = 'warm_throughput')", + apply: "ALTER TABLE tables ADD COLUMN IF NOT EXISTS warm_throughput JSONB", + at_serve: true, + }, + ShapeStep { + what: "add indexes.warm_throughput", + needed: "SELECT EXISTS(SELECT 1 FROM information_schema.tables \ + WHERE table_schema = 'public' AND table_name = 'indexes') \ + AND NOT EXISTS(SELECT 1 FROM information_schema.columns \ + WHERE table_schema = 'public' AND table_name = 'indexes' \ + AND column_name = 'warm_throughput')", + apply: "ALTER TABLE indexes ADD COLUMN IF NOT EXISTS warm_throughput JSONB", + at_serve: true, + }, ]; /// How long a startup shape step waits for its table lock, in seconds. An diff --git a/crates/storage-postgres/src/table_helpers.rs b/crates/storage-postgres/src/table_helpers.rs index 4dfb7f18f..d4d7fcad2 100755 --- a/crates/storage-postgres/src/table_helpers.rs +++ b/crates/storage-postgres/src/table_helpers.rs @@ -8,7 +8,7 @@ use extenddb_core::types::{ ProvisionedThroughputDescription, SseDescription, SseType, TableDescription, TableStatus, }; use extenddb_storage::error::StorageError; -use extenddb_storage::util::{index_arn, stream_arn}; +use extenddb_storage::util::{index_arn, parse_warm_throughput_record, stream_arn}; use extenddb_storage::vector_catalog::{VectorIndexCatalogRow, vector_index_descriptions}; use crate::PostgresEngine; @@ -34,6 +34,7 @@ pub(crate) struct TableRow { pub table_class: Option, pub sse_specification: Option, pub on_demand_throughput: Option, + pub warm_throughput: Option, } /// Row type for index metadata queries. @@ -46,6 +47,7 @@ pub(crate) struct IndexRow { pub projection: serde_json::Value, pub index_status: String, pub provisioned_throughput: Option, + pub warm_throughput: Option, } /// A `vector_indexes` catalog row, in `FromRow` field order. @@ -112,7 +114,8 @@ impl PostgresEngine { EXTRACT(EPOCH FROM creation_date_time)::FLOAT8 as creation_epoch, table_size_bytes, item_count, table_arn, table_id, deletion_protection_enabled, stream_label, - table_class, sse_specification, on_demand_throughput + table_class, sse_specification, on_demand_throughput, + warm_throughput FROM tables WHERE account_id = $1 AND table_name = $2", ) .bind(account_id) @@ -125,7 +128,7 @@ impl PostgresEngine { let index_rows: Vec = sqlx::query_as( r"SELECT index_name, index_id, index_type, key_schema, projection, - index_status, provisioned_throughput + index_status, provisioned_throughput, warm_throughput FROM indexes WHERE table_id = $1", ) .bind(&row.table_id) @@ -193,6 +196,10 @@ impl PostgresEngine { .transpose() .map_err(|e: serde_json::Error| StorageError::Internal(e.to_string()))?; + let warm_json = idx + .warm_throughput + .as_ref() + .map(serde_json::Value::to_string); gsis.push(GsiDescription { index_name: idx.index_name.clone(), key_schema: ks, @@ -213,6 +220,8 @@ impl PostgresEngine { &row.table_name, &idx.index_name, ), + warm_throughput: None, + warm_throughput_record: parse_warm_throughput_record(warm_json.as_deref())?, }); } else { lsis.push(LsiDescription { @@ -330,6 +339,9 @@ impl PostgresEngine { on_demand_throughput: row .on_demand_throughput .and_then(|v| serde_json::from_value(v).ok()), + warm_throughput_record: parse_warm_throughput_record( + row.warm_throughput.map(|v| v.to_string()).as_deref(), + )?, // Vector indexes are read separately and applied by the caller: this // builder takes `index_rows` only, and the two callers differ in // whether they need them. `build_table_description` fills them in; diff --git a/crates/storage-postgres/src/update_table.rs b/crates/storage-postgres/src/update_table.rs index dec99a93b..9a3474414 100755 --- a/crates/storage-postgres/src/update_table.rs +++ b/crates/storage-postgres/src/update_table.rs @@ -7,7 +7,9 @@ use extenddb_core::types::{ AttributeDefinition, BillingMode, KeySchemaElement, TableDescription, UpdateTableInput, }; use extenddb_storage::error::StorageError; -use extenddb_storage::util::{building_refuses, effective_attribute_definitions}; +use extenddb_storage::util::{ + building_refuses, effective_attribute_definitions, warm_throughput_record_for_create, +}; use crate::PostgresEngine; use crate::data::gsi_build::{ @@ -430,6 +432,10 @@ impl PostgresEngine { .map_err(|e| StorageError::Internal(e.to_string()))?; } + // WarmThroughput on the table and on updated indexes: stored as sent, + // nothing acts on it. Validated by the engine against the current values. + Self::store_warm_throughput_updates(&mut tx, account_id, &table_id, &input).await?; + // Apply GSI updates (create/delete). let mut created_index_ids: Vec = Vec::new(); let mut deleted_index_ids: Vec = Vec::new(); @@ -465,12 +471,20 @@ impl PostgresEngine { .transpose() .map_err(|e| StorageError::Internal(e.to_string()))?; + let gsi_warm = warm_throughput_record_for_create( + create.warm_throughput.as_ref(), + Self::control_plane_delay_secs(&mut tx).await?, + ) + .as_ref() + .map(serde_json::to_value) + .transpose() + .map_err(|e| StorageError::Internal(e.to_string()))?; let index_id = uuid::Uuid::new_v4().to_string(); sqlx::query( r"INSERT INTO indexes (table_id, index_name, index_id, index_type, key_schema, projection, - index_status, provisioned_throughput) - VALUES ($1, $2, $3, 'GSI', $4, $5, 'CREATING', $6)", + index_status, provisioned_throughput, warm_throughput) + VALUES ($1, $2, $3, 'GSI', $4, $5, 'CREATING', $6, $7)", ) .bind(&table_id) .bind(&create.index_name) @@ -478,6 +492,7 @@ impl PostgresEngine { .bind(&gsi_ks) .bind(&gsi_proj) .bind(&gsi_pt) + .bind(&gsi_warm) .execute(&mut *tx) .await .map_err(|e| StorageError::Internal(e.to_string()))?; @@ -1087,6 +1102,11 @@ impl PostgresEngine { } } + // The build outlasts a new GSI's WarmThroughput UPDATING window; start + // it again so the response reports it as the service does. + self.restart_created_gsi_warm_throughput(&table_id, &input) + .await?; + self.build_table_description(account_id, &input.table_name) .await } diff --git a/crates/storage-postgres/src/warm_throughput.rs b/crates/storage-postgres/src/warm_throughput.rs new file mode 100644 index 000000000..8d43d398d --- /dev/null +++ b/crates/storage-postgres/src/warm_throughput.rs @@ -0,0 +1,168 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! Stored `WarmThroughput` for `PostgresEngine`: the control-plane delay the +//! `UPDATING` window takes, and the `UpdateTable` merges. +//! +//! The record is written as sent and read back by DescribeTable; nothing +//! pre-warms or scales on it, and it has no effect on throughput, throttling +//! or storage. See `extenddb_core::types::WarmThroughputRecord`. + +use extenddb_core::types::UpdateTableInput; +use extenddb_storage::error::StorageError; +use extenddb_storage::util::{ + warm_throughput_record_for_create, warm_throughput_record_for_update, +}; + +use crate::PostgresEngine; + +impl PostgresEngine { + /// `control_plane_delay_seconds` from the settings table (0.25 when + /// unset): the window a fresh record reports `UPDATING` for, the same + /// one the CREATING to ACTIVE flip uses. + /// + /// The two windows are timed by different clocks: the CREATING to ACTIVE + /// flip is scheduled from the database clock (`NOW()` in the create + /// INSERT), the record's `UPDATING` deadline is the application clock + /// (`epoch_seconds_now()`). On a database host whose clock differs from + /// the server's, the member can read `UPDATING` after the table is + /// `ACTIVE`, or `ACTIVE` before, by the difference. + pub(crate) async fn control_plane_delay_secs( + conn: &mut sqlx::PgConnection, + ) -> Result { + sqlx::query_scalar( + "SELECT COALESCE((SELECT value::FLOAT8 FROM settings \ + WHERE key = 'control_plane_delay_seconds'), 0.25)", + ) + .fetch_one(&mut *conn) + .await + .map_err(|e| StorageError::Internal(e.to_string())) + } + + /// Store the request's table-level `WarmThroughput` and each + /// `GlobalSecondaryIndexUpdates.Update.WarmThroughput`, merged over what + /// is stored, inside the caller's catalog transaction. The engine has + /// already validated the values against the current ones. + pub(crate) async fn store_warm_throughput_updates( + tx: &mut sqlx::Transaction<'_, sqlx::Postgres>, + account_id: &str, + table_id: &str, + input: &UpdateTableInput, + ) -> Result<(), StorageError> { + let gsi_updates: Vec<_> = input + .global_secondary_index_updates + .iter() + .flatten() + .filter_map(|u| u.update.as_ref()) + .filter_map(|u| u.warm_throughput.as_ref().map(|wt| (&u.index_name, wt))) + .collect(); + if input.warm_throughput.is_none() && gsi_updates.is_empty() { + return Ok(()); + } + let delay_secs = Self::control_plane_delay_secs(tx).await?; + let to_json = |record| { + serde_json::to_value(record).map_err(|e| StorageError::Internal(e.to_string())) + }; + + if let Some(spec) = &input.warm_throughput { + let existing: Option = sqlx::query_scalar( + "SELECT warm_throughput FROM tables WHERE account_id = $1 AND table_name = $2", + ) + .bind(account_id) + .bind(&input.table_name) + .fetch_one(&mut **tx) + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + let existing = existing.map(|v| v.to_string()); + let record = warm_throughput_record_for_update(existing.as_deref(), spec, delay_secs)?; + sqlx::query( + "UPDATE tables SET warm_throughput = $1 WHERE account_id = $2 AND table_name = $3", + ) + .bind(to_json(record)?) + .bind(account_id) + .bind(&input.table_name) + .execute(&mut **tx) + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + } + + for (index_name, spec) in gsi_updates { + let existing: Option> = sqlx::query_scalar( + "SELECT warm_throughput FROM indexes \ + WHERE table_id = $1 AND index_name = $2 AND index_type = 'GSI'", + ) + .bind(table_id) + .bind(index_name) + .fetch_optional(&mut **tx) + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + let Some(existing) = existing else { + return Err(StorageError::IndexNotFound(index_name.clone())); + }; + let existing = existing.map(|v| v.to_string()); + let record = warm_throughput_record_for_update(existing.as_deref(), spec, delay_secs)?; + sqlx::query( + "UPDATE indexes SET warm_throughput = $1 WHERE table_id = $2 AND index_name = $3", + ) + .bind(to_json(record)?) + .bind(table_id) + .bind(index_name) + .execute(&mut **tx) + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + } + Ok(()) + } + + /// Start the `UPDATING` window of each GSI this `UpdateTable` created with + /// `WarmThroughput` again, now that its build has run. + /// + /// The row is inserted with the window starting at the insert, inside the + /// catalog transaction, and the build that follows waits at least + /// `GRACE_PUBLISH` (1 s) before it publishes the index, longer than the + /// default 0.25 s window. Without this the response, which is described + /// after the build, would report the new index's `WarmThroughput` `ACTIVE` + /// where the service reports it `UPDATING`. + pub(crate) async fn restart_created_gsi_warm_throughput( + &self, + table_id: &str, + input: &UpdateTableInput, + ) -> Result<(), StorageError> { + let creates: Vec<_> = input + .global_secondary_index_updates + .iter() + .flatten() + .filter_map(|u| u.create.as_ref()) + .filter(|c| c.warm_throughput.is_some()) + .collect(); + if creates.is_empty() { + return Ok(()); + } + let mut conn = self + .pool + .acquire() + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + let delay_secs = Self::control_plane_delay_secs(&mut conn).await?; + for create in creates { + let Some(record) = + warm_throughput_record_for_create(create.warm_throughput.as_ref(), delay_secs) + else { + continue; + }; + let json = + serde_json::to_value(record).map_err(|e| StorageError::Internal(e.to_string()))?; + sqlx::query( + "UPDATE indexes SET warm_throughput = $1 \ + WHERE table_id = $2 AND index_name = $3 AND index_type = 'GSI'", + ) + .bind(json) + .bind(table_id) + .bind(&create.index_name) + .execute(&mut *conn) + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + } + Ok(()) + } +} diff --git a/crates/storage-postgres/tests/key_collation.rs b/crates/storage-postgres/tests/key_collation.rs index 4988d17eb..9847a70cb 100644 --- a/crates/storage-postgres/tests/key_collation.rs +++ b/crates/storage-postgres/tests/key_collation.rs @@ -180,6 +180,7 @@ fn composite_input(table: &str) -> CreateTableInput { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }]), ..Default::default() } diff --git a/crates/storage-postgres/tests/twi_conflict.rs b/crates/storage-postgres/tests/twi_conflict.rs index 5d7c6fa14..826cea567 100644 --- a/crates/storage-postgres/tests/twi_conflict.rs +++ b/crates/storage-postgres/tests/twi_conflict.rs @@ -365,6 +365,7 @@ async fn gsi_table(s: &Scratch) -> TableKeyInfo { non_key_attributes: None, }, provisioned_throughput: None, + warm_throughput: None, }; s.engine .create_table( diff --git a/crates/storage-postgres/tests/vector_control_plane.rs b/crates/storage-postgres/tests/vector_control_plane.rs index 6118f9a0a..5f49bfa2b 100644 --- a/crates/storage-postgres/tests/vector_control_plane.rs +++ b/crates/storage-postgres/tests/vector_control_plane.rs @@ -342,6 +342,8 @@ fn update_input(table: &str) -> UpdateTableInput { on_demand_throughput: None, table_throughput_mode: None, vector_index_updates: None, + warm_throughput: None, + sse_specification: None, } } @@ -2909,6 +2911,7 @@ async fn update_table_refuses_a_vector_index_named_like_an_existing_gsi() { key_schema: hash_key("gsipk"), projection: projection(ProjectionType::All), provisioned_throughput: None, + warm_throughput: None, }]); s.engine .create_table(ACCOUNT, input) @@ -2975,6 +2978,7 @@ async fn update_table_refuses_a_gsi_named_like_an_existing_vector_index() { key_schema: hash_key("pk"), projection: projection(ProjectionType::All), provisioned_throughput: None, + warm_throughput: None, }), update: None, delete: None, diff --git a/crates/storage-sqlite/src/backup.rs b/crates/storage-sqlite/src/backup.rs index 4cea564e2..bc249ef8c 100644 --- a/crates/storage-sqlite/src/backup.rs +++ b/crates/storage-sqlite/src/backup.rs @@ -210,6 +210,7 @@ impl SqliteEngine { key_schema, projection, provisioned_throughput: throughput_from_catalog(ipt.as_ref()), + warm_throughput: None, }); } } @@ -2163,12 +2164,17 @@ mod restore_tests { .expect("old backup item"); } - // The dry run names the rebuild, with the row count. + // The dry run names the rebuild, with the row count, and the two + // WarmThroughput columns the 0.0.3 schema predates. assert_eq!( crate::schema::pending_shape_migrations(&engine.pool) .await .expect("pending"), - vec!["rebuild backup_items (3 rows)".to_owned()] + vec![ + "rebuild backup_items (3 rows)".to_owned(), + "add tables.warm_throughput".to_owned(), + "add indexes.warm_throughput".to_owned(), + ] ); let (log_guard, logs) = crate::schema::log_capture::capture(); crate::schema::apply(&engine.pool).await.expect("migrate"); diff --git a/crates/storage-sqlite/src/bootstrapper.rs b/crates/storage-sqlite/src/bootstrapper.rs index 293a39c8d..218cc876e 100644 --- a/crates/storage-sqlite/src/bootstrapper.rs +++ b/crates/storage-sqlite/src/bootstrapper.rs @@ -675,9 +675,14 @@ mod migrate_report_tests { bootstrap.read_catalog_version().await.expect("version"), Some(bootstrap.expected_catalog_version()) ); + // The 0.0.3 schema also predates the stored WarmThroughput columns. assert_eq!( bootstrap.pending_data_migrations().await.expect("pending"), - vec!["rebuild backup_items (2 rows)".to_owned()] + vec![ + "rebuild backup_items (2 rows)".to_owned(), + "add tables.warm_throughput".to_owned(), + "add indexes.warm_throughput".to_owned(), + ] ); bootstrap.run_data_migrations().await.expect("apply"); assert!( diff --git a/crates/storage-sqlite/src/create_table.rs b/crates/storage-sqlite/src/create_table.rs index ae4ae05e7..2bb49959c 100644 --- a/crates/storage-sqlite/src/create_table.rs +++ b/crates/storage-sqlite/src/create_table.rs @@ -17,7 +17,10 @@ use extenddb_core::types::{ ProvisionedThroughputDescription, SseDescription, SseType, TableDescription, TableStatus, }; use extenddb_storage::error::StorageError; -use extenddb_storage::util::{index_arn, stream_arn, table_arn}; +use extenddb_storage::util::{ + index_arn, stream_arn, table_arn, warm_throughput_record_for_create, + warm_throughput_record_json, +}; use crate::sqlite_util::{format_epoch_seconds, format_timestamp, is_unique_violation}; use crate::store::SqliteEngine; @@ -120,6 +123,16 @@ impl SqliteEngine { .and_then(|v| v.parse::().ok()) .unwrap_or(0.25); + // Stored as sent and reported back; nothing acts on it (see the + // schema comment on `tables.warm_throughput`). UPDATING for the same + // delay the CREATING to ACTIVE flip takes. + let warm_record = + warm_throughput_record_for_create(input.warm_throughput.as_ref(), delay_secs); + let warm_json = warm_record + .as_ref() + .map(warm_throughput_record_json) + .transpose()?; + let now = time::OffsetDateTime::now_utc(); let creation_ts = format_timestamp(now); #[allow(clippy::cast_precision_loss)] @@ -209,8 +222,8 @@ impl SqliteEngine { (account_id, table_name, key_schema, attribute_definitions, billing_mode, \ provisioned_throughput, stream_specification, table_status, creation_date_time, \ table_arn, table_id, deletion_protection_enabled, status_transition_at, \ - table_class, sse_specification, on_demand_throughput) \ - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", + table_class, sse_specification, on_demand_throughput, warm_throughput) \ + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", ) .bind(account_id) .bind(&input.table_name) @@ -228,6 +241,7 @@ impl SqliteEngine { .bind(&input.table_class) .bind(&sse_json) .bind(&on_demand_json) + .bind(&warm_json) .execute(&mut *tx) .await; if let Err(e) = inserted { @@ -278,6 +292,7 @@ impl SqliteEngine { // GSI / LSI metadata. let mut gsi_ids: Vec = Vec::new(); + let mut gsi_warm_records = Vec::new(); if let Some(gsis) = &input.global_secondary_indexes { for gsi in gsis { let ks = serde_json::to_string(&gsi.key_schema) @@ -298,12 +313,19 @@ impl SqliteEngine { }) .transpose() .map_err(|e| StorageError::Internal(e.to_string()))?; + let gsi_warm = + warm_throughput_record_for_create(gsi.warm_throughput.as_ref(), delay_secs); + let gsi_warm_json = gsi_warm + .as_ref() + .map(warm_throughput_record_json) + .transpose()?; + gsi_warm_records.push(gsi_warm); let index_id = uuid::Uuid::new_v4().to_string(); sqlx::query( "INSERT INTO indexes \ (table_id, index_name, index_id, index_type, key_schema, projection, \ - index_status, provisioned_throughput) \ - VALUES (?, ?, ?, 'GSI', ?, ?, 'ACTIVE', ?)", + index_status, provisioned_throughput, warm_throughput) \ + VALUES (?, ?, ?, 'GSI', ?, ?, 'ACTIVE', ?, ?)", ) .bind(&table_id) .bind(&gsi.index_name) @@ -311,6 +333,7 @@ impl SqliteEngine { .bind(&ks) .bind(&proj) .bind(&pt) + .bind(&gsi_warm_json) .execute(&mut *tx) .await .map_err(|e| StorageError::Internal(e.to_string()))?; @@ -515,7 +538,8 @@ impl SqliteEngine { let gsis = input.global_secondary_indexes.as_ref().map(|gs| { gs.iter() - .map(|g| GsiDescription { + .zip(gsi_warm_records) + .map(|(g, warm)| GsiDescription { index_name: g.index_name.clone(), key_schema: g.key_schema.clone(), projection: g.projection.clone(), @@ -541,6 +565,8 @@ impl SqliteEngine { &input.table_name, &g.index_name, ), + warm_throughput: None, + warm_throughput_record: warm, }) .collect() }); @@ -654,6 +680,9 @@ impl SqliteEngine { .as_ref() .map(|tc| serde_json::json!({ "TableClass": tc })), on_demand_throughput: input.on_demand_throughput, + // Derived by the engine from the record; see core. + warm_throughput: None, + warm_throughput_record: warm_record, // Every field is populated deliberately, with no `..Default::default()` // spread. This response is the complete description of what was just // created, so a new core field should break this site and force a diff --git a/crates/storage-sqlite/src/lib.rs b/crates/storage-sqlite/src/lib.rs index 3953fc350..4b5823aea 100644 --- a/crates/storage-sqlite/src/lib.rs +++ b/crates/storage-sqlite/src/lib.rs @@ -45,6 +45,7 @@ mod table_helpers; mod update_table; mod vector_bench; mod vector_search; +mod warm_throughput; mod worker; mod workers; diff --git a/crates/storage-sqlite/src/schema.rs b/crates/storage-sqlite/src/schema.rs index d04ee5b56..36700da11 100644 --- a/crates/storage-sqlite/src/schema.rs +++ b/crates/storage-sqlite/src/schema.rs @@ -65,6 +65,11 @@ CREATE TABLE IF NOT EXISTS tables ( table_class TEXT, sse_specification TEXT, on_demand_throughput TEXT, + -- WarmThroughput as the client set it (JSON, see + -- extenddb_core::types::WarmThroughputRecord). Stored and reported only: + -- nothing pre-warms or scales on it, and it has no effect on throughput, + -- throttling or storage. + warm_throughput TEXT, PRIMARY KEY (account_id, table_name), CONSTRAINT tables_table_id_unique UNIQUE (table_id) ); @@ -84,6 +89,8 @@ CREATE TABLE IF NOT EXISTS indexes ( index_status TEXT NOT NULL DEFAULT 'ACTIVE', provisioned_throughput TEXT, propagation_delay_ms INTEGER, + -- Same stored-only WarmThroughput record as tables.warm_throughput. + warm_throughput TEXT, PRIMARY KEY (table_id, index_name), CONSTRAINT indexes_table_id_fkey FOREIGN KEY (table_id) REFERENCES tables(table_id) ON DELETE CASCADE, @@ -508,6 +515,7 @@ async fn refuse_unwritable_catalog(pool: &SqlitePool) -> OpResult<()> { async fn converge_table_shapes(pool: &SqlitePool) -> OpResult<()> { migrate_backup_items(pool).await?; migrate_table_restores(pool).await?; + migrate_warm_throughput_columns(pool).await?; Ok(()) } @@ -535,9 +543,40 @@ pub async fn pending_shape_migrations(pool: &SqlitePool) -> OpResult if table_restores_lacks_source_table_arn(pool).await? { pending.push("add table_restores.source_table_arn".to_owned()); } + for table in WARM_THROUGHPUT_TABLES { + if lacks_warm_throughput(pool, table).await? { + pending.push(format!("add {table}.warm_throughput")); + } + } Ok(pending) } +/// The catalog tables that carry a stored `WarmThroughput` record. +const WARM_THROUGHPUT_TABLES: [&str; 2] = ["tables", "indexes"]; + +async fn lacks_warm_throughput(pool: &SqlitePool, table: &'static str) -> OpResult { + Ok(table_exists(pool, table).await? && !has_column(pool, table, "warm_throughput").await?) +} + +/// Add `warm_throughput` to `tables` and `indexes` created by an earlier +/// 0.0.4 build. Rows it wrote keep NULL, which DescribeTable reports as the +/// defaults for the billing mode. +async fn migrate_warm_throughput_columns(pool: &SqlitePool) -> OpResult<()> { + for table in WARM_THROUGHPUT_TABLES { + if !lacks_warm_throughput(pool, table).await? { + continue; + } + tracing::info!("catalog upgrade: adding {table}.warm_throughput"); + sqlx::query(&format!( + "ALTER TABLE {table} ADD COLUMN warm_throughput TEXT" + )) + .execute(pool) + .await + .map_err(|e| OpError::Internal(format!("add {table}.warm_throughput: {e}")))?; + } + Ok(()) +} + async fn has_column(pool: &SqlitePool, table: &'static str, column: &str) -> OpResult { sqlx::query_scalar("SELECT EXISTS(SELECT 1 FROM pragma_table_info(?) WHERE name = ?)") .bind(table) diff --git a/crates/storage-sqlite/src/table_helpers.rs b/crates/storage-sqlite/src/table_helpers.rs index bf9288436..f76e20a49 100644 --- a/crates/storage-sqlite/src/table_helpers.rs +++ b/crates/storage-sqlite/src/table_helpers.rs @@ -11,7 +11,7 @@ use extenddb_core::types::{ SseDescription, SseType, TableDescription, TableStatus, }; use extenddb_storage::error::StorageError; -use extenddb_storage::util::{index_arn, stream_arn}; +use extenddb_storage::util::{index_arn, parse_warm_throughput_record, stream_arn}; use extenddb_storage::vector_catalog::{VectorIndexCatalogRow, vector_index_descriptions}; use crate::data; @@ -38,6 +38,7 @@ pub(crate) struct TableRow { pub table_class: Option, pub sse_specification: Option, pub on_demand_throughput: Option, + pub warm_throughput: Option, } /// Catalog `indexes` row. @@ -51,6 +52,7 @@ pub(crate) struct IndexRow { pub projection: String, pub index_status: String, pub provisioned_throughput: Option, + pub warm_throughput: Option, } /// A `vector_indexes` catalog row, in `FromRow` field order. @@ -105,11 +107,11 @@ pub(crate) const TABLE_COLUMNS: &str = "table_name, key_schema, attribute_defini billing_mode, provisioned_throughput, stream_specification, table_status, \ creation_date_time, table_size_bytes, item_count, table_arn, table_id, \ deletion_protection_enabled, stream_label, table_class, sse_specification, \ - on_demand_throughput"; + on_demand_throughput, warm_throughput"; /// Columns selected for an `IndexRow`, in `FromRow` field order. pub(crate) const INDEX_COLUMNS: &str = "index_name, index_id, index_type, key_schema, \ - projection, index_status, provisioned_throughput"; + projection, index_status, provisioned_throughput, warm_throughput"; fn parse_json(s: &str, ctx: &str) -> Result { serde_json::from_str(s).map_err(|e| StorageError::Internal(format!("{ctx}: {e}"))) @@ -212,6 +214,10 @@ impl SqliteEngine { &row.table_name, &idx.index_name, ), + warm_throughput: None, + warm_throughput_record: parse_warm_throughput_record( + idx.warm_throughput.as_deref(), + )?, }); } else { lsis.push(LsiDescription { @@ -325,6 +331,7 @@ impl SqliteEngine { .on_demand_throughput .as_deref() .and_then(|s| serde_json::from_str(s).ok()), + warm_throughput_record: parse_warm_throughput_record(row.warm_throughput.as_deref())?, // Any core field this backend does not populate takes its default, so // adding one does not break this build. ..Default::default() diff --git a/crates/storage-sqlite/src/update_table.rs b/crates/storage-sqlite/src/update_table.rs index 8adff6173..7ceeec523 100644 --- a/crates/storage-sqlite/src/update_table.rs +++ b/crates/storage-sqlite/src/update_table.rs @@ -18,7 +18,10 @@ use extenddb_core::types::{ TableDescription, UpdateTableInput, VectorIndexSpecification, }; use extenddb_storage::error::StorageError; -use extenddb_storage::util::{building_refuses, effective_attribute_definitions}; +use extenddb_storage::util::{ + building_refuses, effective_attribute_definitions, warm_throughput_record_for_create, + warm_throughput_record_json, +}; use crate::store::SqliteEngine; @@ -357,6 +360,10 @@ impl SqliteEngine { } } + // WarmThroughput on the table and on updated indexes: stored as sent, + // nothing acts on it. Validated by the engine against the current values. + Self::store_warm_throughput_updates(&mut tx, account_id, &table_id, &input).await?; + // GSI create/delete. let mut created: Vec = Vec::new(); let mut deleted: Vec = Vec::new(); @@ -404,12 +411,19 @@ impl SqliteEngine { }) .transpose() .map_err(|e| StorageError::Internal(e.to_string()))?; + let warm = warm_throughput_record_for_create( + create.warm_throughput.as_ref(), + Self::control_plane_delay_secs(&mut tx).await?, + ) + .as_ref() + .map(warm_throughput_record_json) + .transpose()?; let index_id = uuid::Uuid::new_v4().to_string(); sqlx::query( "INSERT INTO indexes \ (table_id, index_name, index_id, index_type, key_schema, projection, \ - index_status, provisioned_throughput) \ - VALUES (?, ?, ?, 'GSI', ?, ?, 'CREATING', ?)", + index_status, provisioned_throughput, warm_throughput) \ + VALUES (?, ?, ?, 'GSI', ?, ?, 'CREATING', ?, ?)", ) .bind(&table_id) .bind(&create.index_name) @@ -417,6 +431,7 @@ impl SqliteEngine { .bind(&ks) .bind(&proj) .bind(&pt) + .bind(&warm) .execute(&mut *tx) .await .map_err(|e| StorageError::Internal(e.to_string()))?; diff --git a/crates/storage-sqlite/src/warm_throughput.rs b/crates/storage-sqlite/src/warm_throughput.rs new file mode 100644 index 000000000..944ce56b4 --- /dev/null +++ b/crates/storage-sqlite/src/warm_throughput.rs @@ -0,0 +1,308 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! The `UpdateTable` side of stored `WarmThroughput` for `SqliteEngine`. +//! +//! The record is written as sent and read back by DescribeTable; nothing +//! pre-warms or scales on it, and it has no effect on throughput, throttling +//! or storage. See `extenddb_core::types::WarmThroughputRecord`. + +use extenddb_core::types::UpdateTableInput; +use extenddb_storage::error::StorageError; +use extenddb_storage::util::{warm_throughput_record_for_update, warm_throughput_record_json}; + +use crate::store::SqliteEngine; + +impl SqliteEngine { + /// `control_plane_delay_seconds` from the settings table (0.25 when + /// unset): the window a fresh record reports `UPDATING` for, the same + /// one the CREATING to ACTIVE flip uses. + pub(crate) async fn control_plane_delay_secs( + conn: &mut sqlx::SqliteConnection, + ) -> Result { + Ok(sqlx::query_scalar::<_, String>( + "SELECT value FROM settings WHERE key = 'control_plane_delay_seconds'", + ) + .fetch_optional(&mut *conn) + .await + .map_err(|e| StorageError::Internal(e.to_string()))? + .and_then(|v| v.parse::().ok()) + .unwrap_or(0.25)) + } + + /// Store the request's table-level `WarmThroughput` and each + /// `GlobalSecondaryIndexUpdates.Update.WarmThroughput`, merged over what + /// is stored, inside the caller's catalog transaction. The engine has + /// already validated the values against the current ones. + pub(crate) async fn store_warm_throughput_updates( + tx: &mut sqlx::Transaction<'_, sqlx::Sqlite>, + account_id: &str, + table_id: &str, + input: &UpdateTableInput, + ) -> Result<(), StorageError> { + let gsi_updates: Vec<_> = input + .global_secondary_index_updates + .iter() + .flatten() + .filter_map(|u| u.update.as_ref()) + .filter_map(|u| u.warm_throughput.as_ref().map(|wt| (&u.index_name, wt))) + .collect(); + if input.warm_throughput.is_none() && gsi_updates.is_empty() { + return Ok(()); + } + let delay_secs = Self::control_plane_delay_secs(tx).await?; + + if let Some(spec) = &input.warm_throughput { + let existing: Option = sqlx::query_scalar( + "SELECT warm_throughput FROM tables WHERE account_id = ? AND table_name = ?", + ) + .bind(account_id) + .bind(&input.table_name) + .fetch_one(&mut **tx) + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + let record = warm_throughput_record_for_update(existing.as_deref(), spec, delay_secs)?; + sqlx::query( + "UPDATE tables SET warm_throughput = ? WHERE account_id = ? AND table_name = ?", + ) + .bind(warm_throughput_record_json(&record)?) + .bind(account_id) + .bind(&input.table_name) + .execute(&mut **tx) + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + } + + for (index_name, spec) in gsi_updates { + let existing: Option> = sqlx::query_scalar( + "SELECT warm_throughput FROM indexes \ + WHERE table_id = ? AND index_name = ? AND index_type = 'GSI'", + ) + .bind(table_id) + .bind(index_name) + .fetch_optional(&mut **tx) + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + let Some(existing) = existing else { + return Err(StorageError::IndexNotFound(index_name.clone())); + }; + let record = warm_throughput_record_for_update(existing.as_deref(), spec, delay_secs)?; + sqlx::query( + "UPDATE indexes SET warm_throughput = ? WHERE table_id = ? AND index_name = ?", + ) + .bind(warm_throughput_record_json(&record)?) + .bind(table_id) + .bind(index_name) + .execute(&mut **tx) + .await + .map_err(|e| StorageError::Internal(e.to_string()))?; + } + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use crate::SqliteEngine; + use extenddb_core::types::{WarmThroughputStatus, epoch_seconds_now}; + use extenddb_storage::TableEngine; + use serde_json::json; + + async fn engine_with_delay(delay: &str) -> SqliteEngine { + let engine = SqliteEngine::new(":memory:", 1, "us-east-1", 409_600) + .await + .expect("engine"); + crate::schema::apply(&engine.pool).await.expect("schema"); + sqlx::query("UPDATE settings SET value = ? WHERE key = 'control_plane_delay_seconds'") + .bind(delay) + .execute(&engine.pool) + .await + .expect("delay"); + sqlx::query("INSERT INTO accounts (account_id, account_name) VALUES (?, 'default')") + .bind(ACCOUNT) + .execute(&engine.pool) + .await + .expect("account"); + engine + } + + const ACCOUNT: &str = "000000000000"; + + async fn describe(engine: &SqliteEngine) -> extenddb_core::types::TableDescription { + let mut desc = engine + .describe_table( + ACCOUNT, + extenddb_core::types::DescribeTableInput { + table_name: "t".to_owned(), + }, + ) + .await + .expect("describe"); + desc.prepare_for_response(); + desc + } + + fn on_demand_with_gsi( + warm: Option, + ) -> extenddb_core::types::CreateTableInput { + let mut v = json!({ + "TableName": "t", + "KeySchema": [{"AttributeName": "pk", "KeyType": "HASH"}], + "AttributeDefinitions": [ + {"AttributeName": "pk", "AttributeType": "S"}, + {"AttributeName": "g", "AttributeType": "S"} + ], + "BillingMode": "PAY_PER_REQUEST", + "GlobalSecondaryIndexes": [{ + "IndexName": "gsi", + "KeySchema": [{"AttributeName": "g", "KeyType": "HASH"}], + "Projection": {"ProjectionType": "ALL"} + }] + }); + if let Some(w) = warm { + v["WarmThroughput"] = w; + } + serde_json::from_value(v).expect("input") + } + + /// No request member: the on-demand defaults, ACTIVE, on the table and the + /// index, once the table is ACTIVE (delay 0 makes it so at once). + #[tokio::test] + async fn an_untouched_on_demand_table_reports_the_defaults_once_active() { + let engine = engine_with_delay("0").await; + engine + .create_table_impl(ACCOUNT, on_demand_with_gsi(None), false) + .await + .expect("create"); + let desc = describe(&engine).await; + let wt = desc.warm_throughput.expect("table member"); + assert_eq!( + ( + wt.read_units_per_second, + wt.write_units_per_second, + wt.status + ), + (Some(12_000), Some(4_000), WarmThroughputStatus::Active) + ); + let gsi = &desc.global_secondary_indexes.expect("gsis")[0]; + let gwt = gsi.warm_throughput.expect("index member"); + assert_eq!( + (gwt.read_units_per_second, gwt.write_units_per_second), + (Some(12_000), Some(4_000)) + ); + } + + /// The stored record survives the round trip and an UpdateTable merges + /// over it: the deadline makes it UPDATING for the delay, then ACTIVE. + #[tokio::test] + async fn a_stored_record_is_read_back_and_an_update_merges_over_it() { + let engine = engine_with_delay("0").await; + engine + .create_table_impl( + ACCOUNT, + on_demand_with_gsi(Some(json!({"ReadUnitsPerSecond": 20000}))), + false, + ) + .await + .expect("create"); + let desc = describe(&engine).await; + let wt = desc.warm_throughput.expect("table member"); + assert_eq!( + ( + wt.read_units_per_second, + wt.write_units_per_second, + wt.status + ), + (Some(20_000), Some(4_000), WarmThroughputStatus::Active) + ); + + // A long delay so the UPDATING window is observable. + sqlx::query("UPDATE settings SET value = '3600' WHERE key = 'control_plane_delay_seconds'") + .execute(&engine.pool) + .await + .expect("delay"); + let update: extenddb_core::types::UpdateTableInput = serde_json::from_value(json!({ + "TableName": "t", + "WarmThroughput": {"WriteUnitsPerSecond": 8000}, + "GlobalSecondaryIndexUpdates": [{"Update": { + "IndexName": "gsi", + "WarmThroughput": {"ReadUnitsPerSecond": 24000, "WriteUnitsPerSecond": 4000} + }}] + })) + .expect("update"); + // While UPDATING the previous pair is reported (as the service does). + let mut returned = engine.update_table(ACCOUNT, update).await.expect("update"); + returned.prepare_for_response(); + let wt = returned.warm_throughput.expect("table member"); + assert_eq!( + ( + wt.read_units_per_second, + wt.write_units_per_second, + wt.status + ), + (Some(20_000), Some(4_000), WarmThroughputStatus::Updating) + ); + let gsi = &returned.global_secondary_indexes.expect("gsis")[0]; + let gwt = gsi.warm_throughput.expect("index member"); + assert_eq!( + ( + gwt.read_units_per_second, + gwt.write_units_per_second, + gwt.status + ), + (Some(12_000), Some(4_000), WarmThroughputStatus::Updating) + ); + // Past the deadline the same record reads ACTIVE. + let mut later = engine + .describe_table( + ACCOUNT, + extenddb_core::types::DescribeTableInput { + table_name: "t".to_owned(), + }, + ) + .await + .expect("describe"); + later.populate_warm_throughput(epoch_seconds_now() + 3601.0); + let settled = later.warm_throughput.expect("member"); + assert_eq!( + ( + settled.read_units_per_second, + settled.write_units_per_second, + settled.status + ), + (Some(20_000), Some(8_000), WarmThroughputStatus::Active) + ); + let gsi = &later.global_secondary_indexes.expect("gsis")[0]; + let gwt = gsi.warm_throughput.expect("index member"); + assert_eq!( + (gwt.read_units_per_second, gwt.write_units_per_second), + (Some(24_000), Some(4_000)) + ); + } + + /// An Update naming an index the table does not have is IndexNotFound. + #[tokio::test] + async fn an_update_of_a_missing_index_is_index_not_found() { + let engine = engine_with_delay("0").await; + engine + .create_table_impl(ACCOUNT, on_demand_with_gsi(None), false) + .await + .expect("create"); + let update: extenddb_core::types::UpdateTableInput = serde_json::from_value(json!({ + "TableName": "t", + "GlobalSecondaryIndexUpdates": [{"Update": { + "IndexName": "nope", + "WarmThroughput": {"ReadUnitsPerSecond": 24000} + }}] + })) + .expect("update"); + let err = engine + .update_table(ACCOUNT, update) + .await + .expect_err("refused"); + assert!( + matches!(err, extenddb_storage::error::StorageError::IndexNotFound(ref n) if n == "nope"), + "{err:?}" + ); + } +} diff --git a/crates/storage/src/backup_definition.rs b/crates/storage/src/backup_definition.rs index 411b22ab6..b25d150f1 100644 --- a/crates/storage/src/backup_definition.rs +++ b/crates/storage/src/backup_definition.rs @@ -493,6 +493,7 @@ mod tests { read_capacity_units: 3, write_capacity_units: 4, }), + warm_throughput: None, }], local_secondary_indexes: Vec::new(), table_class: Some("STANDARD_INFREQUENT_ACCESS".to_owned()), diff --git a/crates/storage/src/util/mod.rs b/crates/storage/src/util/mod.rs index a762738e7..c9bb7a502 100755 --- a/crates/storage/src/util/mod.rs +++ b/crates/storage/src/util/mod.rs @@ -10,6 +10,7 @@ mod arn; mod backup_id; mod catalog_version; mod key; +mod warm_throughput; pub use arn::{ ARN_PARTITION, format_stream_label, index_arn, new_stream_label, parse_stream_arn, stream_arn, @@ -27,6 +28,10 @@ pub use key::{ merge_attribute_definitions, parse_sk, pk_to_text, recover_sort_key_definitions, sk_column, sk_column_n, sk_info, }; +pub use warm_throughput::{ + parse_warm_throughput_record, warm_throughput_record_for_create, + warm_throughput_record_for_update, warm_throughput_record_json, warm_throughput_updating_until, +}; /// Whether an UpdateTable's GSI actions must be refused because the GSIs named /// in `building` are `CREATING`: any Create, or a Delete of a GSI that is not diff --git a/crates/storage/src/util/warm_throughput.rs b/crates/storage/src/util/warm_throughput.rs new file mode 100644 index 000000000..8050e0985 --- /dev/null +++ b/crates/storage/src/util/warm_throughput.rs @@ -0,0 +1,148 @@ +// Copyright 2026 ExtendDB contributors +// SPDX-License-Identifier: Apache-2.0 + +//! Persisting `WarmThroughput` records, shared by every backend. +//! +//! A backend stores one JSON value per table and per global secondary index +//! (see `extenddb_core::types::WarmThroughputRecord`) and loads it back into +//! the description's `warm_throughput_record`; the engine derives the wire +//! member from it. Nothing else reads the value: it does not pre-warm, scale, +//! throttle or size anything. + +use extenddb_core::types::{WarmThroughput, WarmThroughputRecord, epoch_seconds_now}; + +use crate::error::StorageError; + +/// The epoch second until which a record written now reports `UPDATING`: +/// `delay_secs` from now, the control-plane delay that also times the +/// CREATING to ACTIVE flip. Zero delay means ACTIVE at once. +#[must_use] +pub fn warm_throughput_updating_until(delay_secs: f64) -> f64 { + epoch_seconds_now() + delay_secs.max(0.0) +} + +/// The record a create path stores for a request's `WarmThroughput`, or +/// `None` when the client sent none (the defaults are then reported). +#[must_use] +pub fn warm_throughput_record_for_create( + spec: Option<&WarmThroughput>, + delay_secs: f64, +) -> Option { + spec.map(|s| WarmThroughputRecord::from_request(s, warm_throughput_updating_until(delay_secs))) +} + +/// The record an update path stores over the stored JSON (`None` when the +/// resource never had one). +/// +/// # Errors +/// +/// `Internal` when the stored JSON does not parse. +pub fn warm_throughput_record_for_update( + existing_json: Option<&str>, + spec: &WarmThroughput, + delay_secs: f64, +) -> Result { + let existing = parse_warm_throughput_record(existing_json)?; + let now = epoch_seconds_now(); + Ok(WarmThroughputRecord::apply_update( + existing, + spec, + now + delay_secs.max(0.0), + now, + )) +} + +/// The stored JSON of a record. +/// +/// # Errors +/// +/// `Internal` on a serialization failure. +pub fn warm_throughput_record_json(record: &WarmThroughputRecord) -> Result { + serde_json::to_string(record) + .map_err(|e| StorageError::Internal(format!("warm throughput record: {e}"))) +} + +/// A record from its stored JSON; `None` in, `None` out. +/// +/// # Errors +/// +/// `Internal` when the text does not parse as a record. +pub fn parse_warm_throughput_record( + json: Option<&str>, +) -> Result, StorageError> { + json.map(|text| { + serde_json::from_str(text) + .map_err(|e| StorageError::Internal(format!("warm throughput record: {e}"))) + }) + .transpose() +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_create_without_the_member_stores_nothing() { + assert_eq!(warm_throughput_record_for_create(None, 0.25), None); + } + + #[test] + fn a_create_with_the_member_stores_it_updating_for_the_delay() { + let spec = WarmThroughput { + read_units_per_second: Some(20_000), + write_units_per_second: None, + }; + let record = warm_throughput_record_for_create(Some(&spec), 5.0).expect("stored"); + assert_eq!(record.read_units_per_second, Some(20_000)); + assert_eq!(record.write_units_per_second, None); + assert!(record.is_updating_at(epoch_seconds_now() + 4.0)); + assert!(!record.is_updating_at(epoch_seconds_now() + 6.0)); + } + + #[test] + fn an_update_merges_into_the_stored_json_and_round_trips() { + let stored = warm_throughput_record_json(&WarmThroughputRecord { + read_units_per_second: Some(100), + write_units_per_second: Some(50), + previous_read_units_per_second: None, + previous_write_units_per_second: None, + updating_until: 0.0, + }) + .expect("json"); + let updated = warm_throughput_record_for_update( + Some(&stored), + &WarmThroughput { + read_units_per_second: None, + write_units_per_second: Some(75), + }, + 0.0, + ) + .expect("merged"); + assert_eq!(updated.read_units_per_second, Some(100)); + assert_eq!(updated.write_units_per_second, Some(75)); + let again = parse_warm_throughput_record(Some( + &warm_throughput_record_json(&updated).expect("json"), + )) + .expect("parses") + .expect("present"); + assert_eq!( + (again.read_units_per_second, again.write_units_per_second), + ( + updated.read_units_per_second, + updated.write_units_per_second + ) + ); + // serde_json's float parser is not exact to the last bit without its + // float_roundtrip feature; a microsecond is far below what the status + // check can see. + assert!((again.updating_until - updated.updating_until).abs() < 1e-6); + } + + #[test] + fn garbage_in_the_column_is_an_internal_error_not_a_default() { + let err = parse_warm_throughput_record(Some("{\"x\":")).expect_err("refused"); + assert!( + matches!(err, StorageError::Internal(m) if m.starts_with("warm throughput record:")) + ); + } +} diff --git a/devtools/run-tests b/devtools/run-tests index 421a97b3a..17da66a7a 100755 --- a/devtools/run-tests +++ b/devtools/run-tests @@ -65,6 +65,7 @@ RUN_RUST_INTEGRATION=false RUN_PYTEST=false RUN_EXTERNAL=false RUN_COMPREHENSIVE=false +RUN_TERRAFORM=false RUN_CATALOG_CHECK=false # Options @@ -88,6 +89,8 @@ Suites (at least one required): --pytest Python integration tests --external External Java test suite --comprehensive Comprehensive Python suite + --terraform Terraform apply of tests/terraform/warm-throughput (installs + the pinned Terraform unless TERRAFORM names a binary) --all all suites --catalog-check post-test catalog integrity check @@ -115,7 +118,8 @@ while [[ $# -gt 0 ]]; do --pytest) RUN_PYTEST=true; shift ;; --external) RUN_EXTERNAL=true; shift ;; --comprehensive) RUN_COMPREHENSIVE=true; shift ;; - --all) RUN_RUST=true; RUN_RUST_INTEGRATION=true; RUN_PYTEST=true; RUN_EXTERNAL=true; RUN_COMPREHENSIVE=true; shift ;; + --terraform) RUN_TERRAFORM=true; shift ;; + --all) RUN_RUST=true; RUN_RUST_INTEGRATION=true; RUN_PYTEST=true; RUN_EXTERNAL=true; RUN_COMPREHENSIVE=true; RUN_TERRAFORM=true; shift ;; --release) RELEASE=true; shift ;; --filter) FILTER="$2"; shift 2 ;; --catalog-check) RUN_CATALOG_CHECK=true; shift ;; @@ -155,8 +159,8 @@ if [[ -z "$TARGET" ]]; then fi # --- Validate: at least one suite --- -if ! $RUN_RUST && ! $RUN_RUST_INTEGRATION && ! $RUN_PYTEST && ! $RUN_EXTERNAL && ! $RUN_COMPREHENSIVE && ! $RUN_CATALOG_CHECK; then - echo "error: at least one suite flag is required (--rust, --rust-integration, --pytest, --external, --comprehensive, --all, --catalog-check)" +if ! $RUN_RUST && ! $RUN_RUST_INTEGRATION && ! $RUN_PYTEST && ! $RUN_EXTERNAL && ! $RUN_COMPREHENSIVE && ! $RUN_TERRAFORM && ! $RUN_CATALOG_CHECK; then + echo "error: at least one suite flag is required (--rust, --rust-integration, --pytest, --external, --comprehensive, --terraform, --all, --catalog-check)" exit 1 fi @@ -218,7 +222,7 @@ validate_real_dynamodb() { # Only validate credentials for integration suites NEEDS_INTEGRATION=false -if $RUN_PYTEST || $RUN_EXTERNAL || $RUN_COMPREHENSIVE || $RUN_CATALOG_CHECK || $RUN_RUST_INTEGRATION; then +if $RUN_PYTEST || $RUN_EXTERNAL || $RUN_COMPREHENSIVE || $RUN_TERRAFORM || $RUN_CATALOG_CHECK || $RUN_RUST_INTEGRATION; then NEEDS_INTEGRATION=true fi @@ -552,6 +556,22 @@ if $RUN_COMPREHENSIVE; then echo "" fi +# --- Terraform (the provider's WarmThroughput wait, issue #462) --- +if $RUN_TERRAFORM; then + OUTFILE="discussions/test-terraform-${HASH}.txt" + echo "=== Terraform apply → ${OUTFILE} ===" + if [[ "$TARGET" == "real-dynamodb" ]]; then + echo " skipped: the configuration points at an ExtendDB endpoint" + elif EXTENDDB_TEST_ENDPOINT="$EXTENDDB_TEST_ENDPOINT" \ + tests/terraform/warm-throughput/run.sh 2>&1 | tee "$OUTFILE"; then + echo " ✓ Terraform apply passed" + else + echo " ✗ Terraform apply failed" + PASS=false + fi + echo "" +fi + # --- CLI lifecycle tests (postgres-only; run last — they start/stop their own servers) --- if $RUN_PYTEST \ && [[ "$TARGET" != "real-dynamodb" \ diff --git a/docs/differences-from-dynamodb.md b/docs/differences-from-dynamodb.md index 2d6469b7e..a8d2f4ac8 100755 --- a/docs/differences-from-dynamodb.md +++ b/docs/differences-from-dynamodb.md @@ -145,6 +145,7 @@ The same on all four backends. |------|----------|------| | Provisioned throughput | Token bucket per table/partition | Token bucket per table/partition, matching DynamoDB's burst and refill behavior | | On-demand capacity | Automatic scaling | Fixed initial burst capacity (4000 WCU / 12000 RCU), no auto-scaling | +| Warm throughput | Pre-provisions partitions to the requested read and write units; raising it is a long-running operation (`Status: UPDATING` for minutes, the previous values reported meanwhile); a request above the account's table maximum is `LimitExceededException` | Stored and reported with the same shape, defaults and validation so clients that read it (the Terraform AWS provider since 6.13.0) work: the previous values are reported with `Status: UPDATING` for `control_plane_delay_seconds` after a change, then the new values `ACTIVE`; a decrease is measured against the reported values, as on the service. Nothing acts on the values: no pre-warming and no scaling at the database level. The account maximum is not known to ExtendDB and is not enforced. A provisioned table whose warm throughput was never set, or was set on one side only, reports its new units at once after a capacity change, where the service reports the old units until the table is `ACTIVE`. Switching a table to `PAY_PER_REQUEST` stores 12000 read and 4000 write units where its warm throughput is lower, and the stored values stay after a switch back to `PROVISIONED`, as on the service; the response to the switch reads the new values with `Status: UPDATING` for `control_plane_delay_seconds`, where the service answers with the previous values and `Status: ACTIVE` | | Throttling | Always on; throttles requests that exceed provisioned/burst capacity. No setting to disable | Configurable via `throttling_enabled` runtime setting (default: `true`) | ## Operations Not Implemented diff --git a/docs/getting-started.md b/docs/getting-started.md index 2f22f158a..ddfdcfdff 100755 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -292,6 +292,10 @@ By default, control plane operations (CreateTable, DeleteTable) emulate real Dyn control_plane_delay_seconds 10 ``` +### Warm Throughput + +`WarmThroughput` on `CreateTable`, `UpdateTable` and `GlobalSecondaryIndexUpdates` is stored and reported on `DescribeTable` with the shape, defaults (12000 read and 4000 write units on an on-demand table, the provisioned units on a provisioned one) and validation of Amazon DynamoDB, and `Status` reads `UPDATING` for the control plane delay after a change, then `ACTIVE`. Nothing acts on the values: there is no pre-warming and no scaling at the database level, and they have no effect on throughput, throttling or storage. + ### Credential Import Controls whether `extenddb manage import-access-key` is allowed (default: `true`). diff --git a/docs/manuals/01-architecture-guide.md b/docs/manuals/01-architecture-guide.md index c709f3de5..6402f592e 100755 --- a/docs/manuals/01-architecture-guide.md +++ b/docs/manuals/01-architecture-guide.md @@ -172,7 +172,7 @@ extenddb uses a dual-database architecture: - **Catalog database** (e.g., `extenddb_catalog`): Stores table metadata, account/user/group/role/policy definitions, access keys, settings, stream metadata, and metrics. Shared across all accounts. - **Data database** (e.g., `extenddb_catalog_data`): Stores user items, GSI/LSI data, and stream records. Each table gets its own PostgreSQL table. -The catalog version is 0.0.4 on PostgreSQL, SQLite, and Cassandra. PostgreSQL and SQLite store it in the `settings` table under the key `catalog_version` and check it at startup; Cassandra has no startup check (see the Upgrade Manual). +The catalog version is 0.0.4 on PostgreSQL and SQLite and 0.0.5 on Cassandra (the stored warm throughput columns, Cassandra migration V006). PostgreSQL and SQLite store it in the `settings` table under the key `catalog_version` and check it at startup; Cassandra has no startup check (see the Upgrade Manual). The MongoDB backend tracks its own catalog version, 0.0.2. A mismatch between the compiled-in version and the stored one prevents a PostgreSQL or SQLite server from starting; run `extenddb migrate` to upgrade. diff --git a/docs/manuals/07-upgrade-manual.md b/docs/manuals/07-upgrade-manual.md index 23c1cb6f5..5673a6d72 100755 --- a/docs/manuals/07-upgrade-manual.md +++ b/docs/manuals/07-upgrade-manual.md @@ -202,7 +202,20 @@ To discard data written after the upgrade as well, restore `` from its ## Version History -### Catalog 0.0.4 (Current) +### Catalog 0.0.5 (Cassandra only) + +Adds the stored `WarmThroughput` record: `V006__warm_throughput.cql` adds a nullable `warm_throughput` text column to `tables` and to `indexes` and stamps `catalog_version` 0.0.5. Nothing acts on the value; it is stored and reported by `DescribeTable` (see the differences document). PostgreSQL and SQLite add the same two columns at `serve` start (and in `extenddb migrate`) without a version change, so their catalog stays 0.0.4. + +A Cassandra server from this release started before `extenddb migrate --yes` serves every table and item operation and reports the default warm throughput; a `CreateTable` or `UpdateTable` that names `WarmThroughput` fails with an internal error (`Undefined column name warm_throughput`) until the catalog is migrated, an `UpdateTable` that switches a table to `PAY_PER_REQUEST` succeeds without storing the on-demand warm throughput (the reported value still follows the on-demand minimum while the table is on-demand), and CreateBackup, RestoreTableFromBackup and DeleteBackup are refused as for V005. Run: + +``` +extenddb migrate --config extenddb.toml --cassandra-user # exit 3, lists V006 +extenddb migrate --config extenddb.toml --cassandra-user --yes # applies V006 +``` + +V006 only adds columns and the version row; a 0.0.4 Cassandra binary keeps working against a 0.0.5 catalog. + +### Catalog 0.0.4 (Current on PostgreSQL and SQLite) Adds the table definition a backup records and the registration of a running restore: diff --git a/tests/python/test_warm_throughput.py b/tests/python/test_warm_throughput.py new file mode 100644 index 000000000..93ffa31ed --- /dev/null +++ b/tests/python/test_warm_throughput.py @@ -0,0 +1,446 @@ +# Copyright 2026 ExtendDB contributors +# SPDX-License-Identifier: Apache-2.0 + +"""WarmThroughput on tables and global secondary indexes. + +Every assertion here is a row measured against Amazon DynamoDB (us-east-1, +2026-10-23). The file runs against ExtendDB and against the service with the +same assertions; the two differ only in timing, which the `is_extenddb` +fixture gates: ExtendDB reports `Status: UPDATING` for the control-plane delay +(a fraction of a second by default), the service for minutes, so the "returns +to ACTIVE" wait runs only against ExtendDB. + +ExtendDB stores and reports the values. Nothing acts on them. +""" + +from __future__ import annotations + +import pytest +from botocore.exceptions import ClientError + +from warm_throughput_helpers import ( # noqa: F401 (tables is a fixture) + DECREASE, + DELETION_PROTECTION_ONLY, + GSI, + LSI, + ON_DEMAND_DEFAULTS, + ON_DEMAND_MIN, + ON_DEMAND_NEEDS_BILLING, + ONLY_OPERATION, + TABLE_CLASS_ONLY, + _create, + _sentence, + _validation, + _wait_all_active, + _wait_warm_active, + tables, +) + + +class TestWarmThroughputLifecycle: + def test_on_demand_table_absent_while_creating_defaults_once_active( + self, dynamodb_client, tables + ): + """Rows: CreateTable response and first DescribeTable carry no member while + CREATING; ACTIVE reads {12000, 4000, ACTIVE} on the table and every GSI; + a local secondary index carries none.""" + name, created = _create( + dynamodb_client, + tables, + BillingMode="PAY_PER_REQUEST", + GlobalSecondaryIndexes=[GSI], + LocalSecondaryIndexes=[LSI], + ) + if created["TableStatus"] == "CREATING": + assert "WarmThroughput" not in created + assert all("WarmThroughput" not in g for g in created["GlobalSecondaryIndexes"]) + first = dynamodb_client.describe_table(TableName=name)["Table"] + if first["TableStatus"] == "CREATING": + assert "WarmThroughput" not in first + active = _wait_all_active(dynamodb_client, name) + assert active["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "ACTIVE"} + gsi = active["GlobalSecondaryIndexes"][0] + assert gsi["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "ACTIVE"} + assert "WarmThroughput" not in active["LocalSecondaryIndexes"][0] + + def test_provisioned_table_and_index_default_to_their_units(self, dynamodb_client, tables): + """Row: a 1/1 provisioned table with a 1/1 GSI reads {1, 1, ACTIVE} on both.""" + name, created = _create( + dynamodb_client, + tables, + BillingMode="PROVISIONED", + ProvisionedThroughput={"ReadCapacityUnits": 1, "WriteCapacityUnits": 1}, + GlobalSecondaryIndexes=[ + {**GSI, "ProvisionedThroughput": {"ReadCapacityUnits": 1, "WriteCapacityUnits": 1}} + ], + ) + if created["TableStatus"] == "CREATING": + assert "WarmThroughput" not in created + active = _wait_all_active(dynamodb_client, name) + expected = {"ReadUnitsPerSecond": 1, "WriteUnitsPerSecond": 1, "Status": "ACTIVE"} + assert active["WarmThroughput"] == expected + assert active["GlobalSecondaryIndexes"][0]["WarmThroughput"] == expected + + def test_update_table_without_warm_throughput_reports_it_active( + self, dynamodb_client, tables + ): + """Row: an UpdateTable that only touches deletion protection carries the + member with Status ACTIVE in its response.""" + name, _ = _create(dynamodb_client, tables, BillingMode="PAY_PER_REQUEST") + _wait_all_active(dynamodb_client, name) + resp = dynamodb_client.update_table(TableName=name, DeletionProtectionEnabled=False) + assert resp["TableDescription"]["WarmThroughput"] == { + **ON_DEMAND_DEFAULTS, + "Status": "ACTIVE", + } + + def test_create_with_warm_throughput_is_present_at_once_and_partial_is_echoed( + self, dynamodb_client, tables + ): + """Rows C1 and C4: a request with WarmThroughput is answered with the member + at once, Status UPDATING, as requested (a read-only request echoes only the + read side in the CreateTable response), and reads back filled once ACTIVE.""" + name, created = _create( + dynamodb_client, + tables, + BillingMode="PAY_PER_REQUEST", + WarmThroughput={"ReadUnitsPerSecond": 20000}, + ) + if created["TableStatus"] == "CREATING": + assert created["WarmThroughput"] == {"ReadUnitsPerSecond": 20000, "Status": "UPDATING"} + else: + assert created["WarmThroughput"]["ReadUnitsPerSecond"] == 20000 + _wait_all_active(dynamodb_client, name) + active = _wait_warm_active(dynamodb_client, name, timeout=900) + assert active["WarmThroughput"] == { + "ReadUnitsPerSecond": 20000, + "WriteUnitsPerSecond": 4000, + "Status": "ACTIVE", + } + + both, created = _create( + dynamodb_client, + tables, + BillingMode="PAY_PER_REQUEST", + WarmThroughput={"ReadUnitsPerSecond": 12000, "WriteUnitsPerSecond": 4000}, + GlobalSecondaryIndexes=[ + {**GSI, "WarmThroughput": {"ReadUnitsPerSecond": 12000, "WriteUnitsPerSecond": 4000}} + ], + ) + if created["TableStatus"] == "CREATING": + assert created["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "UPDATING"} + assert created["GlobalSecondaryIndexes"][0]["WarmThroughput"] == { + **ON_DEMAND_DEFAULTS, + "Status": "UPDATING", + } + active = _wait_all_active(dynamodb_client, both) + active = _wait_warm_active(dynamodb_client, both, timeout=900) + assert active["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "ACTIVE"} + assert active["GlobalSecondaryIndexes"][0]["WarmThroughput"] == { + **ON_DEMAND_DEFAULTS, + "Status": "ACTIVE", + } + + def test_update_warm_throughput_goes_updating_then_active( + self, dynamodb_client, tables, is_extenddb + ): + """Rows U1, U2, U3, U5, U6, S2a, S2b and S2c: UpdateTable with WarmThroughput + is accepted and the response reports the PREVIOUS values with Status + UPDATING while TableStatus and IndexStatus stay ACTIVE; inside the window a + request carrying those previous values is accepted (U3); once settled the new + values read ACTIVE and the same request is a decrease (S2b). Only the waits + are gated: ExtendDB returns to ACTIVE within the control plane delay, the + service after minutes, so the settled assertions run on ExtendDB only.""" + name, _ = _create( + dynamodb_client, + tables, + BillingMode="PAY_PER_REQUEST", + GlobalSecondaryIndexes=[GSI], + ) + _wait_all_active(dynamodb_client, name) + + # U1: the same values as current, and nothing else in the request. + resp = dynamodb_client.update_table(TableName=name, WarmThroughput=ON_DEMAND_DEFAULTS) + t = resp["TableDescription"] + assert t["TableStatus"] == "ACTIVE" + assert t["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "UPDATING"} + if is_extenddb: + _wait_warm_active(dynamodb_client, name) + + # U2 and S2a: higher values; the response carries the previous pair. + higher = {"ReadUnitsPerSecond": 24000, "WriteUnitsPerSecond": 8000} + t = dynamodb_client.update_table(TableName=name, WarmThroughput=higher)["TableDescription"] + assert t["TableStatus"] == "ACTIVE" + assert t["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "UPDATING"} + described = dynamodb_client.describe_table(TableName=name)["Table"]["WarmThroughput"] + if described["Status"] == "UPDATING": + assert described == {**ON_DEMAND_DEFAULTS, "Status": "UPDATING"} + # U3: inside the window the previous values are not a decrease. + t = dynamodb_client.update_table(TableName=name, WarmThroughput=ON_DEMAND_DEFAULTS)[ + "TableDescription" + ] + assert t["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "UPDATING"} + if is_extenddb: + _wait_warm_active(dynamodb_client, name) + dynamodb_client.update_table(TableName=name, WarmThroughput=higher) + if is_extenddb: + active = _wait_warm_active(dynamodb_client, name) + assert active["WarmThroughput"] == {**higher, "Status": "ACTIVE"} + + # S2b: on the settled table the previous values are a decrease. + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table(TableName=name, WarmThroughput=ON_DEMAND_DEFAULTS) + _validation(exc, _sentence("ReadUnitsPerSecond", "table", DECREASE)) + # S2c: the same values re-sent report themselves UPDATING. + t = dynamodb_client.update_table(TableName=name, WarmThroughput=higher)[ + "TableDescription" + ] + assert t["WarmThroughput"] == {**higher, "Status": "UPDATING"} + _wait_warm_active(dynamodb_client, name) + + # U5: read alone; the write side keeps its value. + dynamodb_client.update_table(TableName=name, WarmThroughput={"ReadUnitsPerSecond": 30000}) + active = _wait_warm_active(dynamodb_client, name) + assert active["WarmThroughput"] == { + "ReadUnitsPerSecond": 30000, + "WriteUnitsPerSecond": 8000, + "Status": "ACTIVE", + } + + # U6: a GSI through GlobalSecondaryIndexUpdates.Update (same values, so + # the previous and new pairs agree on both targets). + resp = dynamodb_client.update_table( + TableName=name, + GlobalSecondaryIndexUpdates=[ + {"Update": {"IndexName": "gsi", "WarmThroughput": ON_DEMAND_DEFAULTS}} + ], + ) + gsi = resp["TableDescription"]["GlobalSecondaryIndexes"][0] + assert gsi["IndexStatus"] == "ACTIVE" + assert gsi["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "UPDATING"} + assert resp["TableDescription"]["TableStatus"] == "ACTIVE" + if is_extenddb: + active = _wait_warm_active(dynamodb_client, name) + assert active["GlobalSecondaryIndexes"][0]["WarmThroughput"] == { + **ON_DEMAND_DEFAULTS, + "Status": "ACTIVE", + } + + def test_capacity_change_leaves_a_set_warm_throughput_alone(self, dynamodb_client, tables): + """Rows C6 and U8: a provisioned 5/5 table created with WarmThroughput 100/50 + keeps 100/50 ACTIVE after its capacity moves to 10/10.""" + name, created = _create( + dynamodb_client, + tables, + BillingMode="PROVISIONED", + ProvisionedThroughput={"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}, + WarmThroughput={"ReadUnitsPerSecond": 100, "WriteUnitsPerSecond": 50}, + ) + if created["TableStatus"] == "CREATING": + assert created["WarmThroughput"] == { + "ReadUnitsPerSecond": 100, + "WriteUnitsPerSecond": 50, + "Status": "UPDATING", + } + _wait_all_active(dynamodb_client, name) + _wait_warm_active(dynamodb_client, name, timeout=900) + resp = dynamodb_client.update_table( + TableName=name, + ProvisionedThroughput={"ReadCapacityUnits": 10, "WriteCapacityUnits": 10}, + ) + expected = {"ReadUnitsPerSecond": 100, "WriteUnitsPerSecond": 50, "Status": "ACTIVE"} + assert resp["TableDescription"]["WarmThroughput"] == expected + _wait_all_active(dynamodb_client, name) + assert dynamodb_client.describe_table(TableName=name)["Table"]["WarmThroughput"] == expected + + +class TestWarmThroughputValidation: + def test_on_demand_create_below_the_minimum(self, dynamodb_client, tables): + """Rows C2, C3 and C7: the on-demand minimum, read checked before write.""" + for warm, member in ( + ({"ReadUnitsPerSecond": 11999, "WriteUnitsPerSecond": 4000}, "ReadUnitsPerSecond"), + ({"ReadUnitsPerSecond": 12000, "WriteUnitsPerSecond": 3999}, "WriteUnitsPerSecond"), + ({"ReadUnitsPerSecond": 0, "WriteUnitsPerSecond": 0}, "ReadUnitsPerSecond"), + ): + with pytest.raises(ClientError) as exc: + _create(dynamodb_client, tables, BillingMode="PAY_PER_REQUEST", WarmThroughput=warm) + _validation(exc, _sentence(member, "table", ON_DEMAND_MIN)) + + def test_on_demand_gsi_create_below_the_minimum_names_the_index(self, dynamodb_client, tables): + """Rows G1 and G2: the index form of the on-demand sentence.""" + for warm, member in ( + ({"ReadUnitsPerSecond": 11999, "WriteUnitsPerSecond": 4000}, "ReadUnitsPerSecond"), + ({"ReadUnitsPerSecond": 12000, "WriteUnitsPerSecond": 3999}, "WriteUnitsPerSecond"), + ): + with pytest.raises(ClientError) as exc: + _create( + dynamodb_client, + tables, + BillingMode="PAY_PER_REQUEST", + GlobalSecondaryIndexes=[{**GSI, "WarmThroughput": warm}], + ) + _validation(exc, _sentence(member, "index gsi", ON_DEMAND_MIN)) + + def test_provisioned_create_below_the_units(self, dynamodb_client, tables): + """Row C5 (read form measured; the write form is the same sentence with the + member names substituted).""" + with pytest.raises(ClientError) as exc: + _create( + dynamodb_client, + tables, + BillingMode="PROVISIONED", + ProvisionedThroughput={"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}, + WarmThroughput={"ReadUnitsPerSecond": 1, "WriteUnitsPerSecond": 1}, + ) + _validation( + exc, + _sentence( + "ReadUnitsPerSecond", "table", + " is lower than ReadCapacityUnits of ProvisionedThroughput", + ), + ) + # P1: the write form. + with pytest.raises(ClientError) as exc: + _create( + dynamodb_client, + tables, + BillingMode="PROVISIONED", + ProvisionedThroughput={"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}, + WarmThroughput={"ReadUnitsPerSecond": 5, "WriteUnitsPerSecond": 4}, + ) + _validation( + exc, + _sentence( + "WriteUnitsPerSecond", "table", + " is lower than WriteCapacityUnits of ProvisionedThroughput", + ), + ) + # G3: the index form, on a 5/5 GSI. + with pytest.raises(ClientError) as exc: + _create( + dynamodb_client, + tables, + BillingMode="PROVISIONED", + ProvisionedThroughput={"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}, + GlobalSecondaryIndexes=[{ + **GSI, + "ProvisionedThroughput": {"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}, + "WarmThroughput": {"ReadUnitsPerSecond": 1, "WriteUnitsPerSecond": 1}, + }], + ) + _validation( + exc, + _sentence( + "ReadUnitsPerSecond", "index gsi", + " is lower than ReadCapacityUnits of ProvisionedThroughput", + ), + ) + # P2: equal to the units is accepted and reads back {5, 5, ACTIVE}. + name, _ = _create( + dynamodb_client, + tables, + BillingMode="PROVISIONED", + ProvisionedThroughput={"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}, + WarmThroughput={"ReadUnitsPerSecond": 5, "WriteUnitsPerSecond": 5}, + ) + _wait_all_active(dynamodb_client, name) + active = _wait_warm_active(dynamodb_client, name, timeout=900) + assert active["WarmThroughput"] == { + "ReadUnitsPerSecond": 5, + "WriteUnitsPerSecond": 5, + "Status": "ACTIVE", + } + + def test_update_below_current_is_a_decrease(self, dynamodb_client, tables): + """Rows U4 and U7: below the current value on either billing mode.""" + od, _ = _create(dynamodb_client, tables, BillingMode="PAY_PER_REQUEST") + _wait_all_active(dynamodb_client, od) + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table( + TableName=od, WarmThroughput={"ReadUnitsPerSecond": 11999, "WriteUnitsPerSecond": 4000} + ) + _validation(exc, _sentence("ReadUnitsPerSecond", "table", DECREASE)) + # UT1: the write form. + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table( + TableName=od, WarmThroughput={"ReadUnitsPerSecond": 12000, "WriteUnitsPerSecond": 3999} + ) + _validation(exc, _sentence("WriteUnitsPerSecond", "table", DECREASE)) + + pv, _ = _create( + dynamodb_client, + tables, + BillingMode="PROVISIONED", + ProvisionedThroughput={"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}, + WarmThroughput={"ReadUnitsPerSecond": 100, "WriteUnitsPerSecond": 50}, + ) + _wait_all_active(dynamodb_client, pv) + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table( + TableName=pv, WarmThroughput={"ReadUnitsPerSecond": 2, "WriteUnitsPerSecond": 2} + ) + _validation(exc, _sentence("ReadUnitsPerSecond", "table", DECREASE)) + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table(TableName=pv, WarmThroughput={"WriteUnitsPerSecond": 49}) + _validation(exc, _sentence("WriteUnitsPerSecond", "table", DECREASE)) + + def test_gsi_update_below_current_names_the_index(self, dynamodb_client, tables): + """Rows UG1 and UG2: the index form of the decrease sentence, both members.""" + name, _ = _create( + dynamodb_client, tables, BillingMode="PAY_PER_REQUEST", GlobalSecondaryIndexes=[GSI] + ) + _wait_all_active(dynamodb_client, name) + for warm, member in ( + ({"ReadUnitsPerSecond": 11999, "WriteUnitsPerSecond": 4000}, "ReadUnitsPerSecond"), + ({"ReadUnitsPerSecond": 12000, "WriteUnitsPerSecond": 3999}, "WriteUnitsPerSecond"), + ): + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table( + TableName=name, + GlobalSecondaryIndexUpdates=[{"Update": {"IndexName": "gsi", "WarmThroughput": warm}}], + ) + _validation(exc, _sentence(member, "index gsi", DECREASE)) + + def test_table_warm_throughput_beside_another_change(self, dynamodb_client, tables): + """The per-companion rules, measured on an on-demand table with one GSI + (values otherwise valid): TableClass and DeletionProtection companions + answer their own "must be the only operation" sentence; a + StreamSpecification companion, and BillingMode with ProvisionedThroughput, + answer the WarmThroughput sentence; an OnDemandThroughput companion without + BillingMode answers its own sentence; a GlobalSecondaryIndexUpdates Update + carrying the index's WarmThroughput is accepted and both go UPDATING.""" + name, _ = _create( + dynamodb_client, tables, BillingMode="PAY_PER_REQUEST", GlobalSecondaryIndexes=[GSI] + ) + _wait_all_active(dynamodb_client, name) + cases = [ + ({"TableClass": "STANDARD"}, TABLE_CLASS_ONLY), + ({"DeletionProtectionEnabled": False}, DELETION_PROTECTION_ONLY), + ({"OnDemandThroughput": {"MaxReadRequestUnits": 100000}}, ON_DEMAND_NEEDS_BILLING), + ({"StreamSpecification": {"StreamEnabled": False}}, ONLY_OPERATION), + ( + { + "BillingMode": "PROVISIONED", + "ProvisionedThroughput": {"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}, + }, + ONLY_OPERATION, + ), + ] + for extra, expected in cases: + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table( + TableName=name, WarmThroughput=ON_DEMAND_DEFAULTS, **extra + ) + _validation(exc, expected) + + resp = dynamodb_client.update_table( + TableName=name, + WarmThroughput=ON_DEMAND_DEFAULTS, + GlobalSecondaryIndexUpdates=[ + {"Update": {"IndexName": "gsi", "WarmThroughput": ON_DEMAND_DEFAULTS}} + ], + ) + t = resp["TableDescription"] + assert t["TableStatus"] == "ACTIVE" + assert t["WarmThroughput"]["Status"] == "UPDATING" + assert t["GlobalSecondaryIndexes"][0]["WarmThroughput"]["Status"] == "UPDATING" diff --git a/tests/python/test_warm_throughput_backup.py b/tests/python/test_warm_throughput_backup.py new file mode 100644 index 000000000..40ebbe9cd --- /dev/null +++ b/tests/python/test_warm_throughput_backup.py @@ -0,0 +1,121 @@ +# Copyright 2026 ExtendDB contributors +# SPDX-License-Identifier: Apache-2.0 + +"""WarmThroughput and backups (rows S6a to S6d, measured against Amazon DynamoDB +on 2026-10-23): a backup records no warm throughput, a restored table and its +indexes report the on-demand defaults whatever the source had set, and a +GlobalSecondaryIndexOverride entry carrying WarmThroughput is refused. The file +runs against ExtendDB and the service with the same assertions.""" + +from __future__ import annotations + +import time + +import pytest +from botocore.exceptions import ClientError + +from helpers import unique_name +from warm_throughput_helpers import ( # noqa: F401 (tables is a fixture) + GSI, + ON_DEMAND_DEFAULTS, + _create, + _validation, + _wait_all_active, + _wait_warm_active, + tables, +) + +GSI_OVERRIDE_WARM_THROUGHPUT = ( + "One or more parameter values were invalid: WarmThroughput is not supported on Global " + "Secondary Indexes via RestoreTableFromBackup" +) + + +def _wait_backup_available(client, arn: str, timeout: float = 900.0) -> dict: + deadline = time.monotonic() + timeout + while True: + desc = client.describe_backup(BackupArn=arn)["BackupDescription"] + if desc["BackupDetails"]["BackupStatus"] == "AVAILABLE": + return desc + if time.monotonic() > deadline: + raise TimeoutError(f"backup {arn} did not become AVAILABLE") + time.sleep(1) + + +def test_a_restored_table_reports_the_defaults_and_a_gsi_override_cannot_set_it( + dynamodb_client, tables +): + """Rows S6a to S6d.""" + name, _ = _create( + dynamodb_client, + tables, + BillingMode="PAY_PER_REQUEST", + WarmThroughput={"ReadUnitsPerSecond": 30000, "WriteUnitsPerSecond": 8000}, + GlobalSecondaryIndexes=[ + {**GSI, "WarmThroughput": {"ReadUnitsPerSecond": 24000, "WriteUnitsPerSecond": 8000}} + ], + ) + _wait_all_active(dynamodb_client, name) + _wait_warm_active(dynamodb_client, name, timeout=900) + # A just-created table refuses CreateBackup with + # ContinuousBackupsUnavailableException for a short while on the service + # (backups are still being enabled); ExtendDB accepts at once. Retried the + # way test_backup_restore_fidelity.py does. + deadline = time.monotonic() + 60 + while True: + try: + arn = dynamodb_client.create_backup( + TableName=name, BackupName=f"wt-{int(time.time())}" + )["BackupDetails"]["BackupArn"] + break + except ClientError as e: + code = e.response["Error"]["Code"] + if code != "ContinuousBackupsUnavailableException" or time.monotonic() >= deadline: + raise + time.sleep(2) + try: + # S6b: the backup records no warm throughput. + desc = _wait_backup_available(dynamodb_client, arn) + assert "WarmThroughput" not in desc["SourceTableDetails"] + for gsi in desc.get("SourceTableFeatureDetails", {}).get("GlobalSecondaryIndexes", []): + assert "WarmThroughput" not in gsi + + # S6c: no member while CREATING; the defaults once ACTIVE, on the table + # and the index, although the source had set larger values. + target = unique_name("wt-restore") + tables.append(target) + restored = dynamodb_client.restore_table_from_backup( + TargetTableName=target, BackupArn=arn + )["TableDescription"] + if restored["TableStatus"] == "CREATING": + assert "WarmThroughput" not in restored + assert all("WarmThroughput" not in g for g in restored["GlobalSecondaryIndexes"]) + _wait_all_active(dynamodb_client, target) + active = _wait_warm_active(dynamodb_client, target, timeout=900) + assert active["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "ACTIVE"} + assert active["GlobalSecondaryIndexes"][0]["WarmThroughput"] == { + **ON_DEMAND_DEFAULTS, + "Status": "ACTIVE", + } + + # S6d: an override entry may not carry WarmThroughput. + with pytest.raises(ClientError) as exc: + dynamodb_client.restore_table_from_backup( + TargetTableName=unique_name("wt-restore"), + BackupArn=arn, + GlobalSecondaryIndexOverride=[ + { + **GSI, + "WarmThroughput": { + "ReadUnitsPerSecond": 24000, + "WriteUnitsPerSecond": 8000, + }, + } + ], + ) + _validation(exc, GSI_OVERRIDE_WARM_THROUGHPUT) + finally: + try: + dynamodb_client.delete_backup(BackupArn=arn) + except ClientError: + pass diff --git a/tests/python/test_warm_throughput_updates.py b/tests/python/test_warm_throughput_updates.py new file mode 100644 index 000000000..3592a0bf3 --- /dev/null +++ b/tests/python/test_warm_throughput_updates.py @@ -0,0 +1,420 @@ +# Copyright 2026 ExtendDB contributors +# SPDX-License-Identifier: Apache-2.0 + +"""UpdateTable shapes around WarmThroughput measured in the review rounds of +2026-10-23: the GlobalSecondaryIndexUpdates element rules, the companions a +table-level WarmThroughput may and may not have, a GSI Create carrying +WarmThroughput, the empty request on CreateTable, and the defaults after +capacity and billing changes. Row labels refer to the service-r1, r2 and r3 +runs. The file runs against ExtendDB and the service; only waits and the +pre-existing IndexStatus divergence are gated on `is_extenddb`. +""" + +from __future__ import annotations + +import pytest +from botocore.exceptions import ClientError + +from helpers import unique_name +from warm_throughput_helpers import ( # noqa: F401 (tables is a fixture) + ATTRS, + DECREASE, + DELETION_PROTECTION_ONLY, + GSI, + ON_DEMAND_DEFAULTS, + ON_DEMAND_MIN, + ON_DEMAND_NEEDS_BILLING, + ONLY_OPERATION, + TABLE_CLASS_ONLY, + _create, + _sentence, + _validation, + _wait_all_active, + _wait_warm_active, + tables, +) + +ONE_ACTION = ( + "One or more parameter values were invalid: Only one global secondary index action is " + "allowed per GlobalSecondaryIndexUpdate object" +) +ONE_UPDATE_PER_INDEX = ( + "One or more parameter values were invalid: Only one global secondary index update per " + "index is allowed simultaneously. Index: gsi" +) +ON_DEMAND_UPDATE_MEMBERS = ( + "One or more parameter values were invalid: The only Updates for index: gsi when " + "TableThroughputMode is PAY_PER_REQUEST can be to OnDemandThroughput, WarmThroughput" +) +SSE_ONLY = ( + "One or more parameter values were invalid: Server-Side Encryption modification must be " + "the only operation in the request" +) +DELETE_WITH_WARM = ( + "One or more parameter values were invalid: Delete global secondary index cannot be " + "specified when updating WarmThroughput" +) +GSI_TWO = { + "IndexName": "gsi-two", + "KeySchema": [{"AttributeName": "G", "KeyType": "HASH"}], + "Projection": {"ProjectionType": "KEYS_ONLY"}, +} + + +def _on_demand_with_gsi(client, tables) -> str: + name, _ = _create(client, tables, BillingMode="PAY_PER_REQUEST", GlobalSecondaryIndexes=[GSI]) + _wait_all_active(client, name) + return name + + +class TestGsiUpdateElements: + def test_an_element_with_two_actions_is_refused(self, dynamodb_client, tables): + """Rows V1a and V1b: Update plus Create, and Create plus Delete (the Create + keys on a defined attribute, so only the element rule can refuse it).""" + name = _on_demand_with_gsi(dynamodb_client, tables) + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table( + TableName=name, + AttributeDefinitions=ATTRS, + GlobalSecondaryIndexUpdates=[{ + "Update": {"IndexName": "gsi", "WarmThroughput": ON_DEMAND_DEFAULTS}, + "Create": GSI_TWO, + }], + ) + _validation(exc, ONE_ACTION) + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table( + TableName=name, + AttributeDefinitions=ATTRS, + GlobalSecondaryIndexUpdates=[{"Create": GSI_TWO, "Delete": {"IndexName": "gsi"}}], + ) + _validation(exc, ONE_ACTION) + # T3b: Update plus Delete of the same index in one element answers the + # per-index sentence. + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table( + TableName=name, + GlobalSecondaryIndexUpdates=[{ + "Update": {"IndexName": "gsi", "WarmThroughput": ON_DEMAND_DEFAULTS}, + "Delete": {"IndexName": "gsi"}, + }], + ) + _validation(exc, ONE_UPDATE_PER_INDEX) + t = dynamodb_client.describe_table(TableName=name)["Table"] + assert [g["IndexName"] for g in t["GlobalSecondaryIndexes"]] == ["gsi"] + + def test_two_elements_on_one_index_are_refused(self, dynamodb_client, tables): + """Row V3c: Delete gsi and Update gsi WarmThroughput in two elements.""" + name = _on_demand_with_gsi(dynamodb_client, tables) + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table( + TableName=name, + GlobalSecondaryIndexUpdates=[ + {"Delete": {"IndexName": "gsi"}}, + {"Update": {"IndexName": "gsi", "WarmThroughput": ON_DEMAND_DEFAULTS}}, + ], + ) + _validation(exc, ONE_UPDATE_PER_INDEX) + + def test_an_update_without_warm_throughput_names_the_billing_mode( + self, dynamodb_client, tables + ): + """Rows T3d and T3e: an Update carrying a ProvisionedThroughput, or nothing, + on an on-demand table.""" + name = _on_demand_with_gsi(dynamodb_client, tables) + for update in ( + {"IndexName": "gsi", "ProvisionedThroughput": {"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}}, + {"IndexName": "gsi"}, + ): + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table( + TableName=name, GlobalSecondaryIndexUpdates=[{"Update": update}] + ) + _validation(exc, ON_DEMAND_UPDATE_MEMBERS) + + +INDEX_CHANGE_WITH_STREAM = ( + "One or more parameter values were invalid: You cannot create or delete index while " + "changing stream status" +) +STREAM_ON = {"StreamEnabled": True, "StreamViewType": "NEW_IMAGE"} +PROVISIONED_5 = {"ReadCapacityUnits": 5, "WriteCapacityUnits": 5} + + +def _refused(client, expected: str, **kwargs) -> None: + with pytest.raises(ClientError) as exc: + client.update_table(**kwargs) + _validation(exc, expected) + + +class TestWarmThroughputCompanions: + """The companions a table-level WarmThroughput may not have (rows S1a to S1d, + S9a, T4a to T4d, V2, V3a, V3b, V4). Every request here is refused, so one + table serves all of them.""" + + def test_sse_and_the_ordering_of_the_member_sentences(self, dynamodb_client, tables): + name = _on_demand_with_gsi(dynamodb_client, tables) + sse = {"Enabled": True} + cases = [ + ({"SSESpecification": sse}, SSE_ONLY), + # SSE outranks TableClass (S1a), DeletionProtection (S1b), Streams (S1c). + ({"SSESpecification": sse, "TableClass": "STANDARD_INFREQUENT_ACCESS"}, SSE_ONLY), + ({"SSESpecification": sse, "DeletionProtectionEnabled": False}, SSE_ONLY), + ({"SSESpecification": sse, "StreamSpecification": STREAM_ON}, SSE_ONLY), + # DeletionProtection outranks Streams (S1d). + ( + {"DeletionProtectionEnabled": False, "StreamSpecification": STREAM_ON}, + DELETION_PROTECTION_ONLY, + ), + # V4: the triple answers the TableClass sentence. + ( + { + "TableClass": "STANDARD_INFREQUENT_ACCESS", + "DeletionProtectionEnabled": False, + "StreamSpecification": STREAM_ON, + }, + TABLE_CLASS_ONLY, + ), + ] + for extra, expected in cases: + _refused(dynamodb_client, expected, TableName=name, WarmThroughput=ON_DEMAND_DEFAULTS, **extra) + + def test_on_demand_throughput_with_and_without_billing_mode(self, dynamodb_client, tables): + name = _on_demand_with_gsi(dynamodb_client, tables) + odt = {"MaxReadRequestUnits": 100000} + cases = [ + # T4a and T4d: the same sentence with and without BillingMode. + ({"BillingMode": "PAY_PER_REQUEST", "OnDemandThroughput": odt}, ON_DEMAND_NEEDS_BILLING), + ({"OnDemandThroughput": odt}, ON_DEMAND_NEEDS_BILLING), + # T4b and T4c: TableClass and DeletionProtection outrank it. + ({"TableClass": "STANDARD_INFREQUENT_ACCESS", "OnDemandThroughput": odt}, TABLE_CLASS_ONLY), + ({"DeletionProtectionEnabled": False, "OnDemandThroughput": odt}, DELETION_PROTECTION_ONLY), + ] + for extra, expected in cases: + _refused(dynamodb_client, expected, TableName=name, WarmThroughput=ON_DEMAND_DEFAULTS, **extra) + + def test_provisioned_throughput_alone_is_not_an_allowed_companion(self, dynamodb_client, tables): + """Row V2.""" + name = _on_demand_with_gsi(dynamodb_client, tables) + _refused( + dynamodb_client, + ONLY_OPERATION, + TableName=name, + WarmThroughput=ON_DEMAND_DEFAULTS, + ProvisionedThroughput=PROVISIONED_5, + ) + + def test_index_delete_is_refused_and_the_index_stays(self, dynamodb_client, tables): + """Rows S9a, V3a and V3b: Delete beside a table WarmThroughput; TableClass + outranks it; with Streams the service names the stream change.""" + name = _on_demand_with_gsi(dynamodb_client, tables) + delete = [{"Delete": {"IndexName": "gsi"}}] + _refused( + dynamodb_client, DELETE_WITH_WARM, TableName=name, + WarmThroughput=ON_DEMAND_DEFAULTS, GlobalSecondaryIndexUpdates=delete, + ) + _refused( + dynamodb_client, TABLE_CLASS_ONLY, TableName=name, + WarmThroughput=ON_DEMAND_DEFAULTS, GlobalSecondaryIndexUpdates=delete, + TableClass="STANDARD_INFREQUENT_ACCESS", + ) + _refused( + dynamodb_client, INDEX_CHANGE_WITH_STREAM, TableName=name, + WarmThroughput=ON_DEMAND_DEFAULTS, GlobalSecondaryIndexUpdates=delete, + StreamSpecification=STREAM_ON, + ) + t = _wait_all_active(dynamodb_client, name) + assert [g["IndexName"] for g in t["GlobalSecondaryIndexes"]] == ["gsi"] + assert t["GlobalSecondaryIndexes"][0]["WarmThroughput"] == { + **ON_DEMAND_DEFAULTS, "Status": "ACTIVE" + } + + def test_missing_table(self, dynamodb_client): + """Rows S10, T5a, T5c: a bad value alone, and an index Update alone, are + ResourceNotFoundException. Rows S10b, T5b, T5d: a companion is answered + first. Only the code is compared for the not-found message.""" + missing = unique_name("wt-missing") + for kwargs in ( + {"WarmThroughput": {"ReadUnitsPerSecond": 11999, "WriteUnitsPerSecond": 4000}}, + {"WarmThroughput": {"ReadUnitsPerSecond": 1}}, + { + "GlobalSecondaryIndexUpdates": [ + {"Update": {"IndexName": "gsi", "WarmThroughput": ON_DEMAND_DEFAULTS}} + ] + }, + ): + with pytest.raises(ClientError) as exc: + dynamodb_client.update_table(TableName=missing, **kwargs) + assert exc.value.response["Error"]["Code"] == "ResourceNotFoundException" + for extra, expected in ( + ({"TableClass": "STANDARD_INFREQUENT_ACCESS"}, TABLE_CLASS_ONLY), + ({"StreamSpecification": STREAM_ON}, ONLY_OPERATION), + ({"DeletionProtectionEnabled": False}, DELETION_PROTECTION_ONLY), + ): + _refused( + dynamodb_client, expected, TableName=missing, + WarmThroughput=ON_DEMAND_DEFAULTS, **extra, + ) + + +class TestWarmThroughputCreateShapes: + def test_create_with_an_empty_warm_throughput_reports_no_member_while_creating( + self, dynamodb_client, tables + ): + """Row T1: `WarmThroughput: {}` on CreateTable is accepted, the response + while CREATING carries no member, and the defaults read once ACTIVE.""" + name, created = _create( + dynamodb_client, tables, BillingMode="PAY_PER_REQUEST", WarmThroughput={} + ) + if created["TableStatus"] == "CREATING": + assert "WarmThroughput" not in created + active = _wait_all_active(dynamodb_client, name) + active = _wait_warm_active(dynamodb_client, name, timeout=900) + assert active["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "ACTIVE"} + + def test_gsi_create_with_warm_throughput_in_update_table( + self, dynamodb_client, tables, is_extenddb + ): + """Rows S8a and V5: a GSI created through GlobalSecondaryIndexUpdates with + WarmThroughput reports the values as sent with Status UPDATING, then the + same values ACTIVE. The service also reports the new index CREATING and the + table UPDATING; ExtendDB reports both ACTIVE at once (the index build is + immediate), so those two statuses are asserted on the service only.""" + name = _on_demand_with_gsi(dynamodb_client, tables) + sent = {"ReadUnitsPerSecond": 30000, "WriteUnitsPerSecond": 8000} + resp = dynamodb_client.update_table( + TableName=name, + AttributeDefinitions=ATTRS, + GlobalSecondaryIndexUpdates=[{"Create": {**GSI_TWO, "WarmThroughput": sent}}], + ) + t = resp["TableDescription"] + by_name = {g["IndexName"]: g for g in t["GlobalSecondaryIndexes"]} + assert by_name["gsi-two"]["WarmThroughput"] == {**sent, "Status": "UPDATING"} + assert by_name["gsi"]["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "ACTIVE"} + if not is_extenddb: + assert t["TableStatus"] == "UPDATING" + assert by_name["gsi-two"]["IndexStatus"] == "CREATING" + described = dynamodb_client.describe_table(TableName=name)["Table"] + new = {g["IndexName"]: g for g in described["GlobalSecondaryIndexes"]}["gsi-two"] + if new["WarmThroughput"]["Status"] == "UPDATING": + assert new["WarmThroughput"] == {**sent, "Status": "UPDATING"} + _wait_all_active(dynamodb_client, name) + settled = _wait_warm_active(dynamodb_client, name, timeout=900) + by_name = {g["IndexName"]: g for g in settled["GlobalSecondaryIndexes"]} + assert by_name["gsi-two"]["WarmThroughput"] == {**sent, "Status": "ACTIVE"} + assert by_name["gsi"]["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "ACTIVE"} + + def test_gsi_create_below_the_minimum_names_the_index(self, dynamodb_client, tables): + """Row S8b.""" + name = _on_demand_with_gsi(dynamodb_client, tables) + _refused( + dynamodb_client, + _sentence("ReadUnitsPerSecond", "index gsi3", ON_DEMAND_MIN), + TableName=name, + AttributeDefinitions=ATTRS, + GlobalSecondaryIndexUpdates=[{"Create": { + **GSI_TWO, + "IndexName": "gsi3", + "WarmThroughput": {"ReadUnitsPerSecond": 11999, "WriteUnitsPerSecond": 4000}, + }}], + ) + + +class TestWarmThroughputAcrossBillingChanges: + """The values a provisioned table reports around capacity and billing mode + changes (rows S3, S7a to S7c, T7a to T7d). Statuses the service reports while + the table is UPDATING for a capacity or billing change, and the previous + values it answers with, are asserted on the service only: ExtendDB changes + capacity and billing mode with the table ACTIVE, and answers a switch to + PAY_PER_REQUEST with the on-demand values.""" + + def test_switch_to_on_demand_raises_a_low_warm_throughput( + self, dynamodb_client, tables, is_extenddb + ): + """Rows T7a to T7d and S7c: a read-only record {100} on a provisioned 5/5 + table reads {100, 5}, follows a capacity change to {100, 10}, and reads + {12000, 4000} once the table is on-demand; the on-demand values are then + not a decrease.""" + name, _ = _create( + dynamodb_client, + tables, + BillingMode="PROVISIONED", + ProvisionedThroughput=PROVISIONED_5, + WarmThroughput={"ReadUnitsPerSecond": 100}, + ) + _wait_all_active(dynamodb_client, name) + active = _wait_warm_active(dynamodb_client, name, timeout=900) + assert active["WarmThroughput"] == { + "ReadUnitsPerSecond": 100, "WriteUnitsPerSecond": 5, "Status": "ACTIVE" + } + dynamodb_client.update_table( + TableName=name, + ProvisionedThroughput={"ReadCapacityUnits": 10, "WriteCapacityUnits": 10}, + ) + _wait_all_active(dynamodb_client, name) + assert dynamodb_client.describe_table(TableName=name)["Table"]["WarmThroughput"] == { + "ReadUnitsPerSecond": 100, "WriteUnitsPerSecond": 10, "Status": "ACTIVE" + } + resp = dynamodb_client.update_table(TableName=name, BillingMode="PAY_PER_REQUEST") + if not is_extenddb: + assert resp["TableDescription"]["WarmThroughput"] == { + "ReadUnitsPerSecond": 100, "WriteUnitsPerSecond": 10, "Status": "ACTIVE" + } + _wait_all_active(dynamodb_client, name) + active = _wait_warm_active(dynamodb_client, name, timeout=900) + assert active["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "ACTIVE"} + resp = dynamodb_client.update_table(TableName=name, WarmThroughput=ON_DEMAND_DEFAULTS) + assert resp["TableDescription"]["WarmThroughput"] == { + **ON_DEMAND_DEFAULTS, "Status": "UPDATING" + } + + def test_never_set_warm_throughput_across_capacity_and_billing_changes( + self, dynamodb_client, tables, is_extenddb + ): + """Rows S3, S7a and S7b: a provisioned 5/5 table that never set warm + throughput reads {5, 5}, then {10, 10} after a capacity change, then + {12000, 4000} on-demand, and keeps {12000, 4000} after the switch back to + provisioned.""" + name, _ = _create( + dynamodb_client, tables, BillingMode="PROVISIONED", ProvisionedThroughput=PROVISIONED_5 + ) + _wait_all_active(dynamodb_client, name) + active = _wait_warm_active(dynamodb_client, name, timeout=900) + assert active["WarmThroughput"] == { + "ReadUnitsPerSecond": 5, "WriteUnitsPerSecond": 5, "Status": "ACTIVE" + } + + # S3: the service answers the capacity change with the old units. + resp = dynamodb_client.update_table( + TableName=name, + ProvisionedThroughput={"ReadCapacityUnits": 10, "WriteCapacityUnits": 10}, + ) + if not is_extenddb: + assert resp["TableDescription"]["WarmThroughput"] == { + "ReadUnitsPerSecond": 5, "WriteUnitsPerSecond": 5, "Status": "ACTIVE" + } + _wait_all_active(dynamodb_client, name) + assert dynamodb_client.describe_table(TableName=name)["Table"]["WarmThroughput"] == { + "ReadUnitsPerSecond": 10, "WriteUnitsPerSecond": 10, "Status": "ACTIVE" + } + + # S7a: on-demand. + resp = dynamodb_client.update_table(TableName=name, BillingMode="PAY_PER_REQUEST") + if not is_extenddb: + assert resp["TableDescription"]["WarmThroughput"] == { + "ReadUnitsPerSecond": 10, "WriteUnitsPerSecond": 10, "Status": "ACTIVE" + } + _wait_all_active(dynamodb_client, name) + active = _wait_warm_active(dynamodb_client, name, timeout=900) + assert active["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "ACTIVE"} + + # S7b: back to provisioned; the on-demand values stay. + resp = dynamodb_client.update_table( + TableName=name, BillingMode="PROVISIONED", ProvisionedThroughput=PROVISIONED_5 + ) + assert resp["TableDescription"]["WarmThroughput"] == { + **ON_DEMAND_DEFAULTS, "Status": "ACTIVE" + } + _wait_all_active(dynamodb_client, name) + active = _wait_warm_active(dynamodb_client, name, timeout=900) + assert active["WarmThroughput"] == {**ON_DEMAND_DEFAULTS, "Status": "ACTIVE"} diff --git a/tests/python/warm_throughput_helpers.py b/tests/python/warm_throughput_helpers.py new file mode 100644 index 000000000..bd9204b4a --- /dev/null +++ b/tests/python/warm_throughput_helpers.py @@ -0,0 +1,124 @@ +# Copyright 2026 ExtendDB contributors +# SPDX-License-Identifier: Apache-2.0 + +"""Shared constants, waits and the `tables` fixture for the WarmThroughput +wire tests (`test_warm_throughput.py`, `test_warm_throughput_updates.py`). +Every sentence here was measured against Amazon DynamoDB on 2026-10-23.""" + +from __future__ import annotations + +import time + +import pytest + +from helpers import unique_name, wait_for_active, wait_for_deleted + +ON_DEMAND_DEFAULTS = {"ReadUnitsPerSecond": 12000, "WriteUnitsPerSecond": 4000} +PREFIX = "One or more parameter values were invalid: Requested " +ON_DEMAND_MIN = ( + " is lower than initial throughput for OnDemand. See: " + "https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/" + "on-demand-capacity-mode.html#on-demand-capacity-mode-initial" +) +DECREASE = " is lower than current WarmThroughput, decreasing WarmThroughput is not supported" +ONLY_OPERATION = ( + "One or more parameter values were invalid: WarmThroughput must be the only operation " + "in the request" +) +TABLE_CLASS_ONLY = ( + "One or more parameter values were invalid: TableClass modification must be the only " + "operation in the request" +) +DELETION_PROTECTION_ONLY = ( + "One or more parameter values were invalid: DeletionProtection modification must be the " + "only operation in the request" +) +ON_DEMAND_NEEDS_BILLING = ( + "One or more parameter values were invalid: OnDemandThroughput can only be combined with " + "BillingMode in an UpdateTable request" +) + + +def _sentence(member: str, subject: str, suffix: str) -> str: + """`Requested for WarmThroughput for `.""" + return f"{PREFIX}{member} for WarmThroughput for {subject}{suffix}" + +PK = [{"AttributeName": "PK", "KeyType": "HASH"}, {"AttributeName": "SK", "KeyType": "RANGE"}] +ATTRS = [ + {"AttributeName": "PK", "AttributeType": "S"}, + {"AttributeName": "SK", "AttributeType": "S"}, + {"AttributeName": "G", "AttributeType": "S"}, +] +GSI = { + "IndexName": "gsi", + "KeySchema": [{"AttributeName": "G", "KeyType": "HASH"}], + "Projection": {"ProjectionType": "ALL"}, +} +LSI = { + "IndexName": "lsi", + "KeySchema": [{"AttributeName": "PK", "KeyType": "HASH"}, {"AttributeName": "G", "KeyType": "RANGE"}], + "Projection": {"ProjectionType": "KEYS_ONLY"}, +} + + +def _wait_all_active(client, name: str, timeout: float = 900.0) -> dict: + """The description once the table and every GSI are ACTIVE.""" + wait_for_active(client, name, timeout) + deadline = time.monotonic() + timeout + while True: + t = client.describe_table(TableName=name)["Table"] + if all(g["IndexStatus"] == "ACTIVE" for g in t.get("GlobalSecondaryIndexes", [])): + return t + if time.monotonic() > deadline: + raise TimeoutError(f"indexes of {name} did not become ACTIVE") + time.sleep(1) + + +def _wait_warm_active(client, name: str, timeout: float = 30.0) -> dict: + """The description once the table's and every GSI's WarmThroughput is ACTIVE.""" + deadline = time.monotonic() + timeout + while True: + t = client.describe_table(TableName=name)["Table"] + statuses = [t["WarmThroughput"]["Status"]] + [ + g["WarmThroughput"]["Status"] for g in t.get("GlobalSecondaryIndexes", []) + ] + if all(s == "ACTIVE" for s in statuses): + return t + if time.monotonic() > deadline: + raise TimeoutError(f"WarmThroughput of {name} stayed {statuses}") + time.sleep(0.2) + + +def _validation(exc_info, expected: str) -> None: + err = exc_info.value.response["Error"] + assert err["Code"] == "ValidationException", err + assert err["Message"] == expected, err["Message"] + + +@pytest.fixture() +def tables(dynamodb_client): + """Register created table names for deletion after the test.""" + names: list[str] = [] + yield names + for name in names: + try: + dynamodb_client.update_table(TableName=name, DeletionProtectionEnabled=False) + except Exception: + pass + try: + dynamodb_client.delete_table(TableName=name) + wait_for_deleted(dynamodb_client, name, 600) + except Exception: + pass + + +def _create(client, tables, **kwargs) -> tuple[str, dict]: + name = unique_name("wt") + tables.append(name) + # The G attribute is defined only when an index keys on it: an unused + # definition is refused by the service and by ExtendDB. + indexed = "GlobalSecondaryIndexes" in kwargs or "LocalSecondaryIndexes" in kwargs + attrs = ATTRS if indexed else ATTRS[:2] + args = {"TableName": name, "KeySchema": PK, "AttributeDefinitions": attrs} + args.update(kwargs) + return name, client.create_table(**args)["TableDescription"] diff --git a/tests/terraform/install-terraform.sh b/tests/terraform/install-terraform.sh new file mode 100755 index 000000000..ef87d6818 --- /dev/null +++ b/tests/terraform/install-terraform.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash +# Copyright 2026 ExtendDB contributors +# SPDX-License-Identifier: Apache-2.0 +# +# Install a pinned Terraform release into a directory, checking the zip +# against the checksum pinned below and against the SHA256SUMS file HashiCorp +# publishes for the release. Prints the path of the binary. +# +# tests/terraform/install-terraform.sh [DIR] (default: $TMPDIR/terraform-) +# +# Same version and checksum for local runs and for CI (the integration +# workflows call this script), so a run cannot silently pick up a different +# Terraform. + +set -euo pipefail + +TERRAFORM_VERSION="1.17.0" +# sha256 of terraform_1.17.0_linux_amd64.zip, from +# https://releases.hashicorp.com/terraform/1.17.0/terraform_1.17.0_SHA256SUMS +TERRAFORM_SHA256="c168cab7c637a8769fdb90089ddd19de701bc6ff8aca64dc2e2185fe46f59f67" + +dir="${1:-${TMPDIR:-/tmp}/terraform-${TERRAFORM_VERSION}}" +mkdir -p "$dir" +if [[ -x "$dir/terraform" ]] && "$dir/terraform" version | grep -q "Terraform v${TERRAFORM_VERSION}\$"; then + echo "$dir/terraform" + exit 0 +fi + +zip="terraform_${TERRAFORM_VERSION}_linux_amd64.zip" +base="https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}" +curl -sSf -o "$dir/$zip" "$base/$zip" +curl -sSf -o "$dir/SHA256SUMS" "$base/terraform_${TERRAFORM_VERSION}_SHA256SUMS" + +published=$(awk -v f="$zip" '$2 == f { print $1 }' "$dir/SHA256SUMS") +actual=$(sha256sum "$dir/$zip" | awk '{ print $1 }') +if [[ "$published" != "$TERRAFORM_SHA256" ]]; then + echo "error: published checksum for $zip is $published, pinned $TERRAFORM_SHA256" >&2 + exit 1 +fi +if [[ "$actual" != "$TERRAFORM_SHA256" ]]; then + echo "error: $zip has checksum $actual, pinned $TERRAFORM_SHA256" >&2 + exit 1 +fi + +unzip -o -q "$dir/$zip" -d "$dir" +rm -f "$dir/$zip" "$dir/SHA256SUMS" +echo "$dir/terraform" diff --git a/tests/terraform/warm-throughput/main.tf b/tests/terraform/warm-throughput/main.tf new file mode 100644 index 000000000..f52a8498c --- /dev/null +++ b/tests/terraform/warm-throughput/main.tf @@ -0,0 +1,95 @@ +# Copyright 2026 ExtendDB contributors +# SPDX-License-Identifier: Apache-2.0 +# +# The configuration from hashicorp/terraform-provider-aws#44344 (the table +# whose apply hung against an emulator that reported no WarmThroughput), plus +# a second table with a global secondary index, against the ExtendDB under +# test. Since provider 6.13.0 the aws_dynamodb_table resource waits for +# WarmThroughput.Status to read ACTIVE after every table and index create or +# update; run.sh asserts the apply completes and a second plan has no changes. + +terraform { + required_providers { + aws = { + source = "hashicorp/aws" + version = "6.70.0" + } + } +} + +variable "endpoint" { + description = "The ExtendDB endpoint, for example https://127.0.0.1:18443" + type = string +} + +variable "access_key" { + type = string + sensitive = true +} + +variable "secret_key" { + type = string + sensitive = true +} + +variable "insecure" { + description = "Accept the server's self-signed TLS certificate" + type = bool + default = true +} + +provider "aws" { + region = "us-east-1" + access_key = var.access_key + secret_key = var.secret_key + insecure = var.insecure + + skip_credentials_validation = true + skip_metadata_api_check = true + skip_requesting_account_id = true + + endpoints { + dynamodb = var.endpoint + } +} + +resource "aws_dynamodb_table" "example" { + name = "example" + billing_mode = "PAY_PER_REQUEST" + + hash_key = "PK" + range_key = "SK" + + attribute { + name = "PK" + type = "S" + } + + attribute { + name = "SK" + type = "S" + } +} + +resource "aws_dynamodb_table" "with_index" { + name = "example-with-index" + billing_mode = "PAY_PER_REQUEST" + + hash_key = "PK" + + attribute { + name = "PK" + type = "S" + } + + attribute { + name = "G" + type = "S" + } + + global_secondary_index { + name = "gsi" + hash_key = "G" + projection_type = "ALL" + } +} diff --git a/tests/terraform/warm-throughput/run.sh b/tests/terraform/warm-throughput/run.sh new file mode 100755 index 000000000..e41483c48 --- /dev/null +++ b/tests/terraform/warm-throughput/run.sh @@ -0,0 +1,89 @@ +#!/usr/bin/env bash +# Copyright 2026 ExtendDB contributors +# SPDX-License-Identifier: Apache-2.0 +# +# Apply tests/terraform/warm-throughput/main.tf against an ExtendDB and assert +# the apply completes within five minutes and a second plan has no changes. +# The tables are destroyed afterwards. +# +# Environment: +# EXTENDDB_TEST_ENDPOINT the server, for example https://127.0.0.1:18443 +# AWS_ACCESS_KEY_ID a key the server accepts +# AWS_SECRET_ACCESS_KEY +# TERRAFORM the terraform binary (default: install-terraform.sh) +# +# tests/terraform/warm-throughput/run.sh +# +# Exit 0 when the apply and the plan both pass; non-zero otherwise, with the +# failing step named. + +set -euo pipefail + +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +endpoint="${EXTENDDB_TEST_ENDPOINT:?set EXTENDDB_TEST_ENDPOINT}" +: "${AWS_ACCESS_KEY_ID:?set AWS_ACCESS_KEY_ID}" +: "${AWS_SECRET_ACCESS_KEY:?set AWS_SECRET_ACCESS_KEY}" +terraform="${TERRAFORM:-$("$here/../install-terraform.sh")}" +[[ -x "$terraform" ]] || { echo "FAIL: terraform binary $terraform not found or not executable"; exit 1; } + +# A scratch working copy, so the repository never holds provider binaries, +# lock files or state. +work="$(mktemp -d "${TMPDIR:-/tmp}/extenddb-tf-warm-throughput.XXXXXX")" +# The tables use fixed names, so a run that leaves them behind makes the next +# apply fail with ResourceInUseException: remove them on every exit once the +# apply has started (a failed assertion included). +applied=0 +cleanup() { + if [[ $applied -eq 1 ]]; then + timeout 300 "$terraform" destroy -auto-approve -no-color >/dev/null 2>&1 || true + fi + rm -rf "$work" +} +trap cleanup EXIT +cp "$here/main.tf" "$work/" +cd "$work" + +export TF_IN_AUTOMATION=1 +export TF_INPUT=0 +export TF_VAR_endpoint="$endpoint" +export TF_VAR_access_key="$AWS_ACCESS_KEY_ID" +export TF_VAR_secret_key="$AWS_SECRET_ACCESS_KEY" +# Keep the provider's own credential chain out of it. +unset AWS_PROFILE AWS_SESSION_TOKEN + +echo "=== terraform init ($("$terraform" version | head -1))" +"$terraform" init -no-color >init.log 2>&1 || { cat init.log; echo "FAIL: terraform init"; exit 1; } + +echo "=== terraform apply (5 minute limit)" +applied=1 +set +e +timeout 300 "$terraform" apply -auto-approve -no-color >apply.log 2>&1 +apply_status=$? +set -e +if [[ $apply_status -ne 0 ]]; then + cat apply.log + if [[ $apply_status -eq 124 ]]; then + echo "FAIL: terraform apply did not finish within 300 s (the provider waits for WarmThroughput.Status ACTIVE)" + else + echo "FAIL: terraform apply exited $apply_status" + fi + exit 1 +fi +grep -E "Apply complete! Resources: 2 added" apply.log || { cat apply.log; echo "FAIL: expected 2 resources added"; exit 1; } + +echo "=== terraform plan (must report no changes)" +set +e +timeout 300 "$terraform" plan -detailed-exitcode -no-color >plan.log 2>&1 +plan_status=$? +set -e +if [[ $plan_status -ne 0 ]]; then + cat plan.log + echo "FAIL: terraform plan exited $plan_status (2 means the applied tables drifted from the configuration)" + exit 1 +fi +grep -E "No changes" plan.log || { cat plan.log; echo "FAIL: terraform plan did not report No changes"; exit 1; } + +echo "=== terraform destroy" +timeout 300 "$terraform" destroy -auto-approve -no-color >destroy.log 2>&1 || { cat destroy.log; echo "FAIL: terraform destroy"; exit 1; } +applied=0 +echo "PASS: apply completed, second plan had no changes, tables destroyed"