Skip to content

Key schema definition checks for CreateTable and UpdateTable - #595

Open
LeeroyHannigan wants to merge 4 commits into
fix/517-519-520-stream-update-table-sentencesfrom
fix/index-key-definition-sentences
Open

LeeroyHannigan wants to merge 4 commits into
fix/517-519-520-stream-update-table-sentencesfrom
fix/index-key-definition-sentences

Conversation

@LeeroyHannigan

Copy link
Copy Markdown
Collaborator

Fixes #554. Fixes #555. Stacked on #548, which owns the UpdateTable sentences this changes.

CreateTable and an UpdateTable GSI Create now check AttributeDefinitions against each key schema with DynamoDB's rules and sentences, and refuse a name defined twice.

What the service answers

Measured on 2026-10-25 and on 2026-10-26 at 01:27, 01:29, 02:27 and 02:34 UTC in us-east-1, each request twice with the same answer. No request created or changed a table: every CreateTable named an existing table, and every UpdateTable carried a key with no definition or a duplicate the service refuses.

  • A table billing fault (throughput given with PAY_PER_REQUEST, or missing with PROVISIONED) is reported before any definition fault.
  • A name defined twice answers Attribute Name is duplicated: <name>, naming the first name whose second definition comes first ([b, a, b, a] names b). It comes before a missing key and applies to two definitions of one name with different types. The service checks it after the rule that one attribute cannot be both HASH and RANGE and after the length rule below, and before the other key schema checks.
  • The table's key schema is checked first, then each LSI's, then each GSI's.
  • A schema with more keys than there are definitions answers Invalid KeySchema: Some index key attribute have no definition. This is the rule behind fix(streams, update-table): CreationRequestDateTime; measured UpdateTable sentences and validation order #548's empty-member row and the "two or more missing keys" row of The Keys and AttributeDefinitions lists in the undefined-index-key sentence differ from the service #554: definitions [pk] for keys g, r answer it, while [pk, z] for the same keys answer the listing sentence below.
  • Otherwise, a key with no definition answers One or more parameter values were invalid: Some index key attributes are not defined in AttributeDefinitions. Keys: [...], AttributeDefinitions: [...], with every key of that schema and every definition.
  • On UpdateTable, a definition that redefines the table's own key with another type beside a valid Create is accepted, and the table keeps its type (UpdateTable accepts a redefined table key type and a duplicated AttributeDefinitions entry #555's first row).

The service orders the two lists in a way that is neither the request's nor alphabetical and does not depend on the attribute types: [pk, g] renders [pk, g], [g, pk] renders [g, pk], [pk, r] renders [r, pk], and keys hk, rk render [rk, hk]. ExtendDB lists keys in key schema order and definitions in request order.

What changed

The redefinition row needed no code change: the backends already keep the stored definition of a name the table has.

Not changed here

Measured differences that exist on #548's head and on main as well:

  • CreateTable checks the key schema shape (first element not HASH, second not RANGE, one name as HASH and RANGE) and the vector index attribute rules before the table billing, the duplicate and the key definition checks; the service checks the billing, the duplicate and the definitions first. UpdateTable checks vector index faults before the duplicate and the GSI listing.
  • The texts for a first element that is not HASH, a second that is not RANGE, PROVISIONED without throughput, and one name as table HASH and RANGE given as many definitions as keys differ from the service's.

Tests

tests/test_key_schema_definitions.py: 31 dual-target wire tests from the measurements. The lists in the listing sentence are compared as sets. The redefinition test waits for its index to be active before the table is deleted.

Run locally against SQLite:

  • the new file at the head: 31 passed; on fix(streams, update-table): CreationRequestDateTime; measured UpdateTable sentences and validation order #548's head: 27 failed, 4 passed. The 4 that pass there are the one-key UpdateTable listing ([g] against [pk]), the one-key CreateTable listing ([pk] against [x]), the billing fault beside a duplicate (fix(streams, update-table): CreationRequestDateTime; measured UpdateTable sentences and validation order #548 has no duplicate check) and the redefinition row.
  • at the head, every file under tests/ matching update_table, create_table, table_operations, gsi, lsi, throughput or billing, with the new file: 101 passed, 8 skipped, and 2 failed in tests/test_gsi_async.py, which calls the settings CLI at ./target/release/extenddb, a path these runs do not provide; it passes when the CLI is there.
  • at the head, tests/python files matching table, gsi, lsi, index or stream: 104 passed.
  • cargo +1.97.0 fmt --all -- --check, cargo +1.97.0 clippy --all-targets -- -D warnings, cargo test --workspace --exclude extenddb-storage-cassandra on stable and on 1.88.0 (1,775 passed each), and cargo test -p extenddb-storage-cassandra --lib (61 passed).

CI runs the new file in run-integration, run-integration-sqlite, mongodb-pytest and cassandra-pytest.

Files

crates/core/src/validation/mod.rs goes from 3,805 to 3,804 lines and crates/engine/src/update_table.rs from 384 to 355. The new checks are in the 169-line key_definitions.rs.

Merging

This is based on #548's branch. After #548 merges, retarget it to main.

git merge-tree --write-tree against the 22 open PRs that change validation/mod.rs, update_table.rs or update_table_tests.rs adds conflicts with five. #481 and #485 each declare a module and a pub use on the lines this adds at the top of validation/mod.rs; keep both sides. #399, #507 and #531 each add a row to the Control Plane table of docs/differences-from-dynamodb.md after the same line as this change's row; keep both rows. The other conflicts are the same with or without this change; #474 moves validate_gsi_updates out of update_table.rs, so it carries the new GSI Create branch to its new place.

CreateTable and an UpdateTable GSI Create refused an undefined key attribute
with one sentence that named only the first missing key. Amazon DynamoDB
(measured 2026-10-25 and 2026-10-26 at 01:30 and 01:36 UTC, each request
twice) applies these rules:

- A name defined twice in AttributeDefinitions answers `Attribute Name is
  duplicated: <name>`, naming the first name whose second definition comes
  first, before any key check. ExtendDB accepted duplicates.
- The table's key schema is checked first, then each LSI's, then each GSI's.
  A schema with more keys than there are definitions answers `Invalid
  KeySchema: Some index key attribute have no definition`; otherwise a key
  with no definition answers `Some index key attributes are not defined in
  AttributeDefinitions`, listing every key of that schema and every
  definition.

The two checks live in the new validation/key_definitions.rs and both
operations call them. The service orders the two lists in a way that follows
neither the request nor the alphabet; ExtendDB lists them in request order.
The UpdateTable unit test row that expected the listing sentence for one
definition and two keys now expects the key schema sentence, as measured.
Dual-target wire tests from the 2026-10-25 and 2026-10-26 measurements:
UpdateTable GSI Create with a missing hash or range key, both keys, fewer
definitions than keys and names defined twice; CreateTable with missing keys
on the table, an LSI and GSIs in each order, fewer definitions than keys and
names defined twice; and an UpdateTable that redefines the table's own key
with another type, which keeps the stored type. The lists in the sentence are
compared as sets, since the service's order is not the request's.
…order documented

Amazon DynamoDB reports a CreateTable billing fault (throughput with
PAY_PER_REQUEST, or none with PROVISIONED) before a duplicated or missing
definition (measured 2026-10-26 at 02:27 and 02:34 UTC, each case twice).
The billing check now runs before the definition checks.

The duplicate check compared every definition with every earlier one, and
UpdateTable runs it once per GSI Create: 8,000 definitions with 200 Creates
took 22.9 s. It now makes one pass with a set of seen names and still names
the first name whose second definition comes first.

The module comment states where the service checks a duplicate (after the
billing checks, the one-name-twice rule and the length rule) and where
ExtendDB does (after the key schema shape checks).
docs/differences-from-dynamodb.md records the order of the two lists in the
listing sentence.
CreateTable with PAY_PER_REQUEST and throughput beside a duplicated name, a
missing key, or fewer definitions than keys answers the billing sentence
(measured 2026-10-26 at 02:27 and 02:34 UTC). The redefinition test waits
for its new index to be active before the table is deleted, since the
service refuses DeleteTable while an index is being created, and writes an
item with the stored key type. The measurement times in the docstring are
corrected to 01:27 and 01:29 UTC, plus the two later runs.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants