[LeeroyHannigan]

Summary

On PostgreSQL, CreateTable with U+0000 inside a key schema or attribute definition name answered 500 at the catalog insert, because jsonb rejects the \^@ escape (#367). The service answers 200 and returns the name byte for byte. On main the same byte in a table name answered 500 Internal error during authorization on DescribeTable, UpdateTable and DeleteTable, and 500 on PutItem, GetItem, Query, Scan and the batch and transaction forms, because the authorization key info and resource tag lookups hand the name to PostgreSQL as a TEXT parameter; the service answers 400 with the member pattern sentence at tableName.

Two changes, one per cause.

Catalog escape (PostgreSQL)

catalog_json.rs writes the seven catalog columns that carry attribute names (tables.key_schema, tables.attribute_definitions, indexes.key_schema, indexes.projection, backups.key_schema, backups.attribute_definitions, backup_definitions.definition) through an order-preserving escape (U+0000 as U+0001 U+0001, U+0001 as U+0001 U+0002) and reads them back through its inverse at every decode site (table describe, index resolution, stream describe, backup describe, restore source and target). The alphabet is the one #368 uses for item data, so one decoder reads both. DescribeTable, DescribeStream, DescribeBackup and a restored table return every name byte-identical; measured against the service, a backup and restore of a table with a U+0000 GSI key attribute name round-trips the same way.

Migration, catalog 0.0.5

An earlier release stored a name containing U+0001 raw, which the new decoder would read as an escape pair. 004_escape_catalog_control_chars.sql (catalog 0.0.5) is preceded by a code step, catalog_escape.rs, that re-encodes every document holding a raw control character in those seven columns, one transaction per table with a (migration, table) marker committed with the rewrite, so an interrupted run resumes with the tables not yet marked and no row is visited twice. The escape is not repeatable (an escaped U+0001 and a raw pair are the same bytes), so the step refuses to run while any other session holds the catalog database, naming the sessions; sessions whose application_name is extenddb-migrate (the migrate command's own) are left out, and EXTENDDB_MIGRATE_IGNORE_CONNECTIONS=1 skips the check for idle pooler connections. serve refuses a 0.0.4 catalog with the version sentence and refuses a catalog stamped 0.0.5 that lacks the ledger record, naming the file. After this release a table whose key attribute, index key or projected attribute names contain U+0000 or U+0001 on PostgreSQL is not readable by an earlier release; the upgrade manual carries the sequence and that sentence.

Walked end to end with the release binary: a 0.0.4 catalog built from the previous release's migration files, seeded raw by SQL with a<U+0001><U+0001>z as the key attribute name, g<U+0001><U+0002>k as a GSI key and n<U+0001><U+0001> as a projected name in all seven columns; serve refuses (expected 0.0.5, found 0.0.4); migrate previews Pending: catalog 0.0.4 -> 0.0.5; data migrations [catalog 004_escape_catalog_control_chars.sql] and exits 3; migrate --yes re-encodes (stored text a\^A\^Az to a\^A\^B\^A\^Bz) and stamps 0.0.5; the names come back byte-identical through DescribeTable, DescribeBackup and a restore; a second run changes no byte in any of the seven columns; a psql session held open makes it refuse, and it runs once the session closes. init on a fresh catalog records the migration with zero updates per table.

Composition with #368 (fix/nul-in-strings): same alphabet, disjoint tables. #368's 004_escape_control_chars rewrites the data database's item columns and backup_items; this one rewrites the seven catalog columns; neither reads the other's tables, so they run in either order. Both bump CATALOG_VERSION to 0.0.5 with a different 004 filename, so whichever lands second resolves five hunks: storage-postgres/src/bootstrapper.rs (the application name constant and the trait method, which crates/app/src/migrate_connection.rs shares byte for byte), lib.rs (the version doc comment), migrations.rs (the entry and the count tripwire), tests/catalog_converge.rs (the 004 file the scratch applies) and docs/manuals/07-upgrade-manual.md. The resolution: one 004 SQL file stamping 0.0.5 (or the second renumbered to 005 and 0.0.6), one MIGRATE_APPLICATION_NAME, and the exclusive-access check covering both databases; each migration's own record keeps protecting its own tables. Every other file merges clean (git merge-tree).

Table name and ARN check before authorization (every backend)

crates/server/src/table_name_gate.rs and arn_gate.rs check every table name and ARN a request carries before the key info and tag lookups, with the sentence the service was measured to give:

• the member pattern at tableName, targetTableName (RestoreTableFromBackup, RestoreTableToPointInTime), sourceTableName, tableCreationParameters.tableName (ImportTable), exclusiveStartTableName, indexName (Query, Scan, UpdateTable create and delete) and transactItems.<n>.member.<op>.tableName; when a bare name also fails a length bound, 2 validation errors detected: lists the length clause first, joined by ; ;
• the ARN map-key constraint list at requestItems (BatchGetItem, BatchWriteItem);
• for an arn: value in a table name field: Table name of ARN must satisfy ... for a plain table ARN with a bad table segment, Invalid subscriber ID or Invalid AWS region for a bad account or region segment, and the Valid ARN format is 'arn:<awsPartition>:<vendor>:<region>:<subscriber>:resourceType/resourceName' ... frame for anything else (another service, an index or stream suffix, too few fields); measured today, an existing table's own index ARN or stream ARN in TableName answers that frame on Query, GetItem, Scan and DescribeTable, where main answered ResourceNotFoundException;
• Invalid TableArn: Invalid Table ARN for TableArn (Export) and Invalid Request: Invalid TableArn: Invalid Table ARN for SourceTableArn (point-in-time restore); Invalid StreamArn for a stream ARN with a bad table segment or a U+0000 inside the label; Invalid Backup ARN for a backup ARN with a bad table segment (U+0000 in the id keeps the engine's Invalid Request: BackupArn is not valid); Invalid TableArn: Invalid ResourceArn provided as input <arn> for the tag operations; ListStreams's own TableName must be at least 3 characters long and at most 255 characters long.

The parsed RequestItems map keeps request order (serde_json preserve_order through feature unification), so the first offending key is reported, as measured. A DescribeTable with a foreign-service ARN is left to the engine (the service answers AccessDeniedException there; the engine's ValidationException stands, noted in the module).

Tests

• Live service: tests/python/test_nul_metadata_names.py (25 rows), test_nul_table_name_fields.py (18), test_nul_arn_shapes.py (6): 49 of 49 against the service today; the sentences in the files are the measured ones, byte for byte where the service gives the whole sentence (service-367-*.jsonl measurements of 2026-10-24).
• The same 49 rows against this branch on PostgreSQL, SQLite (dev mode), MongoDB (mongo:7 replica set) and Cassandra (cassandra:4.1): 49 of 49 each. MongoDB and Cassandra already store a U+0000 key attribute name on main; the gate rows are the change there. Controls: with the gate limited to TableName (an earlier state of this branch) 12 of the 18 field rows fail, and with the ARN classification removed 5 of the 6 ARN rows fail.
• crates/storage-postgres/tests/catalog_nul_names.rs (2): the catalog round trip of U+0000 in key schema, attribute definition, GSI and LSI key and projected names, asserting the stored form (escaped), DescribeTable, DescribeStream, DescribeBackup, the restore source and a restored table's names byte for byte; two scratch databases dropped from Drop.
• crates/storage-postgres/tests/catalog_escape_migration.rs (3): the migration walk above, idempotence, the record-less refusal, the exclusive-access refusal and release.
• catalog_converge.rs (5) adapted to 004; doc_version_literals (3) for the eight version literals; unit tests 12 across the two gate modules and 2 in catalog_escape.
• Full PostgreSQL storage-level suite against a pgvector:pg16 container: 108 passed in 11 files. Existing wire set (test_tables, test_gsi, test_control_plane, test_error_handling, test_batch, test_items, test_tagging, test_query_scan, test_transactions, test_streams; 181): 181 on PostgreSQL, SQLite and MongoDB; 175 on Cassandra, where the six test_gsi.py::TestGSIQuery rows return zero rows with an origin/main Cassandra binary against the same container too (pre-existing; this branch changes no Cassandra code).
• cargo fmt, clippy 1.97.0 with -D warnings across the workspace, 1.88.0 build: clean. Unit: server 214, engine 178, storage-postgres 61, app 32, storage 125. No dependency change. New files under 600 lines except catalog_nul_names.rs (615, a test file); pre-existing files over 600 that grew: migrations.rs, lib.rs, bootstrapper.rs (storage-postgres), cmd_migrate.rs, storage/src/bootstrapper.rs, backup_engine.rs, the upgrade manual.
• .github/workflows/integration.yml: the PostgreSQL storage-level list carries catalog_nul_names and catalog_escape_migration.

Merge notes

• Store U+0000 in strings on PostgreSQL and MongoDB #368 (fix/nul-in-strings): the five hunks above; resolution stated.
• docs/differences-from-dynamodb.md gains a Control Plane row for the catalog encoding.

Not in this PR

• U+0000 inside item attribute values and names on PostgreSQL and MongoDB: Store U+0000 in strings on PostgreSQL and MongoDB #368.
• Vector index catalog columns are not part of the escape (no attribute names).
• The Cassandra GSI propagation gap above (pre-existing, noted on storage-cassandra: asynchronous GSI writes are never applied under serve (default 10 ms delay) #407).
• DescribeTable with a foreign-service ARN answers ValidationException where the service answers AccessDeniedException.

Closes #367

Rebase on #401

#401 added three readers of the escaped catalog columns, and the rebased branch decodes them through from_catalog_json like the others: the table's attribute definitions that every write now reads with the index list (data/index.rs), the base key schema and attribute definitions UpdateTable reads before it creates a GSI's data table (update_table.rs), and the four columns the GSI recovery pass reads (data/gsi_build.rs).