[LeeroyHannigan]

On Cassandra, a Scan now returns every item of a table that holds partitions whose items have all been deleted. Before, the Scan stopped early with no LastEvaluatedKey, and CreateBackup, export, and the TTL backfill and audit stopped early the same way, because they page through the same storage function.

This PR fixes no open issue. #611 fixes the same defect in Query.

What happened

crates/storage-cassandra/src/data/scan.rs asks Cassandra for LIMIT limit + 1 rows and skips each row without item_data. Such rows are:

• the static-only row Cassandra returns for a partition whose items have all been deleted (DeleteItem sets the static column partition_max_delete_timestamp), on a table with a sort key;
• the row a TTL claim on an absent key writes, with only prepared_txn_id and prepared_txn_timestamp set, on any table;
• the row a transaction prepare leaves when its UPDATE ... IF prepared_txn_id = null lands on an item deleted after the transaction read it.

Scan then decided "more pages" from the number of items it kept. A read that met such rows came back with fewer items than its LIMIT and no LastEvaluatedKey, so every caller took it as the end of the table.

What the service answers

Measured on 2026-10-26 in us-east-1, each case twice with the same answer.

• At 15:29 UTC, on a table holding 30 live items written for the measurement among 100 partitions whose one item was put and then deleted: a Scan returns every live item with no Limit, with Limit 1, 7 and 50, with two parallel segments at Limit 3, with Select COUNT and with a FilterExpression. Every case read with ConsistentRead, and the Scan with no Limit was also read eventually consistent.
• ScannedCount counts live items only. With Limit 1, a Scan of the table's 59 live items took 60 pages: 59 with one item and a last one with none.
• At 16:39 UTC, on two new tables (one with a sort key, one hash only) whose 50 items were all deleted: a Scan with no Limit and with Limit 1 and 7 answers Count 0, ScannedCount 0 and no LastEvaluatedKey.

The change

scan.rs decides "more pages" from the rows Cassandra returned, as #611 does for Query. A page that skipped rows can be short but carries a LastEvaluatedKey, and the caller reads on from its last item.

A full read in which no row holds an item has no item to continue from. Scan then reads on after the last row it read: a base-table read now also selects pk and the sort key column, and the next read is pk = ? AND sk > ? for a row with a sort key (a claim row) and token(pk) > token(?) for a row without one (a static-only row, or any row of a hash-only table). Each further read takes up to twice as many rows as the one before and at most max(1000, LIMIT + 1). No read goes back over rows already read.

Cassandra counts a tombstone for each deleted row in the partitions such a read passes, so when those partitions held many deleted rows, a read of 1,000 rows can still reach tombstone_failure_threshold (100,000 by default). A continuation read that Cassandra refuses with a read failure is read again with half as many rows, down to one, and the later reads of the same storage call stay at that size. The first read of a page is not retried: it reads the same rows as the read main makes, and fails where main fails.

Index tables never hold a row without item_data, and their reads are unchanged. The row handling (skipping rows, the position of the last row, the two-query read, the retry) is in the new crates/storage-cassandra/src/data/scan_rows.rs; the two-query read that resumes after an ExclusiveStartKey moved there from scan_paginated, and both paths use it.

Tests

• crates/storage-cassandra/tests/direct/scan_rows_without_item.rs: 6 tests. Thirty live partitions among 300 emptied ones and 20 claim rows, on a table with a sort key and on a hash-only table, paged with no Limit and with Limit 1, 2, 7 and 50, and as two parallel segments; a run of 40 claim rows inside one partition ahead of its item; a run of 5,000 emptied partitions between 5 items, after which the test reads Cassandra's system_views.tombstones_per_read histogram for the table and requires that no read counted 1,500 tombstones or more (a read of 1,001 rows shows as about 1,109); a table with no item left; and a table of 101,000 emptied partitions, more than Cassandra's default tombstone_failure_threshold of 100,000, Scanned with Limit 1,000 and 128.
• crates/storage-cassandra/tests/direct/scan_sort_key_types.rs: 4 tests. Claim rows among items in one partition, with number (38-digit, negative and exponent values), binary (up to 1,024 bytes) and string (multi-byte UTF-8) sort keys, plus 40 emptied partitions, paged with Limit 1, 2, 3, 7 and no Limit; and 2,100 emptied partitions that each held 110 rows, with one item after them in token order, Scanned with Limit 1 and 128.
• tests/test_scan_emptied_partitions.py: 16 dual-target wire tests built from the measurements.

Run locally against Cassandra 4.1 (one node, PasswordAuthenticator) on 2026-10-26:

• the direct tests: the 10 new tests and the 13 in direct/scan.rs pass. With main's scan.rs, 8 of the new tests fail (Limit 1 read 0 of 30 items on the table with a sort key, 3 of 30 on the hash-only table, 0 of 5 after the 5,000-partition run, no item after the 40 claim rows, none of the typed items, and not the item after the 2,100 heavy partitions); the no-item-left and 101,000-partition tests pass on main too.
• with an earlier version of this change, which read again from the same start with twice the LIMIT, the 101,000-partition test failed with READ_TOO_MANY_TOMBSTONES (ReadFailure, answered as 500), and the tombstone check in the 5,000-partition test failed (one read counted about 2,299 tombstones). With a copy of this change whose read size has no cap, the tombstone check failed the same way.
• without the retry at half the rows, the 2,100-partition test failed with READ_TOO_MANY_TOMBSTONES on a continuation read; with it, the Scan reads through to the item.
• the wire file with the runtime settings devtools/run-tests applies in CI: 16 passed at the head on Cassandra and on SQLite. With main's binary on Cassandra, the 5 paging tests on the table with a sort key fail; the hash-only ones pass, since a deleted item of a hash-only table leaves no row.
• 300 live single-item partitions and 30 emptied ones, Scan paged to the end: 300 items at the head with no Limit and with Limit 50; on main, 112 and 44. 1,100 live items and 20 emptied partitions, CreateBackup and RestoreTableFromBackup: the restored table holds 1,100 items at the head and 984 on main.
• cargo +1.97.0 fmt --all -- --check and cargo +1.97.0 clippy --all-targets -- -D warnings: clean. cargo test --workspace --exclude extenddb-storage-cassandra: 1,794 passed on stable and on 1.88.0. cargo test -p extenddb-storage-cassandra --lib: 61 passed.

Not changed here

These answer the same on main:

• The GSI backfill pages with token(pk) > token(last pk), so a partition larger than its 500-row page loses its tail.
• A Query with a sort key condition and an ExclusiveStartKey answers 500 (Cassandra: a Query with a sort key condition and an ExclusiveStartKey answers InternalServerError #488).
• The first read of a page that meets partitions with many deleted rows can fail with READ_TOO_MANY_TOMBSTONES: at 110 deleted rows per partition, a first read of 1,001 rows (the LIMIT of CreateBackup, export and the TTL work) fails on main, and the head makes the same first read without a retry.

Merging

git merge-tree --write-tree against the 14 open PRs that change scan.rs, data/mod.rs or tests/direct_integration.rs:

• fix(cassandra): give every data-path statement a consistency level #591 (consistency levels) gains a conflict in scan.rs. It adds a consistent_read argument to scan_paginated and runs its two queries through query_rows_at. This PR moves those two queries into scan_rows.rs (rows_after_position, which runs each statement through rows_of so the retry can see the driver's error type), so whichever merges second passes consistent_read down to rows_of and sets the read consistency there, which covers the continuation reads as well.
• feat(storage): SnapshotEngine, a consistent table snapshot on every backend [backup/restore 4/4] #395 conflicts with main already, in .github/workflows/integration.yml; this PR adds no conflict to it.
• fix(cassandra): record stream records for transacted writes #576 and fix(query): refuse a Query ExclusiveStartKey the key condition cannot reach #483 each gain a conflict in crates/storage-cassandra/src/data/mod.rs: each adds its own mod line next to mod scan;, as this PR does. Keep both lines. fix(query): bound begins_with on a string sort key by the next prefix #588, stacked on fix(query): refuse a Query ExclusiveStartKey the key condition cannot reach #483, inherits the same conflict.
• perf(storage-cassandra): restore copy writes plain INSERTs, not the LWT path #527, feat: store and report WarmThroughput on tables and global secondary indexes #474, fix(cassandra): a Query of a partition with no items left answers no items #611, fix: answer requests during a table's DELETING window as the service does #550, fix(cassandra): fence concurrent DeleteTable with a lease #598, fix(streams): resolve the shard against its stream inside the GetRecords read #543, fix(streams): disabling a stream closes its shards; enabling makes a new one #515, fix(storage-cassandra): apply max_connections and override CDRS default reconnect policy #457, fix(storage-cassandra): serialize migrations to prevent keyspace contamination #453 and fix(storage-cassandra): index rows follow the base item under every write path #424 merge clean.