Cassandra: a transaction completed by the recovery worker writes no index rows and no stream record #579

Summary

On the Cassandra backend a transaction completed by the recovery worker after a crash writes its items and nothing else: no GSI or LSI index rows and no stream record. Read from recover_commit_op in crates/storage-cassandra/src/data/transactions.rs at main 744ea29; unchanged by #576, which adds the stream record to the live commit path only.

Mechanism

recover_commit_op replays the base mutation from the ledger (UPDATE ... SET item_data = ?, prepared_txn_id = NULL ... for PUT and UPDATE, the two-step delete for DELETE) and calls neither the index sync nor the stream record builder. The ledger's LedgerOp carries the keys, the item data and the pre-commit image (pre_commit_item_data), but not the stream capture (view type, region) or the table's key schema, both of which stream_record_statement and sync_indexes need.

The window is the one between the base-row LWT and the secondary-effects LOGGED BATCH in the live commit path (commit_single_operation then commit_secondary_effects). A crash there leaves the item written and the ledger in COMMITTING; recovery finishes the item and skips the rest. The gap is permanent and silent: a stream consumer's view of that item is stale with nothing to signal it, and an index query misses or mis-returns the item until the next write to it.

Fix shape

Carry the capture on LedgerOp as an optional #[serde(default)] field (the ledger is already versioned by shape in that way for pre_commit_item_data), look the key info up by table_id from the catalog in recovery, and have recover_commit_op call the same commit_secondary_effects the live path calls, with the pre-image from pre_commit_item_data and the row's created_to_prepare and last_committed_txn_timestamp applied through the same old-image rule. A storage test that runs PREPARE and the base LWT, drops the engine before the secondary batch, runs recovery, and asserts the index rows and the stream record are present.

Related: #568 (the live commit path's index arms for Put and Delete), which the recovery change should land after so recovery and the live path share one function.