Repository navigation
GetRecords: AT_SEQUENCE_NUMBER past a closed shard's ending number positions at the last record on the service #521
Description
Summary
On a closed shard holding one record with sequence number S (equal to its EndingSequenceNumber), GetShardIterator AT_SEQUENCE_NUMBER with a number greater than S (measured with S plus 1 and S plus 10^15, twice each) issues an iterator, and GetRecords on it returns that record. AFTER_SEQUENCE_NUMBER with S or greater returns no records. So the service positions an AT_SEQUENCE_NUMBER iterator at the last record at or before the requested number when the number is past the shard's end, rather than after it. ExtendDB returns no records for AT_SEQUENCE_NUMBER past the shard's last record.
Reachable only with a number the client did not obtain from the shard (every number a consumer holds is at or before the ending number), so low priority. Whether the same positioning applies on an open shard, and what the service does for a number between two records, was not measured.
Fix shape
Measure the open-shard and between-records cases first; then, if the rule is "at or before", position AT_SEQUENCE_NUMBER at the greatest record sequence number not exceeding the request. Relevant to #372, which ends the iterator at the ending number.
Found while reviewing #515.
Activity
LeeroyHannigan commentedon Oct 25, 2026
Measured the open-shard and between-records cases on 2026-10-25 (three probe runs, two to three repeats per row, composite-key table so every record shares one shard, 22-digit sequence numbers). The rule is not "at or before".
The service's sequence number has two parts: the leading digits are a write position that advances in steps of 100 per write in these runs (records at
1100...,1300...,1500...), and the remaining 18 digits are a remainder it does not use for positioning.GetShardIterator AT_SEQUENCE_NUMBERwith number N positions as follows.r2 + 1,r2 + 10^15,r1 - 1, remainder all zeros or all nines)1200...between1100...and1300...)1000...,1followed by 21 zeros)1600...with the newest at1500...,2500...)ValidationExceptionInvalid SequenceNumber for the shard1310...with the newest at1500...;1350...,1399...)Invalid SequenceNumber for the shardEndingSequenceNumber + 1on the closed shardAFTER_SEQUENCE_NUMBERwith the last record's own number or any number in its bucket returns no records on both shard states. An iterator issued past the end of an open shard keeps reading: after a fourth write,GetRecordson it returned the new record.So the issue's observation (
AT S + 1andAT S + 10^15return the last record) is the first row: both numbers sit in the last record's bucket. The service never positions after the last record forAT; it either lands on a record's bucket, on the next record for an empty bucket, or refuses.ExtendDB's sequence numbers are a dense per-backend counter (
format!("{value:021}")), so there is no bucket to land in: every number names an exact position and "the number after the last record" is a position with no record. Reproducing the service's table would mean adopting a two-part sequence number (position plus remainder), which is a format change on every backend and the stream wire tests. For the shapes a consumer reaches with numbers it was given, the two agree:ATwith a record's own number returns that record and everything after it on both. The divergences areATwith a number no record carries, which a consumer only has by constructing it.Rows and raw output: three probe scripts, 2026-10-25, us-east-1.