Skip to content

fix(common): make the query timer's start time atomic - #1556

Merged
adsharma merged 1 commit into
LadybugDB:mainfrom
zahariash:fix/query-timer-race
Nov 10, 2026
Merged

adsharma merged 1 commit into
LadybugDB:mainfrom
zahariash:fix/query-timer-race

Conversation

@zahariash

Copy link
Copy Markdown
Contributor

This work was produced with the help of language models.

Follow-up to #1482 for the two points in the review comment.

Timer race. With a query timeout, the waiting thread reads the statement's timer while the thread running IMPORT DATABASE restarts it for each imported statement, and the start time was a plain time_point. On main, TSan reports the race in TaskSchedulerTest.TimeoutInterruptsStatementStartedAfterDeadline (3 of 3 runs) and in an import of 1500 RETURN 1; statements on one thread with a 20 ms timeout. ActiveQuery::reset() also zeroed the timer before startTimer() set it again, so a waiter reading in between would see a long-expired deadline and interrupt the statement that was starting.

Fix. The start time is now a std::atomic, stored with release and loaded with acquire, which compile to the same plain moves as before on x86-64. A static_assert pins its size and alignment to the plain time_point, so ClientContext's layout does not change. reset() no longer zeroes the timer: a waiter reading before startTimer() now sees the previous statement's start, which has expired only if more than the timeout has passed since it. The atomic makes Timer non-copyable, so a redundant timer = Timer() in TimeMetric's constructor is gone; nothing in the tree, the extensions or the Python driver copies a Timer.

Re-check interval. Completion wakes the waiter at once, so the interval delays no wakeup. Past the deadline the waiter reads the clock every 100 ms, sees a statement that started after the deadline within 100 ms, and then sleeps until that statement's own deadline, so only timeouts under 100 ms can overrun. A scratch test that started the next statement at 13 points of the cycle measured this in a Release build: 150 and 300 ms timeouts were interrupted within 1 ms of their deadline, 20 and 50 ms ones at most 96 ms after the statement started. 1000 ms would let sub-second timeouts overrun by up to a second; 10 ms would help only timeouts under 100 ms, with ten times the wakeups. The comment on INTERRUPT_RECHECK_INTERVAL_MS now says this.

Tests. No new test: the scheduler test above reports the race under TSan on every run, so the nightly TSan job will flag it if it comes back, and a timing assertion would be flaky on loaded CI runners. With the change, TSan reports nothing in those two cases or in TaskSchedulerTest.* (3 runs each). Release ctest: 3439 passed, 0 failed; the import, export, large-copy, GDS and timeout tests also pass with MAX_NUM_THREADS=1.

With a query timeout, the thread waiting on a task reads the statement's
timer in getTimeoutRemainingInMS, while the thread running IMPORT
DATABASE restarts it for each imported statement (resetActiveQuery and
startTimer in executeNoLock). Timer's start time was a plain time_point,
so this was a data race. TSan reports it on main in
TaskSchedulerTest.TimeoutInterruptsStatementStartedAfterDeadline and in
an import of many short statements.

The start time is now a std::atomic, stored with release and loaded with
acquire. It has the same size and alignment as the plain time_point (a
static_assert pins this), so ClientContext's layout does not change.

ActiveQuery::reset() no longer zeroes the timer: a waiter reading
between the reset and startTimer() could see a long-expired deadline and
interrupt the statement that was starting. TimeMetric's constructor
drops a redundant timer = Timer(), which the atomic no longer allows.

INTERRUPT_RECHECK_INTERVAL_MS gets a comment on what it bounds and why
it is not shorter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@adsharma
adsharma merged commit 996b92d into LadybugDB:main Nov 10, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants