Skip to content

fix(scan): do not scan external rel tables before pulling child tuple - #1533

Open
Tyagiquamar wants to merge 2 commits into
LadybugDB:mainfrom
Tyagiquamar:fix/1527-arrow-rel-table-first-node-rels
Open

Tyagiquamar wants to merge 2 commits into
LadybugDB:mainfrom
Tyagiquamar:fix/1527-arrow-rel-table-first-node-rels

Conversation

@Tyagiquamar

Copy link
Copy Markdown
Contributor

Summary

Fixes #1527 by preventing ScanRelTable from scanning external relationship tables before the first child tuple (or bound node batch) has been pulled, and ensuring external relationship tables safely return false when their scan state is uninitialized or in TableScanSource::NONE.

Root Cause & Problem

Commit c1dcef7301 introduced an unconditional call to tableInfo.table->initScanState(...) in ScanRelTable::initLocalStateInternal for all non-native RelTable types:

if (typeid(*tableInfo.table) != typeid(RelTable)) {
    auto transaction = transaction::Transaction::Get(*clientContext);
    tableInfo.table->initScanState(transaction, *scanState);
}

That change was added to prevent ForeignRelTable::scanInternal from throwing or crashing when sharedState was null on the very first scan call.

However, unlike the assumption that external rel tables do not need bound node information, ArrowRelTable (both flat and CSR) and IceDiskRelTable use initScanState to read scanState.nodeIDVector and map bound node offsets to positions:

  1. At initLocalStateInternal time, no child tuple has been fetched from children[0]. The bound node vector holds default offset 0.
  2. initScanState maps offset 0 into arrowBoundNodeOffsetToSelPos (or boundNodeOffsets) and sets source = TableScanSource::COMMITTED.
  3. In getNextTuplesInternal, the initial loop executes while (tableInfo.table->scan(transaction, *scanState)). Because source is COMMITTED, scan() scans and emits all relationships for node offset 0 (the first node in the table).
  4. After exhausting node 0's relationships, children[0]->getNextTuple(...) is finally invoked to pull the actual matching tuple (e.g. from a primary key index lookup), re-initializing scanState for the requested node.
  5. Consequently, any primary key match query over an Arrow rel table returned node 0's relationships in addition to the matching node's relationships.

Technical Solution

  1. Removed the premature initScanState call from ScanRelTable::initLocalStateInternal. All rel tables now wait until either fetchNextBoundNodeBatch (in sourceMode) or children[0]->getNextTuple (in non-sourceMode) pulls valid bound nodes before initializing scan state.
  2. In ScanRelTable::getNextTuplesInternal, guarded both scan loops with scanState->source != TableScanSource::NONE to ensure scan() is never called prior to bound node initialization.
  3. Updated ForeignRelTable::scanInternal to safely return false when source == TableScanSource::NONE or sharedState == nullptr instead of throwing a RuntimeException.
  4. Set foreignRelScanState.source = TableScanSource::COMMITTED in ForeignRelTable::initScanState, and reset it to TableScanSource::NONE when the foreign scan produces 0 tuples.
  5. Updated ArrowRelTable::scanInternal and IceDiskRelTable::scanInternal to check scanState.source == TableScanSource::NONE and return false safely when no scan is active.

Test Plan

Added automated regression tests in LadybugDB native test suites:

  • ArrowRelTableTest.MatchPrimaryKeyOverArrowRelTableDoesNotReturnFirstNodeRels in test/api/arrow_rel_table_test.cpp: verifies that a primary key match query on a flat Arrow rel table returns only the requested node's edges and does not emit node 0's edges, and returns 0 rows when matching a node without relationships.
  • ArrowCsrRelTableTest.MatchPrimaryKeyOverArrowCsrRelTableDoesNotReturnFirstNodeRels in test/api/arrow_csr_rel_table_test.cpp: verifies identical behavior for CSR layout Arrow rel tables.

Commit c1dcef7 added an unconditional tableInfo.table->initScanState
call in ScanRelTable::initLocalStateInternal for non-native RelTables
to prevent a crash in ForeignRelTable::scanInternal when sharedState was
null. However, for ArrowRelTable and IceDiskRelTable, initScanState reads
the bound node ID vector to populate bound node mappings. At
initLocalStateInternal time, no child tuple or bound node batch has been
fetched yet, leaving the bound vector at default offset 0. When
getNextTuplesInternal was called, scan() executed against node offset 0
before pulling the actual child tuple, yielding node 0's relationships
in addition to the matching node's relationships.

This fix:
1. Removes the premature initScanState call from
   ScanRelTable::initLocalStateInternal.
2. Guards the scan loops in ScanRelTable::getNextTuplesInternal with
   scanState->source != TableScanSource::NONE.
3. Updates ForeignRelTable::scanInternal, ArrowRelTable::scanInternal,
   and IceDiskRelTable::scanInternal to safely return false when
   source == TableScanSource::NONE or scan state is uninitialized.
4. Marks ForeignRelTableScanState source as COMMITTED during
   initScanState and resets it to NONE upon scan completion.
5. Adds regression tests covering primary key matches over flat and CSR
   Arrow relationship tables.

Fixes LadybugDB#1527.

Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
clang-format-18 in CI expects the ++iceDiskScanState continuation to
stay column-aligned with the open paren, as on main. Local clang-format-19
accepts both, so match main exactly.

Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: primary-key match over an Arrow rel table also returns the first node's rels

1 participant