Skip to content

Stop leaking a telemetry collector on every final flush - #1800

Open
vuanhphung wants to merge 2 commits into
mainfrom
perf/telemetry-collector-leak
Open

vuanhphung wants to merge 2 commits into
mainfrom
perf/telemetry-collector-leak

Conversation

@vuanhphung

@vuanhphung vuanhphung commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Description

TelemetryClient.close() runs when the last connection to a host closes, and it called getOrCreateCollector() for the connection the client was created with. That connection had already closed, and its collector had already been removed, so close() created a new, empty collector that was never removed. Each one keeps its connection context alive. On main, 20 sequential open/close cycles against a real warehouse leave 20 entries in TelemetryCollectorManager, so apps that open a connection per query grow without bound. close() now removes the collector instead and exports it only if one is still there. Export behavior does not change.

#1798 contains the same hunk, because its close-race test checks that no per-connection state is left behind. The hunks are identical, so the two PRs merge cleanly in either order; only NEXT_CHANGELOG.md needs a trivial rebase.

Testing

Two new TelemetryClientTest cases fail on main: close() leaves no collector for the closed connection, and a collector with pending details is still exported and then removed. TelemetryClientTest, TelemetryClientFactoryTest and TelemetryCollectorManagerTest pass. The full jdbc-core suite (3,881 tests) has one failure, ArrowBufferAllocatorTest.testCreateDatabricksBufferAllocator, which also fails on main.

Telemetry Errors

  • Not applicable — this PR does not add or change a telemetry-visible error.
  • Applicable — the error uses DatabricksDriverErrorCode where appropriate, and any
    new code is uniquely numbered and tested.
  • Applicable — its driver/server/user classification is linked, or maintainer help is
    requested because the author cannot access the classification.

This pull request and its description were written by Isaac.

TelemetryClient.close() called getOrCreateCollector() for the connection
it was created with. By then that connection had closed and its
collector had been removed, so close() created a new, empty one that was
never removed: one leaked collector (and the connection context it
holds) per close of a host's last connection. Remove the collector
instead and export it only if it is still there.

Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No issues identified by the review bot.

Resolve the main changes by merging (no force-push).

Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No issues identified by the review bot.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant