Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## Unreleased

- Add `Symbol.asyncDispose` to clients, sessions, and operations, delegating to `close()`. Use `asyncDisposable()` with interface-typed results in TypeScript; existing interfaces and factory return types remain unchanged. (#331)
- CloudFetch retry-exhaustion errors now identify the download and the last failure: the `RetryError` message reads `CloudFetch result download failed: <original message> (last error: <cause>)`. `<cause>` is `HTTP <status>: <statusText>` (or `HTTP <status>` when the status text is empty) for HTTP failures, and `network error <code>`, `network error (<type>)` (e.g. `aborted`, when there is no `code`) or a bare `network error` for network failures (the raw network-error message, which can contain the presigned URL, is not copied). If no cause can be determined, the ` (last error: …)` suffix is omitted. The `RetryError` instance, `errorCode` and `payload` (including the node-fetch `Request` / `Response`) are unchanged. (#471)
- Report untyped NULL columns as STRING in kernel result metadata, matching Thrift. (#470)
- Reject non-scalar query parameter values at bind time with a `ParameterError` on both the Thrift and kernel backends, instead of stringifying them (which could leave a Thrift operation stuck before FINISHED). Newly rejected on the Thrift backend: arrays, plain objects, boxed primitives (e.g. `new String('x')`, `new Number(1)`), functions, symbols, and a `DBSQLParameter` that wraps another `DBSQLParameter`. Primitives, `Date`, `Int64`, and `DBSQLParameter` are still accepted.
Expand Down
29 changes: 29 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,35 @@ client
});
```

### Automatic cleanup with `await using`

The client, sessions, and operations implement `Symbol.asyncDispose`. On scope exit, `await using` calls `close()`,
including when the block throws. Disposal follows the existing `close()` behavior, and cleanup errors propagate.
If both the block and a disposal fail, `await using` throws a `SuppressedError` containing both errors.
Node.js 24+ supports
`await using` natively; on Node.js 22, compile it with TypeScript 5.2+. TypeScript also needs the disposable
declarations (such as the global `AsyncDisposable` type) to compile `await using`: if your typings don't already
provide them, add `"esnext.disposable"` (or `"esnext"`) to `compilerOptions.lib` in your `tsconfig.json`, otherwise the
example below can fail to compile even with TypeScript 5.2+.

`openSession()` and the session's operation methods are typed as returning the `IDBSQLSession` and `IOperation`
interfaces, which don't require `Symbol.asyncDispose` (so existing custom implementations keep compiling). In
TypeScript, wrap their results with `asyncDisposable()`: it returns the driver's own sessions and operations unchanged,
typed as disposable.

```typescript
import { DBSQLClient, asyncDisposable } from '@databricks/sql';

async function main() {
await using client = new DBSQLClient();
await client.connect({ host, path, token });

await using session = asyncDisposable(await client.openSession());
await using operation = asyncDisposable(await session.executeStatement('SELECT "Hello, World!"'));
console.table(await operation.fetchAll());
} // disposes `operation`, `session`, then `client`
```

## Configuration

See [CONNECTION_PARAMETERS.md](CONNECTION_PARAMETERS.md) for every connection,
Expand Down
9 changes: 9 additions & 0 deletions lib/DBSQLClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ import MetricsAggregator from './telemetry/MetricsAggregator';
import { DriverConfiguration, DRIVER_NAME, TelemetryEventType, DEFAULT_TELEMETRY_CONFIG } from './telemetry/types';
import { safeEmit, isTelemetryDisabledByEnv } from './telemetry/telemetryUtils';
import driverVersion from './version';
// Declares `Symbol.asyncDispose` for the emitted typings of `[Symbol.asyncDispose]()` below
import './contracts/asyncDispose';

function prependSlash(str: string): string {
if (str.length > 0 && str.charAt(0) !== '/') {
Expand Down Expand Up @@ -951,6 +953,13 @@ export default class DBSQLClient extends EventEmitter implements IDBSQLClient, I
this.authProvider = undefined;
}

/**
* Closes the client (see {@link close}); called by `await using client = ...`
*/
public async [Symbol.asyncDispose](): Promise<void> {
await this.close();
}

public getConfig(): ClientConfig {
return this.config;
}
Expand Down
10 changes: 10 additions & 0 deletions lib/DBSQLOperation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ import { synthesizeThriftStatus, synthesizeThriftResultSetMetadata } from './thr
import { mapOperationTypeToTelemetryType } from './telemetry/telemetryTypeMappers';
import ExceptionClassifier from './telemetry/ExceptionClassifier';
import { safeEmit } from './telemetry/telemetryUtils';
// Declares `Symbol.asyncDispose` for the emitted typings of `[Symbol.asyncDispose]()` below
import './contracts/asyncDispose';

function mapNeutralResultFormatToTelemetryType(resultFormat?: ResultFormat): string | undefined {
switch (resultFormat) {
Expand Down Expand Up @@ -233,6 +235,14 @@ export default class DBSQLOperation implements IOperation {
return result;
}

/**
* Closes operation (see {@link close}); called by `await using operation = ...`
* @throws {StatusError}
*/
public async [Symbol.asyncDispose](): Promise<void> {
await this.close();
}

public async finished(options?: FinishedOptions): Promise<void> {
return this.withErrorTelemetry(async () => {
await this.failIfClosed();
Expand Down
10 changes: 10 additions & 0 deletions lib/DBSQLSession.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ import ISessionBackend from './contracts/ISessionBackend';
import IOperationBackend from './contracts/IOperationBackend';
import { numberToInt64 as numberToInt64Impl } from './thrift-backend/ThriftSessionBackend';
import { safeEmit } from './telemetry/telemetryUtils';
// Declares `Symbol.asyncDispose` for the emitted typings of `[Symbol.asyncDispose]()` below
import './contracts/asyncDispose';

// Explicitly promisify a callback-style `pipeline` because `node:stream/promises` is not available in Node 14
const pipeline = util.promisify(stream.pipeline);
Expand Down Expand Up @@ -365,6 +367,14 @@ export default class DBSQLSession implements IDBSQLSession {
}
}

/**
* Closes the session (see {@link close}); called by `await using session = ...`
* @public
*/
public async [Symbol.asyncDispose](): Promise<void> {
await this.close();
}

private wrapOperation(backend: IOperationBackend): DBSQLOperation {
const operation = new DBSQLOperation({
backend,
Expand Down
40 changes: 40 additions & 0 deletions lib/asyncDisposable.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
// Declares `Symbol.asyncDispose` for the emitted typings below
import './contracts/asyncDispose';

/**
* A closeable resource that can also be bound with `await using`.
*/
export type AsyncDisposableResource<T> = T & {
[Symbol.asyncDispose](): Promise<void>;
};

/**
* Returns `resource` typed as async-disposable, so that it can be bound with `await using`.
*
* `connect()`, `openSession()` and the session's operation methods return the `IDBSQLClient` /
* `IDBSQLSession` / `IOperation` interfaces, which don't require `Symbol.asyncDispose` (so custom
* implementations and subclasses written before it was added keep compiling). The driver's own clients,
* sessions, and operations implement it, so they are returned unchanged. A custom implementation without it gets a non-enumerable
* `[Symbol.asyncDispose]()` that calls its `close()` (this throws a `TypeError` if the object isn't
* extensible, e.g. a frozen object).
*
* @example
* await using session = asyncDisposable(await client.openSession());
* await using operation = asyncDisposable(await session.executeStatement('SELECT 1'));
*/
export default function asyncDisposable<T extends { close(): Promise<unknown> }>(
resource: T,
): AsyncDisposableResource<T> {
const candidate = resource as T & { [Symbol.asyncDispose]?: unknown };
if (typeof candidate[Symbol.asyncDispose] !== 'function') {
Object.defineProperty(resource, Symbol.asyncDispose, {
configurable: true,
writable: true,
enumerable: false,
value: async function asyncDispose(this: T): Promise<void> {
await this.close();
},
});
}
return resource as AsyncDisposableResource<T>;
}
9 changes: 9 additions & 0 deletions lib/contracts/asyncDispose.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
// Declares the built-in `Symbol.asyncDispose` (identical to the declaration in TypeScript's
// `esnext.disposable` lib and in `@types/node`) so the published typings work without either.
declare global {
interface SymbolConstructor {
readonly asyncDispose: unique symbol;
}
}

export {};
3 changes: 3 additions & 0 deletions lib/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ import { LogLevel } from './contracts/IDBSQLLogger';
// Re-export types for TypeScript users
export type { default as ITokenProvider } from './connection/auth/tokenProvider/ITokenProvider';

export { default as asyncDisposable } from './asyncDisposable';
export type { AsyncDisposableResource } from './asyncDisposable';

export { LOG_FILE_UNAVAILABLE_WARNING_CODE } from './DBSQLLogger';
export { CircuitBreakerOpenError, CIRCUIT_BREAKER_OPEN_CODE } from './telemetry/CircuitBreaker';
export { TelemetryTerminalError } from './telemetry/DatabricksTelemetryExporter';
Expand Down
43 changes: 43 additions & 0 deletions tests/unit/DBSQLClient.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -579,6 +579,49 @@ describe('DBSQLClient.close', () => {
});
});

describe('DBSQLClient[Symbol.asyncDispose]', () => {
it('should close the client and its sessions', async () => {
const client = new DBSQLClient();
const thriftClient = sinon.spy(new ThriftClientStub());

client['client'] = thriftClient;
client['connectionProvider'] = new ConnectionProviderStub();
client['authProvider'] = new AuthProviderStub();
client['backend'] = new ThriftBackend({ context: client, onConnectionEvent: () => {} });

const session = await client.openSession();
if (!(session instanceof DBSQLSession)) {
throw new Error('Assertion error: expected session to be DBSQLSession');
}
const closeSpy = sinon.spy(client, 'close');

const result = await client[Symbol.asyncDispose]();
expect(result).to.be.undefined;
expect(closeSpy.calledOnce).to.be.true;
expect(session['isOpen']).to.be.false;
expect(thriftClient.CloseSession.calledOnce).to.be.true;
expect(client['backend']).to.be.undefined;
expect(client['client']).to.be.undefined;
});

it('should be a no-op for a closed or never-connected client', async () => {
const closedClient = new DBSQLClient();
const thriftClient = sinon.spy(new ThriftClientStub());
closedClient['client'] = thriftClient;
closedClient['connectionProvider'] = new ConnectionProviderStub();
closedClient['authProvider'] = new AuthProviderStub();
closedClient['backend'] = new ThriftBackend({ context: closedClient, onConnectionEvent: () => {} });
await closedClient.openSession();

await closedClient.close();
await closedClient[Symbol.asyncDispose]();
await closedClient[Symbol.asyncDispose]();
expect(thriftClient.CloseSession.calledOnce).to.be.true;

await new DBSQLClient()[Symbol.asyncDispose]();
});
});

describe('DBSQLClient.createAuthProvider', () => {
it('should use access token auth method', () => {
const client = new DBSQLClient();
Expand Down
40 changes: 40 additions & 0 deletions tests/unit/DBSQLOperation.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -418,6 +418,46 @@ describe('DBSQLOperation', () => {
});
});

describe('Symbol.asyncDispose', () => {
it('should close the operation', async () => {
const context = new ClientContextStub();
const driver = sinon.spy(context.driver);
const operation = createOperationForTest({ handle: operationHandleStub({ hasResultSet: true }), context });

const result = await operation[Symbol.asyncDispose]();
expect(result).to.be.undefined;
expect(operation['closed']).to.be.true;
expect(driver.closeOperation.callCount).to.be.equal(1);
});

it('should be a no-op for an already closed operation', async () => {
const context = new ClientContextStub();
const driver = sinon.spy(context.driver);
const closed = createOperationForTest({ handle: operationHandleStub({ hasResultSet: true }), context });

await closed.close();
await closed[Symbol.asyncDispose]();
await closed[Symbol.asyncDispose]();
expect(driver.closeOperation.callCount).to.be.equal(1);
});

it('should propagate close errors', async () => {
const context = new ClientContextStub();
context.driver.closeOperationResp.status.statusCode = TStatusCode.ERROR_STATUS;
const operation = createOperationForTest({ handle: operationHandleStub({ hasResultSet: true }), context });

try {
await operation[Symbol.asyncDispose]();
expect.fail('It should throw a StatusError');
} catch (e) {
if (e instanceof AssertionError) {
throw e;
}
expect(e).to.be.instanceOf(StatusError);
}
});
});

describe('finished', () => {
[TOperationState.INITIALIZED_STATE, TOperationState.RUNNING_STATE, TOperationState.PENDING_STATE].forEach(
(operationState) => {
Expand Down
45 changes: 45 additions & 0 deletions tests/unit/DBSQLSession.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -760,4 +760,49 @@ describe('DBSQLSession', () => {
);
});
});

describe('Symbol.asyncDispose', () => {
it('should close the session and its operations', async () => {
const context = new ClientContextStub();
const driver = sinon.spy(context.driver);
const session = createSessionForTest({ handle: sessionHandleStub, context });
const operation = await session.executeStatement('SELECT * FROM table');
if (!(operation instanceof DBSQLOperation)) {
expect.fail('Assertion error: operation is not a DBSQLOperation');
}

const result = await session[Symbol.asyncDispose]();
expect(result).to.be.undefined;
expect(session['isOpen']).to.be.false;
expect(operation['closed']).to.be.true;
expect(driver.closeSession.callCount).to.eq(1);
});

it('should be a no-op for an already closed session', async () => {
const context = new ClientContextStub();
const driver = sinon.spy(context.driver);
const session = createSessionForTest({ handle: sessionHandleStub, context });

await session.close();
await session[Symbol.asyncDispose]();
await session[Symbol.asyncDispose]();
expect(driver.closeSession.callCount).to.eq(1);
});

it('should propagate close errors', async () => {
const context = new ClientContextStub();
const session = createSessionForTest({ handle: sessionHandleStub, context });
const error = new Error('close failed');
sinon.stub(context.driver, 'closeSession').rejects(error);

try {
await session[Symbol.asyncDispose]();
expect.fail('It should throw an error');
} catch (e) {
expect(e).to.equal(error);
}
// A failed close still marks the session closed, so disposing again is a no-op.
await session[Symbol.asyncDispose]();
});
});
});
69 changes: 69 additions & 0 deletions tests/unit/asyncDisposable.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
import { expect } from 'chai';
import sinon from 'sinon';
import asyncDisposable from '../../lib/asyncDisposable';
import * as driver from '../../lib';
import DBSQLClient from '../../lib/DBSQLClient';
import Status from '../../lib/dto/Status';
import OperationStub from './.stubs/OperationStub';
import ClientContextStub from './.stubs/ClientContextStub';
import { createSessionForTest } from './.stubs/createSessionForTest';

describe('asyncDisposable', () => {
it('is exported from the package entry point', () => {
expect(driver.asyncDisposable).to.equal(asyncDisposable);
});

it("returns the driver's own resources unchanged", async () => {
const client = new DBSQLClient();
const session = createSessionForTest({
handle: { sessionId: { guid: Buffer.alloc(16), secret: Buffer.alloc(16) } },
context: new ClientContextStub(),
});
const operation = await session.executeStatement('SELECT 1');

for (const resource of [client, session, operation]) {
const dispose = (resource as any)[Symbol.asyncDispose];
expect(asyncDisposable(resource)).to.equal(resource);
expect((resource as any)[Symbol.asyncDispose]).to.equal(dispose);
}

const closeSpy = sinon.spy(operation, 'close');
await asyncDisposable(operation)[Symbol.asyncDispose]();
expect(closeSpy.calledOnce).to.be.true;
});

it('adds a non-enumerable disposer that closes a custom implementation', async () => {
const operation = new OperationStub([]);
const disposable = asyncDisposable(operation);

expect(disposable).to.equal(operation);
expect(Object.getOwnPropertyDescriptor(operation, Symbol.asyncDispose)?.enumerable).to.be.false;
expect(await disposable[Symbol.asyncDispose]()).to.be.undefined;
expect(operation.closed).to.be.true;
});

it('keeps an existing disposer of a custom implementation', async () => {
const dispose = sinon.stub().resolves();
const resource = { close: async () => Status.success(), [Symbol.asyncDispose]: dispose };

await asyncDisposable(resource)[Symbol.asyncDispose]();
expect(dispose.calledOnce).to.be.true;
});

it('propagates close errors from the added disposer', async () => {
const error = new Error('close failed');
const resource = { close: () => Promise.reject(error) };

try {
await asyncDisposable(resource)[Symbol.asyncDispose]();
expect.fail('It should throw an error');
} catch (e) {
expect(e).to.equal(error);
}
});

it('throws a TypeError for a non-extensible custom implementation', () => {
const resource = Object.freeze({ close: async () => Status.success() });
expect(() => asyncDisposable(resource)).to.throw(TypeError);
});
});
Loading
Loading