Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

- CloudFetch retry-exhaustion errors now identify the download and the last failure: the `RetryError` message reads `CloudFetch result download failed: <original message> (last error: <cause>)`. `<cause>` is `HTTP <status>: <statusText>` (or `HTTP <status>` when the status text is empty) for HTTP failures, and `network error <code>`, `network error (<type>)` (e.g. `aborted`, when there is no `code`) or a bare `network error` for network failures (the raw network-error message, which can contain the presigned URL, is not copied). If no cause can be determined, the ` (last error: …)` suffix is omitted. The `RetryError` instance, `errorCode` and `payload` (including the node-fetch `Request` / `Response`) are unchanged. (#471)
- Report untyped NULL columns as STRING in kernel result metadata, matching Thrift. (#470)
- `IOperation.close()` no longer throws when the close request fails after the operation has finished (for example, once its results were read, or when the execute response already reported completion): the failure is logged as a warning and reported in the returned status's `info`, and the operation is still closed locally. Closing an operation that hasn't finished still throws on failure. (#480)
- Reject non-scalar query parameter values at bind time with a `ParameterError` on both the Thrift and kernel backends, instead of stringifying them (which could leave a Thrift operation stuck before FINISHED). Newly rejected on the Thrift backend: arrays, plain objects, boxed primitives (e.g. `new String('x')`, `new Number(1)`), functions, symbols, and a `DBSQLParameter` that wraps another `DBSQLParameter`. Primitives, `Date`, `Int64`, and `DBSQLParameter` are still accepted.
- `DBSQLLogger`: a `filepath` naming an existing directory now writes to `databricks-sql-nodejs.log` inside it, instead of silently producing no logs. (#565)
- `DBSQLLogger`: if the log file can't be opened (including when its directory can't be created, which used to throw from the constructor), the logger now logs to the console only, leaves `transports.file` undefined, and emits a process warning with code `DBSQL_LOG_FILE_UNAVAILABLE`. Previously an unwritable file could silently stall all logging, including the console. (#565)
Expand Down
38 changes: 34 additions & 4 deletions lib/DBSQLOperation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,8 @@ export default class DBSQLOperation implements IOperation {
}

/**
* Closes operation
* Closes operation. Once the operation has finished, a failed close request
* is logged and reported in the returned status's `info` instead of thrown.
* @throws {StatusError}
*/
public async close(): Promise<Status> {
Expand All @@ -223,7 +224,32 @@ export default class DBSQLOperation implements IOperation {
if (this.closed || this.cancelled) {
return Status.success();
}
const result = await this.backend.close();
let result: Status;
try {
result = await this.backend.close();
} catch (error: any) {
// Until the statement finished, closing it is what stops it server-side,
// so the failure is the caller's to handle (and the close can be retried).
// The backend answers from completion state it already holds (including
// direct results), without another status RPC.
if (!this.backend.isFinished()) {
throw error;
}
// Afterwards the close only releases server-side results, which closing
// the session also does: report the failure and finish the teardown.
// `StatusError` doesn't extend `Error`, so read `message` structurally.
const message = typeof error?.message === 'string' ? error.message : String(error);
this.context
.getLogger()
.log(LogLevel.warn, `Failed to close finished operation ${this.id}; it is closed locally. Cause: ${message}`);
// Buffer the error on the statement instead of reporting it as terminal:
// a terminal error finalizes the statement's aggregation immediately,
// and `emitStatementComplete()` below would then start and finalize a
// second, partial one. Buffered, it's flushed with the single
// statement metric.
this.emitErrorEvent(error, { deferToCompletion: true });
result = Status.success([`Failed to close operation: ${message}`]);
}
this.closed = true;

// Emit statement.complete telemetry event
Expand Down Expand Up @@ -436,16 +462,20 @@ export default class DBSQLOperation implements IOperation {
* in the aggregator; retryable errors are buffered until the statement
* completes. All exceptions from this method itself are swallowed at
* debug level — telemetry must never break the driver.
*
* Pass `deferToCompletion` when the caller completes the statement right
* afterwards: the error is then buffered (not terminal) so the statement's
* aggregation is finalized once, by that completion.
*/
private emitErrorEvent(error: Error): void {
private emitErrorEvent(error: Error, options?: { deferToCompletion?: boolean }): void {
safeEmit(this.context, (emitter) => {
emitter.emitError({
statementId: this.id,
sessionId: this.sessionId,
errorName: error.name || 'Error',
errorMessage: error.message || 'Unknown error',
errorStack: error.stack,
isTerminal: ExceptionClassifier.isTerminal(error),
isTerminal: options?.deferToCompletion ? false : ExceptionClassifier.isTerminal(error),
});
});
}
Expand Down
8 changes: 8 additions & 0 deletions lib/contracts/IOperationBackend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,14 @@ export default interface IOperationBackend {
*/
hasResultSet(): boolean;

/**
* Whether the backend already knows the statement finished successfully,
* from state it has observed: the execute response / direct results, a
* status poll, or a completed wait. Synchronous and MUST NOT issue an RPC.
* `false` means execution may still be running server-side.
*/
isFinished(): boolean;

/**
* Fetch the next chunk of result rows.
*
Expand Down
36 changes: 33 additions & 3 deletions lib/kernel/KernelOperationBackend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,11 @@ export default class KernelOperationBackend implements IOperationBackend {
// already-terminal statement. Drives both fetch and result-metadata.
private fetchHandlePromise?: Promise<KernelFetchHandle>;

// Async path only: set once the kernel reported the statement Succeeded (a
// status poll, or `awaitResult()` resolving). The kernel exposes no
// synchronous terminal-state accessor, so the observed success is retained.
private asyncSucceeded = false;

constructor({ asyncStatement, statement, cancellableExecution, context, id }: KernelOperationBackendOptions) {
// Exactly one of the three handle kinds must be supplied.
const providedCount =
Expand Down Expand Up @@ -269,6 +274,19 @@ export default class KernelOperationBackend implements IOperationBackend {
return true;
}

public isFinished(): boolean {
if (this.asyncStatement) {
return this.asyncSucceeded;
}
if (this.cancellableExecution) {
// `blockingStatement` is set only once `result()` resolved a terminal statement.
return this.blockingStatement !== undefined;
}
// Metadata path and the directResults Completed arm: the statement was
// terminal when the backend was constructed.
return true;
}

// ---------------------------------------------------------------------------
// Fetch / metadata (owned by the kernel-results pipeline).
// ---------------------------------------------------------------------------
Expand Down Expand Up @@ -397,6 +415,7 @@ export default class KernelOperationBackend implements IOperationBackend {
// result.
const state = statusStringToOperationState(await this.asyncStatement.status());
if (state === OperationState.Succeeded) {
this.asyncSucceeded = true;
return { state, hasResultSet: true, ...(await this.readRichStatusFields()) };
}
return { state, hasResultSet: true };
Expand Down Expand Up @@ -593,6 +612,11 @@ export default class KernelOperationBackend implements IOperationBackend {

// eslint-disable-next-line no-await-in-loop
const state = statusStringToOperationState(await this.asyncStatement!.status());
// Record success before the progress callback runs, so a callback that
// throws on the Succeeded tick still leaves `isFinished()` true.
if (state === OperationState.Succeeded) {
this.asyncSucceeded = true;
}

if (options?.callback) {
// On the terminal Succeeded tick, carry the rich status fields
Expand Down Expand Up @@ -725,9 +749,15 @@ export default class KernelOperationBackend implements IOperationBackend {
private getFetchHandle(): Promise<KernelFetchHandle> {
if (!this.fetchHandlePromise) {
if (this.asyncStatement) {
this.fetchHandlePromise = this.asyncStatement.awaitResult().catch((err) => {
throw decodeNapiKernelError(err);
}) as Promise<KernelNativeAsyncResultHandle>;
this.fetchHandlePromise = this.asyncStatement.awaitResult().then(
(handle) => {
this.asyncSucceeded = true;
return handle;
},
(err) => {
throw decodeNapiKernelError(err);
},
) as Promise<KernelNativeAsyncResultHandle>;
} else if (this.cancellableExecution) {
// Sync (`runAsync: false`) path: drive the blocking `result()` to the
// terminal `Statement`. Store it on `blockingStatement` so `close()` can
Expand Down
4 changes: 4 additions & 0 deletions lib/thrift-backend/EmptyOperationBackend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ export default class EmptyOperationBackend implements IOperationBackend {
return true;
}

public isFinished(): boolean {
return true;
}

public async fetchChunk(): Promise<Array<object>> {
return [];
}
Expand Down
6 changes: 6 additions & 0 deletions lib/thrift-backend/ThriftOperationBackend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,12 @@ export default class ThriftOperationBackend implements IOperationBackend {
return Boolean(this.emptyResultOnNotFound || this.operationHandle.hasResultSet);
}

public isFinished(): boolean {
// `state` is updated from every status response this backend processes,
// including the direct-results status delivered with the execute response.
return this.state === TOperationState.FINISHED_STATE;
}

public async fetchChunk({
limit,
disableBuffering,
Expand Down
Loading
Loading