docs(clients): document QWP ingestion and queries in the Node.js client - #569
Open
glasstiger wants to merge 26 commits into
Open
glasstiger wants to merge 26 commits into
glasstiger wants to merge 26 commits into
Conversation
…lient Rewrite the Node.js client page as the JavaScript client page for @questdb/nodejs-client 5.0.0 and the new @questdb/browser-client package: pooled ingestion and streaming SQL queries, column types, compiled writers, acknowledgements, transactions, store-and-forward, UDP, failover, error handling, browser session authentication, and migration from ILP and 4.x. Mark Node.js QWP support as beta on the Connect overview and client cards, and note where the JavaScript client differs on the connect-string, store-and-forward, failover, and client-behavior pages. Document the qwp.browser.tls.termination.enabled server setting.
|
🚀 Build success! Latest successful preview: https://preview-569--questdb-documentation.netlify.app/docs/ Commit SHA: a268159
|
The 5.0.0 release ships QWP support as stable. Drop the beta admonition, list the client as stable on the Connect overview and client cards, and state the 5.0.0 minimum version under Requirements.
- IPv4: 0.0.0.0 is rejected, not stored as NULL; pass null instead. - Store-and-forward example: use lazy_connect=on, because the pooled client cannot start while QuestDB is down with initial_connect_retry=async alone. - Terminal rejections: under store-and-forward the rejected batch stays in the journal and every new sender on the slot fails again; document recovery. - Transactions: a borrowed sender's close() commits the open transaction; only a standalone sender rolls back. Clarify that flush() ends the transaction. - target=replica is a strict filter that also fails startup without a replica. - Document QwpSenderCloseTimeoutError from a standalone sender's close(). - TLS verifies against Node's bundled CAs, not the operating system store. - Store-and-forward lock recovery after a crash, including containers. - Query failover: detect re-execution with batchSequence, note that onReplayReset cannot identify the lease and that timeoutMs spans failover. - Add a DEDUP UPSERT KEYS example for at-least-once replay.
Only @questdb/nodejs-client ships in this release, so document the Node.js client alone and drop every mention of the browser client: the Browser applications section, the @questdb/browser-client package, installation, and requirements, the browser-only opaque QwpUpgradeError kind, and the server-side Browser connections section in configuration/qwp.md with its changelog entry. Rename the page and sidebar entry back to Node.js, and refer to the Node.js client on the connect-string, failover, store-and-forward, client-behavior, PGWire, and date-to-timestamp pages.
glasstiger
changed the title
Unacknowledged rows in memory mode may already have reached QuestDB, so say they may be lost rather than that they are lost. Replace the Java-only journal warning: Node.js clients can share an sf_dir, but clients in other languages lock journals with operating-system file locks that the Node.js client does not see, so they may use the directory only after every Node.js client on it has stopped.
Callbacks run one at a time, and credit for a batch is granted only after its callback resolves, which throttles the server only when initialCredit sets a credit window.
- Store trade IDs as VARCHAR in the store-and-forward and full examples, and document the 2,000,000-value per-sender SYMBOL dictionary cap - Correct rowsAffected, cancel(), max_lifetime_ms, and flush() acceptance semantics, and unwrap QwpReconnectExhaustedError in connectionErrors() - Document the typed reconnect fields, logging hooks, and a table of differences from other clients; merge the sender close semantics into one section - State per-client authentication and durable-ack retry behavior on the shared failover, store-and-forward, and QWP pages, and use one wording for the Node.js memory-mode reconnect exception - Make the store-and-forward error policy table client-agnostic and remove the remaining server-deduplication claims - Fix the UDP client list, separate-connection wording, query overview failover links, quick start client sentence, and changelog entry
- State that a terminally rejected batch blocks store-and-forward ingestion for every table, across restarts, and that longArrayColumn() triggers it on current servers - Say which at() rejections lose staged rows, including the next borrower's rows on a failed pooled sender - Document that a locked journal stops connectQwpNodeClient() even with lazy_connect=on, and when lock cleanup may be automated - Document that batches staged before the first connection can exceed the server limit, and recommend sf_max_segment_bytes=1m - Warn that queries without a credit window buffer results in memory - Warn that QWP auto-creates tables without DEDUP; drop sf_dir from the lazy-start example - Correct the typed reconnect object's effect on the first connection - Explain that 8 fast-failing attempts end query failover in seconds, and use maxAttempts: 0 in the full example - Add a pattern for committing source offsets after acknowledgement - Move Read-after-write and the typed reconnect policy next to their prerequisites, add an error state table, and link the 4.x migration notes from the top - Qualify the connect-string recipes that share target=replica, which the Node.js client also applies to ingress
…d .NET The Java and .NET pages still described DROP_AND_CONTINUE and HALT, which contradicted the corrected store-and-forward error table. - Java: list all ten categories and the RETRIABLE, RETRIABLE_OTHER, TERMINAL, and ABANDONED policies, and add getQuarantinedPath() - .NET: document Retriable, Terminal, and Abandoned with their default categories, the accepted on_*_error values (drop now means retry), and that the keys can only make a category stricter - Store-and-forward tuning: log levels follow terminal and retriable policies - Changelog: mention the Java and .NET page updates
- Make Read-after-write self-contained and move journal replay to its own subsection, which polls for the row instead of waiting on recovered ACKs - Correct the batch-size advice and the typed store-and-forward defaults - Document the result row shape, failover defaults, the full bind setter table, cancellation cost without a credit window, and shutdown cancels - Expand error handling: typed catch examples, sender error fields and policies, upgrade error kinds, 401/403 retry rules, and a runbook for a journal blocked by a terminal rejection - List all connection events and warn that a typed egress reconnect object re-enables query failover - Add examples for the standalone Sender, compiled writers, offset commits, transactions, durable ACK, UDP, DDL/DML, cancellation, result views, and query failover; restore a runnable ILP example - Consolidate startup and outage modes into one section and complete the Differences table - Note Node.js full jitter on the failover and store-and-forward pages and stop listing every server rejection as terminal
…ance - Startup: initial_connect_retry=on and reconnect_* keys retry senders only. connectQwpNodeClient() still rejects at once with the default query_pool_min=1; document query_pool_min=0, and lazy_connect=on as initial_connect_retry=async plus query_pool_min=0. Add Node.js notes to the connect-string startup recipe and the lazy_connect entry, and correct the recipe's claim that the budget covers later reconnects. - Batch size: a frame built offline after any background start can exceed the server limit and block replay, with or without sf_dir; recommend sf_max_segment_bytes=1m for every background start. - Transactions: a pooled sender cannot roll back, because close() commits batches that auto-flush already sent. Show the standalone Sender, whose close() rolls back an uncommitted transaction. - Zero-copy views: the example accumulates a total, so set failover=off to stop a re-run from adding replayed batches twice.
- Quick start polls for the row it wrote, so a first run on a fresh table prints it instead of nothing - Document ackTimeoutMs and durableAckTimeoutMs: a timed-out flush() rejects with a plain Error, waitForAcknowledged() with QwpIngressAckTimeoutError, and neither means the batch failed; raise the durable-ACK deadline above the replication throttle window - Explain that tables needing DEDUP must exist before the first ingester runs, and show ALTER TABLE ... DEDUP ENABLE for an auto-created table - Make Startup and outage modes the single source: recommended keys first, a mode table, a caution that default memory mode blocks producers, and bullets instead of conditional chains; cut restatements elsewhere - Add a runnable decimals example and a decimal bind, and move the old decimal anchors back into the section - Add Quarantined journal slots: .unreplayable-N and .failed slots, and replay with retryQwpNodeOrphanSlot() - Show the onError event, state that every named class is exported, that same-host stale locks are reclaimed automatically, and that at() writes an existing table's designated timestamp - Align connect-string, client-failover, and store-and-forward pages with the Node.js startup and role-filter behavior
- Lock recovery: state the actual reclaim rule (same host, recorded process ID no longer in use) and that a container restarted in place keeps failing until the stale lock is removed. Add Node.js cautions to orphan adoption on the store-and-forward concepts and when-to-use pages. - Connection errors: with several endpoints, an authentication rejection surfaces as the QwpUpgradeError itself, not a QwpFailoverError. - Background drainer: describe its real scope (the client's own sender_id slots always, other sender_ids only with drain_orphans=on) in one place and link it from quarantine, connection events, the differences table, and the connect string reference. - Add a full example combining TLS, a token, two endpoints, store-and-forward, DEDUP, error callbacks, an ACK wait, and a failover-safe query. - List every sender error category with its default policy, and branch on the exported constants in the example. - Map typed reconnect fields to their connect-string keys and defaults, and list the reconnect and failover backoff keys inline. - Document writing from request handlers, the exported client classes, close() behavior per mode, staged rows dropped after an append timeout, and replays that return no batches. - QWP client behavior: include terminal server rejections in the async stop conditions, and correct the Java reconnect-loop appendix.
QuestDB Enterprise 3.3.1 lowered the default of replication.primary.throttle.window.duration from 10 seconds to 1 second. - Replication tuning: update the settings table, the default profile, the cost table's default row, the code sample, and the value table - Replication configuration reference: default 1000 - Node.js client: the durable acknowledgement section quoted the old default; link the 60-second figure to the network-efficiency settings - Changelog: note the correction
- Full example: raise failover_max_attempts together with the failover time budget; the default cap of 8 attempts ends query failover within seconds, so the minute-long budget alone did nothing - Query failover: state that against refused connections the attempt cap, not the time budget, usually ends failover - Server shutdown: with failover on (the default), an interrupted query fails over like any lost connection; only failover=off reports CANCELLED, which query.cancel() also produces, so do not retry a query you cancelled - Connection errors: document the QwpReconnectExhaustedError wrapper when the first connection is retried, list the cause shapes as bullets, and add an example that unwraps the cause chain - timestampColumn(): show the "us" default unit and extend the 1970 warning to it - Backpressure: a borrowed sender's close() rejects with the append timeout error, a standalone Sender's close() with QwpSenderCloseTimeoutError after close_flush_timeout_millis - Name QwpIngressNackError, which waitForAcknowledged() rejects with for a terminally rejected batch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Documents QWP support in
@questdb/nodejs-client5.0.0 in one Node.js client page, alongside the other client guides.documentation/connect/clients/nodejs.mdwith a quick start, connection and authentication, row ingestion, streaming queries, acknowledgements, store-and-forward, failover, errors, and ILP migration. The page remains at/docs/connect/clients/nodejs/; pre-existing anchors are retained. No separate operations page or sidebar entry.decimalColumnText()exponent strings, background startup withsf_dir, durable-ACK capability errors, and connect-string behavior.Shared corrections
on_*_errorsupport; clarifies auth retries, per-client defaults, andsf_dirdirectory creation.Verification
yarn buildandyarn testpass.tsc --strictagainst the local 5.0.0 client build.Dependency
Merge after
@questdb/nodejs-client5.0.0 is published.