Repository navigation
Conversation
yesyayen
requested review from
LeeroyHannigan,
amrith,
c33howard,
jcshepherd,
pdf-amzn and
robinnsc
as code owners
August 19, 2026 20:20
yesyayen
force-pushed
the
docs/wasm-browser-playground
branch
from
August 19, 2026 20:24
c624b74 to
d390a53
Compare
LeeroyHannigan
added
the
RFC
yesyayen
force-pushed
the
docs/wasm-browser-playground
branch
4 times, most recently
from
August 19, 2026 21:23
5a6d2ba to
b938f2b
Compare
Design doc for productionizing the browser WASM proof of concept as a v1 playground: the transport-agnostic dispatch seam, real SQLite compiled to wasm, the executor seam and its three-crate split, the invariants the design rests on, the v1 API surface and what an out-of-scope request does, the CI gate and what it cannot catch, and a four-PR rollout. Also fills the design-doc index gaps at 12 and 13, and fixes a doc link in AGENTS.md that pointed at a file which does not exist.
yesyayen
force-pushed
the
docs/wasm-browser-playground
branch
from
August 19, 2026 21:25
b938f2b to
898be0f
Compare
Compilation is not what excludes stream reads or TTL: the narrow seam leaves metadata.rs and stream.rs present and implemented, the latter forced by create_table.rs calling init_stream_shards, so those operations are rejected at the dispatch boundary instead. Merges them with secondary indexes and Transact, which are excluded the same way and for the same reason, and drops the stale claim that the stream read APIs stay stubbed. Records that the allowlist is also what keeps the ttl.rs and streams.rs mappers off the wasm path. States the engine clock swap as the project convention rather than as a defect the port found, since those three lines are correct on native.
workers.rs has 25 statement sites, not 23, so the set that is seamed but never compiles on wasm is 100 rather than 98. Verified at this branch's tree and at upstream/main, and the five-way breakdown that produced 23 (13 sqlx::query, 9 query_as, 2 query_as::<, 1 query_scalar) itself sums to 25. Also moves the decode-path gate from commit 1 to commit 3. On commit 1 the clause was vacuous: that commit carries the executor crate and the extractor and converts nothing by construction, so the gate bound nothing. Commit 3 is the first conversion, which is where it binds.
The exclusion table said the compilation row's trait methods "return the unsupported error", which cannot hold: a module that is not compiled has no method to return anything, and nothing in the workspace produces StorageError::Unsupported today. The row now states what compilation does and names the part that is not settled, since StorageEngine is a supertrait over six traits including BackupEngine and WorkerStore, so a wasm build that omits those modules either narrows the bound it uses or carries wasm-only arms. Also states plainly what the table left implicit. The allowlist equals the documented v1 surface, so it is the operative gate for every row, and the other two rows describe additional properties rather than the cause of rejection. Two readers reached opposite conclusions from this table because it named three mechanisms and did not say which one governs.
The proposal belongs in docs/rfcs/, not docs/design/: it asks for input on a change that touches the storage backend contract and adds a subsystem, which is what docs/rfcs/README.md says an RFC is for, and docs/design/ describes components as built. It now follows the template's section set. Reframed around the target rather than the browser. Compiling the engine to wasm32-unknown-unknown is the work; the playground is one consumer of it, and only the last of the four steps builds the site. Cut from 1,597 lines to 171, against 305 for the MongoDB backend RFC. What went is the argumentation, the per-commit evidence and the process commentary. What stayed, and in one case returned, is what a maintainer has to act on: the six crates that change before the engine compiles for wasm at all, since cargo check --target wasm32-unknown-unknown -p extenddb-engine currently fails in getrandom, mio and uuid without reaching ExtendDB code. Also corrects claims the long version made: an operation allowlist cannot exclude secondary indexes, which arrive as fields on three operations that are all in v1; DescribeEndpoints and DescribeLimits were in neither list; and a non-zero control-plane delay does not hang, it leaves the table CREATING so every data-plane call returns ResourceNotFoundException.
yesyayen
changed the title
LeeroyHannigan
left a comment
LeeroyHannigan
left a comment
Collaborator
There was a problem hiding this comment.
I checked the RFC's statements about the code against main at 744ea29. These hold: the 37-name operation list, axum reaching the engine through extenddb-auth's HeaderMap, moka and its refresh task in extenddb-cache, the six native-only dependencies and the JoinHandle return of spawn_workers in extenddb-storage, the SQLite delay defaults of 0.25 and 10, Uuid::new_v4 in CreateTable, the three SystemTime::now sites in streams.rs, and the two interpolated LIMIT values. One statement is out of date since main moved: the engine now has two more tokio users (inline). I could not run the wasm32 cargo check here; the target is not installed. The branch merges cleanly with main.
| | `extenddb-core` | `time` gains the `wasm-bindgen` feature, for a clock source | | ||
| | `extenddb-auth` | take `HeaderMap` from `http` instead of `axum::http`, on every target, which drops axum from the graph; the workspace resolves one `http` version, so no signature changes | | ||
| | `extenddb-cache` | a pass-through implementation with no `moka` and no refresh task | | ||
| | `extenddb-engine` | `tokio` becomes native-only, which touches one file: `import_export.rs` is its only user in the crate, and that file, plus the `ImportTable` and `ExportTableToPointInTime` arms that reach it, is gated off. The three `SystemTime::now` sites in `streams.rs` move to the `time` crate. `is_known_operation`'s 37-name `matches!` becomes a `const OPERATIONS: &[&str]` the predicate reads, so the allowlist test can assert against it | |
Collaborator
There was a problem hiding this comment.
Major: on main the engine has two more non-test tokio users, tokio::spawn in crates/engine/src/ttl.rs:150 (UpdateTimeToLive) and crates/engine/src/backup.rs:1015 (added by #384). Both serve operations the v1 allowlist excludes, so they can be gated off like import_export.rs; this row should list them, and the cargo check failure list is worth re-running on current main.
|
|
||
| The shim admits only the v1 surface. A `V1Op` newtype constructible through a single | ||
| `parse_v1_op` function is the gate, over one `const V1_OPS: &[&str]`. `is_known_operation` | ||
| (`crates/engine/src/lib.rs:81`) already enumerates every operation `dispatch` accepts; it |
Collaborator
There was a problem hiding this comment.
Minor: is_known_operation is at crates/engine/src/lib.rs:87 on main. Naming the function without a line number keeps this from going stale.
| native-only change reaching for an `sqlx` type or a tokio type inside the seam fails on the | ||
| change that introduces it. | ||
|
|
||
| The crate split needs a companion amendment to RFC-0002, permitting a backend to factor internal |
Collaborator
There was a problem hiding this comment.
Minor: #293 is this amendment. Linking it here makes the dependency explicit.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make the ExtendDB engine compile to
wasm32-unknown-unknown, and ship one consumer of it: a static page where a visitor runs Amazon DynamoDB API calls against a real ExtendDB with nothing installed. Storage is SQLite compiled to wasm. An operation allowlist bounds the v1 surface. Most of the work is the target itself; the playground is one consumer of it.Three things block it, and the RFC proposes one change for each.
sqlxdoes not compile to wasm and is reachable from every statement in the SQLite backend. An executor seam,extenddb-sqlite-exec, takes over statement execution, and the backend splits into a converted engine half and a catalog half that keepssqlx.extenddb-authand tokio throughextenddb-storage, socargo check --target wasm32-unknown-unknown -p extenddb-enginedies ingetrandom,mioanduuidbefore reaching our code.SystemTime::nowassume a runtime the target does not have. The workers never start and the control-plane delays are zeroed so writes complete inline, the import and export arms are gated off, and the clock moves to thetimecrate.The RFC proposes an executor seam in the SQLite backend, a split of that backend into a converted engine half and a catalog half that stays on
sqlx, target-clean versions of six shared crates, and a wasm entry point that admits twelve operations and rejects the rest before dispatch.