From cafc0062cfbeb1e4c2dbb26e748b909da6d3a673 Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 19:09:41 +0200 Subject: [PATCH 01/13] feat: add pubky marketplace test fixture for the wallet journey Adds an opt-in marketplace compose profile with a Pubky testnet, Paykit Server 867fc883 built from pinned source and a purchase driver CLI (pubky-marketplace), on top of the stack's regtest bitcoind and electrs. --- .gitignore | 1 + CHANGELOG.md | 1 + CLAUDE.md | 3 + README.md | 41 ++ docker-compose.yml | 121 ++++ docs/pubky-marketplace.md | 109 +++ marketplace/driver/.dockerignore | 1 + marketplace/driver/Dockerfile | 11 + marketplace/driver/driver.mjs | 791 +++++++++++++++++++++ marketplace/driver/package-lock.json | 122 ++++ marketplace/driver/package.json | 12 + marketplace/homeserver.toml | 34 + marketplace/postgres-init/01-databases.sql | 2 + marketplace/pubky-testnet/Dockerfile | 22 + pubky-marketplace | 155 ++++ 15 files changed, 1426 insertions(+) create mode 100644 docs/pubky-marketplace.md create mode 100644 marketplace/driver/.dockerignore create mode 100644 marketplace/driver/Dockerfile create mode 100644 marketplace/driver/driver.mjs create mode 100644 marketplace/driver/package-lock.json create mode 100644 marketplace/driver/package.json create mode 100644 marketplace/homeserver.toml create mode 100644 marketplace/postgres-init/01-databases.sql create mode 100644 marketplace/pubky-testnet/Dockerfile create mode 100755 pubky-marketplace diff --git a/.gitignore b/.gitignore index ef3db00..4d49d68 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,4 @@ target/ .vscode/ .trezor-user-env/ __pycache__/ +.marketplace/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 969e256..ca4baf6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Preserve LNURL-pay invoice millisatoshi precision by creating invoices with LND `value_msat` instead of truncating callback amounts to sats ### Added +- Pubky marketplace test fixture behind the `marketplace` compose profile: Pubky testnet, Paykit Server `867fc883` built from pinned source, and the `pubky-marketplace` driver CLI (`up`, `seed`, `purchase`, `mine`, `verify`) for the Bitkit marketplace wallet journey; see `docs/pubky-marketplace.md` - Homegate Docker Compose service with dedicated PostgreSQL storage, local homeserver admin mock, and README setup flow - Repo-managed Trezor User Env Docker service and `scripts/trezor-emulator` helper for quickly smoke-testing Bitkit app Trezor PRs - Support `amount_msat` query param in `/generate/bolt11` endpoint for sub-sat precision invoices diff --git a/CLAUDE.md b/CLAUDE.md index 6690a56..26a7349 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,6 +11,7 @@ Bitkit Docker is a complete Docker-based development environment for Bitcoin and - **Electrum Server** - Electrum protocol server - **LNURL Server** - Main web application for LNURL protocols - **VSS Server** - Versioned Storage Service for wallet backup +- **Pubky marketplace fixture** (`marketplace` compose profile) - Pubky testnet, Paykit Server and a purchase driver, run through `./pubky-marketplace` ## Repository Structure @@ -31,6 +32,8 @@ bitkit-docker/ │ ├── middleware/ # Express middleware │ └── utils/ # Utility functions ├── lnd/ # LND configuration and data +├── pubky-marketplace # CLI for the Pubky marketplace fixture (see docs/pubky-marketplace.md) +├── marketplace/ # Fixture assets: Pubky testnet Dockerfile, homeserver config, driver (Node) ├── vss-server/ # VSS server (git submodule) └── sql/ # Database schemas ``` diff --git a/README.md b/README.md index 2755c18..f1ef4e2 100644 --- a/README.md +++ b/README.md @@ -11,6 +11,7 @@ A complete Docker-based development environment for Bitcoin and Lightning Networ - **LDK Backup Server**: Lightning Development Kit backup service - **VSS Server**: Versioned Storage Server for app and ldk-node state backups - **Homegate**: Pubky Homeserver signup gatekeeper with local admin API mock +- **Pubky marketplace fixture** (opt-in `marketplace` profile): Pubky testnet, Paykit Server and a purchase driver for the marketplace wallet journey ## Quick Start @@ -265,6 +266,46 @@ Open the dashboard at `Settings -> Advanced -> Trezor Hardware Wallet`, then che See [docs/trezor-emulator.md](docs/trezor-emulator.md) for helper internals, environment overrides, and troubleshooting commands. +#### Pubky Marketplace Journey + +Use this section for the Bitkit marketplace wallet journey (`journeys/pubky-marketplace` in [bitkit-ios](https://github.com/synonymdev/bitkit-ios/tree/master/journeys/pubky-marketplace) and [bitkit-android](https://github.com/synonymdev/bitkit-android/tree/master/journeys/pubky-marketplace)). The `marketplace` profile adds the integration fixture that the journey lists: a Pubky testnet, Paykit Server `867fc883` (the merge of [pubky/paykit-server#2](https://github.com/pubky/paykit-server/pull/2)) with `/setup` and `x-bitkit-claim=watch-only-account-v1`, and a purchase driver. It reuses this stack's regtest `bitcoind` and Electrum on `tcp://127.0.0.1:60001`. A plain `docker compose up -d` does not start it, and `./pubky-marketplace` starts only the chain and fixture services, so run it without the full stack (Homegate also wants port 6288). It needs Docker, `git`, `curl` and `jq` on the host, and outbound internet during the first build and during setup approval (see the setup relay note in [docs/pubky-marketplace.md](docs/pubky-marketplace.md)). + +```bash +./pubky-marketplace up # fetch pinned sources, build, start, wait until Paykit Server is ready (first build takes a while) +./pubky-marketplace seed # mine to maturity, seller wallet and identity, watch-only setup, headless buyer +./pubky-marketplace verify # whole journey with no wallet app; writes .marketplace/evidence//summary.json +``` + +`verify` creates one purchase, receives the Payment Request as a headless buyer, checks the unsigned and garbage-signed calls fail with 401, pays the derived address, confirms the transaction is the only mempool entry, mines exactly one block, and waits for the signed Paykit status `confirmed`. Run the same steps by hand: + +```bash +./pubky-marketplace purchase --sats 15000 # prints bundle id, derived address, amount, delivery state +./pubky-marketplace receive # headless buyer: Payment Request id, checks lowercase btc and the address +./pubky-marketplace pay # headless buyer pays from the regtest wallet +./pubky-marketplace wait detected # signed Paykit status: detected +./pubky-marketplace mine --bundle # exactly one block, refuses if the purchase is not in the mempool +./pubky-marketplace wait confirmed +./pubky-marketplace status # signed Paykit status and purchase state (completed at 1 confirmation) +``` + +For Bitkit wallets, build each simulator or emulator against this fixture before its first launch: + +- iOS: build with `E2E_BUILD E2E_BACKEND=local E2E_NETWORK=regtest E2E_HOMESERVER_PUBKY=`, where `` is `./pubky-marketplace info | jq -r .homeserver_z32`. Electrum resolves to `tcp://127.0.0.1:60001` with no override. +- Android: build the local E2E backend with the same `E2E_HOMESERVER_PUBKY` in the environment. On each emulator run `adb reverse tcp: tcp:` for 6286, 6287, 6288, 15411 and 15412; the Android journey README covers the emulator's `10.0.2.2` host address. +- In each wallet create a Bitkit-generated Pubky identity (not a Pubky Ring import) and enable contact payments. + +Then, with the buyer wallet: + +```bash +./pubky-marketplace info | jq -r .seller.pubky # save this seller as a contact in the buyer wallet +./pubky-marketplace fund 1000000 # sends coins and mines one funding block +./pubky-marketplace purchase --buyer # the Payment Request appears in the buyer wallet +``` + +Pay the request in the app, then confirm with `./pubky-marketplace mine --bundle `, `./pubky-marketplace wait confirmed` and `./pubky-marketplace status `. The seller in a driver purchase is the fixture's headless seller, because publishing the payment lock needs the seller's identity secret. To check only the watch-only claim in a Bitkit seller wallet, run `./pubky-marketplace setup-url`, open the printed `auth_url` in that wallet (the URL holds a one-time secret, so keep it out of logs and evidence), approve it, then `./pubky-marketplace setup-wait `. + +Remove the fixture with `./pubky-marketplace down`, or start over with `./pubky-marketplace reset`. The Pubky testnet keeps its accounts in memory, so the fixture cannot restart with its state and `down` deletes it (the regtest chain stays). `./pubky-marketplace --help` lists every command. See [docs/pubky-marketplace.md](docs/pubky-marketplace.md) for the pins, ports, roles and what the fixture does not cover. + #### Bech32 LNURL Pay - in `Env.{kt,swift}`, use for REGTEST electrum server: `"tcp://localhost:60001"` diff --git a/docker-compose.yml b/docker-compose.yml index 6e08f72..653abb9 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -330,11 +330,132 @@ services: - ./.trezor-user-env/logs/mcp-screenshots:/trezor-user-env/logs/mcp-screenshots - ./.trezor-user-env/firmware/user_downloaded:/trezor-user-env/src/binaries/firmware/bin/user_downloaded + # --------------------------------------------------------------------------- + # Pubky marketplace fixture (profile "marketplace"). Not started by a plain + # `docker compose up`. Drive it with ./pubky-marketplace; see docs/pubky-marketplace.md. + # bitcoind and electrs above are the chain half of the fixture (Electrum on 60001). + # --------------------------------------------------------------------------- + marketplace-postgres: + profiles: [marketplace] + container_name: marketplace-postgres + image: postgres:16-alpine + restart: "no" # the fixture is disposable: a restarted testnet loses its accounts + environment: + POSTGRES_DB: paykit + POSTGRES_USER: marketplace + POSTGRES_PASSWORD: marketplace + volumes: + - marketplace_postgres_data:/var/lib/postgresql/data + - ./marketplace/postgres-init:/docker-entrypoint-initdb.d:ro + healthcheck: + test: ["CMD-SHELL", "pg_isready -U marketplace -d paykit"] + interval: 2s + timeout: 5s + retries: 30 + + # Pubky Core static testnet: DHT, PKARR relay, HTTP relay and one homeserver. + # The pinned Pubky clients (Bitkit E2E builds, Paykit Server) resolve this + # testnet on fixed localhost ports, so those ports are published unchanged on + # the host loopback and paykit-server and marketplace-driver join this + # container's network namespace. + pubky-testnet: + profiles: [marketplace] + container_name: marketplace-pubky-testnet + image: bitkit-docker/pubky-testnet:f68014c1 + build: + context: ./marketplace/pubky-testnet + restart: "no" # the fixture is disposable: a restarted testnet loses its accounts + depends_on: + marketplace-postgres: + condition: service_healthy + command: ["pubky-testnet", "--homeserver-config", "/etc/pubky-homeserver/config.toml"] + volumes: + - ./marketplace/homeserver.toml:/etc/pubky-homeserver/config.toml:ro + ports: + - "127.0.0.1:6881:6881/udp" + - "127.0.0.1:6881:6881/tcp" + - "127.0.0.1:15411:15411" # PKARR relay + - "127.0.0.1:15412:15412" # HTTP relay + - "127.0.0.1:6286:6286" # homeserver ICANN HTTP + - "127.0.0.1:6287:6287" # homeserver Pubky TLS + - "127.0.0.1:${MARKETPLACE_HOMESERVER_ADMIN_PORT:-16288}:6288" # homeserver admin (6288 is homegate's) + - "127.0.0.1:${MARKETPLACE_PAYKIT_PORT:-3001}:3001" # paykit-server, shares this namespace + + # Paykit Server 867fc883 (merge of pubky/paykit-server#2), built from source + # with the upstream Dockerfile.local. `./pubky-marketplace build` first checks out + # the pinned trees under .marketplace/sources; Dockerfile.local fails closed + # if a tree differs from the pins in Paykit Server's Cargo manifests. + paykit-server: + profiles: [marketplace] + container_name: marketplace-paykit-server + image: bitkit-docker/paykit-server:867fc883 + build: + context: ./.marketplace/sources/paykit-server + dockerfile: Dockerfile.local + additional_contexts: + paykit-lib: ./.marketplace/sources/paykit-rs/paykit-lib + paykit-sdk: ./.marketplace/sources/paykit-rs/paykit-sdk + locks: ./.marketplace/sources/locks + restart: "no" # the fixture is disposable: a restarted testnet loses its accounts + network_mode: service:pubky-testnet + depends_on: + marketplace-postgres: + condition: service_healthy + electrs: + condition: service_started + pubky-testnet: + condition: service_started + environment: + PAYKIT_CONFIG: /state/paykit/paykit-server.toml + PAYKIT_DATABASE_URL: postgres://marketplace:marketplace@marketplace-postgres:5432/paykit + volumes: + - marketplace_state:/state:ro + # The master key and the generated config live in /state/paykit, written by + # `marketplace-driver init`; they never appear in compose or inspect output. + # The driver's seeds sit in /state/secrets (mode 0700, root), out of this + # container's reach. + entrypoint: + - /bin/sh + - -c + - | + until [ -s /state/paykit/paykit-server.toml ] && [ -s /state/paykit/master-key ]; do + echo '[marketplace] waiting for marketplace-driver init' + sleep 1 + done + PAYKIT_MASTER_KEY="$$(cat /state/paykit/master-key)" + export PAYKIT_MASTER_KEY + exec /usr/local/bin/paykit-server + + # The fixture driver. Runs one-shot commands (`docker compose run`) through + # the ./pubky-marketplace wrapper. It carries the paykit-companion-auth and + # paykit-reader-demo helper binaries from the paykit-server image. + marketplace-driver: + profiles: [marketplace] + image: bitkit-docker/marketplace-driver:local + build: + context: ./marketplace/driver + additional_contexts: + paykit: service:paykit-server + network_mode: service:pubky-testnet + depends_on: + - pubky-testnet + environment: + BITCOIN_RPC_URL: http://bitcoind:43782 + BITCOIN_RPC_USER: polaruser + BITCOIN_RPC_PASS: polarpass + PAYKIT_URL: http://127.0.0.1:3001 + volumes: + - marketplace_state:/state + - ./.marketplace/evidence:/evidence + entrypoint: ["node", "/app/driver.mjs"] + volumes: bitcoin_home: postgres_data: lnurl_auth_data: homegate_postgres_data: homegate_data: + marketplace_postgres_data: + marketplace_state: networks: {} diff --git a/docs/pubky-marketplace.md b/docs/pubky-marketplace.md new file mode 100644 index 0000000..27442fb --- /dev/null +++ b/docs/pubky-marketplace.md @@ -0,0 +1,109 @@ +# Pubky marketplace fixture + +Integration fixture for the Bitkit marketplace wallet journey (`journeys/pubky-marketplace` in +bitkit-ios and bitkit-android, tracked by [bitkit-ios#794](https://github.com/synonymdev/bitkit-ios/pull/794) +and [bitkit-android#1338](https://github.com/synonymdev/bitkit-android/pull/1338)). The README section +"Pubky Marketplace Journey" has the commands; this page has what is behind them. + +## What runs + +| Piece | Where | Pin | +| --- | --- | --- | +| Regtest bitcoind and Electrum on `tcp://127.0.0.1:60001` | the stack's `bitcoind` and `electrs` | as in `docker-compose.yml` | +| Pubky Core static testnet: DHT, PKARR relay, HTTP relay, one homeserver with open signup | `pubky-testnet`, built from `marketplace/pubky-testnet/Dockerfile` | pubky-core `f68014c1` | +| Homeserver and Paykit databases | `marketplace-postgres` | `postgres:16-alpine` | +| Paykit Server | `paykit-server`, built from source with the upstream `Dockerfile.local` | pubky/paykit-server `867fc883` (merge of #2), paykit-rs `6b241878`, locks-core `df5ea1b6` | +| Purchase driver | `marketplace-driver`, run by `./pubky-marketplace` | `marketplace/driver/package-lock.json` | + +`./pubky-marketplace build` checks the pinned trees out under `.marketplace/sources` (git ignored) and +fails if a checkout is not at its pin. `Dockerfile.local` then fails closed if a tree differs from the +pins in Paykit Server's Cargo manifests. The pins are at the top of `pubky-marketplace`. + +## Ports + +Published on the host loopback only, because the pinned Pubky clients resolve their local testnet on +these fixed ports: + +| Port | Service | +| --- | --- | +| 6286, 6287 | homeserver (ICANN HTTP, Pubky TLS) | +| 15411, 15412 | PKARR relay, HTTP relay | +| 6881 (tcp and udp) | DHT bootstrap | +| 3001 | Paykit Server (`MARKETPLACE_PAYKIT_PORT`) | +| 16288 | homeserver admin (`MARKETPLACE_HOMESERVER_ADMIN_PORT`; the in-container 6288 is Homegate's host port) | +| 60001 | Electrum, from the base stack | + +`paykit-server` and `marketplace-driver` share the `pubky-testnet` network namespace, as the upstream +Locks compose does, so their Pubky clients reach the testnet on localhost. + +## Roles + +The journey needs a seller, a buyer and a marketplace. The driver can play each of them, and Bitkit +wallets replace the wallet roles in the app journey. + +- **Marketplace (always the driver).** It publishes a `paykit-payment` content lock on the seller's + homeserver and posts a signed `POST /invoices` to Paykit Server as the trusted issuer. The issuer key + is generated at `init`, and its public key is `locks.trusted_public_key` in the generated Paykit + config. Paykit Server's status route is signed the same way. This is the part Locks plays in a full + marketplace; Locks itself is not in this stack. +- **Seller (headless).** The driver holds the seller's Pubky identity, publishes the lock with it, and + completes `/setup` through `paykit-companion-auth`, which approves the same + `watch-only-account-v1` claim Bitkit approves. The seller's spending authority is a wallet seed that + stays in the state volume; Paykit Server receives only the account xpub at `m/84'/1'/0'`. +- **Buyer (headless or Bitkit).** The headless buyer is `paykit-reader-demo` at the `bitkit/wallet` + receiver path, paying from the regtest wallet. A Bitkit buyer is passed as `purchase --buyer `. + +Paykit Server runs at `bitkit/server`. A headless buyer's `receive` succeeding shows the server's +Paykit link to the buyer and the buyer's link back to the seller at those two paths. + +## State and secrets + +Everything lives in the `marketplace_state` volume, and `down` deletes it. + +- `/state/paykit` (readable by the Paykit Server process): generated config and master key. +- `/state/secrets` (root, mode 0700, unreadable by Paykit Server): issuer seed, seller identity seed, + seller wallet seed, buyer identity seed. +- `/state/fixture.json`, `/state/purchases.json`: public facts and the purchase ledger. +- `.marketplace/evidence//summary.json`: `verify` output. It holds public keys, bundle and request + ids, addresses, txids and statuses, and no seed or token. + +The driver never prints a seed or key. `setup-url` prints a one-time auth URL that contains a session +secret; it is meant to be pasted into a wallet, so keep it out of logs and evidence. + +## Lifecycle + +The Pubky testnet keeps homeserver files and its DHT in memory, so its accounts do not survive a +restart while Paykit Server's database still expects them. The fixture is therefore disposable as a +whole: `down` removes the fixture containers, its Postgres and its state volume, and the three +services do not restart on their own. After a crash or a Docker restart, run `./pubky-marketplace +reset`. The regtest chain from the base stack is left alone. + +## Purchase states + +`purchase` returns once Paykit Server has durably accepted the invoice and reports `delivery`: +`queued` or `sent` from the server's outbox metrics, or `failed`. The Payment Request id exists only +in the SDK's delivery, so `receive` (headless buyer) reports it; for a Bitkit buyer read it from the +request row in the app. The purchase state moves `created`, `delivered`, `paid`, `payment_detected`, +`completed`. `completed` means a signed Paykit status of `confirmed` with a matching amount at one or +more confirmations, the gate Locks applies with `minimum_confirmations = 1`. + +The derived address is the seller xpub's external child `0/`, where `n` is the number of earlier +purchases for that seller, checked through `bitcoind deriveaddresses`. A headless `receive` compares it +with the address in the delivered Payment Request. + +## Limits + +- **Setup relay.** The pinned Paykit Server starts the setup sign-in on `https://httprelay.pubky.app`, + not on the local relay, and offers no config to change it. `seed` and a Bitkit seller's approval + therefore need outbound internet. Only that one-time handshake leaves the machine. +- **Seller in the app.** Publishing a lock needs the seller's identity secret, so driver purchases use + the headless seller. A Bitkit wallet that approves `setup-url` becomes another Paykit Server creator + for the watch-only leg, and the driver does not sell as it. Full Locks legacy-connect authority for a + Bitkit seller is out of scope here. +- **No Locks server, no guarded content.** The lock has no guarded resource, and the fixture does not + cover marketplace browsing, content delivery, fiat payment or Hypercolor, which the journey also + excludes. +- **Bitkit apps.** The commands for Bitkit wallets follow the journey's fixture contract. They have not + been run against the apps in this change. +- **Fixed container names.** The base services keep their fixed container names, so another checkout's + stack with the same names must be removed first. diff --git a/marketplace/driver/.dockerignore b/marketplace/driver/.dockerignore new file mode 100644 index 0000000..3c3629e --- /dev/null +++ b/marketplace/driver/.dockerignore @@ -0,0 +1 @@ +node_modules diff --git a/marketplace/driver/Dockerfile b/marketplace/driver/Dockerfile new file mode 100644 index 0000000..5ba7619 --- /dev/null +++ b/marketplace/driver/Dockerfile @@ -0,0 +1,11 @@ +FROM node:22-bookworm-slim@sha256:d649c27dae7ba0137b3cef5dd75baa422c08dc3d9e3fc0c23dfb172dc3cc6436 +WORKDIR /app +COPY package.json package-lock.json ./ +RUN npm ci --omit=dev --ignore-scripts --no-audit --no-fund +# Helper binaries from the pinned Paykit Server image: the watch-only companion +# approval (the wallet's role in the setup flow) and the reader demo (a headless +# buyer). Both are Debian bookworm builds, the same base as this image. +COPY --from=paykit /usr/local/bin/paykit-companion-auth /usr/local/bin/paykit-reader-demo /usr/local/bin/ +# The helpers build their HTTP clients from the system CA store. +COPY --from=paykit /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt +COPY driver.mjs ./ diff --git a/marketplace/driver/driver.mjs b/marketplace/driver/driver.mjs new file mode 100644 index 0000000..8957514 --- /dev/null +++ b/marketplace/driver/driver.mjs @@ -0,0 +1,791 @@ +#!/usr/bin/env node +// Purchase driver for the Pubky marketplace test fixture. +// +// It plays the marketplace side of the journey against the fixture's Paykit +// Server: it publishes a payment lock on the seller's homeserver and asks +// Paykit Server for an invoice as the trusted issuer (the role Locks plays in a +// full marketplace). Roles it can also stand in for, so the whole journey runs +// without a wallet app: the seller (watch-only setup) and the buyer (receive +// and pay). Bitkit wallets take the buyer and seller roles in the app journey. +// +// Secrets stay in /state/secrets (root, 0700). Paykit Server only ever sees the +// seller's account xpub. Nothing here prints a seed, a key or a token. + +import { spawn } from 'node:child_process'; +import { createPrivateKey, randomBytes, sign } from 'node:crypto'; +import { chmod, mkdir, readFile, rm, writeFile } from 'node:fs/promises'; +import { existsSync } from 'node:fs'; + +import { blake3 } from '@noble/hashes/blake3'; +import { HDKey } from '@scure/bip32'; +import { Keypair, Pubky, PublicKey } from '@synonymdev/pubky'; + +const STATE = '/state'; +const SECRETS = `${STATE}/secrets`; +const PAYKIT_DIR = `${STATE}/paykit`; +const FIXTURE_FILE = `${STATE}/fixture.json`; +const PURCHASES_FILE = `${STATE}/purchases.json`; +const EVIDENCE_DIR = '/evidence'; + +const PAYKIT_URL = process.env.PAYKIT_URL ?? 'http://127.0.0.1:3001'; +const RPC_URL = process.env.BITCOIN_RPC_URL ?? 'http://bitcoind:43782'; +const RPC_AUTH = `${process.env.BITCOIN_RPC_USER ?? 'polaruser'}:${process.env.BITCOIN_RPC_PASS ?? 'polarpass'}`; +// The static testnet homeserver key is fixed by Pubky Core. +const HOMESERVER = 'pubky8pinxxgqs41n4aididenw5apqp1urfmzdztr8jt4abrkdn435ewo'; +const SETUP_ORIGIN = 'http://localhost:8080'; +const SERVER_PATH = 'bitkit/server'; +const BUYER_PATH = 'bitkit/wallet'; +const ACCOUNT_INDEX = 0; +const DEFAULT_SATS = 15000; +const EXPECTED_ASSET = 'btc'; +const EXPECTED_ENDPOINT = 'btc-regtest-p2wpkh'; + +const log = (message) => process.stderr.write(`[marketplace] ${message}\n`); +const out = (value) => process.stdout.write(`${JSON.stringify(value, null, 2)}\n`); +const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); +const b64url = (bytes) => Buffer.from(bytes).toString('base64url'); + +class DriverError extends Error {} +const fail = (message) => { + throw new DriverError(message); +}; + +// ---------------------------------------------------------------- state files + +async function readJson(path, fallback) { + try { + return JSON.parse(await readFile(path, 'utf8')); + } catch (error) { + if (error.code === 'ENOENT' && fallback !== undefined) return fallback; + throw error; + } +} + +async function writeJson(path, value) { + await writeFile(path, `${JSON.stringify(value, null, 2)}\n`, { mode: 0o644 }); +} + +async function writeSecret(path, value) { + await writeFile(path, value, { mode: 0o600 }); + await chmod(path, 0o600); +} + +async function readSecret(name) { + const path = `${SECRETS}/${name}`; + if (!existsSync(path)) fail(`missing ${name}; run: ./pubky-marketplace seed`); + return (await readFile(path, 'utf8')).trim(); +} + +async function readFixture() { + const fixture = await readJson(FIXTURE_FILE, null); + if (!fixture?.seller) fail('fixture is not seeded; run: ./pubky-marketplace seed'); + return fixture; +} + +// ------------------------------------------------------------------- bitcoind + +async function rpc(method, params = []) { + const response = await fetch(RPC_URL, { + method: 'POST', + headers: { + 'content-type': 'application/json', + authorization: `Basic ${Buffer.from(RPC_AUTH).toString('base64')}`, + }, + body: JSON.stringify({ jsonrpc: '1.0', id: 'driver', method, params }), + }); + const body = await response.json().catch(() => fail(`bitcoind ${method}: HTTP ${response.status}`)); + if (body.error) fail(`bitcoind ${method}: ${body.error.message}`); + return body.result; +} + +const satsToBtc = (sats) => (Number(sats) / 1e8).toFixed(8); + +async function chainInfo() { + const info = await rpc('getblockchaininfo'); + return { height: info.blocks, hash: info.bestblockhash, chain: info.chain }; +} + +async function mineBlocks(count) { + const address = await rpc('getnewaddress'); + return rpc('generatetoaddress', [count, address]); +} + +async function ensureMatureCoins() { + const { height } = await chainInfo(); + if (height < 101) { + log(`mining ${101 - height} blocks so the regtest wallet has spendable coins`); + await mineBlocks(101 - height); + } +} + +// -------------------------------------------------------------------- Paykit + +async function paykitReady() { + try { + const response = await fetch(`${PAYKIT_URL}/health/ready`); + return { ok: response.status === 200, body: await response.json().catch(() => ({})) }; + } catch { + return { ok: false, body: {} }; + } +} + +async function waitForPaykit(seconds = 180) { + const deadline = Date.now() + seconds * 1000; + for (;;) { + const ready = await paykitReady(); + if (ready.ok) return ready.body; + if (Date.now() > deadline) fail('Paykit Server is not ready; see: ./pubky-marketplace logs paykit-server'); + await sleep(2000); + } +} + +// RFC 8785 canonical JSON for the string, integer, boolean, null, array and +// object values this driver signs and hashes. +function canonical(value) { + if (value === null || typeof value !== 'object') return JSON.stringify(value); + if (Array.isArray(value)) return `[${value.map(canonical).join(',')}]`; + return `{${Object.keys(value) + .sort() + .map((key) => `${JSON.stringify(key)}:${canonical(value[key])}`) + .join(',')}}`; +} + +function keyFromSeed(seed) { + const pkcs8 = Buffer.concat([Buffer.from('302e020100300506032b657004220420', 'hex'), seed]); + return createPrivateKey({ key: pkcs8, format: 'der', type: 'pkcs8' }); +} + +// Paykit Server's business routes accept only bodies signed by the trusted key. +async function signedPost(path, body, { signature } = {}) { + const text = canonical(body); + const issuerSeed = Buffer.from(await readSecret('issuer.seed'), 'base64url'); + const headers = { 'content-type': 'application/json' }; + const value = signature ?? b64url(sign(null, Buffer.from(text), keyFromSeed(issuerSeed))); + if (value !== 'none') headers['x-paykit-signature'] = value; + const response = await fetch(`${PAYKIT_URL}${path}`, { method: 'POST', headers, body: text }); + const raw = await response.text(); + let json = null; + try { + json = raw ? JSON.parse(raw) : null; + } catch { + json = null; + } + return { status: response.status, json, raw }; +} + +async function paykitStatus(purchase) { + const response = await signedPost('/transactions/status', { + bundle_id: purchase.bundle_id, + creator: purchase.seller, + }); + if (response.status !== 200) fail(`Paykit status returned HTTP ${response.status}`); + return response.json; +} + +async function outboxState() { + const text = await (await fetch(`${PAYKIT_URL}/metrics`)).text(); + const value = (name) => + text + .split('\n') + .filter((line) => line.startsWith(name) && !line.startsWith('#')) + .reduce((total, line) => total + Number(line.trim().split(/\s+/).pop()), 0); + return { + depth: value('paykit_outbox_depth'), + permanent_failures: value('paykit_outbox_permanent_failures'), + }; +} + +// ------------------------------------------------------------- Pubky identity + +const pubkyClient = () => Pubky.testnet('localhost'); + +async function signUpIdentity(seed) { + const keypair = Keypair.fromSecret(seed); + const signer = pubkyClient().signer(keypair); + try { + await signer.signup(PublicKey.from(HOMESERVER), null); + } catch (error) { + const message = String(error?.message ?? error).toLowerCase(); + if (!/already|409|conflict/.test(message)) throw error; + await signer.signin(); + } + return keypair.publicKey.toString(); +} + +async function sellerSession() { + const seed = Buffer.from(await readSecret('seller-identity.seed'), 'base64url'); + const signer = pubkyClient().signer(Keypair.fromSecret(seed)); + return signer.signin(); +} + +// ----------------------------------------------------------- helper binaries + +function runHelper(binary, input, env = {}) { + return new Promise((resolve, reject) => { + const child = spawn(binary, [], { env: { ...process.env, ...env }, stdio: ['pipe', 'pipe', 'pipe'] }); + let stdout = ''; + let stderr = ''; + child.stdout.on('data', (chunk) => (stdout += chunk)); + child.stderr.on('data', (chunk) => (stderr += chunk)); + child.on('error', reject); + child.on('close', (code) => resolve({ code, stdout: stdout.trim(), stderr: stderr.trim() })); + child.stdin.end(JSON.stringify(input)); + }); +} + +const readerEnv = (seller) => ({ + PAYKIT_READER_STATE_PATH: `${STATE}/reader/state.bin`, + PAYKIT_READER_PUBKY_TESTNET_HOST: 'localhost', + PAYKIT_READER_RECEIVER_PATH: BUYER_PATH, + PAYKIT_READER_SERVER_PUBKY: seller, + PAYKIT_READER_SERVER_PATH: SERVER_PATH, +}); + +// ------------------------------------------------------------------ encodings + +const CROCKFORD = '0123456789ABCDEFGHJKMNPQRSTVWXYZ'; + +function crockford(bytes) { + let bits = 0; + let value = 0; + let result = ''; + for (const byte of bytes) { + value = (value << 8) | byte; + bits += 8; + while (bits >= 5) { + result += CROCKFORD[(value >>> (bits - 5)) & 31]; + bits -= 5; + } + } + if (bits > 0) result += CROCKFORD[(value << (5 - bits)) & 31]; + return result; +} + +const newBundleId = () => crockford(randomBytes(16)); + +// -------------------------------------------------------------------- commands + +async function init() { + await mkdir(SECRETS, { recursive: true, mode: 0o700 }); + await chmod(SECRETS, 0o700); + await mkdir(PAYKIT_DIR, { recursive: true, mode: 0o755 }); + await mkdir(`${STATE}/reader`, { recursive: true, mode: 0o700 }); + if (!existsSync(`${SECRETS}/issuer.seed`)) { + await writeSecret(`${SECRETS}/issuer.seed`, b64url(randomBytes(32))); + log('created the marketplace issuer key'); + } + const issuer = Keypair.fromSecret(Buffer.from(await readSecret('issuer.seed'), 'base64url')) + .publicKey.toString(); + if (!existsSync(`${PAYKIT_DIR}/master-key`)) { + await writeFile(`${PAYKIT_DIR}/master-key`, b64url(randomBytes(32)), { mode: 0o644 }); + log('created the Paykit Server master key'); + } + const config = `[http] +listen_addr = "0.0.0.0:3001" + +[locks] +trusted_public_key = "${issuer}" + +[setup] +allowed_origins = ["${SETUP_ORIGIN}"] + +[paykit] +receiver_path = "${SERVER_PATH}" +receiver_path_priority = ["bitkit"] +network = "testnet" + +[bitcoin] +network = "regtest" + +[electrum] +endpoint = "tcp://electrs:60001" +poll_interval = "1s" + +[outbox] +poll_interval = "500ms" +`; + const path = `${PAYKIT_DIR}/paykit-server.toml`; + if (!existsSync(path) || (await readFile(path, 'utf8')) !== config) { + await writeFile(path, config, { mode: 0o644 }); + } + out({ status: 'initialized', issuer }); +} + +async function completeSetup(flowId, seconds = 120) { + const deadline = Date.now() + seconds * 1000; + for (;;) { + const response = await fetch(`${PAYKIT_URL}/setup/${flowId}/complete`, { method: 'POST' }); + if (response.status === 200) return; + if (![408, 425, 429, 502, 503, 504].includes(response.status)) { + fail(`setup flow ended with HTTP ${response.status}`); + } + if (Date.now() > deadline) fail('setup flow did not complete in time'); + await sleep(1500); + } +} + +async function beginSetup() { + const state = `marketplace-${Date.now()}`; + const response = await fetch( + `${PAYKIT_URL}/setup?return_to=${encodeURIComponent(SETUP_ORIGIN)}&state=${state}`, + ); + if (response.status !== 200) fail(`GET /setup returned HTTP ${response.status}`); + const html = await response.text(); + const flow = html.match(/const flowId=("(?:[^"\\]|\\.)*");/); + const auth = html.match(/([^<]+)<\/code>/); + if (!flow || !auth) fail('setup page has no flow id or auth URL'); + const authUrl = auth[1] + .replaceAll('&', '&') + .replaceAll('<', '<') + .replaceAll('>', '>') + .replaceAll('"', '"') + .replaceAll(''', "'"); + return { flowId: JSON.parse(flow[1]), authUrl }; +} + +async function setupUrl() { + await waitForPaykit(); + const { flowId, authUrl } = await beginSetup(); + const claim = new URL(authUrl.replace('pubkyauth://', 'http://pubkyauth/')).searchParams.get('x-bitkit-claim'); + out({ + flow_id: flowId, + auth_url: authUrl, + claim, + next: `./pubky-marketplace setup-wait ${flowId}`, + }); +} + +async function setupWait(flowId) { + if (!flowId) fail('usage: setup-wait '); + await completeSetup(flowId, 300); + out({ flow_id: flowId, status: 'complete' }); +} + +async function createBuyer() { + const seed = randomBytes(32); + await writeSecret(`${SECRETS}/buyer-identity.seed`, b64url(seed)); + const buyer = await signUpIdentity(seed); + const fixture = await readFixture(); + await mkdir(`${STATE}/reader`, { recursive: true }); + // A fresh buyer starts with fresh reader state, so exactly one request is actionable. + await rm(`${STATE}/reader/state.bin`, { force: true }); + const prepared = await runHelper( + 'paykit-reader-demo', + { version: 1, operation: 'prepare', reader_secret: b64url(seed) }, + readerEnv(fixture.seller.pubky), + ); + if (prepared.code !== 0) fail(`buyer receiver marker failed: ${prepared.stdout || prepared.stderr}`); + fixture.buyer = { pubky: buyer, receiver_path: BUYER_PATH, kind: 'headless' }; + await writeJson(FIXTURE_FILE, fixture); + log(`headless buyer ready: ${buyer}`); + return fixture.buyer; +} + +async function seed(args) { + const buyerMode = flag(args, '--buyer') ?? 'headless'; + if (!['headless', 'none'].includes(buyerMode)) fail('usage: seed [--buyer headless|none]'); + await waitForPaykit(); + await ensureMatureCoins(); + let fixture = await readJson(FIXTURE_FILE, {}); + if (!fixture.seller) { + // The seller wallet's seed is spending authority and stays in /state/secrets. + // Paykit Server receives only the watch-only account xpub at m/84'/1'/0'. + const walletSeed = randomBytes(32); + await writeSecret(`${SECRETS}/seller-wallet.seed`, walletSeed.toString('hex')); + const account = HDKey.fromMasterSeed(walletSeed, { private: 0x04358394, public: 0x043587cf }).derive( + `m/84'/1'/${ACCOUNT_INDEX}'`, + ); + const xpub = account.publicExtendedKey; + if (!xpub.startsWith('tpub')) fail('seller account key is not a regtest tpub'); + const identitySeed = randomBytes(32); + await writeSecret(`${SECRETS}/seller-identity.seed`, b64url(identitySeed)); + const sellerPubky = await signUpIdentity(identitySeed); + log(`seller identity ${sellerPubky}; completing the watch-only setup`); + const { flowId, authUrl } = await beginSetup(); + const approval = await runHelper('paykit-companion-auth', { + version: 1, + auth_url: authUrl, + creator_secret: b64url(identitySeed), + account_xpub: xpub, + account_index: ACCOUNT_INDEX, + }); + if (approval.code !== 0 || !approval.stdout.includes('"approved"')) { + fail(`companion approval failed: ${approval.stderr || approval.stdout}`); + } + await completeSetup(flowId); + fixture = { + homeserver: HOMESERVER, + seller: { + pubky: sellerPubky, + account_xpub: xpub, + account_index: ACCOUNT_INDEX, + receiver_path: SERVER_PATH, + kind: 'headless', + setup_completed_at: new Date().toISOString(), + }, + }; + await writeJson(FIXTURE_FILE, fixture); + } else { + log('seller already seeded'); + } + if (buyerMode === 'headless' && (!fixture.buyer || flagPresent(args, '--fresh-buyer'))) await createBuyer(); + out(await publicInfo()); +} + +const flagPresent = (args, name) => args.includes(name); +function flag(args, name) { + const at = args.indexOf(name); + return at >= 0 ? args[at + 1] : undefined; +} + +async function publicInfo() { + const fixture = await readJson(FIXTURE_FILE, {}); + const chain = await chainInfo().catch(() => null); + const purchases = await readJson(PURCHASES_FILE, []); + return { + homeserver: HOMESERVER, + homeserver_z32: HOMESERVER.replace(/^pubky/, ''), + pubky_testnet: { + homeserver_http: 'http://localhost:6286', + homeserver_pubky_tls: 'localhost:6287', + pkarr_relay: 'http://localhost:15411', + http_relay: 'http://localhost:15412', + dht: 'localhost:6881', + }, + paykit_server: { url: 'http://localhost:3001', receiver_path: SERVER_PATH, network: 'regtest' }, + electrum: 'tcp://127.0.0.1:60001', + chain, + seller: fixture.seller + ? { + pubky: fixture.seller.pubky, + account_xpub: fixture.seller.account_xpub, + account_index: fixture.seller.account_index, + spending_authority: 'held only in the fixture state volume, never given to Paykit Server', + } + : null, + buyer: fixture.buyer ?? null, + purchases: purchases.length, + latest_purchase: purchases.at(-1) ?? null, + }; +} + +async function fund(args) { + const [address, sats = '1000000'] = args; + if (!address || !address.startsWith('bcrt1')) fail('usage: fund [sats]'); + if (!/^\d+$/.test(sats) || Number(sats) <= 0) fail('sats must be a positive integer'); + await ensureMatureCoins(); + const txid = await rpc('sendtoaddress', [address, satsToBtc(sats)]); + const [block] = await mineBlocks(1); + const chain = await chainInfo(); + out({ txid, sats: Number(sats), block, height: chain.height, tip: chain.hash }); +} + +function lockFor({ seller, sats, issuer }) { + return { + version: 1, + creator: seller, + criteria: [ + { + criterion_id: 'criterion-1', + verifier_type: 'paykit-payment', + params: { recipient_pubky: seller, amount: String(sats), asset: 'BTC' }, + }, + ], + lock_logic: { type: 'all', criteria: ['criterion-1'] }, + access_policy: { requested_credential_ttl_seconds: 900 }, + lock_server: { override: issuer }, + created_at: new Date().toISOString().replace(/\.\d+Z$/, 'Z'), + }; +} + +async function issuerPubky() { + const seed = Buffer.from(await readSecret('issuer.seed'), 'base64url'); + return Keypair.fromSecret(seed).publicKey.toString(); +} + +async function expectedAddress(fixture, child) { + const info = await rpc('getdescriptorinfo', [`wpkh(${fixture.seller.account_xpub}/0/*)`]); + const addresses = await rpc('deriveaddresses', [info.descriptor, [child, child]]); + return addresses[0]; +} + +async function purchase(args) { + const sats = Number(flag(args, '--sats') ?? DEFAULT_SATS); + if (!Number.isInteger(sats) || sats <= 0) fail('--sats must be a positive integer'); + const buyerArg = flag(args, '--buyer') ?? 'headless'; + const fixture = await readFixture(); + let reader; + if (buyerArg === 'headless') { + if (!fixture.buyer) fail('no headless buyer; run: ./pubky-marketplace seed --buyer headless'); + reader = fixture.buyer.pubky; + } else { + reader = buyerArg; + } + await waitForPaykit(); + const purchases = await readJson(PURCHASES_FILE, []); + const childIndex = purchases.filter((entry) => entry.seller === fixture.seller.pubky).length; + + const lock = lockFor({ seller: fixture.seller.pubky, sats, issuer: await issuerPubky() }); + const lockText = canonical(lock); + const lockId = crockford(blake3(Buffer.from(lockText))); + const lockPath = `/pub/locks.app/${lockId}.json`; + const session = await sellerSession(); + await session.storage.putText(lockPath, lockText); + + const bundleId = newBundleId(); + const lockResource = `${fixture.seller.pubky}${lockPath}`; + const response = await signedPost('/invoices', { bundle_id: bundleId, lock_resource: lockResource, reader }); + if (response.status !== 204) { + const code = response.json?.error?.code ? ` ${response.json.error.code}` : ''; + const hint = + response.status === 503 + ? '; the buyer needs a Paykit receiver marker on this homeserver (Bitkit: enable contact payments)' + : ''; + fail(`POST /invoices returned HTTP ${response.status}${code}${hint}`); + } + const record = { + bundle_id: bundleId, + seller: fixture.seller.pubky, + reader, + buyer_kind: buyerArg === 'headless' ? 'headless' : 'external', + amount_sats: sats, + asset: EXPECTED_ASSET, + endpoint: EXPECTED_ENDPOINT, + lock_resource: lockResource, + derived_address: await expectedAddress(fixture, childIndex), + child_index: childIndex, + created_at: new Date().toISOString(), + state: 'created', + }; + purchases.push(record); + await writeJson(PURCHASES_FILE, purchases); + + // The invoice call returns once the delivery intent is durable; the outbox + // worker then hands the Payment Request to the SDK. + let delivery = await outboxState(); + const deadline = Date.now() + 20000; + while (delivery.depth > 0 && delivery.permanent_failures === 0 && Date.now() < deadline) { + await sleep(500); + delivery = await outboxState(); + } + record.delivery = delivery.permanent_failures > 0 ? 'failed' : delivery.depth === 0 ? 'sent' : 'queued'; + await writeJson(PURCHASES_FILE, purchases); + out({ ...record, paykit_status: await paykitStatus(record) }); +} + +async function findPurchase(bundleId) { + const purchases = await readJson(PURCHASES_FILE, []); + const purchase = bundleId ? purchases.find((entry) => entry.bundle_id === bundleId) : purchases.at(-1); + if (!purchase) fail(bundleId ? `unknown bundle ${bundleId}` : 'no purchases yet; run: ./pubky-marketplace purchase'); + return { purchases, purchase }; +} + +async function receive(args) { + const fixture = await readFixture(); + const { purchases, purchase } = await findPurchase(args[0]); + if (purchase.buyer_kind !== 'headless') fail('receive is for the headless buyer; a Bitkit buyer receives in the app'); + const seed = await readSecret('buyer-identity.seed'); + log('waiting for the Payment Request (up to 5 minutes)'); + const result = await runHelper( + 'paykit-reader-demo', + { version: 1, operation: 'receive', reader_secret: seed }, + readerEnv(fixture.seller.pubky), + ); + if (result.code !== 0) fail(`receive failed: ${result.stdout || result.stderr}`); + const request = JSON.parse(result.stdout); + // The pinned reader rejects any endpoint other than btc-regtest-p2wpkh and any + // payload that is not a JSON object with a string value before it projects. + if (request.status !== 'received' || request.asset !== EXPECTED_ASSET) fail('Payment Request is not canonical lowercase btc'); + if (!request.address.startsWith('bcrt1')) fail('Payment Request address is not regtest'); + if (request.amount_sats !== String(purchase.amount_sats)) fail('Payment Request amount does not match the purchase'); + if (request.address !== purchase.derived_address) fail('Payment Request address is not the seller xpub child for this purchase'); + purchase.payment_request_id = request.payment_request_id; + purchase.delivery = 'received'; + purchase.state = 'delivered'; + await writeJson(PURCHASES_FILE, purchases); + out({ + bundle_id: purchase.bundle_id, + payment_request_id: request.payment_request_id, + delivery: 'received', + asset: request.asset, + endpoint: EXPECTED_ENDPOINT, + address: request.address, + amount_sats: Number(request.amount_sats), + address_derived_from_seller_xpub: true, + }); +} + +async function pay(args) { + const { purchases, purchase } = await findPurchase(args[0]); + await ensureMatureCoins(); + const txid = await rpc('sendtoaddress', [purchase.derived_address, satsToBtc(purchase.amount_sats)]); + purchase.txid = txid; + purchase.state = 'paid'; + await writeJson(PURCHASES_FILE, purchases); + const mempool = await rpc('getrawmempool'); + const tx = await rpc('getrawtransaction', [txid, true]); + const match = tx.vout.find((output) => output.scriptPubKey.address === purchase.derived_address); + out({ + bundle_id: purchase.bundle_id, + txid, + mempool_entries: mempool.length, + matched_output_sats: match ? Math.round(match.value * 1e8) : null, + }); +} + +async function statusCommand(args) { + const { purchases, purchase } = await findPurchase(args[0]); + const paykit = await paykitStatus(purchase); + const before = purchase.state; + if (paykit.status === 'confirmed' && paykit.amount_matched && paykit.confirmations >= 1) purchase.state = 'completed'; + else if (paykit.status === 'detected' && paykit.amount_matched) purchase.state = 'payment_detected'; + if (purchase.state !== before) await writeJson(PURCHASES_FILE, purchases); + out({ + bundle_id: purchase.bundle_id, + seller: purchase.seller, + reader: purchase.reader, + payment_request_id: purchase.payment_request_id ?? null, + delivery: purchase.delivery ?? null, + derived_address: purchase.derived_address, + amount_sats: purchase.amount_sats, + txid: purchase.txid ?? null, + paykit_status: paykit, + // Completed is the gate Locks applies with minimum_confirmations = 1. + purchase_state: purchase.state, + }); +} + +async function mine(args) { + const bundle = flag(args, '--bundle'); + const before = await chainInfo(); + const mempoolBefore = await rpc('getrawmempool'); + if (bundle) { + const { purchase } = await findPurchase(bundle); + if (!purchase.txid || !mempoolBefore.includes(purchase.txid)) fail('the purchase transaction is not in the mempool'); + } + const [block] = await mineBlocks(1); + const after = await chainInfo(); + if (after.height !== before.height + 1) fail(`expected exactly one new block, chain moved ${before.height} to ${after.height}`); + out({ + mined: 1, + height: after.height, + block, + mempool_before: mempoolBefore.length, + mempool_after: (await rpc('getrawmempool')).length, + }); +} + +async function waitFor(args) { + const [bundle, target, seconds = '90'] = args; + if (!bundle || !['detected', 'confirmed'].includes(target)) fail('usage: wait [seconds]'); + const { purchase } = await findPurchase(bundle); + const deadline = Date.now() + Number(seconds) * 1000; + for (;;) { + const paykit = await paykitStatus(purchase); + const reached = + target === 'detected' + ? ['detected', 'confirmed'].includes(paykit.status) && paykit.amount_matched + : paykit.status === 'confirmed' && paykit.confirmations >= 1 && paykit.amount_matched; + if (reached) return out({ bundle_id: bundle, paykit_status: paykit }); + if (Date.now() > deadline) fail(`payment never reached ${target}; last status ${JSON.stringify(paykit)}`); + await sleep(1500); + } +} + +// The whole journey with the driver in every wallet role. Each step asserts. +async function verify() { + const evidence = { started_at: new Date().toISOString(), steps: {} }; + const capture = async (name, run) => { + let output = ''; + const original = process.stdout.write.bind(process.stdout); + process.stdout.write = (chunk) => { + output += chunk; + return true; + }; + try { + await run(); + } finally { + process.stdout.write = original; + } + const value = output ? JSON.parse(output) : null; + evidence.steps[name] = value; + log(`${name}: ok`); + return value; + }; + + await capture('seed', () => seed(['--buyer', 'none'])); + delete evidence.steps.seed; // public facts are recorded under seller and buyer + const fixture = await readFixture(); + const buyer = await createBuyer(); + evidence.seller = { pubky: fixture.seller.pubky, account_xpub: fixture.seller.account_xpub, account_index: fixture.seller.account_index }; + evidence.buyer = { pubky: buyer.pubky }; + + const created = await capture('purchase', () => purchase(['--buyer', 'headless'])); + if (created.paykit_status.status !== 'undetected') fail('a new purchase must start undetected'); + const bundle = created.bundle_id; + const invoiceBody = { bundle_id: bundle, lock_resource: created.lock_resource, reader: created.reader }; + + const unsigned = await signedPost('/invoices', invoiceBody, { signature: 'none' }); + const garbage = await signedPost('/invoices', invoiceBody, { signature: b64url(Buffer.alloc(64)) }); + const unsignedStatus = await signedPost('/transactions/status', { bundle_id: bundle, creator: created.seller }, { signature: 'none' }); + if (unsigned.status !== 401 || garbage.status !== 401 || unsignedStatus.status !== 401) { + fail('Paykit Server accepted an unsigned or garbage-signed business call'); + } + evidence.steps.trust_boundary = { unsigned_invoice: 401, garbage_signed_invoice: 401, unsigned_status: 401 }; + log('trust_boundary: ok'); + + const received = await capture('receive', () => receive([bundle])); + const heightBefore = (await chainInfo()).height; + const paid = await capture('pay', () => pay([bundle])); + if (paid.matched_output_sats !== received.amount_sats || paid.mempool_entries !== 1) { + fail('the mempool must hold exactly the purchase transaction with an amount-matched output'); + } + await capture('detected', () => waitFor([bundle, 'detected'])); + const mined = await capture('mine', () => mine(['--bundle', bundle])); + if (mined.height !== heightBefore + 1) fail('exactly one block must confirm the payment'); + await capture('confirmed', () => waitFor([bundle, 'confirmed'])); + const final = await capture('status', () => statusCommand([bundle])); + if (final.purchase_state !== 'completed') fail('purchase did not complete'); + + evidence.finished_at = new Date().toISOString(); + evidence.result = 'passed'; + if (existsSync(EVIDENCE_DIR)) { + const dir = `${EVIDENCE_DIR}/${evidence.started_at.replace(/[:.]/g, '-')}`; + await mkdir(dir, { recursive: true }); + await writeJson(`${dir}/summary.json`, evidence); + evidence.evidence_dir = `.marketplace/evidence/${dir.split('/').pop()}`; + } + out(evidence); +} + +const commands = { + init: () => init(), + seed, + info: async () => out(await publicInfo()), + 'setup-url': () => setupUrl(), + 'setup-wait': (args) => setupWait(args[0]), + fund, + purchase, + receive, + pay, + status: statusCommand, + mine, + wait: waitFor, + verify: () => verify(), +}; + +const [command, ...args] = process.argv.slice(2); +if (!commands[command]) { + process.stderr.write(`unknown driver command: ${command ?? '(none)'}\n`); + process.exit(2); +} +try { + await commands[command](args); +} catch (error) { + if (error instanceof DriverError) { + process.stderr.write(`FAIL: ${error.message}\n`); + process.exit(1); + } + process.stderr.write(`FAIL: ${String(error?.message ?? error)}\n`); + process.exit(1); +} diff --git a/marketplace/driver/package-lock.json b/marketplace/driver/package-lock.json new file mode 100644 index 0000000..853b63b --- /dev/null +++ b/marketplace/driver/package-lock.json @@ -0,0 +1,122 @@ +{ + "name": "bitkit-docker-marketplace-driver", + "version": "0.1.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "bitkit-docker-marketplace-driver", + "version": "0.1.0", + "dependencies": { + "@noble/hashes": "1.8.0", + "@scure/bip32": "1.7.0", + "@synonymdev/pubky": "0.9.3" + } + }, + "node_modules/@noble/curves": { + "version": "1.9.7", + "resolved": "https://registry.npmjs.org/@noble/curves/-/curves-1.9.7.tgz", + "integrity": "sha512-gbKGcRUYIjA3/zCCNaWDciTMFI0dCkvou3TL8Zmy5Nc7sJ47a0jtOeZoTaMxkuqRo9cRhjOdZJXegxYE5FN/xw==", + "license": "MIT", + "dependencies": { + "@noble/hashes": "1.8.0" + }, + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@noble/hashes": { + "version": "1.8.0", + "resolved": "https://registry.npmjs.org/@noble/hashes/-/hashes-1.8.0.tgz", + "integrity": "sha512-jCs9ldd7NwzpgXDIf6P3+NrHh9/sD6CQdxHyjQI+h/6rDNo88ypBxxz45UDuZHz9r3tNz7N/VInSVoVdtXEI4A==", + "license": "MIT", + "engines": { + "node": "^14.21.3 || >=16" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@scure/base": { + "version": "1.2.6", + "resolved": "https://registry.npmjs.org/@scure/base/-/base-1.2.6.tgz", + "integrity": "sha512-g/nm5FgUa//MCj1gV09zTJTaM6KBAHqLN907YVQqf7zC49+DcO4B1so4ZX07Ef10Twr6nuqYEH9GEggFXA4Fmg==", + "license": "MIT", + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@scure/bip32": { + "version": "1.7.0", + "resolved": "https://registry.npmjs.org/@scure/bip32/-/bip32-1.7.0.tgz", + "integrity": "sha512-E4FFX/N3f4B80AKWp5dP6ow+flD1LQZo/w8UnLGYZO674jS6YnYeepycOOksv+vLPSpgN35wgKgy+ybfTb2SMw==", + "license": "MIT", + "dependencies": { + "@noble/curves": "~1.9.0", + "@noble/hashes": "~1.8.0", + "@scure/base": "~1.2.5" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@synonymdev/pubky": { + "version": "0.9.3", + "resolved": "https://registry.npmjs.org/@synonymdev/pubky/-/pubky-0.9.3.tgz", + "integrity": "sha512-3Kl8Ad+yy5moAUc9PrcUypaMhzk5VW7f/bckUMi8mNQU1heEX8sT2VqMM5+lkQN13R9ZWQRV4j4osYGXR3zT6A==", + "license": "MIT", + "dependencies": { + "fetch-cookie": "^3.0.1" + } + }, + "node_modules/fetch-cookie": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/fetch-cookie/-/fetch-cookie-3.2.0.tgz", + "integrity": "sha512-n61pQIxP25C6DRhcJxn7BDzgHP/+S56Urowb5WFxtcRMpU6drqXD90xjyAsVQYsNSNNVbaCcYY1DuHsdkZLuiA==", + "license": "Unlicense", + "dependencies": { + "set-cookie-parser": "^2.4.8", + "tough-cookie": "^6.0.0" + } + }, + "node_modules/set-cookie-parser": { + "version": "2.7.2", + "resolved": "https://registry.npmjs.org/set-cookie-parser/-/set-cookie-parser-2.7.2.tgz", + "integrity": "sha512-oeM1lpU/UvhTxw+g3cIfxXHyJRc/uidd3yK1P242gzHds0udQBYzs3y8j4gCCW+ZJ7ad0yctld8RYO+bdurlvw==", + "license": "MIT" + }, + "node_modules/tldts": { + "version": "7.4.4", + "resolved": "https://registry.npmjs.org/tldts/-/tldts-7.4.4.tgz", + "integrity": "sha512-kFXFK7O4WPextIUAOk8qtnw9dxR9UIXP9CjuH1cTBVBZMDeQcUPgr/IazGiw1B0Yiw5L75gHLWeW4iD793r90g==", + "license": "MIT", + "dependencies": { + "tldts-core": "^7.4.4" + }, + "bin": { + "tldts": "bin/cli.js" + } + }, + "node_modules/tldts-core": { + "version": "7.4.4", + "resolved": "https://registry.npmjs.org/tldts-core/-/tldts-core-7.4.4.tgz", + "integrity": "sha512-vwVLJVvvpslm7vqAH7+XNj/neA/Ynq7DT2EEcMuwc5YzN5XaMyRAqxwU+uX3azZ1FQtB2gvrvnLnAEkvYlVdfg==", + "license": "MIT" + }, + "node_modules/tough-cookie": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-6.0.1.tgz", + "integrity": "sha512-LktZQb3IeoUWB9lqR5EWTHgW/VTITCXg4D21M+lvybRVdylLrRMnqaIONLVb5mav8vM19m44HIcGq4qASeu2Qw==", + "license": "BSD-3-Clause", + "dependencies": { + "tldts": "^7.0.5" + }, + "engines": { + "node": ">=16" + } + } + } +} diff --git a/marketplace/driver/package.json b/marketplace/driver/package.json new file mode 100644 index 0000000..2b6f5c0 --- /dev/null +++ b/marketplace/driver/package.json @@ -0,0 +1,12 @@ +{ + "name": "bitkit-docker-marketplace-driver", + "version": "0.1.0", + "private": true, + "type": "module", + "description": "Purchase driver for the Pubky marketplace test fixture", + "dependencies": { + "@noble/hashes": "1.8.0", + "@scure/bip32": "1.7.0", + "@synonymdev/pubky": "0.9.3" + } +} diff --git a/marketplace/homeserver.toml b/marketplace/homeserver.toml new file mode 100644 index 0000000..7c588bd --- /dev/null +++ b/marketplace/homeserver.toml @@ -0,0 +1,34 @@ +# Homeserver config for the marketplace fixture's Pubky testnet. Open signup so +# Bitkit's generated identities and the fixture's headless identities can sign +# up without a token. Local-only credentials. +[general] +database_url = "postgres://marketplace:marketplace@marketplace-postgres:5432/pubky_homeserver" +signup_mode = "open" + +[drive] +pubky_listen_socket = "0.0.0.0:6287" +icann_listen_socket = "0.0.0.0:6286" + +[storage] +type = "file_system" + +[admin] +enabled = true +listen_socket = "0.0.0.0:6288" +admin_password = "admin" + +[metrics] +enabled = false +listen_socket = "0.0.0.0:6289" + +[pkdns] +public_ip = "127.0.0.1" +public_pubky_tls_port = 6287 +public_icann_http_port = 6286 +icann_domain = "localhost" +user_keys_republisher_interval = 14400 +dht_request_timeout_ms = 2000 + +[logging] +level = "info" +module_levels = ["pubky_homeserver=debug", "tower_http=debug"] diff --git a/marketplace/postgres-init/01-databases.sql b/marketplace/postgres-init/01-databases.sql new file mode 100644 index 0000000..d5a1de5 --- /dev/null +++ b/marketplace/postgres-init/01-databases.sql @@ -0,0 +1,2 @@ +-- POSTGRES_DB creates "paykit"; the homeserver needs its own database. +CREATE DATABASE pubky_homeserver; diff --git a/marketplace/pubky-testnet/Dockerfile b/marketplace/pubky-testnet/Dockerfile new file mode 100644 index 0000000..5a0cf1c --- /dev/null +++ b/marketplace/pubky-testnet/Dockerfile @@ -0,0 +1,22 @@ +# Pubky Core static testnet (DHT, PKARR relay, HTTP relay, homeserver) for the +# marketplace fixture. Same build as pubky/locks docker/pubky-testnet.Dockerfile +# at ba49a777, pinned to the Pubky Core revision that Paykit Server 867fc883 and +# Bitkit's local testnet client are exercised against. +FROM rust:1.89.0-bookworm AS builder + +ARG PUBKY_CORE_REV=f68014c111af0458e6a321e2d87a12479bfb3218 +WORKDIR /usr/src/pubky-core +RUN git clone --filter=blob:none https://github.com/pubky/pubky-core.git . \ + && git checkout --detach "${PUBKY_CORE_REV}" +# The pinned Pubky Core revision locks quinn-proto 0.11.14, affected by +# RUSTSEC-2026-0185. Keep this precise override until its lockfile advances. +RUN cargo update -p quinn-proto --precise 0.11.15 \ + && cargo build --release -p pubky-testnet --bin pubky-testnet + +FROM debian:bookworm-slim +RUN apt-get update \ + && apt-get install -y --no-install-recommends ca-certificates \ + && rm -rf /var/lib/apt/lists/* +COPY --from=builder /usr/src/pubky-core/target/release/pubky-testnet /usr/local/bin/pubky-testnet +EXPOSE 6881 15411 15412 6286 6287 6288 +CMD ["pubky-testnet"] diff --git a/pubky-marketplace b/pubky-marketplace new file mode 100755 index 0000000..28b6510 --- /dev/null +++ b/pubky-marketplace @@ -0,0 +1,155 @@ +#!/usr/bin/env bash +# Pubky marketplace test fixture for Bitkit's marketplace wallet journey. +# Docs: docs/pubky-marketplace.md and the "Pubky marketplace fixture" section of README.md. + +set -euo pipefail + +CLI_NAME="$(basename "$0")" +cd "$(dirname "$0")" + +# Pinned upstream revisions (Paykit Server 867fc883 is the merge of pubky/paykit-server#2). +# Pubky Core is pinned in marketplace/pubky-testnet/Dockerfile. +PAYKIT_SERVER_REV=867fc883125c7b89a7b712c2551619cccdfdc0f7 +PAYKIT_RS_REV=6b241878a9bba5cecea919c0298c3f90624be6ff # paykit-server's paykit-lib and paykit-sdk pin +LOCKS_REV=df5ea1b6d8dcdec3a9b5a915c3f57bca69d75c8a # paykit-server's locks-core pin + +SOURCES=.marketplace/sources +PAYKIT_PORT="${MARKETPLACE_PAYKIT_PORT:-3001}" +COMPOSE=(docker compose --profile marketplace) +CHAIN_SERVICES=(bitcoind bitcoinsetup electrs) +FIXTURE_SERVICES=(marketplace-postgres pubky-testnet paykit-server) + +show_help() { + cat < [options] +Stack: + build Fetch the pinned sources and build the images + up Build if needed, start the chain and the fixture, wait until ready + ps Show the fixture containers and Paykit Server readiness + logs [service] Follow the fixture logs + down Remove the fixture containers and state (the chain stays) + reset down, then up: a fresh testnet, Paykit database and seller +Fixture: + seed [--buyer headless|none] Mine to maturity, create the seller wallet and identity, complete the + watch-only setup, optionally create a headless buyer + info Public fixture facts: homeserver key, seller, buyer, ports + setup-url New watch-only setup auth URL for a Bitkit wallet to approve + setup-wait Wait until an approved setup flow completes + fund
[sats] Send regtest coins (default 1000000 sats) and mine one block +Purchase: + purchase [--sats N] [--buyer headless|] + Create one purchase; print bundle id, address and delivery state + receive Headless buyer: receive the Payment Request and validate it + pay Headless buyer: pay the derived address from the regtest wallet + status Signed Paykit status and purchase state + mine [--bundle ] Mine exactly one block; print height, hash and confirmed purchases + wait [seconds] + Wait for a signed Paykit status + verify Whole headless journey with assertions; writes .marketplace/evidence/ +EOF +} + +compose() { "${COMPOSE[@]}" "$@"; } + +driver() { compose run --rm --no-deps --quiet-build -T marketplace-driver "$@"; } + +project_name() { docker compose config --format json | jq -r '.name'; } + +fetch_pinned() { + local url="$1" dir="$2" rev="$3" + if [ ! -d "$dir/.git" ]; then + mkdir -p "$dir" + git -C "$dir" init -q + git -C "$dir" remote add origin "$url" + fi + if [ "$(git -C "$dir" rev-parse -q --verify HEAD 2>/dev/null || true)" != "$rev" ]; then + git -C "$dir" fetch -q --depth 1 origin "$rev" + git -C "$dir" checkout -q --detach FETCH_HEAD + fi + if [ "$(git -C "$dir" rev-parse HEAD)" != "$rev" ]; then + echo "$dir is not at $rev" >&2 + exit 1 + fi + echo "pinned $dir @ ${rev:0:8}" +} + +fetch_sources() { + fetch_pinned https://github.com/pubky/paykit-server.git "$SOURCES/paykit-server" "$PAYKIT_SERVER_REV" + fetch_pinned https://github.com/pubky/paykit-rs.git "$SOURCES/paykit-rs" "$PAYKIT_RS_REV" + fetch_pinned https://github.com/pubky/locks.git "$SOURCES/locks" "$LOCKS_REV" +} + +build() { + fetch_sources + compose build pubky-testnet paykit-server + compose build marketplace-driver +} + +images_present() { + docker image inspect bitkit-docker/pubky-testnet:f68014c1 bitkit-docker/paykit-server:867fc883 \ + bitkit-docker/marketplace-driver:local >/dev/null 2>&1 +} + +wait_ready() { + local deadline=$(($(date +%s) + ${1:-300})) + until [ "$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1:${PAYKIT_PORT}/health/ready" || true)" = "200" ]; do + if [ "$(date +%s)" -gt "$deadline" ]; then + echo "Paykit Server did not become ready; see: ${CLI_NAME} logs paykit-server" >&2 + exit 1 + fi + sleep 2 + done + curl -fsS "http://127.0.0.1:${PAYKIT_PORT}/health/ready" + echo +} + +up() { + mkdir -p .marketplace/evidence + images_present || build + compose up -d "${CHAIN_SERVICES[@]}" marketplace-postgres pubky-testnet + driver init + compose up -d paykit-server + wait_ready + echo "Electrum: tcp://127.0.0.1:60001" + echo "Next: ${CLI_NAME} seed" +} + +if [ $# -eq 0 ] || [ "$1" = "-h" ] || [ "$1" = "--help" ]; then + show_help + exit 0 +fi + +command="$1" +shift + +case "$command" in + build) build ;; + up) up ;; + ps) + compose ps "${CHAIN_SERVICES[@]}" "${FIXTURE_SERVICES[@]}" || true + curl -s "http://127.0.0.1:${PAYKIT_PORT}/health/ready" || echo "Paykit Server: not reachable" + echo + ;; + logs) compose logs -f --tail 100 "${@:-paykit-server}" ;; + down) + # The Pubky testnet keeps its homeserver files and DHT in memory, so it cannot + # restart with its accounts intact. The fixture is disposable as a whole. + project="$(project_name)" + compose rm -sfv "${FIXTURE_SERVICES[@]}" marketplace-driver + docker volume rm -f "${project}_marketplace_postgres_data" "${project}_marketplace_state" >/dev/null + rm -rf .marketplace/evidence + ;; + reset) + "$0" down + up + ;; + seed | info | setup-url | setup-wait | fund | purchase | receive | pay | status | mine | wait | verify) + driver "$command" "$@" + ;; + *) + echo "Unknown command: $command" >&2 + show_help >&2 + exit 1 + ;; +esac From 9b039ef29bba656e2a3af9da0c5217ed3ec2099b Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 21:15:23 +0200 Subject: [PATCH 02/13] docs: use imperative mood in the marketplace changelog entry --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ca4baf6..f93382c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,7 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Preserve LNURL-pay invoice millisatoshi precision by creating invoices with LND `value_msat` instead of truncating callback amounts to sats ### Added -- Pubky marketplace test fixture behind the `marketplace` compose profile: Pubky testnet, Paykit Server `867fc883` built from pinned source, and the `pubky-marketplace` driver CLI (`up`, `seed`, `purchase`, `mine`, `verify`) for the Bitkit marketplace wallet journey; see `docs/pubky-marketplace.md` +- Add Pubky marketplace test fixture behind the `marketplace` compose profile: Pubky testnet, Paykit Server `867fc883` built from pinned source, and the `pubky-marketplace` driver CLI (`up`, `seed`, `purchase`, `mine`, `verify`) for the Bitkit marketplace wallet journey; see `docs/pubky-marketplace.md` - Homegate Docker Compose service with dedicated PostgreSQL storage, local homeserver admin mock, and README setup flow - Repo-managed Trezor User Env Docker service and `scripts/trezor-emulator` helper for quickly smoke-testing Bitkit app Trezor PRs - Support `amount_msat` query param in `/generate/bolt11` endpoint for sub-sat precision invoices From b048460afb1e920c43e9d959442dad430bbe377b Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 21:20:25 +0200 Subject: [PATCH 03/13] fix: pin paykit server to the revision that emits pubky grant auth urls --- CHANGELOG.md | 2 +- README.md | 4 ++-- docker-compose.yml | 12 +++++++----- docs/pubky-marketplace.md | 26 +++++++++++++++++++++----- marketplace/driver/driver.mjs | 20 +++++++++++++++++--- marketplace/pubky-testnet/Dockerfile | 4 ++-- pubky-marketplace | 27 +++++++++++++++++++++------ 7 files changed, 71 insertions(+), 24 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f93382c..d70dcac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,7 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Preserve LNURL-pay invoice millisatoshi precision by creating invoices with LND `value_msat` instead of truncating callback amounts to sats ### Added -- Add Pubky marketplace test fixture behind the `marketplace` compose profile: Pubky testnet, Paykit Server `867fc883` built from pinned source, and the `pubky-marketplace` driver CLI (`up`, `seed`, `purchase`, `mine`, `verify`) for the Bitkit marketplace wallet journey; see `docs/pubky-marketplace.md` +- Add Pubky marketplace test fixture behind the `marketplace` compose profile: Pubky testnet, Paykit Server `722ef268` built from pinned source, and the `pubky-marketplace` driver CLI (`up`, `seed`, `purchase`, `mine`, `verify`) for the Bitkit marketplace wallet journey; see `docs/pubky-marketplace.md` - Homegate Docker Compose service with dedicated PostgreSQL storage, local homeserver admin mock, and README setup flow - Repo-managed Trezor User Env Docker service and `scripts/trezor-emulator` helper for quickly smoke-testing Bitkit app Trezor PRs - Support `amount_msat` query param in `/generate/bolt11` endpoint for sub-sat precision invoices diff --git a/README.md b/README.md index f1ef4e2..797c992 100644 --- a/README.md +++ b/README.md @@ -268,7 +268,7 @@ See [docs/trezor-emulator.md](docs/trezor-emulator.md) for helper internals, env #### Pubky Marketplace Journey -Use this section for the Bitkit marketplace wallet journey (`journeys/pubky-marketplace` in [bitkit-ios](https://github.com/synonymdev/bitkit-ios/tree/master/journeys/pubky-marketplace) and [bitkit-android](https://github.com/synonymdev/bitkit-android/tree/master/journeys/pubky-marketplace)). The `marketplace` profile adds the integration fixture that the journey lists: a Pubky testnet, Paykit Server `867fc883` (the merge of [pubky/paykit-server#2](https://github.com/pubky/paykit-server/pull/2)) with `/setup` and `x-bitkit-claim=watch-only-account-v1`, and a purchase driver. It reuses this stack's regtest `bitcoind` and Electrum on `tcp://127.0.0.1:60001`. A plain `docker compose up -d` does not start it, and `./pubky-marketplace` starts only the chain and fixture services, so run it without the full stack (Homegate also wants port 6288). It needs Docker, `git`, `curl` and `jq` on the host, and outbound internet during the first build and during setup approval (see the setup relay note in [docs/pubky-marketplace.md](docs/pubky-marketplace.md)). +Use this section for the Bitkit marketplace wallet journey (`journeys/pubky-marketplace` in [bitkit-ios](https://github.com/synonymdev/bitkit-ios/tree/master/journeys/pubky-marketplace) and [bitkit-android](https://github.com/synonymdev/bitkit-android/tree/master/journeys/pubky-marketplace)). The `marketplace` profile adds the integration fixture that the journey lists: a Pubky testnet, Paykit Server `722ef268` (v0.1.0-rc4) with `/setup` and `x-bitkit-claim=watch-only-account-v1`, and a purchase driver. That revision emits the Pubky grant auth URL (`pubkyauth://signin_grant` with `cid` and `cpk`) that the apps' Paykit SDK (0.1.0-rc55) requires; the merge of [pubky/paykit-server#2](https://github.com/pubky/paykit-server/pull/2) (`867fc883`) emits the legacy URL, which the apps reject. It reuses this stack's regtest `bitcoind` and Electrum on `tcp://127.0.0.1:60001`. A plain `docker compose up -d` does not start it, and `./pubky-marketplace` starts only the chain and fixture services, so run it without the full stack (Homegate also wants port 6288). It needs Docker, `git`, `curl` and `jq` on the host, and outbound internet during the first build and during setup approval (see the setup relay note in [docs/pubky-marketplace.md](docs/pubky-marketplace.md)). ```bash ./pubky-marketplace up # fetch pinned sources, build, start, wait until Paykit Server is ready (first build takes a while) @@ -302,7 +302,7 @@ Then, with the buyer wallet: ./pubky-marketplace purchase --buyer # the Payment Request appears in the buyer wallet ``` -Pay the request in the app, then confirm with `./pubky-marketplace mine --bundle `, `./pubky-marketplace wait confirmed` and `./pubky-marketplace status `. The seller in a driver purchase is the fixture's headless seller, because publishing the payment lock needs the seller's identity secret. To check only the watch-only claim in a Bitkit seller wallet, run `./pubky-marketplace setup-url`, open the printed `auth_url` in that wallet (the URL holds a one-time secret, so keep it out of logs and evidence), approve it, then `./pubky-marketplace setup-wait `. +Pay the request in the app, then confirm with `./pubky-marketplace mine --bundle `, `./pubky-marketplace wait confirmed` and `./pubky-marketplace status `. The seller in a driver purchase is the fixture's headless seller, because publishing the payment lock needs the seller's identity secret. To check only the watch-only claim in a Bitkit seller wallet, run `./pubky-marketplace setup-url`, open the printed `auth_url` in that wallet (the URL holds a one-time secret, so keep it out of logs and evidence), approve it, then `./pubky-marketplace setup-wait `. The URL is a `pubkyauth://signin_grant?...` link. On Android open it with `adb shell am start -a android.intent.action.VIEW -d ''`; iOS has no `pubkyauth` handler, so open the printed `ios_url` (`bitkit://pubky-auth/setup?`) with `xcrun simctl openurl booted ''`. Remove the fixture with `./pubky-marketplace down`, or start over with `./pubky-marketplace reset`. The Pubky testnet keeps its accounts in memory, so the fixture cannot restart with its state and `down` deletes it (the regtest chain stays). `./pubky-marketplace --help` lists every command. See [docs/pubky-marketplace.md](docs/pubky-marketplace.md) for the pins, ports, roles and what the fixture does not cover. diff --git a/docker-compose.yml b/docker-compose.yml index 653abb9..06685c8 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -381,14 +381,16 @@ services: - "127.0.0.1:${MARKETPLACE_HOMESERVER_ADMIN_PORT:-16288}:6288" # homeserver admin (6288 is homegate's) - "127.0.0.1:${MARKETPLACE_PAYKIT_PORT:-3001}:3001" # paykit-server, shares this namespace - # Paykit Server 867fc883 (merge of pubky/paykit-server#2), built from source - # with the upstream Dockerfile.local. `./pubky-marketplace build` first checks out - # the pinned trees under .marketplace/sources; Dockerfile.local fails closed - # if a tree differs from the pins in Paykit Server's Cargo manifests. + # Paykit Server 722ef268 (v0.1.0-rc4), built from source with the upstream + # Dockerfile.local. Its setup flow emits the Pubky grant auth URL (cid and cpk) + # that the apps' Paykit SDK requires. `./pubky-marketplace build` first checks + # out the pinned trees under .marketplace/sources and confirms that the tree's + # Cargo.lock resolves paykit-rs and locks to them; Dockerfile.local then fails + # closed if a tree differs from the pins in Paykit Server's Cargo manifests. paykit-server: profiles: [marketplace] container_name: marketplace-paykit-server - image: bitkit-docker/paykit-server:867fc883 + image: bitkit-docker/paykit-server:722ef268 build: context: ./.marketplace/sources/paykit-server dockerfile: Dockerfile.local diff --git a/docs/pubky-marketplace.md b/docs/pubky-marketplace.md index 27442fb..452b6a7 100644 --- a/docs/pubky-marketplace.md +++ b/docs/pubky-marketplace.md @@ -12,12 +12,27 @@ and [bitkit-android#1338](https://github.com/synonymdev/bitkit-android/pull/1338 | Regtest bitcoind and Electrum on `tcp://127.0.0.1:60001` | the stack's `bitcoind` and `electrs` | as in `docker-compose.yml` | | Pubky Core static testnet: DHT, PKARR relay, HTTP relay, one homeserver with open signup | `pubky-testnet`, built from `marketplace/pubky-testnet/Dockerfile` | pubky-core `f68014c1` | | Homeserver and Paykit databases | `marketplace-postgres` | `postgres:16-alpine` | -| Paykit Server | `paykit-server`, built from source with the upstream `Dockerfile.local` | pubky/paykit-server `867fc883` (merge of #2), paykit-rs `6b241878`, locks-core `df5ea1b6` | +| Paykit Server | `paykit-server`, built from source with the upstream `Dockerfile.local` | pubky/paykit-server `722ef268` (v0.1.0-rc4), paykit-rs `9b56a0ea` (v0.1.0-rc48), locks-core `8502ef79` (v0.1.0-rc1) | | Purchase driver | `marketplace-driver`, run by `./pubky-marketplace` | `marketplace/driver/package-lock.json` | `./pubky-marketplace build` checks the pinned trees out under `.marketplace/sources` (git ignored) and -fails if a checkout is not at its pin. `Dockerfile.local` then fails closed if a tree differs from the -pins in Paykit Server's Cargo manifests. The pins are at the top of `pubky-marketplace`. +fails if a checkout is not at its pin or if the Paykit Server tree's `Cargo.lock` does not lock paykit-rs +and locks-core to those revisions. `Dockerfile.local` then fails closed if a tree differs from the pins in +Paykit Server's Cargo manifests. The pins are at the top of `pubky-marketplace`. + +### Why this Paykit Server revision + +The apps ship Paykit SDK `0.1.0-rc55` (bitkit-ios and bitkit-android at their 2026-09-29 heads). Its setup +approval accepts only the Pubky grant auth URL: `pubkyauth://signin_grant` with `cid` and `cpk`. Paykit Server +`867fc883` (the merge of pubky/paykit-server#2) is built on paykit-rs rc43 and emits the legacy +`pubkyauth://signin?caps&relay&secret&x-bitkit-claim` URL, which both apps reject ("Missing query parameter +cid"). Paykit Server adopted grant URLs with paykit-rs rc48, and `722ef268` (v0.1.0-rc4) is the newest +merged revision. It keeps `/setup` and `x-bitkit-claim=watch-only-account-v1`. Paykit Server pins paykit-rs +rc48, three releases before the apps' rc55, and no setup, auth or companion-claim code changed between them; +the J1 device run on 2026-09-29 already delivered requests from a paykit-rs rc43 server to rc55 apps. The +driver's `setup-url` refuses any auth URL that is not `signin_grant` with `cid` and `cpk`, so a wrong pin +fails before it reaches a wallet. Unmerged Paykit Server branches move to paykit-rs rc56; they are not +pinned here. ## Ports @@ -103,7 +118,8 @@ with the address in the delivered Payment Request. - **No Locks server, no guarded content.** The lock has no guarded resource, and the fixture does not cover marketplace browsing, content delivery, fiat payment or Hypercolor, which the journey also excludes. -- **Bitkit apps.** The commands for Bitkit wallets follow the journey's fixture contract. They have not - been run against the apps in this change. +- **Bitkit apps.** The commands for Bitkit wallets follow the journey's fixture contract. The buyer legs + ran on both apps in the J1 device run on 2026-09-29, against the previous Paykit Server pin. The seller + setup with the current pin has been run headlessly (`seed` and `verify`) and not yet against an app. - **Fixed container names.** The base services keep their fixed container names, so another checkout's stack with the same names must be removed first. diff --git a/marketplace/driver/driver.mjs b/marketplace/driver/driver.mjs index 8957514..d299381 100644 --- a/marketplace/driver/driver.mjs +++ b/marketplace/driver/driver.mjs @@ -33,6 +33,8 @@ const RPC_AUTH = `${process.env.BITCOIN_RPC_USER ?? 'polaruser'}:${process.env.B // The static testnet homeserver key is fixed by Pubky Core. const HOMESERVER = 'pubky8pinxxgqs41n4aididenw5apqp1urfmzdztr8jt4abrkdn435ewo'; const SETUP_ORIGIN = 'http://localhost:8080'; +// The grant client id Paykit Server requires in its config and puts in the setup auth URL as cid. +const PAYKIT_CLIENT_ID = 'app.paykit.server'; const SERVER_PATH = 'bitkit/server'; const BUYER_PATH = 'bitkit/wallet'; const ACCOUNT_INDEX = 0; @@ -290,6 +292,7 @@ trusted_public_key = "${issuer}" allowed_origins = ["${SETUP_ORIGIN}"] [paykit] +client_id = "${PAYKIT_CLIENT_ID}" receiver_path = "${SERVER_PATH}" receiver_path_priority = ["bitkit"] network = "testnet" @@ -324,6 +327,8 @@ async function completeSetup(flowId, seconds = 120) { } } +const authParams = (authUrl) => new URL(authUrl.replace('pubkyauth://', 'http://pubkyauth/')).searchParams; + async function beginSetup() { const state = `marketplace-${Date.now()}`; const response = await fetch( @@ -332,7 +337,7 @@ async function beginSetup() { if (response.status !== 200) fail(`GET /setup returned HTTP ${response.status}`); const html = await response.text(); const flow = html.match(/const flowId=("(?:[^"\\]|\\.)*");/); - const auth = html.match(/([^<]+)<\/code>/); + const auth = html.match(//); if (!flow || !auth) fail('setup page has no flow id or auth URL'); const authUrl = auth[1] .replaceAll('&', '&') @@ -340,17 +345,26 @@ async function beginSetup() { .replaceAll('>', '>') .replaceAll('"', '"') .replaceAll(''', "'"); + // The apps' Paykit SDK accepts only the Pubky grant protocol. A server that emits the legacy + // pubkyauth://signin URL (no cid or cpk) is the wrong pin, so stop here instead of at the wallet. + const params = authParams(authUrl); + if (!authUrl.startsWith('pubkyauth://signin_grant?') || !params.get('cid') || !params.get('cpk')) { + fail('setup auth URL is not a Pubky grant URL with cid and cpk; check the Paykit Server pin'); + } return { flowId: JSON.parse(flow[1]), authUrl }; } async function setupUrl() { await waitForPaykit(); const { flowId, authUrl } = await beginSetup(); - const claim = new URL(authUrl.replace('pubkyauth://', 'http://pubkyauth/')).searchParams.get('x-bitkit-claim'); + const params = authParams(authUrl); out({ flow_id: flowId, auth_url: authUrl, - claim, + // iOS has no pubkyauth handler: hand the same query to Bitkit as bitkit://pubky-auth/setup?. + ios_url: `bitkit://pubky-auth/setup?${authUrl.slice(authUrl.indexOf('?') + 1)}`, + client_id: params.get('cid'), + claim: params.get('x-bitkit-claim'), next: `./pubky-marketplace setup-wait ${flowId}`, }); } diff --git a/marketplace/pubky-testnet/Dockerfile b/marketplace/pubky-testnet/Dockerfile index 5a0cf1c..63a4ab7 100644 --- a/marketplace/pubky-testnet/Dockerfile +++ b/marketplace/pubky-testnet/Dockerfile @@ -1,7 +1,7 @@ # Pubky Core static testnet (DHT, PKARR relay, HTTP relay, homeserver) for the # marketplace fixture. Same build as pubky/locks docker/pubky-testnet.Dockerfile -# at ba49a777, pinned to the Pubky Core revision that Paykit Server 867fc883 and -# Bitkit's local testnet client are exercised against. +# at ba49a777, pinned to the Pubky Core revision that Bitkit's local testnet +# client and Paykit Server 722ef268 (Pubky 0.11 grant auth) are exercised against. FROM rust:1.89.0-bookworm AS builder ARG PUBKY_CORE_REV=f68014c111af0458e6a321e2d87a12479bfb3218 diff --git a/pubky-marketplace b/pubky-marketplace index 28b6510..0ec77fe 100755 --- a/pubky-marketplace +++ b/pubky-marketplace @@ -7,11 +7,14 @@ set -euo pipefail CLI_NAME="$(basename "$0")" cd "$(dirname "$0")" -# Pinned upstream revisions (Paykit Server 867fc883 is the merge of pubky/paykit-server#2). +# Pinned upstream revisions. Paykit Server 722ef268 is v0.1.0-rc4 (pubky/paykit-server master). Its setup +# flow emits the Pubky grant auth URL (pubkyauth://signin_grant with cid and cpk) that the apps' Paykit SDK +# (0.1.0-rc55) requires, and still carries x-bitkit-claim=watch-only-account-v1. Earlier revisions such as +# 867fc883 emit the legacy pubkyauth://signin URL, which the apps reject. # Pubky Core is pinned in marketplace/pubky-testnet/Dockerfile. -PAYKIT_SERVER_REV=867fc883125c7b89a7b712c2551619cccdfdc0f7 -PAYKIT_RS_REV=6b241878a9bba5cecea919c0298c3f90624be6ff # paykit-server's paykit-lib and paykit-sdk pin -LOCKS_REV=df5ea1b6d8dcdec3a9b5a915c3f57bca69d75c8a # paykit-server's locks-core pin +PAYKIT_SERVER_REV=722ef26834d8a4fc849de1d883eb296106dd2006 +PAYKIT_RS_REV=9b56a0eacd6874137370fa79ec0f40b809140809 # v0.1.0-rc48, paykit-server's paykit-lib and paykit-sdk pin +LOCKS_REV=8502ef79c443c640976a2a901b80c5e717319149 # v0.1.0-rc1, paykit-server's locks-core pin SOURCES=.marketplace/sources PAYKIT_PORT="${MARKETPLACE_PAYKIT_PORT:-3001}" @@ -21,7 +24,7 @@ FIXTURE_SERVICES=(marketplace-postgres pubky-testnet paykit-server) show_help() { cat < [options] Stack: build Fetch the pinned sources and build the images @@ -74,10 +77,22 @@ fetch_pinned() { echo "pinned $dir @ ${rev:0:8}" } +# The paykit-server tree resolves paykit-rs and locks by tag through its Cargo.lock, and its Dockerfile.local +# swaps those for the checkouts below. Fail here, before a long build, if a tag does not resolve to our pins. +check_lock_pins() { + local lock="$SOURCES/paykit-server/Cargo.lock" + if ! grep -Fq "source = \"git+https://github.com/pubky/paykit-rs.git?tag=v0.1.0-rc48#$PAYKIT_RS_REV\"" "$lock" || + ! grep -Fq "source = \"git+https://github.com/pubky/locks.git?tag=v0.1.0-rc1#$LOCKS_REV\"" "$lock"; then + echo "$lock does not lock paykit-rs $PAYKIT_RS_REV and locks $LOCKS_REV" >&2 + exit 1 + fi +} + fetch_sources() { fetch_pinned https://github.com/pubky/paykit-server.git "$SOURCES/paykit-server" "$PAYKIT_SERVER_REV" fetch_pinned https://github.com/pubky/paykit-rs.git "$SOURCES/paykit-rs" "$PAYKIT_RS_REV" fetch_pinned https://github.com/pubky/locks.git "$SOURCES/locks" "$LOCKS_REV" + check_lock_pins } build() { @@ -87,7 +102,7 @@ build() { } images_present() { - docker image inspect bitkit-docker/pubky-testnet:f68014c1 bitkit-docker/paykit-server:867fc883 \ + docker image inspect bitkit-docker/pubky-testnet:f68014c1 bitkit-docker/paykit-server:722ef268 \ bitkit-docker/marketplace-driver:local >/dev/null 2>&1 } From 53bbc68ffa88492f9e099655ccf5456b6de4e5b0 Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 21:25:28 +0200 Subject: [PATCH 04/13] fix: write driver evidence as the host user so down and reset succeed --- docs/pubky-marketplace.md | 5 +++-- marketplace/driver/driver.mjs | 17 ++++++++++++++++- pubky-marketplace | 21 +++++++++++++++++++-- 3 files changed, 38 insertions(+), 5 deletions(-) diff --git a/docs/pubky-marketplace.md b/docs/pubky-marketplace.md index 452b6a7..2989995 100644 --- a/docs/pubky-marketplace.md +++ b/docs/pubky-marketplace.md @@ -79,8 +79,9 @@ Everything lives in the `marketplace_state` volume, and `down` deletes it. - `/state/secrets` (root, mode 0700, unreadable by Paykit Server): issuer seed, seller identity seed, seller wallet seed, buyer identity seed. - `/state/fixture.json`, `/state/purchases.json`: public facts and the purchase ledger. -- `.marketplace/evidence//summary.json`: `verify` output. It holds public keys, bundle and request - ids, addresses, txids and statuses, and no seed or token. +- `.marketplace/evidence//summary.json`: `verify` output, owned by the user who ran the wrapper (the driver + hands it over from its root container). It holds public keys, bundle and request ids, addresses, txids and + statuses, and no seed or token. `down` removes it, through a container if an older run left root-owned files. The driver never prints a seed or key. `setup-url` prints a one-time auth URL that contains a session secret; it is meant to be pasted into a wallet, so keep it out of logs and evidence. diff --git a/marketplace/driver/driver.mjs b/marketplace/driver/driver.mjs index d299381..32d2ae4 100644 --- a/marketplace/driver/driver.mjs +++ b/marketplace/driver/driver.mjs @@ -13,7 +13,7 @@ import { spawn } from 'node:child_process'; import { createPrivateKey, randomBytes, sign } from 'node:crypto'; -import { chmod, mkdir, readFile, rm, writeFile } from 'node:fs/promises'; +import { chmod, chown, mkdir, readFile, rm, writeFile } from 'node:fs/promises'; import { existsSync } from 'node:fs'; import { blake3 } from '@noble/hashes/blake3'; @@ -67,6 +67,20 @@ async function writeJson(path, value) { await writeFile(path, `${JSON.stringify(value, null, 2)}\n`, { mode: 0o644 }); } +// Files on the host mounts belong to the user who ran the wrapper, not to this container's root. +async function handToHost(...paths) { + const uid = Number(process.env.MARKETPLACE_HOST_UID); + const gid = Number(process.env.MARKETPLACE_HOST_GID); + if (!Number.isInteger(uid) || !Number.isInteger(gid)) return; + for (const path of paths) { + try { + await chown(path, uid, gid); + } catch (error) { + log(`could not hand ${path} to the host user: ${error.code ?? error.message}`); + } + } +} + async function writeSecret(path, value) { await writeFile(path, value, { mode: 0o600 }); await chmod(path, 0o600); @@ -767,6 +781,7 @@ async function verify() { const dir = `${EVIDENCE_DIR}/${evidence.started_at.replace(/[:.]/g, '-')}`; await mkdir(dir, { recursive: true }); await writeJson(`${dir}/summary.json`, evidence); + await handToHost(dir, `${dir}/summary.json`); evidence.evidence_dir = `.marketplace/evidence/${dir.split('/').pop()}`; } out(evidence); diff --git a/pubky-marketplace b/pubky-marketplace index 0ec77fe..5c50872 100755 --- a/pubky-marketplace +++ b/pubky-marketplace @@ -55,7 +55,24 @@ EOF compose() { "${COMPOSE[@]}" "$@"; } -driver() { compose run --rm --no-deps --quiet-build -T marketplace-driver "$@"; } +# The driver runs as root inside its container (it owns the root-only state volume). It hands the files it +# writes to the host mounts (.marketplace/evidence) to the calling user, so the host can remove them later. +driver() { + mkdir -p .marketplace/evidence + compose run --rm --no-deps --quiet-build -T \ + -e MARKETPLACE_HOST_UID="$(id -u)" -e MARKETPLACE_HOST_GID="$(id -g)" marketplace-driver "$@" +} + +# Older runs left root-owned evidence that the host user cannot delete. Empty it through the driver image +# (root in the container), then remove the directory itself. +remove_evidence() { + [ -e .marketplace/evidence ] || return 0 + if ! rm -rf .marketplace/evidence 2>/dev/null; then + docker run --rm -v "$PWD/.marketplace:/host" --entrypoint find bitkit-docker/marketplace-driver:local \ + /host/evidence -mindepth 1 -delete + rm -rf .marketplace/evidence + fi +} project_name() { docker compose config --format json | jq -r '.name'; } @@ -153,7 +170,7 @@ case "$command" in project="$(project_name)" compose rm -sfv "${FIXTURE_SERVICES[@]}" marketplace-driver docker volume rm -f "${project}_marketplace_postgres_data" "${project}_marketplace_state" >/dev/null - rm -rf .marketplace/evidence + remove_evidence ;; reset) "$0" down From 6c2dd4e18f5dcac6a70e8f2371fb78d39d699bab Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 21:25:33 +0200 Subject: [PATCH 05/13] fix: find the purchase transaction for any buyer in mine --bundle --- README.md | 2 +- docs/pubky-marketplace.md | 6 ++++ marketplace/driver/driver.mjs | 53 +++++++++++++++++++++++++++++++++-- 3 files changed, 58 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 797c992..2e6497e 100644 --- a/README.md +++ b/README.md @@ -283,7 +283,7 @@ Use this section for the Bitkit marketplace wallet journey (`journeys/pubky-mark ./pubky-marketplace receive # headless buyer: Payment Request id, checks lowercase btc and the address ./pubky-marketplace pay # headless buyer pays from the regtest wallet ./pubky-marketplace wait detected # signed Paykit status: detected -./pubky-marketplace mine --bundle # exactly one block, refuses if the purchase is not in the mempool +./pubky-marketplace mine --bundle # exactly one block, refuses if the purchase is not in the mempool (finds an app buyer's payment by address and amount) ./pubky-marketplace wait confirmed ./pubky-marketplace status # signed Paykit status and purchase state (completed at 1 confirmation) ``` diff --git a/docs/pubky-marketplace.md b/docs/pubky-marketplace.md index 2989995..8434eca 100644 --- a/docs/pubky-marketplace.md +++ b/docs/pubky-marketplace.md @@ -107,6 +107,12 @@ The derived address is the seller xpub's external child `0/`, where `n` is th purchases for that seller, checked through `bitcoind deriveaddresses`. A headless `receive` compares it with the address in the delivered Payment Request. +`mine --bundle` mines one block only after it finds the purchase transaction in the mempool. The headless buyer's +`pay` records its txid; for a buyer that pays from an app, `mine --bundle` looks for the transaction that pays +the derived address the purchase's exact amount, records its txid in the ledger and prints it. `status` does the +same for a payment that is already confirmed (it searches the last 50 blocks). More than one matching +transaction is an error, and no match makes `mine --bundle` refuse. + ## Limits - **Setup relay.** The pinned Paykit Server starts the setup sign-in on `https://httprelay.pubky.app`, diff --git a/marketplace/driver/driver.mjs b/marketplace/driver/driver.mjs index 32d2ae4..65f2b2e 100644 --- a/marketplace/driver/driver.mjs +++ b/marketplace/driver/driver.mjs @@ -537,6 +537,47 @@ async function expectedAddress(fixture, child) { return addresses[0]; } +// The transaction that pays a purchase, whoever paid it. The headless buyer's `pay` records its txid; a +// Bitkit buyer pays from the app, so the ledger has none. Match on the purchase's derived address and amount +// instead: in the mempool first, then in the latest blocks (a confirmed payment is no longer in the mempool). +const RECENT_BLOCKS = 50; + +async function findPaymentTx(purchase, { mempoolOnly = false } = {}) { + const pays = (tx) => + tx.vout.some( + (output) => + output.scriptPubKey?.address === purchase.derived_address && + Math.round(output.value * 1e8) === purchase.amount_sats, + ); + const found = []; + for (const txid of await rpc('getrawmempool')) { + // A transaction can leave the mempool between the two calls; skip it then. + const tx = await rpc('getrawtransaction', [txid, true]).catch(() => null); + if (tx && pays(tx)) found.push({ txid, confirmed: false }); + } + if (!found.length && !mempoolOnly) { + const { height } = await chainInfo(); + for (let at = height; at > Math.max(0, height - RECENT_BLOCKS) && !found.length; at--) { + const block = await rpc('getblock', [await rpc('getblockhash', [at]), 2]); + for (const tx of block.tx) if (pays(tx)) found.push({ txid: tx.txid, confirmed: true, height: at }); + } + } + if (found.length > 1) fail(`more than one transaction pays ${purchase.derived_address}: ${found.map((tx) => tx.txid).join(', ')}`); + return found[0] ?? null; +} + +// Remember the payment transaction in the ledger. Returns it, or null when the payment is not on chain yet. +async function recordPaymentTx(purchases, purchase, options) { + const tx = purchase.txid && !options?.mempoolOnly ? { txid: purchase.txid } : await findPaymentTx(purchase, options); + if (!tx) return null; + if (purchase.txid !== tx.txid) { + purchase.txid = tx.txid; + if (['created', 'delivered'].includes(purchase.state)) purchase.state = 'paid'; + await writeJson(PURCHASES_FILE, purchases); + } + return tx; +} + async function purchase(args) { const sats = Number(flag(args, '--sats') ?? DEFAULT_SATS); if (!Number.isInteger(sats) || sats <= 0) fail('--sats must be a positive integer'); @@ -664,6 +705,8 @@ async function pay(args) { async function statusCommand(args) { const { purchases, purchase } = await findPurchase(args[0]); const paykit = await paykitStatus(purchase); + // A Bitkit buyer pays from the app, so learn the txid from the chain (non-fatal: nothing to find before payment). + if (!purchase.txid) await recordPaymentTx(purchases, purchase).catch((error) => log(`no payment txid yet: ${error.message}`)); const before = purchase.state; if (paykit.status === 'confirmed' && paykit.amount_matched && paykit.confirmations >= 1) purchase.state = 'completed'; else if (paykit.status === 'detected' && paykit.amount_matched) purchase.state = 'payment_detected'; @@ -687,9 +730,14 @@ async function mine(args) { const bundle = flag(args, '--bundle'); const before = await chainInfo(); const mempoolBefore = await rpc('getrawmempool'); + let payment = null; if (bundle) { - const { purchase } = await findPurchase(bundle); - if (!purchase.txid || !mempoolBefore.includes(purchase.txid)) fail('the purchase transaction is not in the mempool'); + const { purchases, purchase } = await findPurchase(bundle); + // The headless buyer's txid is in the ledger; for any other buyer find the payment by address and amount. + payment = purchase.txid && mempoolBefore.includes(purchase.txid) + ? { txid: purchase.txid } + : await recordPaymentTx(purchases, purchase, { mempoolOnly: true }); + if (!payment) fail('the purchase transaction is not in the mempool'); } const [block] = await mineBlocks(1); const after = await chainInfo(); @@ -698,6 +746,7 @@ async function mine(args) { mined: 1, height: after.height, block, + ...(payment ? { bundle_id: bundle, txid: payment.txid } : {}), mempool_before: mempoolBefore.length, mempool_after: (await rpc('getrawmempool')).length, }); From 80118c96c376b1fd1e22c468248eb917b4eb3306 Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 21:26:03 +0200 Subject: [PATCH 06/13] feat: add peers command reporting seller and buyer link state --- README.md | 2 + docs/pubky-marketplace.md | 16 +++++++ marketplace/driver/driver.mjs | 90 +++++++++++++++++++++++++++++++++++ pubky-marketplace | 5 +- 4 files changed, 112 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 2e6497e..01e4ac6 100644 --- a/README.md +++ b/README.md @@ -299,7 +299,9 @@ Then, with the buyer wallet: ```bash ./pubky-marketplace info | jq -r .seller.pubky # save this seller as a contact in the buyer wallet ./pubky-marketplace fund 1000000 # sends coins and mines one funding block +./pubky-marketplace peers --buyer # journey step 14: seller setup ready, both receiver markers published ./pubky-marketplace purchase --buyer # the Payment Request appears in the buyer wallet +./pubky-marketplace peers --buyer --wait 60 # after the purchase: Paykit Server's link to the buyer is connected ``` Pay the request in the app, then confirm with `./pubky-marketplace mine --bundle `, `./pubky-marketplace wait confirmed` and `./pubky-marketplace status `. The seller in a driver purchase is the fixture's headless seller, because publishing the payment lock needs the seller's identity secret. To check only the watch-only claim in a Bitkit seller wallet, run `./pubky-marketplace setup-url`, open the printed `auth_url` in that wallet (the URL holds a one-time secret, so keep it out of logs and evidence), approve it, then `./pubky-marketplace setup-wait `. The URL is a `pubkyauth://signin_grant?...` link. On Android open it with `adb shell am start -a android.intent.action.VIEW -d ''`; iOS has no `pubkyauth` handler, so open the printed `ios_url` (`bitkit://pubky-auth/setup?`) with `xcrun simctl openurl booted ''`. diff --git a/docs/pubky-marketplace.md b/docs/pubky-marketplace.md index 8434eca..d28e9a1 100644 --- a/docs/pubky-marketplace.md +++ b/docs/pubky-marketplace.md @@ -113,6 +113,22 @@ the derived address the purchase's exact amount, records its txid in the ledger same for a payment that is already confirmed (it searches the last 50 blocks). More than one matching transaction is an error, and no match makes `mine --bundle` refuse. +## Linked peers + +`peers [--buyer ] [--bundle ] [--wait ]` is the fixture's answer to journey step 14. It +prints, for the seller and for the buyer (default: the latest purchase's buyer, else the headless buyer): + +- the public Paykit receiver marker each identity publishes (`bitkit/server` for the seller, `bitkit/wallet` for + the buyer), which shows contact payments are on; +- the seller's Paykit Server setup authority (`ready`, `setup_required` or `unavailable`); +- Paykit Server's persisted link state to the buyer for the purchase's reader binding (`none`, `handshake`, + `connected`, `recovery_required` or `blocked`), and for the headless buyer its own view of the link. + +`ready_for_purchase` is true when both markers are published and the seller's setup is ready. `linked` is true +only when Paykit Server reports the link `connected`. Paykit Server keeps that state per purchase and exposes no +per-peer query, so before a buyer's first purchase `server_side` reads `no_purchase_yet` and `linked` is false; +use `ready_for_purchase` there and `peers --wait 60` after `purchase`, which exits 1 if the link never connects. + ## Limits - **Setup relay.** The pinned Paykit Server starts the setup sign-in on `https://httprelay.pubky.app`, diff --git a/marketplace/driver/driver.mjs b/marketplace/driver/driver.mjs index 65f2b2e..e14e1cc 100644 --- a/marketplace/driver/driver.mjs +++ b/marketplace/driver/driver.mjs @@ -769,6 +769,95 @@ async function waitFor(args) { } } +// Linked-peer report for journey step 14: "the fixture reports the seller and buyer as linked peers". +// It reads what the fixture can see: each side's public Paykit receiver marker, the seller's setup +// authority, and Paykit Server's persisted peer state for a purchase's reader binding. +async function receiverMarker(pubky, receiverPath) { + const path = `/pub/paykit/v0/${receiverPath}/receiver.json`; + const storage = pubkyClient().publicStorage; + if (!(await storage.exists(`${pubky}${path}`))) return { path, present: false }; + return { path, present: true, marker: await storage.getJson(`${pubky}${path}`) }; +} + +async function peerReport({ fixture, purchases, buyerArg, bundleArg }) { + const reader = buyerArg ?? (bundleArg ? undefined : purchases.at(-1)?.reader) ?? fixture.buyer?.pubky; + if (!reader) fail('no buyer; pass --buyer or run: ./pubky-marketplace seed --buyer headless'); + const purchase = bundleArg + ? purchases.find((entry) => entry.bundle_id === bundleArg) + : purchases.filter((entry) => entry.reader === reader).at(-1); + if (bundleArg && !purchase) fail(`unknown bundle ${bundleArg}`); + const headless = fixture.buyer?.pubky === reader; + + const setup = await signedPost('/setup/status', { creator: fixture.seller.pubky }); + let serverSide = 'no_purchase_yet'; + if (purchase) { + const state = await signedPost('/connections/status', { creator: fixture.seller.pubky, bundle_id: purchase.bundle_id }); + serverSide = state.status === 200 ? state.json.state : `unavailable_http_${state.status}`; + } + let buyerSide = null; + if (headless) { + const inspected = await runHelper( + 'paykit-reader-demo', + { version: 1, operation: 'inspect', reader_secret: await readSecret('buyer-identity.seed') }, + readerEnv(fixture.seller.pubky), + ); + buyerSide = inspected.code === 0 ? JSON.parse(inspected.stdout).connection_state : 'unavailable'; + } + const sellerMarker = await receiverMarker(fixture.seller.pubky, SERVER_PATH); + const buyerMarker = await receiverMarker(reader, BUYER_PATH); + const sellerReady = setup.status === 200 && setup.json?.status === 'ready' && sellerMarker.present; + return { + seller: { + pubky: fixture.seller.pubky, + kind: fixture.seller.kind, + receiver_path: SERVER_PATH, + setup: setup.status === 200 ? setup.json.status : `unavailable_http_${setup.status}`, + receiver_marker: sellerMarker, + }, + buyer: { + pubky: reader, + kind: headless ? 'headless' : 'external', + receiver_path: BUYER_PATH, + receiver_marker: buyerMarker, + }, + link: { + bundle_id: purchase?.bundle_id ?? null, + // Paykit Server's view of its link to the buyer: none, handshake, connected, recovery_required or blocked. + // The server keeps it per purchase, so before the first purchase for this buyer it reads no_purchase_yet. + server_side: serverSide, + // The headless buyer's own view of its link to the server; a Bitkit buyer shows this in the app. + buyer_side: buyerSide, + }, + // Both identities publish their receiver markers and the seller's setup authority is usable: a purchase can be delivered. + ready_for_purchase: sellerReady && buyerMarker.present, + // Paykit Server holds a live link to the buyer. + linked: sellerReady && buyerMarker.present && serverSide === 'connected', + }; +} + +async function peers(args) { + const fixture = await readFixture(); + const purchases = await readJson(PURCHASES_FILE, []); + const waitSeconds = Number(flag(args, '--wait') ?? 0); + if (!Number.isFinite(waitSeconds) || waitSeconds < 0) fail('usage: peers [--buyer ] [--bundle ] [--wait ]'); + await waitForPaykit(); + const deadline = Date.now() + waitSeconds * 1000; + for (;;) { + const report = await peerReport({ + fixture, + purchases: await readJson(PURCHASES_FILE, purchases), + buyerArg: flag(args, '--buyer'), + bundleArg: flag(args, '--bundle'), + }); + if (report.linked || !waitSeconds || Date.now() > deadline) { + out(report); + if (waitSeconds && !report.linked) fail(`seller and buyer are not linked after ${waitSeconds}s`); + return; + } + await sleep(2000); + } +} + // The whole journey with the driver in every wallet role. Each step asserts. async function verify() { const evidence = { started_at: new Date().toISOString(), steps: {} }; @@ -848,6 +937,7 @@ const commands = { pay, status: statusCommand, mine, + peers, wait: waitFor, verify: () => verify(), }; diff --git a/pubky-marketplace b/pubky-marketplace index 5c50872..42211f2 100755 --- a/pubky-marketplace +++ b/pubky-marketplace @@ -40,6 +40,9 @@ Fixture: setup-url New watch-only setup auth URL for a Bitkit wallet to approve setup-wait Wait until an approved setup flow completes fund
[sats] Send regtest coins (default 1000000 sats) and mine one block + peers [--buyer ] [--bundle ] [--wait ] + Report the seller and buyer as linked peers: receiver markers, setup + authority and Paykit Server's link state (linked is true once connected) Purchase: purchase [--sats N] [--buyer headless|] Create one purchase; print bundle id, address and delivery state @@ -176,7 +179,7 @@ case "$command" in "$0" down up ;; - seed | info | setup-url | setup-wait | fund | purchase | receive | pay | status | mine | wait | verify) + seed | info | setup-url | setup-wait | fund | peers | purchase | receive | pay | status | mine | wait | verify) driver "$command" "$@" ;; *) From 890722e2835de6dfd19616675d664e28194396e2 Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 21:26:10 +0200 Subject: [PATCH 07/13] docs: fix the marketplace readme ios extra-args and android reverse ports --- README.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 01e4ac6..cdd5d07 100644 --- a/README.md +++ b/README.md @@ -268,7 +268,7 @@ See [docs/trezor-emulator.md](docs/trezor-emulator.md) for helper internals, env #### Pubky Marketplace Journey -Use this section for the Bitkit marketplace wallet journey (`journeys/pubky-marketplace` in [bitkit-ios](https://github.com/synonymdev/bitkit-ios/tree/master/journeys/pubky-marketplace) and [bitkit-android](https://github.com/synonymdev/bitkit-android/tree/master/journeys/pubky-marketplace)). The `marketplace` profile adds the integration fixture that the journey lists: a Pubky testnet, Paykit Server `722ef268` (v0.1.0-rc4) with `/setup` and `x-bitkit-claim=watch-only-account-v1`, and a purchase driver. That revision emits the Pubky grant auth URL (`pubkyauth://signin_grant` with `cid` and `cpk`) that the apps' Paykit SDK (0.1.0-rc55) requires; the merge of [pubky/paykit-server#2](https://github.com/pubky/paykit-server/pull/2) (`867fc883`) emits the legacy URL, which the apps reject. It reuses this stack's regtest `bitcoind` and Electrum on `tcp://127.0.0.1:60001`. A plain `docker compose up -d` does not start it, and `./pubky-marketplace` starts only the chain and fixture services, so run it without the full stack (Homegate also wants port 6288). It needs Docker, `git`, `curl` and `jq` on the host, and outbound internet during the first build and during setup approval (see the setup relay note in [docs/pubky-marketplace.md](docs/pubky-marketplace.md)). +Use this section for the Bitkit marketplace wallet journey (`journeys/pubky-marketplace` in [bitkit-ios](https://github.com/synonymdev/bitkit-ios/tree/master/journeys/pubky-marketplace) and [bitkit-android](https://github.com/synonymdev/bitkit-android/tree/master/journeys/pubky-marketplace)). The `marketplace` profile adds the integration fixture that the journey lists: a Pubky testnet, Paykit Server `722ef268` (v0.1.0-rc4) with `/setup` and `x-bitkit-claim=watch-only-account-v1`, and a purchase driver. That revision emits the Pubky grant auth URL (`pubkyauth://signin_grant` with `cid` and `cpk`) that the apps' Paykit SDK (0.1.0-rc55) requires; the merge of [pubky/paykit-server#2](https://github.com/pubky/paykit-server/pull/2) (`867fc883`) emits the legacy URL, which the apps reject. It reuses this stack's regtest `bitcoind` and Electrum on `tcp://127.0.0.1:60001`. A plain `docker compose up -d` does not start it, and `./pubky-marketplace` starts only the chain and fixture services, so run it without the full stack. It needs Docker, `git`, `curl` and `jq` on the host, and outbound internet during the first build and during setup approval (see the setup relay note in [docs/pubky-marketplace.md](docs/pubky-marketplace.md)). ```bash ./pubky-marketplace up # fetch pinned sources, build, start, wait until Paykit Server is ready (first build takes a while) @@ -290,8 +290,17 @@ Use this section for the Bitkit marketplace wallet journey (`journeys/pubky-mark For Bitkit wallets, build each simulator or emulator against this fixture before its first launch: -- iOS: build with `E2E_BUILD E2E_BACKEND=local E2E_NETWORK=regtest E2E_HOMESERVER_PUBKY=`, where `` is `./pubky-marketplace info | jq -r .homeserver_z32`. Electrum resolves to `tcp://127.0.0.1:60001` with no override. -- Android: build the local E2E backend with the same `E2E_HOMESERVER_PUBKY` in the environment. On each emulator run `adb reverse tcp: tcp:` for 6286, 6287, 6288, 15411 and 15412; the Android journey README covers the emulator's `10.0.2.2` host address. +- iOS: build with the local E2E backend and the fixture homeserver key, passing each build setting as its own `--extra-args` element. Putting them all in one quoted string makes xcodebuild read the whole string as the value of the first setting, so `E2E_HOMESERVER_PUBKY` never reaches `Info.plist`. Electrum resolves to `tcp://127.0.0.1:60001` with no override. + + ```bash + xcodebuildmcp simulator build-and-run --simulator-id \ + --extra-args 'SWIFT_ACTIVE_COMPILATION_CONDITIONS=$(inherited) E2E_BUILD' \ + --extra-args 'E2E_BACKEND=local' \ + --extra-args 'E2E_NETWORK=regtest' \ + --extra-args "E2E_HOMESERVER_PUBKY=$(./pubky-marketplace info | jq -r .homeserver_z32)" + ``` + +- Android: build the local E2E backend with the same `E2E_HOMESERVER_PUBKY` in the environment. On each emulator run `adb reverse tcp: tcp:` for 6286, 6287, 15411 and 15412 (the homeserver admin port is published on 16288 and no app uses it); the Android journey README covers the emulator's `10.0.2.2` host address. - In each wallet create a Bitkit-generated Pubky identity (not a Pubky Ring import) and enable contact payments. Then, with the buyer wallet: @@ -304,7 +313,7 @@ Then, with the buyer wallet: ./pubky-marketplace peers --buyer --wait 60 # after the purchase: Paykit Server's link to the buyer is connected ``` -Pay the request in the app, then confirm with `./pubky-marketplace mine --bundle `, `./pubky-marketplace wait confirmed` and `./pubky-marketplace status `. The seller in a driver purchase is the fixture's headless seller, because publishing the payment lock needs the seller's identity secret. To check only the watch-only claim in a Bitkit seller wallet, run `./pubky-marketplace setup-url`, open the printed `auth_url` in that wallet (the URL holds a one-time secret, so keep it out of logs and evidence), approve it, then `./pubky-marketplace setup-wait `. The URL is a `pubkyauth://signin_grant?...` link. On Android open it with `adb shell am start -a android.intent.action.VIEW -d ''`; iOS has no `pubkyauth` handler, so open the printed `ios_url` (`bitkit://pubky-auth/setup?`) with `xcrun simctl openurl booted ''`. +Pay the request in the app, then confirm with `./pubky-marketplace mine --bundle `, `./pubky-marketplace wait confirmed` and `./pubky-marketplace status `. The seller in a driver purchase is the fixture's headless seller, because publishing the payment lock needs the seller's identity secret. To check only the watch-only claim in a Bitkit seller wallet, run `./pubky-marketplace setup-url`, open the printed `auth_url` in that wallet (the URL holds a one-time secret, so keep it out of logs and evidence), approve it, then `./pubky-marketplace setup-wait `. The URL is a `pubkyauth://signin_grant?...` link. On Android open it with `adb shell am start -a android.intent.action.VIEW -d ''`; iOS has no `pubkyauth` handler, so open the printed `ios_url` (`bitkit://pubky-auth/setup?`) with `xcrun simctl openurl ''`. Remove the fixture with `./pubky-marketplace down`, or start over with `./pubky-marketplace reset`. The Pubky testnet keeps its accounts in memory, so the fixture cannot restart with its state and `down` deletes it (the regtest chain stays). `./pubky-marketplace --help` lists every command. See [docs/pubky-marketplace.md](docs/pubky-marketplace.md) for the pins, ports, roles and what the fixture does not cover. From 7ec467313c6c6d4139d25d1cb2e1f7fe61051e7d Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 21:51:08 +0200 Subject: [PATCH 08/13] build: move the marketplace driver to @synonymdev/pubky 0.10.0 0.10.0 is the release cut five days after the pinned Pubky Core homeserver (f68014c1) and adds the grant auth flow that a Bitkit seller approval needs. 0.11.0 upgrades pkarr to v8 and the relay to v2 past that homeserver, so it is not used. Sessions are grants there, so signin names its client. --- marketplace/driver/driver.mjs | 6 ++++-- marketplace/driver/package-lock.json | 8 ++++---- marketplace/driver/package.json | 2 +- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/marketplace/driver/driver.mjs b/marketplace/driver/driver.mjs index e14e1cc..be1ccaa 100644 --- a/marketplace/driver/driver.mjs +++ b/marketplace/driver/driver.mjs @@ -35,6 +35,8 @@ const HOMESERVER = 'pubky8pinxxgqs41n4aididenw5apqp1urfmzdztr8jt4abrkdn435ewo'; const SETUP_ORIGIN = 'http://localhost:8080'; // The grant client id Paykit Server requires in its config and puts in the setup auth URL as cid. const PAYKIT_CLIENT_ID = 'app.paykit.server'; +// Pubky 0.10 sessions are grants, so a signin names its client. +const HEADLESS_CLIENT_ID = 'marketplace.fixture'; const SERVER_PATH = 'bitkit/server'; const BUYER_PATH = 'bitkit/wallet'; const ACCOUNT_INDEX = 0; @@ -223,7 +225,7 @@ async function signUpIdentity(seed) { } catch (error) { const message = String(error?.message ?? error).toLowerCase(); if (!/already|409|conflict/.test(message)) throw error; - await signer.signin(); + await signer.signin(HEADLESS_CLIENT_ID); } return keypair.publicKey.toString(); } @@ -231,7 +233,7 @@ async function signUpIdentity(seed) { async function sellerSession() { const seed = Buffer.from(await readSecret('seller-identity.seed'), 'base64url'); const signer = pubkyClient().signer(Keypair.fromSecret(seed)); - return signer.signin(); + return signer.signin(HEADLESS_CLIENT_ID); } // ----------------------------------------------------------- helper binaries diff --git a/marketplace/driver/package-lock.json b/marketplace/driver/package-lock.json index 853b63b..f91044f 100644 --- a/marketplace/driver/package-lock.json +++ b/marketplace/driver/package-lock.json @@ -10,7 +10,7 @@ "dependencies": { "@noble/hashes": "1.8.0", "@scure/bip32": "1.7.0", - "@synonymdev/pubky": "0.9.3" + "@synonymdev/pubky": "0.10.0" } }, "node_modules/@noble/curves": { @@ -64,9 +64,9 @@ } }, "node_modules/@synonymdev/pubky": { - "version": "0.9.3", - "resolved": "https://registry.npmjs.org/@synonymdev/pubky/-/pubky-0.9.3.tgz", - "integrity": "sha512-3Kl8Ad+yy5moAUc9PrcUypaMhzk5VW7f/bckUMi8mNQU1heEX8sT2VqMM5+lkQN13R9ZWQRV4j4osYGXR3zT6A==", + "version": "0.10.0", + "resolved": "https://registry.npmjs.org/@synonymdev/pubky/-/pubky-0.10.0.tgz", + "integrity": "sha512-XlyTQ0yYo/3Jk/M3Xw1x4gMskK2SeAG0n0aO54BZyNQq774bwMBX8W/vYzARKVs1WlNY/RdJbANlZeT2YQePgg==", "license": "MIT", "dependencies": { "fetch-cookie": "^3.0.1" diff --git a/marketplace/driver/package.json b/marketplace/driver/package.json index 2b6f5c0..587c8ba 100644 --- a/marketplace/driver/package.json +++ b/marketplace/driver/package.json @@ -7,6 +7,6 @@ "dependencies": { "@noble/hashes": "1.8.0", "@scure/bip32": "1.7.0", - "@synonymdev/pubky": "0.9.3" + "@synonymdev/pubky": "0.10.0" } } From 3d9b13f91a7a8466581e35ed7d2ebcc4f61d0241 Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 21:55:23 +0200 Subject: [PATCH 09/13] feat: sell as a bitkit-approved seller in the marketplace driver seller-auth prints a pubkyauth signin_grant request for /pub/locks.app/ with the Android deep link and the iOS paste route, waits for the seller wallet to approve it and keeps the grant session. purchase --seller bitkit publishes the payment lock with that session, so Paykit Server pays the wallet's own xpub child. A Bitkit seller's xpub never reaches the fixture, so the payout address comes from the Payment Request, mine --address or the exact-amount output. setup-wait, peers, status and mine work for it. --- marketplace/driver/driver.mjs | 345 ++++++++++++++++++++++++++++------ pubky-marketplace | 20 +- 2 files changed, 303 insertions(+), 62 deletions(-) diff --git a/marketplace/driver/driver.mjs b/marketplace/driver/driver.mjs index be1ccaa..97f7ed0 100644 --- a/marketplace/driver/driver.mjs +++ b/marketplace/driver/driver.mjs @@ -7,6 +7,9 @@ // full marketplace). Roles it can also stand in for, so the whole journey runs // without a wallet app: the seller (watch-only setup) and the buyer (receive // and pay). Bitkit wallets take the buyer and seller roles in the app journey. +// A Bitkit seller approves two Pubky grants: the Paykit setup (`setup-url`) and +// a `/pub/locks.app/` write grant for the marketplace (`seller-auth`), and the +// driver publishes the payment lock with that grant session. // // Secrets stay in /state/secrets (root, 0700). Paykit Server only ever sees the // seller's account xpub. Nothing here prints a seed, a key or a token. @@ -18,7 +21,7 @@ import { existsSync } from 'node:fs'; import { blake3 } from '@noble/hashes/blake3'; import { HDKey } from '@scure/bip32'; -import { Keypair, Pubky, PublicKey } from '@synonymdev/pubky'; +import { AuthFlowKind, Keypair, Pubky, PublicKey } from '@synonymdev/pubky'; const STATE = '/state'; const SECRETS = `${STATE}/secrets`; @@ -37,6 +40,13 @@ const SETUP_ORIGIN = 'http://localhost:8080'; const PAYKIT_CLIENT_ID = 'app.paykit.server'; // Pubky 0.10 sessions are grants, so a signin names its client. const HEADLESS_CLIENT_ID = 'marketplace.fixture'; +// The write grant a Bitkit seller approves for the marketplace (the role Locks plays). The apps show the +// client id as "Requester ID" and the path as the requested permission. +const LOCKS_CLIENT_ID = 'locks.app'; +const LOCKS_CAPS = '/pub/locks.app/:rw'; +// The testnet's own HTTP relay; wallets reach it on localhost like the homeserver (Android: adb reverse 15412). +const LOCKS_RELAY = 'http://localhost:15412/inbox/'; +const BITKIT_SESSION_SECRET = 'bitkit-seller.session'; const SERVER_PATH = 'bitkit/server'; const BUYER_PATH = 'bitkit/wallet'; const ACCOUNT_INDEX = 0; @@ -230,10 +240,33 @@ async function signUpIdentity(seed) { return keypair.publicKey.toString(); } -async function sellerSession() { - const seed = Buffer.from(await readSecret('seller-identity.seed'), 'base64url'); - const signer = pubkyClient().signer(Keypair.fromSecret(seed)); - return signer.signin(HEADLESS_CLIENT_ID); +// A write session on the seller's /pub/locks.app/, for publishing the payment lock. The headless seller signs +// in with its own key. A Bitkit seller has no key here: the session is the grant its wallet approved in +// `seller-auth`, restored from the state volume (each restore mints a fresh short-lived bearer). +async function sellerSession(seller) { + if (seller.kind === 'headless') { + const seed = Buffer.from(await readSecret('seller-identity.seed'), 'base64url'); + return pubkyClient().signer(Keypair.fromSecret(seed)).signin(HEADLESS_CLIENT_ID); + } + const session = await pubkyClient().restoreSession(await readSecret(BITKIT_SESSION_SECRET)); + if (session.info.publicKey.toString() !== seller.pubky) fail('the stored Bitkit seller session belongs to another identity; run: ./pubky-marketplace seller-auth'); + return session; +} + +// The seller record for `--seller`: the headless seller (default), the Bitkit seller, or its approved pubky. +function pickSeller(fixture, which = 'headless') { + if (which === 'headless') return fixture.seller; + const bitkit = fixture.bitkit_seller; + if (!bitkit) fail('no Bitkit seller; run: ./pubky-marketplace seller-auth'); + if (which === 'bitkit' || which === bitkit.pubky) return bitkit; + fail(`--seller must be headless, bitkit or ${bitkit.pubky}`); +} + +const sellerOf = (fixture, pubky) => [fixture.seller, fixture.bitkit_seller].find((entry) => entry?.pubky === pubky) ?? fixture.seller; + +async function setupStatus(pubky) { + const response = await signedPost('/setup/status', { creator: pubky }); + return response.status === 200 ? response.json.status : `unavailable_http_${response.status}`; } // ----------------------------------------------------------- helper binaries @@ -370,6 +403,21 @@ async function beginSetup() { return { flowId: JSON.parse(flow[1]), authUrl }; } +// The watch-only setup approval: the wallet's role in Paykit Server's /setup flow. The wallet gives the server +// its account xpub with the companion claim, then approves the setup grant with its Pubky identity. +async function approveSetupAs(authUrl, identitySeed, xpub, accountIndex) { + const approval = await runHelper('paykit-companion-auth', { + version: 1, + auth_url: authUrl, + creator_secret: b64url(identitySeed), + account_xpub: xpub, + account_index: accountIndex, + }); + if (approval.code !== 0 || !approval.stdout.includes('"approved"')) { + fail(`companion approval failed: ${approval.stderr || approval.stdout}`); + } +} + async function setupUrl() { await waitForPaykit(); const { flowId, authUrl } = await beginSetup(); @@ -377,7 +425,9 @@ async function setupUrl() { out({ flow_id: flowId, auth_url: authUrl, - // iOS has no pubkyauth handler: hand the same query to Bitkit as bitkit://pubky-auth/setup?. + // Android opens the grant URL directly. iOS has no pubkyauth handler: hand the same query to Bitkit as + // bitkit://pubky-auth/setup?. + android: `adb shell am start -a android.intent.action.VIEW -d '${authUrl}'`, ios_url: `bitkit://pubky-auth/setup?${authUrl.slice(authUrl.indexOf('?') + 1)}`, client_id: params.get('cid'), claim: params.get('x-bitkit-claim'), @@ -385,13 +435,34 @@ async function setupUrl() { }); } +// When a Bitkit seller has approved the marketplace grant, the setup must have been approved by the same +// identity: Paykit Server reports the setup per creator. async function setupWait(flowId) { if (!flowId) fail('usage: setup-wait '); await completeSetup(flowId, 300); - out({ flow_id: flowId, status: 'complete' }); + const fixture = await readJson(FIXTURE_FILE, {}); + const bitkit = fixture.bitkit_seller; + if (!bitkit) { + return out({ + flow_id: flowId, + status: 'complete', + next: './pubky-marketplace seller-auth (the marketplace grant, approved by the same wallet)', + }); + } + let status = await setupStatus(bitkit.pubky); + for (let attempt = 0; status !== 'ready' && attempt < 10; attempt++) { + await sleep(1500); + status = await setupStatus(bitkit.pubky); + } + if (status !== 'ready') { + fail(`setup completed but the Bitkit seller ${bitkit.pubky} is ${status}: the wallet that approved the setup is not the one that approved seller-auth`); + } + bitkit.setup_completed_at ??= new Date().toISOString(); + await writeJson(FIXTURE_FILE, fixture); + out({ flow_id: flowId, status: 'complete', seller: bitkit.pubky, paykit_setup: 'ready' }); } -async function createBuyer() { +async function createBuyer(sellerPubky) { const seed = randomBytes(32); await writeSecret(`${SECRETS}/buyer-identity.seed`, b64url(seed)); const buyer = await signUpIdentity(seed); @@ -402,7 +473,7 @@ async function createBuyer() { const prepared = await runHelper( 'paykit-reader-demo', { version: 1, operation: 'prepare', reader_secret: b64url(seed) }, - readerEnv(fixture.seller.pubky), + readerEnv(sellerPubky ?? fixture.seller.pubky), ); if (prepared.code !== 0) fail(`buyer receiver marker failed: ${prepared.stdout || prepared.stderr}`); fixture.buyer = { pubky: buyer, receiver_path: BUYER_PATH, kind: 'headless' }; @@ -432,16 +503,7 @@ async function seed(args) { const sellerPubky = await signUpIdentity(identitySeed); log(`seller identity ${sellerPubky}; completing the watch-only setup`); const { flowId, authUrl } = await beginSetup(); - const approval = await runHelper('paykit-companion-auth', { - version: 1, - auth_url: authUrl, - creator_secret: b64url(identitySeed), - account_xpub: xpub, - account_index: ACCOUNT_INDEX, - }); - if (approval.code !== 0 || !approval.stdout.includes('"approved"')) { - fail(`companion approval failed: ${approval.stderr || approval.stdout}`); - } + await approveSetupAs(authUrl, identitySeed, xpub, ACCOUNT_INDEX); await completeSetup(flowId); fixture = { homeserver: HOMESERVER, @@ -493,6 +555,15 @@ async function publicInfo() { spending_authority: 'held only in the fixture state volume, never given to Paykit Server', } : null, + bitkit_seller: fixture.bitkit_seller + ? { + pubky: fixture.bitkit_seller.pubky, + kind: fixture.bitkit_seller.kind, + marketplace_grant: `${fixture.bitkit_seller.client_id} ${fixture.bitkit_seller.capabilities}`, + setup_completed_at: fixture.bitkit_seller.setup_completed_at ?? null, + spending_authority: 'held only in the seller wallet; the fixture holds a marketplace grant session and no xpub', + } + : null, buyer: fixture.buyer ?? null, purchases: purchases.length, latest_purchase: purchases.at(-1) ?? null, @@ -533,38 +604,72 @@ async function issuerPubky() { return Keypair.fromSecret(seed).publicKey.toString(); } -async function expectedAddress(fixture, child) { - const info = await rpc('getdescriptorinfo', [`wpkh(${fixture.seller.account_xpub}/0/*)`]); +async function expectedAddress(xpub, child) { + const info = await rpc('getdescriptorinfo', [`wpkh(${xpub}/0/*)`]); const addresses = await rpc('deriveaddresses', [info.descriptor, [child, child]]); return addresses[0]; } +// Where a purchase is paid. A headless seller's address is derived from its xpub before the purchase is +// delivered. A Bitkit seller's xpub never leaves its wallet and Paykit Server exposes neither the xpub nor the +// derived address, so the fixture learns the address from the delivered Payment Request (`receive`), from the +// operator (`mine --address`) or from the transaction that pays the exact amount. +const paymentAddress = (purchase) => purchase.derived_address ?? purchase.payout_address ?? null; + +async function validatePayoutAddress(address) { + const info = await rpc('validateaddress', [address]); + if (!info.isvalid || !info.iswitness || info.witness_version !== 0 || info.witness_program?.length !== 40) { + fail(`${address} is not a regtest native SegWit (p2wpkh) address`); + } +} + +async function setPayoutAddress(purchases, purchase, address, source) { + if (purchase.derived_address) { + if (address !== purchase.derived_address) fail('the address is not the seller xpub child for this purchase'); + return; + } + if (purchase.payout_address === address) return; + if (purchase.payout_address) fail(`the purchase already has payout address ${purchase.payout_address}`); + await validatePayoutAddress(address); + purchase.payout_address = address; + purchase.payout_address_source = source; + await writeJson(PURCHASES_FILE, purchases); +} + // The transaction that pays a purchase, whoever paid it. The headless buyer's `pay` records its txid; a -// Bitkit buyer pays from the app, so the ledger has none. Match on the purchase's derived address and amount +// Bitkit buyer pays from the app, so the ledger has none. Match on the purchase's address and amount // instead: in the mempool first, then in the latest blocks (a confirmed payment is no longer in the mempool). +// Without a known address, match the one p2wpkh output of exactly the amount. const RECENT_BLOCKS = 50; async function findPaymentTx(purchase, { mempoolOnly = false } = {}) { - const pays = (tx) => - tx.vout.some( + const address = paymentAddress(purchase); + const paid = (tx) => + tx.vout.find( (output) => - output.scriptPubKey?.address === purchase.derived_address && - Math.round(output.value * 1e8) === purchase.amount_sats, + Math.round(output.value * 1e8) === purchase.amount_sats && + (address ? output.scriptPubKey?.address === address : output.scriptPubKey?.type === 'witness_v0_keyhash'), ); const found = []; for (const txid of await rpc('getrawmempool')) { // A transaction can leave the mempool between the two calls; skip it then. const tx = await rpc('getrawtransaction', [txid, true]).catch(() => null); - if (tx && pays(tx)) found.push({ txid, confirmed: false }); + const output = tx && paid(tx); + if (output) found.push({ txid, confirmed: false, address: output.scriptPubKey.address }); } if (!found.length && !mempoolOnly) { const { height } = await chainInfo(); for (let at = height; at > Math.max(0, height - RECENT_BLOCKS) && !found.length; at--) { const block = await rpc('getblock', [await rpc('getblockhash', [at]), 2]); - for (const tx of block.tx) if (pays(tx)) found.push({ txid: tx.txid, confirmed: true, height: at }); + for (const tx of block.tx) { + const output = paid(tx); + if (output) found.push({ txid: tx.txid, confirmed: true, height: at, address: output.scriptPubKey.address }); + } } } - if (found.length > 1) fail(`more than one transaction pays ${purchase.derived_address}: ${found.map((tx) => tx.txid).join(', ')}`); + if (found.length > 1) { + fail(`more than one transaction pays ${address ?? `${purchase.amount_sats} sats`} (${found.map((tx) => tx.txid).join(', ')}); pass --address `); + } return found[0] ?? null; } @@ -572,11 +677,18 @@ async function findPaymentTx(purchase, { mempoolOnly = false } = {}) { async function recordPaymentTx(purchases, purchase, options) { const tx = purchase.txid && !options?.mempoolOnly ? { txid: purchase.txid } : await findPaymentTx(purchase, options); if (!tx) return null; + let changed = false; if (purchase.txid !== tx.txid) { purchase.txid = tx.txid; if (['created', 'delivered'].includes(purchase.state)) purchase.state = 'paid'; - await writeJson(PURCHASES_FILE, purchases); + changed = true; + } + if (tx.address && !paymentAddress(purchase)) { + purchase.payout_address = tx.address; + purchase.payout_address_source = 'amount_match'; + changed = true; } + if (changed) await writeJson(PURCHASES_FILE, purchases); return tx; } @@ -585,6 +697,12 @@ async function purchase(args) { if (!Number.isInteger(sats) || sats <= 0) fail('--sats must be a positive integer'); const buyerArg = flag(args, '--buyer') ?? 'headless'; const fixture = await readFixture(); + const seller = pickSeller(fixture, flag(args, '--seller') ?? 'headless'); + await waitForPaykit(); + if (seller.kind !== 'headless') { + const setup = await setupStatus(seller.pubky); + if (setup !== 'ready') fail(`the Bitkit seller's Paykit setup is ${setup}; run: ./pubky-marketplace setup-url`); + } let reader; if (buyerArg === 'headless') { if (!fixture.buyer) fail('no headless buyer; run: ./pubky-marketplace seed --buyer headless'); @@ -592,19 +710,18 @@ async function purchase(args) { } else { reader = buyerArg; } - await waitForPaykit(); const purchases = await readJson(PURCHASES_FILE, []); - const childIndex = purchases.filter((entry) => entry.seller === fixture.seller.pubky).length; + const childIndex = purchases.filter((entry) => entry.seller === seller.pubky).length; - const lock = lockFor({ seller: fixture.seller.pubky, sats, issuer: await issuerPubky() }); + const lock = lockFor({ seller: seller.pubky, sats, issuer: await issuerPubky() }); const lockText = canonical(lock); const lockId = crockford(blake3(Buffer.from(lockText))); const lockPath = `/pub/locks.app/${lockId}.json`; - const session = await sellerSession(); + const session = await sellerSession(seller); await session.storage.putText(lockPath, lockText); const bundleId = newBundleId(); - const lockResource = `${fixture.seller.pubky}${lockPath}`; + const lockResource = `${seller.pubky}${lockPath}`; const response = await signedPost('/invoices', { bundle_id: bundleId, lock_resource: lockResource, reader }); if (response.status !== 204) { const code = response.json?.error?.code ? ` ${response.json.error.code}` : ''; @@ -616,15 +733,17 @@ async function purchase(args) { } const record = { bundle_id: bundleId, - seller: fixture.seller.pubky, + seller: seller.pubky, + seller_kind: seller.kind, reader, buyer_kind: buyerArg === 'headless' ? 'headless' : 'external', amount_sats: sats, asset: EXPECTED_ASSET, endpoint: EXPECTED_ENDPOINT, lock_resource: lockResource, - derived_address: await expectedAddress(fixture, childIndex), - child_index: childIndex, + // Only a seller whose xpub the fixture holds has a derived address it can check. + derived_address: seller.kind === 'headless' ? await expectedAddress(seller.account_xpub, childIndex) : null, + child_index: seller.kind === 'headless' ? childIndex : null, created_at: new Date().toISOString(), state: 'created', }; @@ -652,7 +771,6 @@ async function findPurchase(bundleId) { } async function receive(args) { - const fixture = await readFixture(); const { purchases, purchase } = await findPurchase(args[0]); if (purchase.buyer_kind !== 'headless') fail('receive is for the headless buyer; a Bitkit buyer receives in the app'); const seed = await readSecret('buyer-identity.seed'); @@ -660,7 +778,7 @@ async function receive(args) { const result = await runHelper( 'paykit-reader-demo', { version: 1, operation: 'receive', reader_secret: seed }, - readerEnv(fixture.seller.pubky), + readerEnv(purchase.seller), ); if (result.code !== 0) fail(`receive failed: ${result.stdout || result.stderr}`); const request = JSON.parse(result.stdout); @@ -669,7 +787,9 @@ async function receive(args) { if (request.status !== 'received' || request.asset !== EXPECTED_ASSET) fail('Payment Request is not canonical lowercase btc'); if (!request.address.startsWith('bcrt1')) fail('Payment Request address is not regtest'); if (request.amount_sats !== String(purchase.amount_sats)) fail('Payment Request amount does not match the purchase'); - if (request.address !== purchase.derived_address) fail('Payment Request address is not the seller xpub child for this purchase'); + // A Bitkit seller's xpub is not here: the delivered address is checked to be a regtest p2wpkh address and + // kept as the address the purchase must be paid to. Whether it belongs to the seller's wallet shows in the app. + await setPayoutAddress(purchases, purchase, request.address, 'payment_request'); purchase.payment_request_id = request.payment_request_id; purchase.delivery = 'received'; purchase.state = 'delivered'; @@ -682,20 +802,25 @@ async function receive(args) { endpoint: EXPECTED_ENDPOINT, address: request.address, amount_sats: Number(request.amount_sats), - address_derived_from_seller_xpub: true, + address_derived_from_seller_xpub: purchase.derived_address ? true : null, + address_check: purchase.derived_address + ? 'equals the seller xpub child for this purchase' + : 'regtest p2wpkh only; the fixture has no xpub for a Bitkit seller', }); } async function pay(args) { const { purchases, purchase } = await findPurchase(args[0]); await ensureMatureCoins(); - const txid = await rpc('sendtoaddress', [purchase.derived_address, satsToBtc(purchase.amount_sats)]); + const address = paymentAddress(purchase); + if (!address) fail('the payment address is not known yet; run: ./pubky-marketplace receive '); + const txid = await rpc('sendtoaddress', [address, satsToBtc(purchase.amount_sats)]); purchase.txid = txid; purchase.state = 'paid'; await writeJson(PURCHASES_FILE, purchases); const mempool = await rpc('getrawmempool'); const tx = await rpc('getrawtransaction', [txid, true]); - const match = tx.vout.find((output) => output.scriptPubKey.address === purchase.derived_address); + const match = tx.vout.find((output) => output.scriptPubKey.address === address); out({ bundle_id: purchase.bundle_id, txid, @@ -719,7 +844,10 @@ async function statusCommand(args) { reader: purchase.reader, payment_request_id: purchase.payment_request_id ?? null, delivery: purchase.delivery ?? null, - derived_address: purchase.derived_address, + seller_kind: purchase.seller_kind ?? 'headless', + derived_address: purchase.derived_address ?? null, + payout_address: paymentAddress(purchase), + payout_address_source: purchase.derived_address ? 'seller_xpub' : (purchase.payout_address_source ?? null), amount_sats: purchase.amount_sats, txid: purchase.txid ?? null, paykit_status: paykit, @@ -733,13 +861,17 @@ async function mine(args) { const before = await chainInfo(); const mempoolBefore = await rpc('getrawmempool'); let payment = null; + let payoutAddress = null; if (bundle) { const { purchases, purchase } = await findPurchase(bundle); + const address = flag(args, '--address'); + if (address) await setPayoutAddress(purchases, purchase, address, 'operator'); // The headless buyer's txid is in the ledger; for any other buyer find the payment by address and amount. payment = purchase.txid && mempoolBefore.includes(purchase.txid) ? { txid: purchase.txid } : await recordPaymentTx(purchases, purchase, { mempoolOnly: true }); if (!payment) fail('the purchase transaction is not in the mempool'); + payoutAddress = paymentAddress(purchase); } const [block] = await mineBlocks(1); const after = await chainInfo(); @@ -748,7 +880,7 @@ async function mine(args) { mined: 1, height: after.height, block, - ...(payment ? { bundle_id: bundle, txid: payment.txid } : {}), + ...(payment ? { bundle_id: bundle, txid: payment.txid, payout_address: payoutAddress } : {}), mempool_before: mempoolBefore.length, mempool_after: (await rpc('getrawmempool')).length, }); @@ -781,19 +913,23 @@ async function receiverMarker(pubky, receiverPath) { return { path, present: true, marker: await storage.getJson(`${pubky}${path}`) }; } -async function peerReport({ fixture, purchases, buyerArg, bundleArg }) { - const reader = buyerArg ?? (bundleArg ? undefined : purchases.at(-1)?.reader) ?? fixture.buyer?.pubky; +async function peerReport({ fixture, purchases, buyerArg, bundleArg, sellerArg }) { + // The seller is --seller, else the seller of the bundle or latest purchase, else the headless seller. + const chosen = sellerArg ? pickSeller(fixture, sellerArg) : null; + const ofSeller = (entry) => !chosen || entry.seller === chosen.pubky; + const reader = buyerArg ?? (bundleArg ? undefined : purchases.filter(ofSeller).at(-1)?.reader) ?? fixture.buyer?.pubky; if (!reader) fail('no buyer; pass --buyer or run: ./pubky-marketplace seed --buyer headless'); const purchase = bundleArg ? purchases.find((entry) => entry.bundle_id === bundleArg) - : purchases.filter((entry) => entry.reader === reader).at(-1); + : purchases.filter((entry) => entry.reader === reader && ofSeller(entry)).at(-1); if (bundleArg && !purchase) fail(`unknown bundle ${bundleArg}`); + const seller = chosen ?? sellerOf(fixture, purchase?.seller); const headless = fixture.buyer?.pubky === reader; - const setup = await signedPost('/setup/status', { creator: fixture.seller.pubky }); + const setup = await signedPost('/setup/status', { creator: seller.pubky }); let serverSide = 'no_purchase_yet'; if (purchase) { - const state = await signedPost('/connections/status', { creator: fixture.seller.pubky, bundle_id: purchase.bundle_id }); + const state = await signedPost('/connections/status', { creator: seller.pubky, bundle_id: purchase.bundle_id }); serverSide = state.status === 200 ? state.json.state : `unavailable_http_${state.status}`; } let buyerSide = null; @@ -801,17 +937,17 @@ async function peerReport({ fixture, purchases, buyerArg, bundleArg }) { const inspected = await runHelper( 'paykit-reader-demo', { version: 1, operation: 'inspect', reader_secret: await readSecret('buyer-identity.seed') }, - readerEnv(fixture.seller.pubky), + readerEnv(seller.pubky), ); buyerSide = inspected.code === 0 ? JSON.parse(inspected.stdout).connection_state : 'unavailable'; } - const sellerMarker = await receiverMarker(fixture.seller.pubky, SERVER_PATH); + const sellerMarker = await receiverMarker(seller.pubky, SERVER_PATH); const buyerMarker = await receiverMarker(reader, BUYER_PATH); const sellerReady = setup.status === 200 && setup.json?.status === 'ready' && sellerMarker.present; return { seller: { - pubky: fixture.seller.pubky, - kind: fixture.seller.kind, + pubky: seller.pubky, + kind: seller.kind, receiver_path: SERVER_PATH, setup: setup.status === 200 ? setup.json.status : `unavailable_http_${setup.status}`, receiver_marker: sellerMarker, @@ -841,7 +977,7 @@ async function peers(args) { const fixture = await readFixture(); const purchases = await readJson(PURCHASES_FILE, []); const waitSeconds = Number(flag(args, '--wait') ?? 0); - if (!Number.isFinite(waitSeconds) || waitSeconds < 0) fail('usage: peers [--buyer ] [--bundle ] [--wait ]'); + if (!Number.isFinite(waitSeconds) || waitSeconds < 0) fail('usage: peers [--seller headless|bitkit|] [--buyer ] [--bundle ] [--wait ]'); await waitForPaykit(); const deadline = Date.now() + waitSeconds * 1000; for (;;) { @@ -850,6 +986,7 @@ async function peers(args) { purchases: await readJson(PURCHASES_FILE, purchases), buyerArg: flag(args, '--buyer'), bundleArg: flag(args, '--bundle'), + sellerArg: flag(args, '--seller'), }); if (report.linked || !waitSeconds || Date.now() > deadline) { out(report); @@ -927,12 +1064,105 @@ async function verify() { out(evidence); } +// ---------------------------------------------------------------- Bitkit seller + +const androidOpen = (authUrl) => `adb shell am start -a android.intent.action.VIEW -d '${authUrl}'`; + +// The marketplace's request for a write grant on the seller's /pub/locks.app/, shown to the seller as a Pubky +// auth request (the role Locks plays). The flow polls the relay as long as this process runs. +async function startLocksGrant(relay) { + const flow = await pubkyClient().startGrantAuthFlow(LOCKS_CAPS, AuthFlowKind.signin(), { clientId: LOCKS_CLIENT_ID, relay }); + const authUrl = flow.authorizationUrl; + if (!authUrl.startsWith('pubkyauth://signin_grant?') || !authParams(authUrl).get('cpk')) { + fail('the marketplace grant URL is not a Pubky grant URL with cpk; check the @synonymdev/pubky version'); + } + return { flow, authUrl }; +} + +async function awaitGrant(flow, seconds) { + const deadline = Date.now() + seconds * 1000; + for (;;) { + const session = await flow.tryPollOnce(); + if (session) return session; + if (Date.now() > deadline) fail(`no approval within ${seconds}s; run seller-auth again for a fresh request`); + await sleep(1000); + } +} + +// A grant covers the lock directory when one of its capabilities is a write on that path or a parent of it. +const writesLocks = (capabilities) => + capabilities.some((cap) => { + const at = cap.lastIndexOf(':'); + return cap.slice(at + 1).includes('w') && '/pub/locks.app/'.startsWith(cap.slice(0, at)); + }); + +// Keep the approved grant as the seller's write session and record the identity that approved it. +async function adoptBitkitSeller(session, kind) { + const capabilities = session.info.capabilities; + if (!writesLocks(capabilities)) fail(`the approved grant does not allow writing ${LOCKS_CAPS}: ${JSON.stringify(capabilities)}`); + const pubky = session.info.publicKey.toString(); + const secret = await session.exportLocalSecret(); + const restored = await pubkyClient().restoreSession(secret); + if (restored.info.publicKey.toString() !== pubky) fail('the exported grant session restores to another identity'); + await writeSecret(`${SECRETS}/${BITKIT_SESSION_SECRET}`, secret); + const fixture = await readFixture(); + const previous = fixture.bitkit_seller?.pubky === pubky ? fixture.bitkit_seller : {}; + const paykitSetup = await setupStatus(pubky); + fixture.bitkit_seller = { + ...previous, + pubky, + kind, + client_id: LOCKS_CLIENT_ID, + capabilities: LOCKS_CAPS, + marketplace_grant_at: new Date().toISOString(), + ...(paykitSetup === 'ready' ? { setup_completed_at: previous.setup_completed_at ?? new Date().toISOString() } : {}), + }; + await writeJson(FIXTURE_FILE, fixture); + return { seller: fixture.bitkit_seller, paykitSetup }; +} + +// Print the marketplace grant request for the Bitkit seller wallet, wait for its approval and keep the grant +// session that `purchase --seller bitkit` publishes the payment lock with. +async function sellerAuth(args) { + const relay = flag(args, '--relay') ?? LOCKS_RELAY; + const seconds = Number(flag(args, '--timeout') ?? 300); + if (!Number.isFinite(seconds) || seconds <= 0) fail('usage: seller-auth [--relay ] [--timeout ]'); + await readFixture(); + const { flow, authUrl } = await startLocksGrant(relay); + out({ + status: 'awaiting_approval', + client_id: LOCKS_CLIENT_ID, + capabilities: LOCKS_CAPS, + relay, + auth_url: authUrl, + android: androidOpen(authUrl), + // iOS registers no pubkyauth handler and bitkit://pubky-auth/setup accepts only the setup request: put + // auth_url on the simulator's clipboard, then Scan QR Code, Paste QR Code (E2E builds: Enter QRCode String). + ios: { + clipboard: `printf %s '${authUrl}' | xcrun simctl pbcopy `, + in_app: 'Scan QR Code sheet, then Paste QR Code', + }, + wait_seconds: seconds, // the relay keeps the request for about 5 minutes + }); + const session = await awaitGrant(flow, seconds); + const { seller, paykitSetup } = await adoptBitkitSeller(session, 'bitkit'); + out({ + status: 'approved', + seller: seller.pubky, + client_id: seller.client_id, + capabilities: session.info.capabilities, + paykit_setup: paykitSetup, + next: paykitSetup === 'ready' ? './pubky-marketplace purchase --seller bitkit' : './pubky-marketplace setup-url', + }); +} + const commands = { init: () => init(), seed, info: async () => out(await publicInfo()), 'setup-url': () => setupUrl(), 'setup-wait': (args) => setupWait(args[0]), + 'seller-auth': sellerAuth, fund, purchase, receive, @@ -951,6 +1181,9 @@ if (!commands[command]) { } try { await commands[command](args); + // A grant flow keeps its relay poll pending; leave once stdout is flushed instead of waiting on it. + await new Promise((resolve) => process.stdout.write('', resolve)); + process.exit(0); } catch (error) { if (error instanceof DriverError) { process.stderr.write(`FAIL: ${error.message}\n`); diff --git a/pubky-marketplace b/pubky-marketplace index 42211f2..af688d7 100755 --- a/pubky-marketplace +++ b/pubky-marketplace @@ -38,18 +38,26 @@ Fixture: watch-only setup, optionally create a headless buyer info Public fixture facts: homeserver key, seller, buyer, ports setup-url New watch-only setup auth URL for a Bitkit wallet to approve - setup-wait Wait until an approved setup flow completes + setup-wait Wait until an approved setup flow completes; with a Bitkit seller + recorded, check the same wallet approved it + seller-auth [--relay ] [--timeout ] + Bitkit seller: print the marketplace write grant (/pub/locks.app/) for the + seller wallet to approve, with the Android and iOS handoff, wait for it + and keep the grant session for purchase --seller bitkit fund
[sats] Send regtest coins (default 1000000 sats) and mine one block - peers [--buyer ] [--bundle ] [--wait ] + peers [--seller headless|bitkit|] [--buyer ] [--bundle ] [--wait ] Report the seller and buyer as linked peers: receiver markers, setup authority and Paykit Server's link state (linked is true once connected) Purchase: - purchase [--sats N] [--buyer headless|] - Create one purchase; print bundle id, address and delivery state + purchase [--sats N] [--buyer headless|] [--seller headless|bitkit|] + Create one purchase; print bundle id, address and delivery state. A Bitkit + seller (after seller-auth and setup) is paid at its wallet's derived address receive Headless buyer: receive the Payment Request and validate it pay Headless buyer: pay the derived address from the regtest wallet status Signed Paykit status and purchase state - mine [--bundle ] Mine exactly one block; print height, hash and confirmed purchases + mine [--bundle ] [--address ] + Mine exactly one block; print height, hash and the purchase transaction. + --address names the payout address of a Bitkit seller's purchase wait [seconds] Wait for a signed Paykit status verify Whole headless journey with assertions; writes .marketplace/evidence/ @@ -179,7 +187,7 @@ case "$command" in "$0" down up ;; - seed | info | setup-url | setup-wait | fund | peers | purchase | receive | pay | status | mine | wait | verify) + seed | info | setup-url | setup-wait | seller-auth | fund | peers | purchase | receive | pay | status | mine | wait | verify) driver "$command" "$@" ;; *) From 9a02070e0d5b2f0ba25a104da0ab6022184f5352 Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 21:55:27 +0200 Subject: [PATCH 10/13] test: add a bitkit seller self-test with a headless wallet stand-in verify-bitkit-seller approves the marketplace grant with approveAuthRequest from a stand-in's own keypair and the watch-only setup with the companion claim, checks a purchase before setup is refused, then has a headless buyer pay and asserts the payout goes to the address derived from the stand-in's xpub. verify stays on the headless seller. --- marketplace/driver/driver.mjs | 114 +++++++++++++++++++++++++++++----- pubky-marketplace | 4 +- 2 files changed, 103 insertions(+), 15 deletions(-) diff --git a/marketplace/driver/driver.mjs b/marketplace/driver/driver.mjs index 97f7ed0..14d4225 100644 --- a/marketplace/driver/driver.mjs +++ b/marketplace/driver/driver.mjs @@ -47,6 +47,8 @@ const LOCKS_CAPS = '/pub/locks.app/:rw'; // The testnet's own HTTP relay; wallets reach it on localhost like the homeserver (Android: adb reverse 15412). const LOCKS_RELAY = 'http://localhost:15412/inbox/'; const BITKIT_SESSION_SECRET = 'bitkit-seller.session'; +// Bitkit gives every setup request a fresh BIP84 account, starting at index 1. +const STANDIN_ACCOUNT_INDEX = 1; const SERVER_PATH = 'bitkit/server'; const BUYER_PATH = 'bitkit/wallet'; const ACCOUNT_INDEX = 0; @@ -997,10 +999,9 @@ async function peers(args) { } } -// The whole journey with the driver in every wallet role. Each step asserts. -async function verify() { - const evidence = { started_at: new Date().toISOString(), steps: {} }; - const capture = async (name, run) => { +// Runs a command and records its JSON output in the evidence under `name`. +function evidenceCapture(evidence) { + return async (name, run) => { let output = ''; const original = process.stdout.write.bind(process.stdout); process.stdout.write = (chunk) => { @@ -1017,6 +1018,25 @@ async function verify() { log(`${name}: ok`); return value; }; +} + +async function writeEvidence(evidence, suffix = '') { + evidence.finished_at = new Date().toISOString(); + evidence.result = 'passed'; + if (existsSync(EVIDENCE_DIR)) { + const dir = `${EVIDENCE_DIR}/${evidence.started_at.replace(/[:.]/g, '-')}${suffix}`; + await mkdir(dir, { recursive: true }); + await writeJson(`${dir}/summary.json`, evidence); + await handToHost(dir, `${dir}/summary.json`); + evidence.evidence_dir = `.marketplace/evidence/${dir.split('/').pop()}`; + } + out(evidence); +} + +// The whole journey with the driver in every wallet role. Each step asserts. +async function verify() { + const evidence = { started_at: new Date().toISOString(), steps: {} }; + const capture = evidenceCapture(evidence); await capture('seed', () => seed(['--buyer', 'none'])); delete evidence.steps.seed; // public facts are recorded under seller and buyer @@ -1052,16 +1072,7 @@ async function verify() { const final = await capture('status', () => statusCommand([bundle])); if (final.purchase_state !== 'completed') fail('purchase did not complete'); - evidence.finished_at = new Date().toISOString(); - evidence.result = 'passed'; - if (existsSync(EVIDENCE_DIR)) { - const dir = `${EVIDENCE_DIR}/${evidence.started_at.replace(/[:.]/g, '-')}`; - await mkdir(dir, { recursive: true }); - await writeJson(`${dir}/summary.json`, evidence); - await handToHost(dir, `${dir}/summary.json`); - evidence.evidence_dir = `.marketplace/evidence/${dir.split('/').pop()}`; - } - out(evidence); + await writeEvidence(evidence); } // ---------------------------------------------------------------- Bitkit seller @@ -1156,6 +1167,80 @@ async function sellerAuth(args) { }); } +// The Bitkit seller path with a headless Pubky client standing in for the wallet: it approves the marketplace +// grant with approveAuthRequest from its own keypair and the Paykit setup with the companion claim, as the app +// does, then a headless buyer pays the purchase and the payout must land on the address derived from the +// stand-in's xpub. The fixture never uses that xpub to pick the address: it only checks the result. +async function verifyBitkitSeller() { + const evidence = { mode: 'bitkit-seller-standin', started_at: new Date().toISOString(), steps: {} }; + const capture = evidenceCapture(evidence); + + await capture('seed', () => seed(['--buyer', 'none'])); + delete evidence.steps.seed; + const existing = (await readFixture()).bitkit_seller; + if (existing?.kind === 'bitkit') fail('a real Bitkit seller is recorded; run ./pubky-marketplace reset before the self-test'); + + const identitySeed = randomBytes(32); + const account = HDKey.fromMasterSeed(randomBytes(32), { private: 0x04358394, public: 0x043587cf }).derive( + `m/84'/1'/${STANDIN_ACCOUNT_INDEX}'`, + ); + const xpub = account.publicExtendedKey; + const standin = await signUpIdentity(identitySeed); + evidence.seller = { pubky: standin, account_xpub: xpub, account_index: STANDIN_ACCOUNT_INDEX, kind: 'standin' }; + + // Approval 1: the marketplace grant, over the relay like a wallet's approval. + const { flow, authUrl } = await startLocksGrant(LOCKS_RELAY); + await pubkyClient().signer(Keypair.fromSecret(identitySeed)).approveAuthRequest(authUrl); + const session = await awaitGrant(flow, 60); + const adopted = await adoptBitkitSeller(session, 'standin'); + if (adopted.seller.pubky !== standin) fail('the grant was adopted for another identity'); + if (adopted.paykitSetup === 'ready') fail('a fresh seller must not have a Paykit setup yet'); + evidence.steps.marketplace_grant = { seller: standin, capabilities: session.info.capabilities, paykit_setup: adopted.paykitSetup }; + log('marketplace_grant: ok'); + + // A purchase before the Paykit setup is refused. + const refused = await purchase(['--seller', 'bitkit']).then( + () => null, + (error) => error.message, + ); + if (!refused?.includes('setup')) fail(`a purchase without the Paykit setup must be refused, got: ${refused}`); + evidence.steps.setup_required = { refused }; + log('setup_required: ok'); + + // Approval 2: the watch-only setup by the same identity. + const { flowId, authUrl: setupAuthUrl } = await beginSetup(); + await approveSetupAs(setupAuthUrl, identitySeed, xpub, STANDIN_ACCOUNT_INDEX); + await capture('setup', () => setupWait(flowId)); + if ((await readFixture()).bitkit_seller.setup_completed_at === undefined) fail('setup was not recorded for the Bitkit seller'); + + const buyer = await createBuyer(standin); + evidence.buyer = { pubky: buyer.pubky }; + const created = await capture('purchase', () => purchase(['--seller', 'bitkit', '--buyer', 'headless'])); + if (created.seller !== standin || created.derived_address !== null) fail('a Bitkit seller purchase has no derived address in the ledger'); + if (created.paykit_status.status !== 'undetected') fail('a new purchase must start undetected'); + const bundle = created.bundle_id; + const expected = await expectedAddress(xpub, 0); + + const received = await capture('receive', () => receive([bundle])); + if (received.address !== expected) fail(`the Payment Request pays ${received.address}, not the stand-in xpub's ${expected}`); + const heightBefore = (await chainInfo()).height; + const paid = await capture('pay', () => pay([bundle])); + const tx = await rpc('getrawtransaction', [paid.txid, true]); + const output = tx.vout.find((entry) => entry.scriptPubKey.address === expected); + if (Math.round(output?.value * 1e8) !== received.amount_sats || paid.mempool_entries !== 1) { + fail('the mempool must hold exactly the purchase transaction paying the stand-in xpub address'); + } + await capture('detected', () => waitFor([bundle, 'detected'])); + const mined = await capture('mine', () => mine(['--bundle', bundle])); + if (mined.height !== heightBefore + 1) fail('exactly one block must confirm the payment'); + await capture('confirmed', () => waitFor([bundle, 'confirmed'])); + const final = await capture('status', () => statusCommand([bundle])); + if (final.purchase_state !== 'completed' || final.payout_address !== expected) fail('purchase did not complete on the stand-in xpub address'); + evidence.payout = { address: expected, derived_from: 'stand-in seller xpub, external child 0/0', txid: paid.txid }; + + await writeEvidence(evidence, '-bitkit-seller'); +} + const commands = { init: () => init(), seed, @@ -1172,6 +1257,7 @@ const commands = { peers, wait: waitFor, verify: () => verify(), + 'verify-bitkit-seller': () => verifyBitkitSeller(), }; const [command, ...args] = process.argv.slice(2); diff --git a/pubky-marketplace b/pubky-marketplace index af688d7..f90d376 100755 --- a/pubky-marketplace +++ b/pubky-marketplace @@ -61,6 +61,8 @@ Purchase: wait [seconds] Wait for a signed Paykit status verify Whole headless journey with assertions; writes .marketplace/evidence/ + verify-bitkit-seller Bitkit seller path with a headless stand-in for the wallet (both approvals, + purchase, payout to the stand-in xpub address); writes .marketplace/evidence/ EOF } @@ -187,7 +189,7 @@ case "$command" in "$0" down up ;; - seed | info | setup-url | setup-wait | seller-auth | fund | peers | purchase | receive | pay | status | mine | wait | verify) + seed | info | setup-url | setup-wait | seller-auth | fund | peers | purchase | receive | pay | status | mine | wait | verify | verify-bitkit-seller) driver "$command" "$@" ;; *) From a267b7cc7523137d31c17f997ac0ba894b0346fa Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 21:56:50 +0200 Subject: [PATCH 11/13] docs: document the bitkit seller flow in the marketplace fixture Explain why the seller approves two Pubky requests, what each app accepts and shows, the iOS and Android handoff, and what the fixture can and cannot verify about a Bitkit seller's payout. --- CHANGELOG.md | 2 +- README.md | 19 +++++++- docs/pubky-marketplace.md | 91 +++++++++++++++++++++++++++++++++------ 3 files changed, 97 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d70dcac..88eb03a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,7 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Preserve LNURL-pay invoice millisatoshi precision by creating invoices with LND `value_msat` instead of truncating callback amounts to sats ### Added -- Add Pubky marketplace test fixture behind the `marketplace` compose profile: Pubky testnet, Paykit Server `722ef268` built from pinned source, and the `pubky-marketplace` driver CLI (`up`, `seed`, `purchase`, `mine`, `verify`) for the Bitkit marketplace wallet journey; see `docs/pubky-marketplace.md` +- Add Pubky marketplace test fixture behind the `marketplace` compose profile: Pubky testnet, Paykit Server `722ef268` built from pinned source, and the `pubky-marketplace` driver CLI (`up`, `seed`, `purchase`, `mine`, `verify`, `seller-auth`) for the Bitkit marketplace wallet journey with a headless or Bitkit-approved seller; see `docs/pubky-marketplace.md` - Homegate Docker Compose service with dedicated PostgreSQL storage, local homeserver admin mock, and README setup flow - Repo-managed Trezor User Env Docker service and `scripts/trezor-emulator` helper for quickly smoke-testing Bitkit app Trezor PRs - Support `amount_msat` query param in `/generate/bolt11` endpoint for sub-sat precision invoices diff --git a/README.md b/README.md index cdd5d07..bc3c63b 100644 --- a/README.md +++ b/README.md @@ -313,7 +313,24 @@ Then, with the buyer wallet: ./pubky-marketplace peers --buyer --wait 60 # after the purchase: Paykit Server's link to the buyer is connected ``` -Pay the request in the app, then confirm with `./pubky-marketplace mine --bundle `, `./pubky-marketplace wait confirmed` and `./pubky-marketplace status `. The seller in a driver purchase is the fixture's headless seller, because publishing the payment lock needs the seller's identity secret. To check only the watch-only claim in a Bitkit seller wallet, run `./pubky-marketplace setup-url`, open the printed `auth_url` in that wallet (the URL holds a one-time secret, so keep it out of logs and evidence), approve it, then `./pubky-marketplace setup-wait `. The URL is a `pubkyauth://signin_grant?...` link. On Android open it with `adb shell am start -a android.intent.action.VIEW -d ''`; iOS has no `pubkyauth` handler, so open the printed `ios_url` (`bitkit://pubky-auth/setup?`) with `xcrun simctl openurl ''`. +Pay the request in the app, then confirm with `./pubky-marketplace mine --bundle `, `./pubky-marketplace wait confirmed` and `./pubky-marketplace status `. By default the seller of a purchase is the fixture's headless seller, and `verify` always uses it. + +To make a Bitkit wallet the seller, the wallet approves two Pubky requests for the same identity: the Paykit watch-only setup (it gives Paykit Server the wallet's account xpub, so payouts land in that wallet) and a write grant on `/pub/locks.app/` (the role Locks plays: the driver publishes the payment lock with the granted session). One request cannot carry both, because the apps accept the watch-only claim only for exactly the two Paykit paths. + +```bash +./pubky-marketplace seed --buyer none # once per fixture; the headless seller stays unused +./pubky-marketplace setup-url # open android or ios_url in the seller wallet, approve, then: +./pubky-marketplace setup-wait +./pubky-marketplace seller-auth # prints the marketplace grant request, waits for the approval +./pubky-marketplace purchase --seller bitkit --buyer +./pubky-marketplace mine --bundle # after the buyer pays; add --address if two payments match the amount +./pubky-marketplace wait confirmed +./pubky-marketplace status # payout address, txid, signed Paykit status, purchase state +``` + +Both URLs are `pubkyauth://signin_grant?...` links whose one-time secret must stay out of logs and evidence. On Android open each printed `android` command (`adb shell am start -a android.intent.action.VIEW -d ''`). On iOS the setup request opens through the printed `ios_url` (`xcrun simctl openurl ''`); the marketplace grant has no iOS deep link, so put its `auth_url` on the simulator clipboard (`printf %s '' | xcrun simctl pbcopy `) and use Scan QR Code, then Paste QR Code (E2E builds also have Enter QRCode String). The marketplace request goes through the testnet's HTTP relay on `localhost:15412` (published by the fixture; on Android it is one of the `adb reverse` ports above) and expires after about five minutes; pass `--relay https://httprelay.pubky.app/inbox/` to use the public relay instead. The setup request still uses the public relay and needs outbound internet. + +The fixture cannot check a Bitkit seller's payout address against the wallet's xpub, because Paykit Server keeps the xpub and the derived address to itself. It checks that the Payment Request address is a regtest native SegWit address, that exactly the amount is paid to it on chain, and Paykit Server's signed status (`detected`, then `confirmed` with a matching amount). That the address belongs to the wallet shows in the seller app: its balance rises by the amount and the received activity carries the same txid as `status`. `./pubky-marketplace verify-bitkit-seller` runs the whole path with a headless stand-in for the wallet and does check the payout against the stand-in's xpub. Remove the fixture with `./pubky-marketplace down`, or start over with `./pubky-marketplace reset`. The Pubky testnet keeps its accounts in memory, so the fixture cannot restart with its state and `down` deletes it (the regtest chain stays). `./pubky-marketplace --help` lists every command. See [docs/pubky-marketplace.md](docs/pubky-marketplace.md) for the pins, ports, roles and what the fixture does not cover. diff --git a/docs/pubky-marketplace.md b/docs/pubky-marketplace.md index d28e9a1..9294a4d 100644 --- a/docs/pubky-marketplace.md +++ b/docs/pubky-marketplace.md @@ -13,7 +13,7 @@ and [bitkit-android#1338](https://github.com/synonymdev/bitkit-android/pull/1338 | Pubky Core static testnet: DHT, PKARR relay, HTTP relay, one homeserver with open signup | `pubky-testnet`, built from `marketplace/pubky-testnet/Dockerfile` | pubky-core `f68014c1` | | Homeserver and Paykit databases | `marketplace-postgres` | `postgres:16-alpine` | | Paykit Server | `paykit-server`, built from source with the upstream `Dockerfile.local` | pubky/paykit-server `722ef268` (v0.1.0-rc4), paykit-rs `9b56a0ea` (v0.1.0-rc48), locks-core `8502ef79` (v0.1.0-rc1) | -| Purchase driver | `marketplace-driver`, run by `./pubky-marketplace` | `marketplace/driver/package-lock.json` | +| Purchase driver | `marketplace-driver`, run by `./pubky-marketplace` | `marketplace/driver/package-lock.json`, `@synonymdev/pubky` 0.10.0 | `./pubky-marketplace build` checks the pinned trees out under `.marketplace/sources` (git ignored) and fails if a checkout is not at its pin or if the Paykit Server tree's `Cargo.lock` does not lock paykit-rs @@ -34,6 +34,16 @@ driver's `setup-url` refuses any auth URL that is not `signin_grant` with `cid` fails before it reaches a wallet. Unmerged Paykit Server branches move to paykit-rs rc56; they are not pinned here. +### Why `@synonymdev/pubky` 0.10.0 + +Both the Bitkit seller approval and the headless seller need the grant auth flow, which the driver's earlier +0.9.3 client lacks (it has cookie auth only). The pinned homeserver, Pubky Core `f68014c1` (2026-07-31), sits +between v0.9.3 and v0.10.0 (2026-08-05); the commits between it and v0.10.0 are documentation, callback +parameters and one error-surfacing change. 0.10.0 is therefore the client that matches the homeserver. 0.11.0 and +later upgrade pkarr to v8 and the relay to v2 past that homeserver and are not used until the testnet pin moves. +In 0.10.0 a signin names its client and returns a grant session, so the headless seller signs in as +`marketplace.fixture`. + ## Ports Published on the host loopback only, because the pinned Pubky clients resolve their local testnet on @@ -57,14 +67,17 @@ The journey needs a seller, a buyer and a marketplace. The driver can play each wallets replace the wallet roles in the app journey. - **Marketplace (always the driver).** It publishes a `paykit-payment` content lock on the seller's - homeserver and posts a signed `POST /invoices` to Paykit Server as the trusted issuer. The issuer key + homeserver, with the seller's own session (headless seller) or with the write grant the Bitkit seller approved + (see "Bitkit seller"), and posts a signed `POST /invoices` to Paykit Server as the trusted issuer. The issuer key is generated at `init`, and its public key is `locks.trusted_public_key` in the generated Paykit config. Paykit Server's status route is signed the same way. This is the part Locks plays in a full marketplace; Locks itself is not in this stack. -- **Seller (headless).** The driver holds the seller's Pubky identity, publishes the lock with it, and +- **Seller (headless, the default).** The driver holds the seller's Pubky identity, publishes the lock with it, and completes `/setup` through `paykit-companion-auth`, which approves the same `watch-only-account-v1` claim Bitkit approves. The seller's spending authority is a wallet seed that - stays in the state volume; Paykit Server receives only the account xpub at `m/84'/1'/0'`. + stays in the state volume; Paykit Server receives only the account xpub at `m/84'/1'/0'`. `verify` uses it. +- **Seller (Bitkit).** A Bitkit wallet is the seller through two approvals; the driver holds no key or seed for + it. See "Bitkit seller". - **Buyer (headless or Bitkit).** The headless buyer is `paykit-reader-demo` at the `bitkit/wallet` receiver path, paying from the regtest wallet. A Bitkit buyer is passed as `purchase --buyer `. @@ -77,7 +90,8 @@ Everything lives in the `marketplace_state` volume, and `down` deletes it. - `/state/paykit` (readable by the Paykit Server process): generated config and master key. - `/state/secrets` (root, mode 0700, unreadable by Paykit Server): issuer seed, seller identity seed, - seller wallet seed, buyer identity seed. + seller wallet seed, buyer identity seed, and `bitkit-seller.session`, the `/pub/locks.app/` grant session a + Bitkit seller approved (bearer-equivalent for that path; the grant lasts two years). - `/state/fixture.json`, `/state/purchases.json`: public facts and the purchase ledger. - `.marketplace/evidence//summary.json`: `verify` output, owned by the user who ran the wrapper (the driver hands it over from its root container). It holds public keys, bundle and request ids, addresses, txids and @@ -86,6 +100,53 @@ Everything lives in the `marketplace_state` volume, and `down` deletes it. The driver never prints a seed or key. `setup-url` prints a one-time auth URL that contains a session secret; it is meant to be pasted into a wallet, so keep it out of logs and evidence. +## Bitkit seller + +In the wallet journey the Bitkit seller wallet is the seller: the marketplace acts for the identity that wallet +approves, and payouts land in the wallet. That takes two approvals of the same Pubky identity, in either order: + +| Approval | Fixture command | Requester ID | Permissions | Gives the fixture | +| --- | --- | --- | --- | --- | +| Paykit setup (`x-bitkit-claim=watch-only-account-v1`) | `setup-url`, then `setup-wait ` | `app.paykit.server` | `/pub/paykit/v0/bitkit/server` and `/pub/paykit/v0/private/bitkit/server`, READ, WRITE | Paykit Server holds the wallet's account xpub and derives the payout addresses | +| Marketplace grant | `seller-auth` | `locks.app` | `/pub/locks.app`, READ, WRITE | a session that writes the payment lock to the seller's homeserver | + +One approval cannot carry both. Both apps accept the watch-only claim only when the requested capabilities are +exactly the two Paykit paths (a claim with other capabilities, or those two paths without a claim, is +rejected), and Paykit Server fixes those capabilities. An approval without the claim is an ordinary Pubky +grant request: both apps accept any capabilities and requester ID for it and show them for the user to +approve, so the marketplace grant needs no app change. + +`seller-auth` starts a grant flow (`startGrantAuthFlow` with `/pub/locks.app/:rw`, client id `locks.app`) on the +testnet's HTTP relay, prints the `pubkyauth://signin_grant?caps&relay&secret&cid&cpk` URL, waits up to +`--timeout` seconds (default 300; the relay keeps a request about five minutes) and stores the approved +session under `/state/secrets`. It records the approving identity as the Bitkit seller and reports Paykit's +setup state for it. `setup-wait` then checks the wallet that approved the setup is the same identity, and +`purchase --seller bitkit` (or `--seller `) refuses until it is. `peers --seller bitkit`, +`status`, `wait` and `mine --bundle` work on the purchase's own seller. + +Handoff. Android opens either URL with `adb shell am start -a android.intent.action.VIEW -d ''` +(the app enables its `pubkyauth://signin_grant` handler while it holds a Bitkit-generated Pubky identity). +iOS registers no `pubkyauth` scheme. Its `bitkit://pubky-auth/setup?` handoff requires the claim, so it +opens only the setup request (`ios_url`). The marketplace grant is entered in the app: Scan QR Code, then Paste +QR Code with the URL on the simulator clipboard, or Enter QRCode String in E2E builds. An optional iOS app +change, accepting a claim-less `bitkit://pubky-auth/...` handoff, would let `xcrun simctl openurl` open it +without taps; it is not needed. + +What is verified for the payout, and what is not. Paykit Server exposes no xpub, derived address or txid to +its creators, so for a Bitkit seller the fixture cannot derive the expected address: + +- verified by the fixture: the Payment Request address a headless buyer receives (`receive`) is a regtest + native SegWit address; the mempool holds a transaction paying exactly that address and the purchase amount + (`pay`, `mine --bundle`; for an app buyer the address comes from `mine --address` or from the one p2wpkh + output of exactly the amount); Paykit Server's signed status goes `detected` and then `confirmed` with a + matching amount and one confirmation, which means its own derived address for the invoice received the + payment; the purchase reaches `completed`. +- verified only in the seller app: that the address is derived from the wallet's own xpub. The seller wallet + tracks its watch-only account, so its balance rises by the amount and the received activity carries the + purchase transaction id. +- verified for a stand-in only: `verify-bitkit-seller` approves both requests with a headless client that owns + the xpub, and asserts the Payment Request address and the paid output equal the stand-in's `0/0` child. + ## Lifecycle The Pubky testnet keeps homeserver files and its DHT in memory, so its accounts do not survive a @@ -105,7 +166,9 @@ more confirmations, the gate Locks applies with `minimum_confirmations = 1`. The derived address is the seller xpub's external child `0/`, where `n` is the number of earlier purchases for that seller, checked through `bitcoind deriveaddresses`. A headless `receive` compares it -with the address in the delivered Payment Request. +with the address in the delivered Payment Request. A Bitkit seller's xpub is not in the fixture, so its +purchases have `derived_address` null and a `payout_address` learned from the Payment Request, `mine --address` +or the transaction that pays the exact amount (`payout_address_source` says which). `mine --bundle` mines one block only after it finds the purchase transaction in the mempool. The headless buyer's `pay` records its txid; for a buyer that pays from an app, `mine --bundle` looks for the transaction that pays @@ -132,17 +195,19 @@ use `ready_for_purchase` there and `peers --wait 60` after `purchase`, which exi ## Limits - **Setup relay.** The pinned Paykit Server starts the setup sign-in on `https://httprelay.pubky.app`, - not on the local relay, and offers no config to change it. `seed` and a Bitkit seller's approval - therefore need outbound internet. Only that one-time handshake leaves the machine. -- **Seller in the app.** Publishing a lock needs the seller's identity secret, so driver purchases use - the headless seller. A Bitkit wallet that approves `setup-url` becomes another Paykit Server creator - for the watch-only leg, and the driver does not sell as it. Full Locks legacy-connect authority for a - Bitkit seller is out of scope here. + not on the local relay, and offers no config to change it. `seed` and a Bitkit seller's setup approval + therefore need outbound internet. Only that one-time handshake leaves the machine: the marketplace grant + of a Bitkit seller uses the local relay unless `seller-auth --relay` names another. +- **Locks authority.** A Bitkit seller gives the driver only a write grant on `/pub/locks.app/`, the path Locks + publishes locks under, through the Pubky grant session path. Locks' own connect flow and its other seller + APIs are out of scope. - **No Locks server, no guarded content.** The lock has no guarded resource, and the fixture does not cover marketplace browsing, content delivery, fiat payment or Hypercolor, which the journey also excludes. - **Bitkit apps.** The commands for Bitkit wallets follow the journey's fixture contract. The buyer legs ran on both apps in the J1 device run on 2026-09-29, against the previous Paykit Server pin. The seller - setup with the current pin has been run headlessly (`seed` and `verify`) and not yet against an app. + setup with the current pin has been run headlessly (`seed` and `verify`) and not yet against an app. The + Bitkit seller path (`seller-auth`, `purchase --seller bitkit`) has a headless self-test, + `verify-bitkit-seller`, and has not been run against an app yet. - **Fixed container names.** The base services keep their fixed container names, so another checkout's stack with the same names must be removed first. From 610fa5e2c944fb5cf880af6e3977283ae13cc83a Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 22:59:43 +0200 Subject: [PATCH 12/13] fix: report the bundle's buyer in peers, assert peers in verify and reset from any directory --- docs/pubky-marketplace.md | 3 ++- marketplace/driver/driver.mjs | 25 ++++++++++++++++++++----- pubky-marketplace | 21 ++++++++++++--------- 3 files changed, 34 insertions(+), 15 deletions(-) diff --git a/docs/pubky-marketplace.md b/docs/pubky-marketplace.md index 9294a4d..49ed0ba 100644 --- a/docs/pubky-marketplace.md +++ b/docs/pubky-marketplace.md @@ -179,7 +179,7 @@ transaction is an error, and no match makes `mine --bundle` refuse. ## Linked peers `peers [--buyer ] [--bundle ] [--wait ]` is the fixture's answer to journey step 14. It -prints, for the seller and for the buyer (default: the latest purchase's buyer, else the headless buyer): +prints, for the seller and for the buyer (default: the reader of the purchase `--bundle` selects, else the latest purchase's buyer, else the headless buyer): - the public Paykit receiver marker each identity publishes (`bitkit/server` for the seller, `bitkit/wallet` for the buyer), which shows contact payments are on; @@ -191,6 +191,7 @@ prints, for the seller and for the buyer (default: the latest purchase's buyer, only when Paykit Server reports the link `connected`. Paykit Server keeps that state per purchase and exposes no per-peer query, so before a buyer's first purchase `server_side` reads `no_purchase_yet` and `linked` is false; use `ready_for_purchase` there and `peers --wait 60` after `purchase`, which exits 1 if the link never connects. +`verify` and `verify-bitkit-seller` assert both: `ready_for_purchase` before the purchase and `linked` after `receive`. ## Limits diff --git a/marketplace/driver/driver.mjs b/marketplace/driver/driver.mjs index 14d4225..c9c5cf9 100644 --- a/marketplace/driver/driver.mjs +++ b/marketplace/driver/driver.mjs @@ -919,12 +919,12 @@ async function peerReport({ fixture, purchases, buyerArg, bundleArg, sellerArg } // The seller is --seller, else the seller of the bundle or latest purchase, else the headless seller. const chosen = sellerArg ? pickSeller(fixture, sellerArg) : null; const ofSeller = (entry) => !chosen || entry.seller === chosen.pubky; - const reader = buyerArg ?? (bundleArg ? undefined : purchases.filter(ofSeller).at(-1)?.reader) ?? fixture.buyer?.pubky; + // --bundle selects a purchase, whose reader is the buyer unless --buyer names another one. + const bundlePurchase = bundleArg ? purchases.find((entry) => entry.bundle_id === bundleArg) : undefined; + if (bundleArg && !bundlePurchase) fail(`unknown bundle ${bundleArg}`); + const reader = buyerArg ?? bundlePurchase?.reader ?? purchases.filter(ofSeller).at(-1)?.reader ?? fixture.buyer?.pubky; if (!reader) fail('no buyer; pass --buyer or run: ./pubky-marketplace seed --buyer headless'); - const purchase = bundleArg - ? purchases.find((entry) => entry.bundle_id === bundleArg) - : purchases.filter((entry) => entry.reader === reader && ofSeller(entry)).at(-1); - if (bundleArg && !purchase) fail(`unknown bundle ${bundleArg}`); + const purchase = bundlePurchase ?? purchases.filter((entry) => entry.reader === reader && ofSeller(entry)).at(-1); const seller = chosen ?? sellerOf(fixture, purchase?.seller); const headless = fixture.buyer?.pubky === reader; @@ -1033,6 +1033,14 @@ async function writeEvidence(evidence, suffix = '') { out(evidence); } +// After the buyer received the Payment Request, the bundle's peers must read as linked. +async function assertLinked(capture, bundle) { + const report = await capture('peers_linked', () => peers(['--bundle', bundle, '--wait', '30'])); + if (!report.linked || report.link.bundle_id !== bundle || report.link.server_side !== 'connected') { + fail(`the peers are not linked after receive: ${JSON.stringify(report.link)}`); + } +} + // The whole journey with the driver in every wallet role. Each step asserts. async function verify() { const evidence = { started_at: new Date().toISOString(), steps: {} }; @@ -1045,6 +1053,9 @@ async function verify() { evidence.seller = { pubky: fixture.seller.pubky, account_xpub: fixture.seller.account_xpub, account_index: fixture.seller.account_index }; evidence.buyer = { pubky: buyer.pubky }; + const peersBefore = await capture('peers_before', () => peers([])); + if (!peersBefore.ready_for_purchase || peersBefore.linked) fail('before the purchase the peers must be ready for a purchase and not linked yet'); + const created = await capture('purchase', () => purchase(['--buyer', 'headless'])); if (created.paykit_status.status !== 'undetected') fail('a new purchase must start undetected'); const bundle = created.bundle_id; @@ -1060,6 +1071,7 @@ async function verify() { log('trust_boundary: ok'); const received = await capture('receive', () => receive([bundle])); + await assertLinked(capture, bundle); const heightBefore = (await chainInfo()).height; const paid = await capture('pay', () => pay([bundle])); if (paid.matched_output_sats !== received.amount_sats || paid.mempool_entries !== 1) { @@ -1215,6 +1227,8 @@ async function verifyBitkitSeller() { const buyer = await createBuyer(standin); evidence.buyer = { pubky: buyer.pubky }; + const peersBefore = await capture('peers_before', () => peers(['--seller', 'bitkit'])); + if (!peersBefore.ready_for_purchase || peersBefore.linked) fail('before the purchase the peers must be ready for a purchase and not linked yet'); const created = await capture('purchase', () => purchase(['--seller', 'bitkit', '--buyer', 'headless'])); if (created.seller !== standin || created.derived_address !== null) fail('a Bitkit seller purchase has no derived address in the ledger'); if (created.paykit_status.status !== 'undetected') fail('a new purchase must start undetected'); @@ -1222,6 +1236,7 @@ async function verifyBitkitSeller() { const expected = await expectedAddress(xpub, 0); const received = await capture('receive', () => receive([bundle])); + await assertLinked(capture, bundle); if (received.address !== expected) fail(`the Payment Request pays ${received.address}, not the stand-in xpub's ${expected}`); const heightBefore = (await chainInfo()).height; const paid = await capture('pay', () => pay([bundle])); diff --git a/pubky-marketplace b/pubky-marketplace index f90d376..f0a9e70 100755 --- a/pubky-marketplace +++ b/pubky-marketplace @@ -160,6 +160,16 @@ up() { echo "Next: ${CLI_NAME} seed" } +# The Pubky testnet keeps its homeserver files and DHT in memory, so it cannot +# restart with its accounts intact. The fixture is disposable as a whole. +down() { + local project + project="$(project_name)" + compose rm -sfv "${FIXTURE_SERVICES[@]}" marketplace-driver + docker volume rm -f "${project}_marketplace_postgres_data" "${project}_marketplace_state" >/dev/null + remove_evidence +} + if [ $# -eq 0 ] || [ "$1" = "-h" ] || [ "$1" = "--help" ]; then show_help exit 0 @@ -177,16 +187,9 @@ case "$command" in echo ;; logs) compose logs -f --tail 100 "${@:-paykit-server}" ;; - down) - # The Pubky testnet keeps its homeserver files and DHT in memory, so it cannot - # restart with its accounts intact. The fixture is disposable as a whole. - project="$(project_name)" - compose rm -sfv "${FIXTURE_SERVICES[@]}" marketplace-driver - docker volume rm -f "${project}_marketplace_postgres_data" "${project}_marketplace_state" >/dev/null - remove_evidence - ;; + down) down ;; reset) - "$0" down + down up ;; seed | info | setup-url | setup-wait | seller-auth | fund | peers | purchase | receive | pay | status | mine | wait | verify | verify-bitkit-seller) From c90240746883cd3a4932a3c93e34882289fc7cc2 Mon Sep 17 00:00:00 2001 From: Ovi Trif Date: Tue, 29 Sep 2026 23:12:15 +0200 Subject: [PATCH 13/13] fix: harden the marketplace fixture handoff and reset after the J1 device run --- README.md | 4 +-- docs/pubky-marketplace.md | 43 +++++++++++++++++++++++++++----- marketplace/driver/driver.mjs | 28 +++++++++++++-------- pubky-marketplace | 47 ++++++++++++++++++++++++++++++++--- 4 files changed, 100 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index bc3c63b..55d27f8 100644 --- a/README.md +++ b/README.md @@ -321,14 +321,14 @@ To make a Bitkit wallet the seller, the wallet approves two Pubky requests for t ./pubky-marketplace seed --buyer none # once per fixture; the headless seller stays unused ./pubky-marketplace setup-url # open android or ios_url in the seller wallet, approve, then: ./pubky-marketplace setup-wait -./pubky-marketplace seller-auth # prints the marketplace grant request, waits for the approval +./pubky-marketplace seller-auth # prints the marketplace grant request (first JSON line), waits for the approval (last line) ./pubky-marketplace purchase --seller bitkit --buyer ./pubky-marketplace mine --bundle # after the buyer pays; add --address if two payments match the amount ./pubky-marketplace wait confirmed ./pubky-marketplace status # payout address, txid, signed Paykit status, purchase state ``` -Both URLs are `pubkyauth://signin_grant?...` links whose one-time secret must stay out of logs and evidence. On Android open each printed `android` command (`adb shell am start -a android.intent.action.VIEW -d ''`). On iOS the setup request opens through the printed `ios_url` (`xcrun simctl openurl ''`); the marketplace grant has no iOS deep link, so put its `auth_url` on the simulator clipboard (`printf %s '' | xcrun simctl pbcopy `) and use Scan QR Code, then Paste QR Code (E2E builds also have Enter QRCode String). The marketplace request goes through the testnet's HTTP relay on `localhost:15412` (published by the fixture; on Android it is one of the `adb reverse` ports above) and expires after about five minutes; pass `--relay https://httprelay.pubky.app/inbox/` to use the public relay instead. The setup request still uses the public relay and needs outbound internet. +Both URLs are `pubkyauth://signin_grant?...` links whose one-time secret must stay out of logs and evidence. On Android run each printed `android` command (`adb shell "am start -a android.intent.action.VIEW -d ''"`; keep the double quotes, or the device shell cuts the URL at the first `&`). With several devices or emulators pass `--serial ` to `setup-url` or `seller-auth` and the command gets `-s `. On iOS the setup request opens through the printed `ios_url` (`xcrun simctl openurl ''`); the marketplace grant has no iOS deep link, so put its `auth_url` on the simulator clipboard (`printf %s '' | xcrun simctl pbcopy `) and use Scan QR Code, then Paste QR Code (E2E builds also have Enter QRCode String). The marketplace request goes through the testnet's HTTP relay on `localhost:15412` (published by the fixture; on Android it is one of the `adb reverse` ports above) and expires after about five minutes; pass `--relay https://httprelay.pubky.app/inbox/` to use the public relay instead. The setup request still uses the public relay and needs outbound internet. The fixture cannot check a Bitkit seller's payout address against the wallet's xpub, because Paykit Server keeps the xpub and the derived address to itself. It checks that the Payment Request address is a regtest native SegWit address, that exactly the amount is paid to it on chain, and Paykit Server's signed status (`detected`, then `confirmed` with a matching amount). That the address belongs to the wallet shows in the seller app: its balance rises by the amount and the received activity carries the same txid as `status`. `./pubky-marketplace verify-bitkit-seller` runs the whole path with a headless stand-in for the wallet and does check the payout against the stand-in's xpub. diff --git a/docs/pubky-marketplace.md b/docs/pubky-marketplace.md index 49ed0ba..07e4af5 100644 --- a/docs/pubky-marketplace.md +++ b/docs/pubky-marketplace.md @@ -124,8 +124,20 @@ setup state for it. `setup-wait` then checks the wallet that approved the setup `purchase --seller bitkit` (or `--seller `) refuses until it is. `peers --seller bitkit`, `status`, `wait` and `mine --bundle` work on the purchase's own seller. -Handoff. Android opens either URL with `adb shell am start -a android.intent.action.VIEW -d ''` -(the app enables its `pubkyauth://signin_grant` handler while it holds a Bitkit-generated Pubky identity). +`seller-auth` prints one compact JSON object per line: `awaiting_approval` (with `auth_url`, `android`, `ios`) at +once, then `approved` when the wallet has approved. Read the request from the first line (`... | head -1 | jq +-r .auth_url`, or `jq -r 'select(.status == "awaiting_approval") | .auth_url'` over the stream) and collect both +with `jq -s`. `info` shows the result as `bitkit_seller.marketplace_grant` (`locks.app /pub/locks.app/:rw`) +and `bitkit_seller.setup_completed_at` (null until `setup-wait` has seen the setup complete), next to +`bitkit_seller.pubky` and `kind`; `seller.pubky` is the unused headless seller. + +Handoff. Android opens either URL with the printed `android` command, +`adb shell "am start -a android.intent.action.VIEW -d ''"` (the app enables its +`pubkyauth://signin_grant` handler while it holds a Bitkit-generated Pubky identity). The double quotes around +the whole device command matter: adb hands its arguments to the device shell as one line, so single quotes +outside them are lost and the shell cuts the URL at the first `&`, leaving the app only `caps=...`. With more +than one device or emulator, add `--serial ` to `setup-url` or `seller-auth` +(`adb devices` lists them) and the printed command becomes `adb -s shell "..."`. iOS registers no `pubkyauth` scheme. Its `bitkit://pubky-auth/setup?` handoff requires the claim, so it opens only the setup request (`ios_url`). The marketplace grant is entered in the app: Scan QR Code, then Paste QR Code with the URL on the simulator clipboard, or Enter QRCode String in E2E builds. An optional iOS app @@ -138,7 +150,7 @@ its creators, so for a Bitkit seller the fixture cannot derive the expected addr - verified by the fixture: the Payment Request address a headless buyer receives (`receive`) is a regtest native SegWit address; the mempool holds a transaction paying exactly that address and the purchase amount (`pay`, `mine --bundle`; for an app buyer the address comes from `mine --address` or from the one p2wpkh - output of exactly the amount); Paykit Server's signed status goes `detected` and then `confirmed` with a + output of exactly the amount, both exercised with an app buyer on Android and iOS); Paykit Server's signed status goes `detected` and then `confirmed` with a matching amount and one confirmation, which means its own derived address for the invoice received the payment; the purchase reaches `completed`. - verified only in the seller app: that the address is derived from the wallet's own xpub. The seller wallet @@ -155,12 +167,25 @@ whole: `down` removes the fixture containers, its Postgres and its state volume, services do not restart on their own. After a crash or a Docker restart, run `./pubky-marketplace reset`. The regtest chain from the base stack is left alone. +`up`, `reset` and every driver command fetch the pinned sources under `.marketplace/sources` when a tree is +missing, even if the images exist: the driver image is wired to the paykit-server build context, and compose +refuses to run the driver without it (a fresh clone on a host that built before). Nothing is fetched when the +trees are there. + +## Output + +Command results are JSON on stdout. Progress lines (`reset`, `up` and `down` announce each phase) and compose's own +messages go to stderr. On a host without buildx the wrapper sets `COMPOSE_BAKE=false`, so compose does not print +its "configured to build using Bake, but buildx isn't installed" warning on every run; set `COMPOSE_BAKE` yourself +to keep your own value. `2>&1 | jq` therefore works there, and `seller-auth` prints one object per line (see above). + ## Purchase states `purchase` returns once Paykit Server has durably accepted the invoice and reports `delivery`: `queued` or `sent` from the server's outbox metrics, or `failed`. The Payment Request id exists only -in the SDK's delivery, so `receive` (headless buyer) reports it; for a Bitkit buyer read it from the -request row in the app. The purchase state moves `created`, `delivered`, `paid`, `payment_detected`, +in the SDK's delivery, so `receive` (headless buyer) reports it. For a Bitkit buyer the delivery goes to the +app, and neither Paykit Server's API nor the chain hands the id to the fixture, so `status` prints +`payment_request_id: null`; read the id from the request row in the app. The purchase state moves `created`, `delivered`, `paid`, `payment_detected`, `completed`. `completed` means a signed Paykit status of `confirmed` with a matching amount at one or more confirmations, the gate Locks applies with `minimum_confirmations = 1`. @@ -168,7 +193,9 @@ The derived address is the seller xpub's external child `0/`, where `n` is th purchases for that seller, checked through `bitcoind deriveaddresses`. A headless `receive` compares it with the address in the delivered Payment Request. A Bitkit seller's xpub is not in the fixture, so its purchases have `derived_address` null and a `payout_address` learned from the Payment Request, `mine --address` -or the transaction that pays the exact amount (`payout_address_source` says which). +or the transaction that pays the exact amount (`payout_address_source` says which: `payment_request`, `operator` +or `amount_match`). With an app buyer both `amount_match` and `mine --bundle` without `--address` have been +exercised, on Android and on iOS. `mine --bundle` mines one block only after it finds the purchase transaction in the mempool. The headless buyer's `pay` records its txid; for a buyer that pays from an app, `mine --bundle` looks for the transaction that pays @@ -191,6 +218,10 @@ prints, for the seller and for the buyer (default: the reader of the purchase `- only when Paykit Server reports the link `connected`. Paykit Server keeps that state per purchase and exposes no per-peer query, so before a buyer's first purchase `server_side` reads `no_purchase_yet` and `linked` is false; use `ready_for_purchase` there and `peers --wait 60` after `purchase`, which exits 1 if the link never connects. + +`link.buyer_side` is the buyer's own view of the link and only the headless buyer can be asked. For an app buyer +it reads `not_observable`: the fixture holds no key for the app's end of the link, the app shows that side +(Linked), and `linked` rests on Paykit Server's side (`server_side: connected`) plus the two receiver markers. `verify` and `verify-bitkit-seller` assert both: `ready_for_purchase` before the purchase and `linked` after `receive`. ## Limits diff --git a/marketplace/driver/driver.mjs b/marketplace/driver/driver.mjs index c9c5cf9..2bd7f1f 100644 --- a/marketplace/driver/driver.mjs +++ b/marketplace/driver/driver.mjs @@ -58,6 +58,7 @@ const EXPECTED_ENDPOINT = 'btc-regtest-p2wpkh'; const log = (message) => process.stderr.write(`[marketplace] ${message}\n`); const out = (value) => process.stdout.write(`${JSON.stringify(value, null, 2)}\n`); +const outLine = (value) => process.stdout.write(`${JSON.stringify(value)}\n`); const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); const b64url = (bytes) => Buffer.from(bytes).toString('base64url'); @@ -420,7 +421,7 @@ async function approveSetupAs(authUrl, identitySeed, xpub, accountIndex) { } } -async function setupUrl() { +async function setupUrl(args = []) { await waitForPaykit(); const { flowId, authUrl } = await beginSetup(); const params = authParams(authUrl); @@ -429,7 +430,7 @@ async function setupUrl() { auth_url: authUrl, // Android opens the grant URL directly. iOS has no pubkyauth handler: hand the same query to Bitkit as // bitkit://pubky-auth/setup?. - android: `adb shell am start -a android.intent.action.VIEW -d '${authUrl}'`, + android: androidOpen(authUrl, flag(args, '--serial')), ios_url: `bitkit://pubky-auth/setup?${authUrl.slice(authUrl.indexOf('?') + 1)}`, client_id: params.get('cid'), claim: params.get('x-bitkit-claim'), @@ -934,7 +935,8 @@ async function peerReport({ fixture, purchases, buyerArg, bundleArg, sellerArg } const state = await signedPost('/connections/status', { creator: seller.pubky, bundle_id: purchase.bundle_id }); serverSide = state.status === 200 ? state.json.state : `unavailable_http_${state.status}`; } - let buyerSide = null; + // An app buyer holds its own end of the link inside the app; the fixture has no key for it and cannot inspect it. + let buyerSide = 'not_observable'; if (headless) { const inspected = await runHelper( 'paykit-reader-demo', @@ -965,7 +967,8 @@ async function peerReport({ fixture, purchases, buyerArg, bundleArg, sellerArg } // Paykit Server's view of its link to the buyer: none, handshake, connected, recovery_required or blocked. // The server keeps it per purchase, so before the first purchase for this buyer it reads no_purchase_yet. server_side: serverSide, - // The headless buyer's own view of its link to the server; a Bitkit buyer shows this in the app. + // The headless buyer's own view of its link to the server. For an app buyer it reads not_observable: the app + // shows that side (Linked), and `linked` below rests on Paykit Server's side alone. buyer_side: buyerSide, }, // Both identities publish their receiver markers and the seller's setup authority is usable: a purchase can be delivered. @@ -1089,7 +1092,11 @@ async function verify() { // ---------------------------------------------------------------- Bitkit seller -const androidOpen = (authUrl) => `adb shell am start -a android.intent.action.VIEW -d '${authUrl}'`; +// adb joins the arguments after `shell` and the device shell splits them again, so a URL in single quotes +// outside the double quotes loses its quotes and is cut at the first `&`. The whole device command goes in one +// double-quoted string with the URL single-quoted inside it. With several devices or emulators pass --serial. +const androidOpen = (authUrl, serial) => + `adb${serial ? ` -s ${serial}` : ''} shell "am start -a android.intent.action.VIEW -d '${authUrl}'"`; // The marketplace's request for a write grant on the seller's /pub/locks.app/, shown to the seller as a Pubky // auth request (the role Locks plays). The flow polls the relay as long as this process runs. @@ -1149,16 +1156,17 @@ async function adoptBitkitSeller(session, kind) { async function sellerAuth(args) { const relay = flag(args, '--relay') ?? LOCKS_RELAY; const seconds = Number(flag(args, '--timeout') ?? 300); - if (!Number.isFinite(seconds) || seconds <= 0) fail('usage: seller-auth [--relay ] [--timeout ]'); + if (!Number.isFinite(seconds) || seconds <= 0) fail('usage: seller-auth [--relay ] [--timeout ] [--serial ]'); await readFixture(); const { flow, authUrl } = await startLocksGrant(relay); - out({ + // One compact JSON object per line: the first line is the request, the last one the approval. + outLine({ status: 'awaiting_approval', client_id: LOCKS_CLIENT_ID, capabilities: LOCKS_CAPS, relay, auth_url: authUrl, - android: androidOpen(authUrl), + android: androidOpen(authUrl, flag(args, '--serial')), // iOS registers no pubkyauth handler and bitkit://pubky-auth/setup accepts only the setup request: put // auth_url on the simulator's clipboard, then Scan QR Code, Paste QR Code (E2E builds: Enter QRCode String). ios: { @@ -1169,7 +1177,7 @@ async function sellerAuth(args) { }); const session = await awaitGrant(flow, seconds); const { seller, paykitSetup } = await adoptBitkitSeller(session, 'bitkit'); - out({ + outLine({ status: 'approved', seller: seller.pubky, client_id: seller.client_id, @@ -1260,7 +1268,7 @@ const commands = { init: () => init(), seed, info: async () => out(await publicInfo()), - 'setup-url': () => setupUrl(), + 'setup-url': (args) => setupUrl(args), 'setup-wait': (args) => setupWait(args[0]), 'seller-auth': sellerAuth, fund, diff --git a/pubky-marketplace b/pubky-marketplace index f0a9e70..c2733e3 100755 --- a/pubky-marketplace +++ b/pubky-marketplace @@ -37,13 +37,15 @@ Fixture: seed [--buyer headless|none] Mine to maturity, create the seller wallet and identity, complete the watch-only setup, optionally create a headless buyer info Public fixture facts: homeserver key, seller, buyer, ports - setup-url New watch-only setup auth URL for a Bitkit wallet to approve + setup-url [--serial ] New watch-only setup auth URL for a Bitkit wallet to approve; --serial + adds -s to the printed adb command (needed with several devices) setup-wait Wait until an approved setup flow completes; with a Bitkit seller recorded, check the same wallet approved it - seller-auth [--relay ] [--timeout ] + seller-auth [--relay ] [--timeout ] [--serial ] Bitkit seller: print the marketplace write grant (/pub/locks.app/) for the seller wallet to approve, with the Android and iOS handoff, wait for it - and keep the grant session for purchase --seller bitkit + and keep the grant session for purchase --seller bitkit. Prints one JSON + object per line: awaiting_approval first, approved last fund
[sats] Send regtest coins (default 1000000 sats) and mine one block peers [--seller headless|bitkit|] [--buyer ] [--bundle ] [--wait ] Report the seller and buyer as linked peers: receiver markers, setup @@ -68,9 +70,27 @@ EOF compose() { "${COMPOSE[@]}" "$@"; } +# Progress goes to stderr, so stdout stays the command's own output. +progress() { echo "[${CLI_NAME}] $*" >&2; } + +# The driver image takes the helper binaries from the paykit-server image through `additional_contexts: +# paykit: service:paykit-server`, so every compose command that may build or run the driver prepares the +# paykit-server build context, and compose stops when .marketplace/sources is missing, even with all the images +# present (a fresh clone on a host that built before). Fetch the pinned trees whenever one is missing. This is +# quiet and offline when they are there; keeping the driver's link to that context stays the simpler wiring than +# feeding it the helpers by another route, which would tie the driver image to an image name. +ensure_sources() { + if [ ! -f "$SOURCES/paykit-server/Dockerfile.local" ] || [ ! -d "$SOURCES/paykit-rs/paykit-lib" ] || + [ ! -d "$SOURCES/paykit-rs/paykit-sdk" ] || [ ! -d "$SOURCES/locks" ]; then + progress "fetching the pinned sources under $SOURCES" + fetch_sources >&2 + fi +} + # The driver runs as root inside its container (it owns the root-only state volume). It hands the files it # writes to the host mounts (.marketplace/evidence) to the calling user, so the host can remove them later. driver() { + ensure_sources mkdir -p .marketplace/evidence compose run --rm --no-deps --quiet-build -T \ -e MARKETPLACE_HOST_UID="$(id -u)" -e MARKETPLACE_HOST_GID="$(id -g)" marketplace-driver "$@" @@ -151,10 +171,18 @@ wait_ready() { up() { mkdir -p .marketplace/evidence - images_present || build + ensure_sources + if ! images_present; then + progress "building the images" + build + fi + progress "starting the chain, Postgres and the Pubky testnet" compose up -d "${CHAIN_SERVICES[@]}" marketplace-postgres pubky-testnet + progress "initializing the driver state" driver init + progress "starting Paykit Server" compose up -d paykit-server + progress "waiting for Paykit Server to be ready" wait_ready echo "Electrum: tcp://127.0.0.1:60001" echo "Next: ${CLI_NAME} seed" @@ -165,8 +193,11 @@ up() { down() { local project project="$(project_name)" + progress "removing the fixture containers" compose rm -sfv "${FIXTURE_SERVICES[@]}" marketplace-driver + progress "removing the Postgres and state volumes" docker volume rm -f "${project}_marketplace_postgres_data" "${project}_marketplace_state" >/dev/null + progress "removing the evidence" remove_evidence } @@ -178,6 +209,12 @@ fi command="$1" shift +# Without buildx, compose falls back to the classic builder but says so on every run. That line lands on stderr +# and breaks `2>&1 | jq`. Set COMPOSE_BAKE yourself to keep it as it is. +if [ -z "${COMPOSE_BAKE+x}" ] && ! docker buildx version >/dev/null 2>&1; then + export COMPOSE_BAKE=false +fi + case "$command" in build) build ;; up) up ;; @@ -189,8 +226,10 @@ case "$command" in logs) compose logs -f --tail 100 "${@:-paykit-server}" ;; down) down ;; reset) + progress "reset: down, then up (a first build or a slow host takes minutes)" down up + progress "reset: done; seed next" ;; seed | info | setup-url | setup-wait | seller-auth | fund | peers | purchase | receive | pay | status | mine | wait | verify | verify-bitkit-seller) driver "$command" "$@"