Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,21 @@ jobs:
- name: Audit runtime dependencies
run: yarn audit --groups dependencies
- run: yarn test --runInBand
- name: Start disposable backup replica set
run: |
docker run -d --name keeper-backup-mongo -p 127.0.0.1:27028:27017 \
mongo:7.0@sha256:9854f7139445d766a9523571d6f047530c45547460ffcf8259eb2bf4264632ca \
--replSet keeperQA --bind_ip_all
for attempt in $(seq 1 30); do
if docker exec keeper-backup-mongo mongosh --quiet --eval 'try { rs.status(); } catch (e) { if (e.code !== 94) throw e; rs.initiate({_id: "keeperQA", members: [{_id: 0, host: "127.0.0.1:27017"}]}); } quit(db.hello().isWritablePrimary ? 0 : 1)'; then
break
fi
sleep 2
done
docker exec keeper-backup-mongo mongosh --quiet --eval 'quit(db.hello().isWritablePrimary ? 0 : 1)'
- name: Backup revision and account-isolation regression tests
run: yarn test:backup
- name: Remove disposable backup replica set
if: always()
run: docker rm -f keeper-backup-mongo || true
# yarn test compiles TypeScript before Jest. SonarCloud is deferred.
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Use Node 22 and Yarn 1 with the committed lockfile. Run the local backend in `compose.local.yaml` or with a disposable MongoDB as described in [README.md](README.md). Keep tests and examples self-contained; do not depend on Keeper production accounts, buckets, FCM projects, or credentials.

Before opening a pull request, run `yarn test --runInBand` and the current-source scan in `scripts/check-source-secrets.py` using Gitleaks 8.30.1. Do not commit `.env`, service-account JSON, private keys, request bodies, user data, or screenshots from real users. Synthetic fixtures must be clearly disposable.
Before opening a pull request, run `yarn test --runInBand`, `yarn test:backup` against the disposable `keeperQA` replica set (see README) and the current-source scan in `scripts/check-source-secrets.py` using Gitleaks 8.30.1. Do not commit `.env`, service-account JSON, private keys, request bodies, user data, or screenshots from real users. Synthetic fixtures must be clearly disposable.

Run `yarn audit --groups dependencies` for runtime dependencies; CI requires this check to pass. On 6 October 2026 the committed lockfile had no reported runtime advisories. A full audit also includes two unpatched denial-of-service advisories in Jest's development dependencies: [braces](https://github.com/advisories/GHSA-vfj7-8cjw-p6xm) and [sprintf-js](https://github.com/advisories/GHSA-hp3w-g68c-fv3c). Their inputs here are checked-in test patterns and tool configuration, not HTTP requests. Run contribution tests in a disposable environment without hosted credentials; CI has a 15-minute job limit. These warnings are recorded, not suppressed, and should be reviewed when updating test tools.

Expand Down
26 changes: 17 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Relay is the database, notification and integration backend used by [Bitcoin Kee

## Local backend without hosted credentials

Use Docker Engine and Compose. The local stack has a disposable MongoDB and binds Relay only to loopback. Local mode blocks hosted integration routes.
Use Docker Engine and Compose. The local stack has a disposable single-member MongoDB replica set (`keeper-local`) and binds Relay only to loopback. Backup snapshot and repair use transactions, so a standalone MongoDB cannot serve those routes. Local mode blocks hosted integration routes.

From a checkout of this clean source:

Expand All @@ -15,22 +15,22 @@ curl --fail http://127.0.0.1:3000/health

The health response reports `ready: true`, `mode: local`, and the capabilities available in this stack. Relay HTTP and Socket.IO are on `127.0.0.1:3000` and `127.0.0.1:4002`. The local database has a named volume; `docker compose -f compose.local.yaml down` keeps it. This stack does not send push notifications, sign Apple offers, use the testnet faucet, upload screenshots, or contact hosted purchase and AI services. Unsupported integration routes return `503`.

For a host Node setup, use Node 22, Python 3, Yarn 1 and a **disposable local** MongoDB. Copy `.env.example` to an ignored `.env`, set `LOCAL_DEV=true`, `ENVIRONMENT=DEVELOPMENT`, `BITCOIN_NETWORK=TESTNET`, and a local `DATABASE_URL`, then run `yarn install --frozen-lockfile --ignore-scripts`, `yarn compile`, and `yarn start`. Never copy a hosted environment file into this checkout. Local mode refuses mainnet, non-development environments and nonlocal MongoDB hosts.
For a host Node setup, use Node 22, Python 3, Yarn 1 and a **disposable local single-member replica set** of MongoDB. Copy `.env.example` to an ignored `.env`, set `LOCAL_DEV=true`, `ENVIRONMENT=DEVELOPMENT`, `BITCOIN_NETWORK=TESTNET`, and a local `DATABASE_URL` with the replica-set name in `?replicaSet=...`, then run `yarn install --frozen-lockfile --ignore-scripts`, `yarn compile`, and `yarn start`. Never copy a hosted environment file into this checkout. Local mode refuses mainnet, non-development environments and nonlocal MongoDB hosts.

For app development, configure the mobile app to use these loopback Relay and channel URLs (Android emulator host routing may require `adb reverse`). The optional setup below also starts a local testnet Signing Server. Supported local Relay flows require no production provider credentials.

## Pair with the public Signing Server

Use Docker Compose 2.24.4 or newer for this optional stack. The public Signing Server supports installation, compilation and unit tests on its own. Its local Docker adapter is published in [app PR #7014](https://github.com/KeeperCommunity/bitcoin-keeper/pull/7014), which is still under review. The following commands pin that public adapter and the public Signing Server revision it supports. The app PR's full bootstrap now also pins this public Relay source.
Use Docker Compose 2.24.4 or newer for this optional stack. The public Signing Server supports installation, compilation and unit tests on its own. Its local Docker adapter is published in [app PR #7030](https://github.com/KeeperCommunity/bitcoin-keeper/pull/7030), which is still under review. The following commands pin that public adapter and the public Signing Server revision it supports. The app PR's full bootstrap now also pins this public Relay source.

From inside this Relay checkout:

```sh
RELAY_SOURCE_DIR="$PWD"
git clone --filter=blob:none --sparse --branch codex/contributor-dev-environment https://github.com/KeeperCommunity/bitcoin-keeper.git ../keeper-public-local-adapter
git clone --filter=blob:none --sparse --branch codex/r3-public-contributor-sprint https://github.com/KeeperCommunity/bitcoin-keeper.git ../keeper-public-local-adapter
cd ../keeper-public-local-adapter
git sparse-checkout set dev/local-backend
git checkout 7519452ce32edd6579284ec278c6ae98b796b931
git checkout 58208450c88c0aabd4a44df52630c35843c16484
./dev/local-backend/dev prepare-signing
export SIGNING_SOURCE_DIR="$PWD/dev/local-backend/.sources/signing"
cd "$RELAY_SOURCE_DIR"
Expand All @@ -39,11 +39,19 @@ curl --fail http://127.0.0.1:3000/health
curl --fail http://127.0.0.1:3003/health
```

This prepares Signing Server revision `b82dc4f4a8f75676b70b12545d59c7906556a92b`, verifies the adapter checksums, and starts the services behind a loopback gateway on ports 3000, 4002 and 3003. Relay, Mongo and Signing Server share an internal network with no external egress; only the gateway has host access. Signing Server creates independent disposable development identities in its named volume; preserve that volume together with Mongo when reusing local records. Do not export its contents. Local push delivery and scheduled jobs are disabled; email is captured inside the local signing volume and is never delivered.
This prepares Signing Server revision `3cc7728c27328e7b0358d1f91a32d8d13feba90a`, verifies the adapter checksums, and starts the services behind a loopback gateway on ports 3000, 4002 and 3003. Relay, Mongo and Signing Server share an internal network with no external egress; only the gateway has host access. Signing Server creates independent disposable development identities in its named volume; preserve that volume together with Mongo when reusing local records. Do not export its contents. Local push delivery and scheduled jobs are disabled; email is captured inside the local signing volume and is never delivered.

Stop this stack with `docker compose -f compose.local.yaml -f compose.local-with-signing.yaml down`. This retains the Mongo and signing volumes. The pairing verifies local service availability; bitcoin signing, recovery, inheritance and timer-dependent behavior need their own separate checks. For the complete app contributor stack, run `./dev/local-backend/dev up` from the pinned app checkout above.
Stop this stack with `docker compose -f compose.local.yaml -f compose.local-with-signing.yaml down`. This retains the Mongo and signing volumes. The pairing verifies local service availability; bitcoin signing, recovery, inheritance and timer-dependent behavior need their own separate checks. For the complete app contributor stack, use public contributor PR #7030 at `58208450c88c0aabd4a44df52630c35843c16484` and run `./dev/local-backend/dev up` from that app checkout. Its `dev/local-backend/sources.lock.json` pins Relay `ee935359f22ff7932ea0a62e9084d4b49801f4aa`, which includes the backup snapshot and repair protocol, revision guards, signer-map cleanup, and old-client incremental field aliases. The exact pins describe tested source; source PR approval and a successful local stack do not establish hosted deployment parity.

The paired setup was checked on October 6, 2026 with the public revisions above: both health endpoints, synthetic Relay record create/read/update/clear, Socket.IO polling and WebSocket connections, and rejection of an unauthenticated Signing Server setup request passed. These checks used isolated local ports and disposable volumes.
The earlier public-source paired setup was checked on October 6, 2026 with isolated local ports and disposable volumes: both health endpoints, synthetic Relay record create/read/update/clear, Socket.IO polling and WebSocket connections, and rejection of an unauthenticated Signing Server setup request passed. The app contributor CI verifies its pinned stack from public sources and also exercises backup snapshot/repair, stale revisions, content changing from A to B and back to A, and persistence across service restarts. Check the linked app PR for results at the pinned revision; the earlier setup check does not attest to a newer stack.

## Backup snapshot and repair

`POST /getBackupSnapshot` reads an account's encrypted recovery records in one database snapshot and returns a SHA-256 `revision` covering encrypted content plus a retained account mutation generation, the app image, account-scoped vault images, labels, and any unavailable vault IDs. It does not replace the backup. Invalid account identifiers return `400`; unavailable storage returns a fixed `503` response.

`POST /repairAppBackup` requires the revision just read. A stale revision returns `409` with `BACKUP_CHANGED`; the existing backup stays intact. The generation advances transactionally on backup mutations and survives Delete Backup, so returning from content A to B and back to A never reuses an earlier token. Explicit current-state replacement must use the existing app verification/resubmission flow. Legacy full replacements cannot bypass this requirement after a revisioned repair. Collaborative accounts retain independently encrypted vault copies; no other account's ciphertext is substituted for a missing copy.

The source includes the guarded backup implementation used by the mobile recovery protocol. Publishing it does not deploy a backend or migrate a hosted database. Before any hosted adoption, maintainers must review transaction/replica-set support and the account-scoped unique index with staging fixtures.

## Hosted integrations

Expand All @@ -58,7 +66,7 @@ Deployment and provider permissions are maintainer responsibilities. Do not use

## Checks and contribution

Run `yarn test --runInBand` (which compiles first) and `python3 scripts/check-source-secrets.py` with Gitleaks 8.30.1. The database tests use `READINESS_MONGO_URL` pointing at a disposable MongoDB. See [CONTRIBUTING.md](CONTRIBUTING.md) and [SECURITY.md](SECURITY.md).
Run `yarn test --runInBand` (which compiles first) and `python3 scripts/check-source-secrets.py` with Gitleaks 8.30.1. The database tests use `READINESS_MONGO_URL` pointing at a disposable MongoDB. Run `yarn test:backup` with a disposable loopback replica set named `keeperQA` (default `127.0.0.1:27028`, or set `BACKUP_TEST_MONGO_URL` to its loopback address). These tests create and drop only uniquely named test databases; they never read `.env` or hosted credentials. CI runs both suites. See [CONTRIBUTING.md](CONTRIBUTING.md) and [SECURITY.md](SECURITY.md).

## License

Expand Down
6 changes: 0 additions & 6 deletions compose.local-with-signing.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,6 @@
# Optional pairing with the public SigningServer local adapter.
# Follow README.md to prepare that source before starting this stack.
services:
mongo:
healthcheck:
test: [CMD, mongosh, --quiet, --eval, "quit(db.adminCommand('ping').ok ? 0 : 1)"]
interval: 5s
timeout: 5s
retries: 20
relay:
ports: !reset []
environment:
Expand Down
11 changes: 9 additions & 2 deletions compose.local.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
services:
mongo:
image: mongo:7.0@sha256:9854f7139445d766a9523571d6f047530c45547460ffcf8259eb2bf4264632ca
command: [mongod, --replSet, keeper-local, --bind_ip_all]
healthcheck:
test: [CMD, mongosh, --quiet, --eval, "try { rs.status(); } catch (e) { if (e.code !== 94) throw e; rs.initiate({_id: 'keeper-local', members: [{_id: 0, host: 'mongo:27017'}]}); } quit(db.hello().isWritablePrimary ? 0 : 1)"]
interval: 5s
timeout: 5s
retries: 20
volumes:
- relay-local-mongo:/data/db
networks:
Expand All @@ -10,12 +16,13 @@ services:
context: .
dockerfile: Dockerfile.local
depends_on:
- mongo
mongo:
condition: service_healthy
environment:
LOCAL_DEV: "true"
ENVIRONMENT: DEVELOPMENT
BITCOIN_NETWORK: TESTNET
DATABASE_URL: mongodb://mongo:27017/keeper-relay-local
DATABASE_URL: mongodb://mongo:27017/keeper-relay-local?replicaSet=keeper-local
PORT: "3000"
WS_PORT: "4002"
ports:
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
"dev": "yarn compile && node dist/index.js",
"test": "npm run compile && jest ./dist --detectOpenHandles",
"gcp-build": "tsc -p .",
"test:backup": "node --test tests/*.test.cjs",
"check:secrets": "python3 scripts/check-source-secrets.py"
},
"jest": {
Expand Down
25 changes: 25 additions & 0 deletions src/db.ts
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,18 @@ class Database {
vault: { type: Object, required: true },
});

// Each app encrypts its copy with its own Recovery Key. Collaborative
// participants share the canonical vault ID, but must retain separate images.
private appVaultImageSchema = new Schema({
vaultId: { type: String, required: true, index: true },
vaultShellId: { type: String, index: true },
appId: { type: String, required: true },
isArchived: { type: Boolean, required: true, default: false },
signerIds: [{ type: String, required: true }],
scheme: { type: {}, required: false },
vault: { type: Object, required: true },
});

private appImageSchema = new Schema({
appId: { type: String, required: true, unique: true },
publicId: { type: String, required: true, unique: true },
Expand All @@ -139,6 +151,10 @@ class Database {
subscription: { type: String, required: false },
version: { type: String, required: true },
nodes: [{ type: String, required: false }],
// Once repaired, unversioned full replacements must never overwrite this image.
backupRevisionRequired: { type: Boolean, default: false },
// Retained by Delete Backup, preventing stale A → B → A revisions.
backupGeneration: { type: Number, default: 0, min: 0 },
});

private labelSchema = new Schema({
Expand Down Expand Up @@ -311,6 +327,7 @@ class Database {
{ unique: true },
);
this.articleChunkSchema.index({ ragTimestamp: -1 });
this.appVaultImageSchema.index({ appId: 1, vaultId: 1 }, { unique: true });
this.faucetQuotaSchema.index({ appId: 1, utcDate: 1 }, { unique: true });
this.faucetQuotaSchema.index(
{ createdAt: 1 },
Expand Down Expand Up @@ -349,6 +366,14 @@ class Database {
return mongoose.model("Vault Image", this.vaultImageSchema);
};

public getAppVaultImageModel = () => {
return mongoose.model(
"App Vault Image",
this.appVaultImageSchema,
"appVaultImages",
);
};

public getVaultMapModel = () => {
return mongoose.model("Vault Map", this.vaultMapSchema);
};
Expand Down
1 change: 1 addition & 0 deletions src/localDevelopment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ const localDatabaseRoutes = new Set([
"/getvaultmetadata", "/getsigneridinfo", "/migratexfps", "/modifylabels",
"/getmessages", "/getsubscriptiondetails", "/updatecontactskey",
"/createremotekey", "/getremotekey", "/backupallsignersandvaults",
"/getbackupsnapshot", "/repairappbackup",
"/deletebackup", "/updatecollaborativechannel", "/fetchcollaborativechannel",
"/gethardwarereferrallinks", "/getactivecampaign", "/getadvisors",
]);
Expand Down
Loading
Loading