diff --git a/README.md b/README.md index b8a65aa..e685c13 100644 --- a/README.md +++ b/README.md @@ -47,11 +47,11 @@ The earlier public-source paired setup was checked on October 6, 2026 with isola ## Backup snapshot and repair -`POST /getBackupSnapshot` reads an account's encrypted recovery records in one database snapshot and returns a SHA-256 `revision` covering encrypted content plus a retained account mutation generation, the app image, account-scoped vault images, labels, and any unavailable vault IDs. It does not replace the backup. Invalid account identifiers return `400`; unavailable storage returns a fixed `503` response. +`POST /getBackupSnapshot` reads an account's encrypted recovery records in one database snapshot and returns a SHA-256 `revision` covering encrypted content plus a retained account mutation generation, the app image, account-scoped vault images, labels, and unavailable vault or label IDs. Missing labels and historically shared label IDs are withheld from generic recovery; shared IDs require an operator migration with ownership evidence. It does not replace the backup. Invalid account identifiers return `400`; unavailable storage returns a fixed `503` response. `POST /repairAppBackup` requires the revision just read. A stale revision returns `409` with `BACKUP_CHANGED`; the existing backup stays intact. The generation advances transactionally on backup mutations and survives Delete Backup, so returning from content A to B and back to A never reuses an earlier token. Explicit current-state replacement must use the existing app verification/resubmission flow. Legacy full replacements cannot bypass this requirement after a revisioned repair. Collaborative accounts retain independently encrypted vault copies; no other account's ciphertext is substituted for a missing copy. -The source includes the guarded backup implementation used by the mobile recovery protocol. Publishing it does not deploy a backend or migrate a hosted database. Before any hosted adoption, maintainers must review transaction/replica-set support and the account-scoped unique index with staging fixtures. +The source includes the guarded backup implementation used by the mobile recovery protocol. Publishing it does not deploy a backend or migrate a hosted database. Before any hosted adoption, maintainers must review transaction/replica-set support and the account-scoped unique index with staging fixtures. The shared-label check also needs the `AppImages` multikey index `{ labels: 1 }`: build and verify that index on existing hosted databases before enabling the new snapshot route, then check its query plan and historical shared-reference counts in staging. A source change alone does not guarantee the index exists when automatic index creation is disabled. ## Hosted integrations @@ -68,6 +68,8 @@ Deployment and provider permissions are maintainer responsibilities. Do not use Run `yarn test --runInBand` (which compiles first) and `python3 scripts/check-source-secrets.py` with Gitleaks 8.30.1. The database tests use `READINESS_MONGO_URL` pointing at a disposable MongoDB. Run `yarn test:backup` with a disposable loopback replica set named `keeperQA` (default `127.0.0.1:27028`, or set `BACKUP_TEST_MONGO_URL` to its loopback address). These tests create and drop only uniquely named test databases; they never read `.env` or hosted credentials. CI runs both suites. See [CONTRIBUTING.md](CONTRIBUTING.md) and [SECURITY.md](SECURITY.md). +The optional paired HTTP test uses a compatible public Keeper app checkout for its actual AES encryption/decryption exports. With a disposable loopback MongoDB replica set named `keeperQA`, run `KEEPER_APP_SOURCE=/path/to/bitcoin-keeper BACKUP_TEST_MONGO_URL=mongodb://127.0.0.1:27028 yarn test:automatic-http`. It mounts the actual local-mode allowlist and the exact `/getBackupSnapshot`, `/revisionedBackupMutation` and `/getAppImage` production handler source on loopback Express with the actual Relay service graph, Mongoose models and transactions. Local mode now permits the revisioned mutation route while hosted provider routes remain blocked. The legacy `/getAppImage` endpoint is a restore read only; `prepareRestore(appId)` seeds an actual disposable free-tier App record so the production receipt check uses its normal no-provider branch. No legacy write handler or automatic fallback is mounted. The seven-operation matrix checks canonical acknowledgements, stale CAS conflicts, an old-route 404, malformed acknowledgement (including an unchanged old revision), snapshot metadata faults, and a lost reply after real commits; encrypted readback checks the actual outcome. It reads no `.env`, uses a uniquely named disposable database, and drops only that database afterward. The supporting module also lets the matching app's real Realm/Redux/transport fixture share this HTTP server. Response fault injection does not mock CAS or cryptographic success. Hosted authentication/rate limits, provider bootstrap, native mobile acceptance and production deployment topology are separate checks; this test does not establish them. + ## License Relay is licensed under the [MIT License](LICENSE). Dependencies retain their respective licenses. diff --git a/package.json b/package.json index dc0bc1b..06e2a17 100644 --- a/package.json +++ b/package.json @@ -10,6 +10,7 @@ "test": "npm run compile && jest ./dist --detectOpenHandles", "gcp-build": "tsc -p .", "test:backup": "node --test tests/*.test.cjs", + "test:automatic-http": "node --test tests/integration/automaticBackupHttp.test.cjs", "check:secrets": "python3 scripts/check-source-secrets.py" }, "jest": { diff --git a/src/db.ts b/src/db.ts index 0885707..457230a 100644 --- a/src/db.ts +++ b/src/db.ts @@ -153,6 +153,9 @@ class Database { nodes: [{ type: String, required: false }], // Once repaired, unversioned full replacements must never overwrite this image. backupRevisionRequired: { type: Boolean, default: false }, + // A revisioned incremental writer has claimed this account. Older clients + // must not apply delayed unversioned mutations after that writer commits. + incrementalRevisionRequired: { type: Boolean, default: false }, // Retained by Delete Backup, preventing stale A → B → A revisions. backupGeneration: { type: Number, default: 0, min: 0 }, }); @@ -327,6 +330,8 @@ class Database { { unique: true }, ); this.articleChunkSchema.index({ ragTimestamp: -1 }); + // Snapshot ownership checks must not scan every account for each label. + this.appImageSchema.index({ labels: 1 }); this.appVaultImageSchema.index({ appId: 1, vaultId: 1 }, { unique: true }); this.faucetQuotaSchema.index({ appId: 1, utcDate: 1 }, { unique: true }); this.faucetQuotaSchema.index( diff --git a/src/localDevelopment.ts b/src/localDevelopment.ts index 91828b7..624213a 100644 --- a/src/localDevelopment.ts +++ b/src/localDevelopment.ts @@ -27,7 +27,7 @@ const localDatabaseRoutes = new Set([ "/getvaultmetadata", "/getsigneridinfo", "/migratexfps", "/modifylabels", "/getmessages", "/getsubscriptiondetails", "/updatecontactskey", "/createremotekey", "/getremotekey", "/backupallsignersandvaults", - "/getbackupsnapshot", "/repairappbackup", + "/getbackupsnapshot", "/repairappbackup", "/revisionedbackupmutation", "/deletebackup", "/updatecollaborativechannel", "/fetchcollaborativechannel", "/gethardwarereferrallinks", "/getactivecampaign", "/getadvisors", ]); diff --git a/src/routes/routes.ts b/src/routes/routes.ts index 4aa2420..2b1c60a 100644 --- a/src/routes/routes.ts +++ b/src/routes/routes.ts @@ -77,6 +77,15 @@ const memoryStorage = multer.memoryStorage(); const upload = multer({ memoryStorage }); const screenshotUpload = multer({ storage: multer.memoryStorage() }); +function sendBackupMutationError(res: Response, error: unknown) { + const conflict = error instanceof BackupConflict; + const upgrade = error instanceof BackupUpgradeRequired; + return res.status(conflict || upgrade ? 409 : error instanceof InvalidBackupRequest ? 400 : 503).json({ + updated: false, + error: conflict ? "BACKUP_CHANGED" : upgrade ? "BACKUP_UPGRADE_REQUIRED" : "BACKUP_NOT_UPDATED", + }); +} + export default class Routes { public app: Express; constructor(app: Express) { @@ -348,13 +357,11 @@ export default class Routes { const updated = await bhr.migrateXfp( req.body.appId, req.body.signerChanges, + req.body.expectedRevision, ); res.status(200).json({ updated }); } catch (err) { - res.status(400).json({ - updated: false, - err: err.message, - }); + sendBackupMutationError(res, err); } }); @@ -519,10 +526,11 @@ export default class Routes { hasField("signersObject") ? req.body.signersObject : req.body.signersObjects, req.body.nodes, req.body.replaceNodes === true, + req.body.expectedRevision, ); res.status(200).json(result); } catch (err) { - res.status(400).json(err); + sendBackupMutationError(res, err); } }); @@ -535,10 +543,11 @@ export default class Routes { req.body.appId, req.body.walletIds, req.body.signers, + req.body.expectedRevision, ); res.status(200).json(result); } catch (err) { - res.status(400).json(err); + sendBackupMutationError(res, err); } }); @@ -547,10 +556,10 @@ export default class Routes { return res.status(400).json({ err: "Input param missing - app's id" }); } try { - const result = await bhr.deleteVaults(req.body.appId, req.body.vaults); + const result = await bhr.deleteVaults(req.body.appId, req.body.vaults, req.body.expectedRevision); res.status(200).json(result); } catch (err) { - res.status(400).json(err); + sendBackupMutationError(res, err); } }); @@ -566,6 +575,7 @@ export default class Routes { req.body.appId ?? req.body.appID, req.body.isArchived, req.body.signersData, + req.body.expectedRevision, ); if (updated) { return res.status(200).json({ @@ -588,13 +598,11 @@ export default class Routes { req.body.subscription, req.body.archiveVaultId, req.body.isArchived, + req.body.expectedRevision, ); res.status(200).json(result); - } catch { - res.status(400).json({ - updated: false, - error: "Vault backup update failed", - }); + } catch (error) { + sendBackupMutationError(res, error); } }); @@ -765,13 +773,11 @@ export default class Routes { req.body.appId, req.body?.addLabels, req.body?.deleteLabels, + req.body.expectedRevision, ); res.status(200).json(result); } catch (err) { - res - .status(400) - .json({ updated: false, err: "failed to update the labels" }); - console.log(err); + sendBackupMutationError(res, err); } }); @@ -1066,9 +1072,75 @@ export default class Routes { router.post("/getBackupSnapshot", async (req, res) => { if (typeof req.body.appId !== "string" || !req.body.appId) return res.status(400).json({ error: "Invalid backup account" }); - try { return res.status(200).json(await getBackupSnapshot(req.body.appId)); } + try { return res.status(200).json({ + ...await getBackupSnapshot(req.body.appId), revisionedMutations:true, + }); } catch { return res.status(503).json({ error: "Backup snapshot unavailable" }); } }); + // A distinct route prevents a revisioned client from writing through an old + // Relay instance that would silently ignore expectedRevision. Deploy this + // route to every instance before enabling it in mobile clients. + router.post("/revisionedBackupMutation", async (req, res) => { + const body = req.body || {}; + if (typeof body.appId !== "string" || !body.appId || + typeof body.expectedRevision !== "string" || !/^[a-f0-9]{64}$/.test(body.expectedRevision)) + return res.status(400).json({updated:false,error:"BACKUP_NOT_UPDATED"}); + if (body.appID !== undefined && body.appID !== body.appId) + return res.status(400).json({updated:false,error:"BACKUP_NOT_UPDATED"}); + try { + let result: any; + switch (body.operation) { + case "updateAppImage": + if (("walletsObject" in body && "walletObject" in body) || + ("signersObject" in body && "signersObjects" in body)) + throw new InvalidBackupRequest("Conflicting backup record fields"); + result = await bhr.updateAppImage(body.appId, body.publicId, + "walletsObject" in body ? body.walletsObject : body.walletObject, + body.subscription, body.version, + "signersObject" in body ? body.signersObject : body.signersObjects, + body.nodes, body.replaceNodes === true, body.expectedRevision); + break; + case "migrateXfps": + result = await bhr.migrateXfp(body.appId, body.signerChanges, body.expectedRevision); + break; + case "deleteAppImageEntity": + result = await bhr.deleteAppImageEntity(body.appId, body.walletIds, + body.signers, body.expectedRevision); + break; + case "deleteVaults": + result = await bhr.deleteVaults(body.appId, body.vaults, body.expectedRevision); + break; + case "updateVaultImage": + if (typeof body.vaultId !== "string" || !body.vaultId) + throw new InvalidBackupRequest("Invalid vault ID"); + result = body.isUpdate + ? await bhr.updateVault(body.vaultId, body.vault, body.appId, + body.isArchived, body.signersData, body.expectedRevision) + : await bhr.addVaultImage(body.appId, body.vaultShellId, body.vaultId, + body.scheme, body.vault, body.signersData, body.subscription, + body.archiveVaultId, body.isArchived, body.expectedRevision); + break; + case "modifyLabels": + result = await bhr.modifyLabels(body.appId, body.addLabels, + body.deleteLabels, body.expectedRevision); + break; + case "deleteBackup": + result = await bhr.deleteBackup(body.appId, body.expectedRevision); + break; + case "repairAppBackup": + result = await repairAppBackup(body, true); + break; + default: + throw new InvalidBackupRequest("Unknown backup mutation"); + } + if (result === false || result?.updated === false || + typeof result?.newRevision !== "string" || !/^[a-f0-9]{64}$/.test(result.newRevision)) + return res.status(503).json({updated:false,error:"BACKUP_NOT_UPDATED"}); + return res.status(200).json({updated:true,preconditionApplied:true,newRevision:result.newRevision}); + } catch (error) { + return sendBackupMutationError(res, error); + } + }); router.post("/repairAppBackup", async (req, res) => { try { return res.status(200).json(await repairAppBackup(req.body)); } catch (error) { @@ -1098,11 +1170,10 @@ export default class Routes { return res.status(400).json({ err: "Input param missing - app's id" }); } try { - const result = await bhr.deleteBackup(req.body.appId); + const result = await bhr.deleteBackup(req.body.appId, req.body.expectedRevision); res.status(200).json(result); } catch (err) { - console.log("🚀 ~deleteBackup err:", err); - res.status(400).json(err); + sendBackupMutationError(res, err); } }); diff --git a/src/services/backupSnapshot.ts b/src/services/backupSnapshot.ts index 6c4b80a..6579cab 100644 --- a/src/services/backupSnapshot.ts +++ b/src/services/backupSnapshot.ts @@ -52,18 +52,32 @@ async function snapshot(appId: string, session: mongoose.ClientSession) { const labelIds = appImage.labels || []; const { allVaultImages, unavailableVaultIds } = await accountVaultImages(appId, vaultIds, session); const labels = await db.getLabelModel().find({ id: { $in: labelIds } }).session(session).lean() as any[]; - if (new Set(labelIds).size !== labels.length) { - throw new Error("Incomplete backup; no replacement was made"); - } + const peerReferences = await db.getAppImageModel().find({ + appId:{$ne:appId}, labels:{$in:labelIds}, + }, {labels:1}).session(session).lean() as any[]; + const requestedLabelIds = new Set(labelIds); + const sharedLabelIds = new Set(); + for (const peer of peerReferences) for (const id of peer.labels || []) + if (requestedLabelIds.has(id)) sharedLabelIds.add(id); + // Historical cross-account label deletion could leave a dangling reference. + // A historical shared reference is also ambiguous: the global ciphertext + // cannot be attributed to either account. Withhold it from both recoveries. + // A uniquely owned missing label can be resubmitted by its owner, while a + // shared ID requires an operator migration with ownership evidence. + const foundLabelIds = new Set(labels.map(label => label.id)); + const unavailableLabelIds = labelIds.filter(id => + !foundLabelIds.has(id) || sharedLabelIds.has(id)).sort(); + const availableLabels = labels.filter(label => !sharedLabelIds.has(label.id)); const records = { appId, generation: app ? {identity:String(app._id),version:app.backupGeneration || 0} : null, wallets: appImage.wallets || {}, signers: appImage.signers || {}, nodes: appImage.nodes || [], vaults: allVaultImages.map(v => ({ vaultId: v.vaultId, vault: v.vault, appId: v.appId, isArchived: !!v.isArchived })).sort((a,b) => a.vaultId.localeCompare(b.vaultId)), unavailableVaultIds, - labels: labels.map(l => ({ id:l.id, content:l.content })).sort((a,b) => a.id.localeCompare(b.id)), + labels: availableLabels.map(l => ({ id:l.id, content:l.content })).sort((a,b) => a.id.localeCompare(b.id)), + unavailableLabelIds, }; const revision = createHash("sha256").update(canonical(records)).digest("hex"); - return { appImage, allVaultImages, unavailableVaultIds, labels, revision, exists: !!app }; + return { appImage, allVaultImages, unavailableVaultIds, labels:availableLabels, unavailableLabelIds, revision, exists: !!app }; } let scopedVaultInitialization: Promise; @@ -106,16 +120,52 @@ async function inTransaction(work: (session: mongoose.ClientSession) => Promi } finally { await session.endSession(); } } -async function advanceBackupGeneration(appId: string, session: mongoose.ClientSession) { - await db.getAppImageModel().updateOne({appId}, {$inc:{backupGeneration:1}}, {session}); +async function advanceBackupGeneration(appId: string, session: mongoose.ClientSession, revisionedIncremental = false) { + await db.getAppImageModel().updateOne({appId}, { + $inc:{backupGeneration:1}, + ...(revisionedIncremental ? {$set:{incrementalRevisionRequired:true}} : {}), + }, {session}); } // Legacy incremental writes also share the transaction/generation boundary. -export async function withBackupMutation(appId: string, work: (session: mongoose.ClientSession) => Promise) { +export async function assertBackupMutationRevision( + appId: string, + expectedRevision: string | undefined, + session: mongoose.ClientSession +) { + if (expectedRevision !== undefined && !/^[a-f0-9]{64}$/.test(expectedRevision)) + throw new InvalidBackupRequest("Invalid backup revision"); + if (expectedRevision === undefined) { + const current = await db.getAppImageModel().findOne({appId}).session(session).lean() as any; + if (current?.incrementalRevisionRequired) throw new BackupUpgradeRequired(); + return false; + } + const current = await snapshot(appId, session); + if (current.revision !== expectedRevision) throw new BackupConflict(); + return true; +} + +export async function withBackupMutation( + appId: string, + work: (session: mongoose.ClientSession) => Promise, + expectedRevision?: string, + revisionedIncremental = false +) { assertAccountId(appId); return inTransaction(async session => { + await assertBackupMutationRevision(appId, expectedRevision, session); const result = await work(session); - if (result !== false && (!result || (result as any).updated !== false)) await advanceBackupGeneration(appId, session); + if (result !== false && (!result || (result as any).updated !== false)) { + await advanceBackupGeneration(appId, session, revisionedIncremental && expectedRevision !== undefined); + // Compute the token before committing. A separate read after the commit + // could observe another device's write and let stale ciphertext rebase. + if (revisionedIncremental && expectedRevision !== undefined) { + const newRevision = (await snapshot(appId, session)).revision; + return (result && typeof result === "object" + ? { ...result, newRevision } + : { updated: true, newRevision }) as T; + } + } return result; }); } @@ -142,7 +192,7 @@ export async function getAppBackupVaultImages(appId: string, vaultIds: string[]) // Incremental deletion removes only the requested IDs. Never save a previously // read wallet/key map: a delayed deletion must preserve a newer repair or save. -export async function deleteBackupEntities(appId: string, wallets?: string[], signers?: string[]) { +export async function deleteBackupEntities(appId: string, wallets?: string[], signers?: string[], expectedRevision?: string) { if (typeof appId !== "string" || !appId) throw new InvalidBackupRequest("Invalid backup account"); const safeIds = (ids: unknown) => ids == null || (Array.isArray(ids) && ids.every(id => typeof id === "string" && id.length > 0 && id.length <= 512 && !/[.$]/.test(id) && @@ -150,6 +200,7 @@ export async function deleteBackupEntities(appId: string, wallets?: string[], si )); if (!safeIds(wallets) || !safeIds(signers)) throw new InvalidBackupRequest("Invalid backup record IDs"); return inTransaction(async session => { + const revisioned = await assertBackupMutationRevision(appId, expectedRevision, session); const app = await db.getAppImageModel().findOne({ appId }).session(session).lean(); if (!app) return { updated: false, error: "No app image" }; const removals: Record = {}; @@ -157,9 +208,10 @@ export async function deleteBackupEntities(appId: string, wallets?: string[], si for (const id of signers || []) removals[`signers.${id}`] = ""; if (Object.keys(removals).length) { const result = await db.getAppImageModel().updateOne({ appId }, { $unset: removals }, { session }); - if (result.modifiedCount) await advanceBackupGeneration(appId, session); + if (result.modifiedCount || revisioned) await advanceBackupGeneration(appId, session, revisioned); } - return { updated: true, error: "" }; + else if (revisioned) await advanceBackupGeneration(appId, session, true); + return { updated: true, error: "", ...(revisioned ? {newRevision:(await snapshot(appId, session)).revision} : {}) }; }); } @@ -299,6 +351,7 @@ export async function saveBackupVault(data: any) { } const appId = data.appId || legacy?.appId; if (typeof appId !== "string" || !appId) throw new InvalidBackupRequest("Backup account missing"); + const revisioned = await assertBackupMutationRevision(appId, data.expectedRevision, session); const existing = (await accountVaultImages(appId, [data.vaultId], session)).allVaultImages[0]; if (data.isUpdate && !existing) return {updated:false,error:"No vault image"}; const app = await db.getAppImageModel().findOne({ appId }).session(session).lean(); @@ -320,12 +373,12 @@ export async function saveBackupVault(data: any) { const retainedIds = isArchived ? [] : signersData === undefined ? (existing?.signerIds || []) : signersData.map(s => s.signerId); await pruneRemovedVaultMaps(data.vaultId, (existing?.signerIds || []).filter(id => !retainedIds.includes(id)), session); await db.getAppImageModel().updateOne({ appId }, { $addToSet: { vaults: data.vaultId } }, { session }); - await advanceBackupGeneration(appId, session); - return { updated: true, error: "" }; + await advanceBackupGeneration(appId, session, revisioned); + return { updated: true, error: "", ...(revisioned ? {newRevision:(await snapshot(appId, session)).revision} : {}) }; }); } -export async function deleteBackupVaults(appId: string, vaultIds: string[]) { +export async function deleteBackupVaults(appId: string, vaultIds: string[], expectedRevision?: string) { if (typeof appId !== "string" || !appId || !Array.isArray(vaultIds) || vaultIds.some(id => !validRecordId(id))) throw new InvalidBackupRequest("Invalid vault deletion"); return withBackupMutation(appId, async session => { @@ -341,13 +394,13 @@ export async function deleteBackupVaults(appId: string, vaultIds: string[]) { await removeAccountVaults(appId, authorizedIds, session); await db.getAppImageModel().updateOne({ appId }, { $pull: { vaults: { $in: vaultIds } } }, { session }); return { updated: true }; - }); + }, expectedRevision, true); } // The user's explicit "Delete" choice clears the encrypted backup while // retaining the app identity and subscription record. All related records // change together, so a failed delete cannot leave a partial backup behind. -export async function deleteAppBackup(appId: string) { +export async function deleteAppBackup(appId: string, expectedRevision?: string) { if (typeof appId !== "string" || !appId) throw new InvalidBackupRequest("Invalid backup account"); return withBackupMutation(appId, async session => { const app = await db.getAppImageModel().findOne({ appId }).session(session).lean() as any; @@ -366,7 +419,7 @@ export async function deleteAppBackup(appId: string) { wallets: {}, signers: {}, nodes: [], vaults: [], labels: [], } }, { session }); return { updated: true, error: "" }; - }); + }, expectedRevision, true); } function assertPayload(data: any) { @@ -391,16 +444,20 @@ function assertPayload(data: any) { // This endpoint never silently falls back to the legacy destructive replacement. // Transactions atomically update the referenced records and app image. On a // concurrent-write retry, the token is checked again against the new snapshot. -export async function repairAppBackup(data: any) { - return writeBackup(data, false); +export async function repairAppBackup(data: any, activateIncrementalFence = false) { + return writeBackup(data, false, activateIncrementalFence); } -async function writeBackup(data: any, legacy: boolean) { +async function writeBackup(data: any, legacy: boolean, activateIncrementalFence = false) { assertPayload(data); return withBackupMutation(data.appId, async session => { const current = await snapshot(data.appId, session); if (legacy && current.appImage.backupRevisionRequired) throw new BackupUpgradeRequired(); if (current.revision !== data.expectedRevision) throw new BackupConflict(); + // A missing historical label can only be restored by supplying its exact + // referenced ID. A replacement request must not silently discard it. + if (current.unavailableLabelIds.some(id => !data.labels.some((label: any) => label.id === id))) + throw new BackupConflict(); const includesAll = (oldIds: string[], newIds: string[]) => oldIds.every(id => newIds.includes(id)); // Only an explicit, revisioned repair may remove records. Old full writers // still fail closed, including transaction retries after a newer repair. @@ -450,13 +507,18 @@ async function writeBackup(data: any, legacy: boolean) { } // Updating this document also makes concurrent membership edits conflict // with the transaction instead of being lost in a full-snapshot write. + // A new account already has an empty AppImage from createNewApp. Its first + // explicit repair must persist the recovery subscription on that image, + // not only when an AppImage is inserted by this write. + const subscription = typeof data.subscription === "string" ? { subscription:data.subscription } : {}; await db.getAppImageModel().updateOne({ appId:data.appId }, { $set:{ wallets:data.walletObject, signers:data.signersObject, nodes:data.nodes, vaults:Object.keys(data.vaultObject), labels:data.labels.map(l => l.id), version:data.version, ...(!legacy ? { backupRevisionRequired:true } : {}), - }, $setOnInsert:{ appId:data.appId, publicId:data.publicId, subscription:data.subscription } }, {upsert:true,session}); + ...(!legacy ? subscription : {}), + }, $setOnInsert:{ appId:data.appId, publicId:data.publicId, ...(legacy ? subscription : {}) } }, {upsert:true,session}); return { updated:true }; - }); + }, legacy ? undefined : data.expectedRevision, activateIncrementalFence); } // Old clients have no revision token. Read their starting state, preserve archive @@ -468,6 +530,7 @@ export async function backupLegacyApp(data: any) { const current = await getBackupSnapshot(data.appId); if (current.appImage.backupRevisionRequired) throw new BackupUpgradeRequired(); if (current.unavailableVaultIds.length) throw new BackupConflict(); + if (current.unavailableLabelIds.length) throw new BackupConflict(); const vaultObject = Object.create(null); for (const [id, value] of Object.entries(data.vaultObject || {}) as [string, any][]) { vaultObject[id] = { ...value, vaultId:id, diff --git a/src/services/bhr.ts b/src/services/bhr.ts index c530d50..3964e41 100644 --- a/src/services/bhr.ts +++ b/src/services/bhr.ts @@ -8,6 +8,9 @@ import { deleteBackupVaults, getBackupSnapshot, withBackupMutation, + BackupConflict, + BackupUpgradeRequired, + InvalidBackupRequest, } from "./backupSnapshot"; import { AppImage, AppSubscription } from "../interface"; import db from "../db"; @@ -22,7 +25,8 @@ export const updateAppImage = async ( version, signersObject, nodes, - replaceNodes = false + replaceNodes = false, + expectedRevision?: string ): Promise<{ updated: boolean; error?: string }> => { if (typeof appId !== "string" || !appId) return {updated:false,error:"Invalid backup account"}; const appImageModel: any = db.getAppImageModel(); @@ -71,8 +75,9 @@ export const updateAppImage = async ( await appImageModel.updateOne({ appId }, { $set:changes, $setOnInsert:defaults }, { upsert:true,runValidators:true,session }); } return { updated:true,error:"" }; - }); - } catch { + }, expectedRevision, true); + } catch (error) { + if (error instanceof BackupConflict || (expectedRevision !== undefined && error instanceof BackupUpgradeRequired)) throw error; return { updated:false,error:"Backup update failed" }; } }; @@ -80,19 +85,22 @@ export const updateAppImage = async ( export const deleteAppImageEntity = async ( appId, wallets, - signers + signers, + expectedRevision?: string ): Promise<{ updated: boolean; error?: string }> => { try { - return await deleteBackupEntities(appId, wallets, signers); - } catch { + return await deleteBackupEntities(appId, wallets, signers, expectedRevision); + } catch (error) { + if (error instanceof BackupConflict || (expectedRevision !== undefined && error instanceof BackupUpgradeRequired)) throw error; return { updated: false, error: "Backup deletion failed" }; } }; -export const deleteVaults = async (appId, vaults) => { +export const deleteVaults = async (appId, vaults, expectedRevision?: string) => { try { - return await deleteBackupVaults(appId, vaults); - } catch { + return await deleteBackupVaults(appId, vaults, expectedRevision); + } catch (error) { + if (error instanceof BackupConflict || (expectedRevision !== undefined && error instanceof BackupUpgradeRequired)) throw error; return { updated: false, error: "Vault backup deletion failed" }; } }; @@ -136,7 +144,8 @@ export const addVaultImage = async ( signersData, subscription, archiveVaultId?, - isArchived? + isArchived?, + expectedRevision?: string ) => { try { return await saveBackupVault({ @@ -149,8 +158,10 @@ export const addVaultImage = async ( subscription, archiveVaultId, isArchived, + expectedRevision, }); - } catch { + } catch (error) { + if (error instanceof BackupConflict || (expectedRevision !== undefined && error instanceof BackupUpgradeRequired)) throw error; return { updated: false, error: "Vault backup update failed" }; } }; @@ -163,7 +174,7 @@ export const archiveVault = async (vaultId) => { } }; -export const updateVault = async (vaultId, vault, appId?, isArchived?, signersData?) => { +export const updateVault = async (vaultId, vault, appId?, isArchived?, signersData?, expectedRevision?: string) => { try { const result = await saveBackupVault({ appId, @@ -172,9 +183,11 @@ export const updateVault = async (vaultId, vault, appId?, isArchived?, signersDa isArchived, signersData, isUpdate: true, + expectedRevision, }); - return result?.updated ? {} : false; - } catch { + return result?.updated ? (expectedRevision !== undefined ? {newRevision:result.newRevision} : {}) : false; + } catch (error) { + if (error instanceof BackupConflict || (expectedRevision !== undefined && error instanceof BackupUpgradeRequired)) throw error; return false; } }; @@ -237,7 +250,8 @@ export const getVaultImage = async (vaultId) => { export const getAppImage = async (appId, appversion) => { const snapshot = await getBackupSnapshot(appId); - if (snapshot.unavailableVaultIds.length) throw new Error("Recovery data unavailable"); + if (snapshot.unavailableVaultIds.length || snapshot.unavailableLabelIds.length) + throw new Error("Recovery data unavailable"); const appImage = snapshot.exists ? snapshot.appImage : { ...snapshot.appImage, version: appversion }; @@ -307,7 +321,8 @@ interface SignerChange { export const migrateXfp = async ( appId: string, - signerChanges: SignerChange[] + signerChanges: SignerChange[], + expectedRevision?: string ) => { try { return await withBackupMutation(appId, async session => { @@ -327,14 +342,15 @@ export const migrateXfp = async ( } else { return false; } - }); + }, expectedRevision, true); } catch (err) { + if (err instanceof BackupConflict || (expectedRevision !== undefined && err instanceof BackupUpgradeRequired)) throw err; console.log(err); return false; } }; -export const modifyLabels = async (appId, addLabels, deleteLabels) => { +export const modifyLabels = async (appId, addLabels, deleteLabels, expectedRevision?: string) => { try { return await withBackupMutation(appId, async session => { const appImageModel: any = db.getAppImageModel(); @@ -348,40 +364,50 @@ export const modifyLabels = async (appId, addLabels, deleteLabels) => { if (addLabels && addLabels.length > 0) { const newLabels = []; for (const label of addLabels) { - if (!label || !label.content) { - continue; - } - // Check if label exists + if (!label || typeof label.id !== "string" || !label.id || /[.$]/.test(label.id) || + typeof label.content !== "string" || !label.content) + throw new InvalidBackupRequest("Invalid encrypted label"); + const otherOwner = await appImageModel.findOne({ + appId: {$ne:appId}, labels:label.id, + }).session(session).lean(); + if (otherOwner) throw new BackupConflict(); const existingLabel = await labelModel.findOne({ id: label.id }).session(session); if (existingLabel) { + await labelModel.updateOne({id:label.id}, {$set:{content:label.content}}, {session}); newLabels.push(existingLabel.id); } else { - // need to create that label const createdLabels = await labelModel.create([label], {session}); newLabels.push(createdLabels[0].id); } } appImage.labels = appImage.labels || []; - appImage.labels.push(...newLabels); + for (const id of newLabels) if (!appImage.labels.includes(id)) appImage.labels.push(id); await appImage.save({session}); } if (deleteLabels && deleteLabels.length > 0) { - const deletedLabels = await labelModel.deleteMany({ - id: { $in: deleteLabels }, - }).session(session); - if (appImage.labels && appImage.labels.length > 0) { - appImage.labels = appImage.labels.filter( - (labelId) => !deleteLabels.includes(labelId.toString()) - ); + if (!Array.isArray(deleteLabels) || deleteLabels.some(id => + typeof id !== "string" || !id || /[.$]/.test(id))) + throw new InvalidBackupRequest("Invalid encrypted label IDs"); + const owned = new Set((appImage.labels || []).map(id => id.toString())); + const removed = deleteLabels.filter(id => owned.has(id)); + if (removed.length) { + appImage.labels = appImage.labels.filter(labelId => !removed.includes(labelId.toString())); + await appImage.save({session}); + for (const id of removed) { + const otherOwner = await appImageModel.findOne({ + appId: {$ne:appId}, labels:id, + }).session(session).lean(); + if (!otherOwner) await labelModel.deleteOne({id}).session(session); + } } - await appImage.save({session}); } return { updated: true }; - }); + }, expectedRevision, true); } catch (error) { + if (error instanceof BackupConflict || (expectedRevision !== undefined && error instanceof BackupUpgradeRequired)) throw error; console.error("Error modifying labels:", error); throw new Error("An error occurred while modifying labels"); } diff --git a/src/tests/security/localBackupRoutes.test.ts b/src/tests/security/localBackupRoutes.test.ts index 4c71e07..f72326e 100644 --- a/src/tests/security/localBackupRoutes.test.ts +++ b/src/tests/security/localBackupRoutes.test.ts @@ -7,15 +7,18 @@ it("allows disposable backup routes while hosted integrations remain unavailable app.use(express.json(), localDevelopmentRoutes); app.post("/getBackupSnapshot", (_req, res) => res.json({ revision: "synthetic-revision" })); app.post("/repairAppBackup", (_req, res) => res.json({ updated: true })); + app.post("/revisionedBackupMutation", (_req, res) => res.json({ reachedRevisionedRoute: true })); app.post("/offer", (_req, res) => res.json({ shouldNotReachProvider: true })); const server = await new Promise>(resolve => { const listener = app.listen(0, "127.0.0.1", () => resolve(listener)); }); const baseUrl = `http://127.0.0.1:${(server.address() as AddressInfo).port}`; try { - for (const route of ["getBackupSnapshot", "repairAppBackup"]) { + for (const route of ["getBackupSnapshot", "repairAppBackup", "revisionedBackupMutation"]) { const response = await fetch(`${baseUrl}/${route}`, { method: "POST", body: "{}", headers: { "Content-Type": "application/json" } }); expect(response.status).toBe(200); + if (route === "revisionedBackupMutation") + expect((await response.json()).reachedRevisionedRoute).toBe(true); } const response = await fetch(`${baseUrl}/offer`, { method: "POST" }); expect(response.status).toBe(503); diff --git a/tests/backupSnapshot.test.cjs b/tests/backupSnapshot.test.cjs index d476fd8..43e1ecc 100644 --- a/tests/backupSnapshot.test.cjs +++ b/tests/backupSnapshot.test.cjs @@ -139,9 +139,57 @@ test('legacy full vault save removes a replaced active signer lookup',async()=>{ assert.equal(await map.countDocuments({signerId:'old-legacy-signer'}),0); assert.equal((await map.findOne({signerId:'new-legacy-signer'}).lean()).vaultId,'legacy-replacement'); }); -test('incomplete referenced records fail closed',async()=>{ - const p=await payload('incomplete');await service.repairAppBackup(p);await app.updateOne({appId:'incomplete'},{$set:{labels:['missing-record']}}); - await assert.rejects(service.getBackupSnapshot('incomplete'),/Incomplete backup/); +test('orphan label is visible for repair but cannot be used for generic recovery or discarded',async()=>{ + const p=await payload('incomplete'); + p.labels=[{id:'missing-record',content:'local-encrypted-label'}]; + await service.repairAppBackup(p); + await label.deleteOne({id:'missing-record'}); + const broken=await service.getBackupSnapshot(p.appId); + assert.deepEqual(broken.unavailableLabelIds,['missing-record']); + assert.deepEqual(broken.labels,[]); + await assert.rejects(legacy.getAppImage(p.appId,'2.5.16'),/Recovery data unavailable/); + await assert.rejects(service.repairAppBackup({...p,expectedRevision:broken.revision, + labels:[],replaceCurrentState:true},true),service.BackupConflict); + const restored=await service.repairAppBackup({...p,expectedRevision:broken.revision, + replaceCurrentState:true},true); + assert.equal(restored.updated,true); + const complete=await service.getBackupSnapshot(p.appId); + assert.deepEqual(complete.unavailableLabelIds,[]); + assert.equal(complete.labels[0].content,'local-encrypted-label'); + assert.equal(restored.newRevision,complete.revision); +}); +test('legacy full backup cannot silently replace a dangling label reference',async()=>{ + const p=await payload('legacy-orphan-label'); + p.labels=[{id:'legacy-missing-label',content:'encrypted-label'}]; + delete p.expectedRevision; + assert.equal((await legacy.backupAllSignersAndVaults(p)).updated,true); + await label.deleteOne({id:'legacy-missing-label'}); + await assert.rejects(legacy.backupAllSignersAndVaults(p),/Backup changed/); + assert.deepEqual((await service.getBackupSnapshot(p.appId)).unavailableLabelIds,['legacy-missing-label']); +}); +test('historical shared label reference withholds ambiguous ciphertext from both accounts',async()=>{ + const first=await payload('historical-label-first'); + first.labels=[{id:'historical-shared-label',content:'first-owner-ciphertext'}]; + await service.repairAppBackup(first); + const second=await payload('historical-label-second'); + await service.repairAppBackup(second); + // Simulate an older endpoint attaching the same global label ID to another + // account. Its ciphertext has no trustworthy per-account owner metadata. + await app.updateOne({appId:second.appId},{$addToSet:{labels:'historical-shared-label'}}); + for(const account of [first,second]){ + const snapshot=await service.getBackupSnapshot(account.appId); + assert.deepEqual(snapshot.unavailableLabelIds,['historical-shared-label']); + assert.deepEqual(snapshot.labels,[]); + await assert.rejects(legacy.getAppImage(account.appId,'2.5.16'),/Recovery data unavailable/); + } + const firstSnapshot=await service.getBackupSnapshot(first.appId); + await assert.rejects(service.repairAppBackup({...first, + expectedRevision:firstSnapshot.revision,replaceCurrentState:true},true),service.BackupConflict); + assert.equal((await label.findOne({id:'historical-shared-label'}).lean()).content,'first-owner-ciphertext'); +}); +test('shared-label ownership lookup has its required multikey index',async()=>{ + const indexes=await app.collection.indexes(); + assert.ok(indexes.some(index=>index.key.labels===1)); }); test('malformed encrypted payload cannot create records',async()=>{ const p=await payload('invalid');p.nodes=[{}];await assert.rejects(service.repairAppBackup(p),/Invalid encrypted node/);assert.equal(await app.countDocuments({appId:'invalid'}),0); @@ -152,6 +200,140 @@ test('version-only incremental update succeeds without nodes and preserves recor assert.equal((await legacy.updateAppImage('version',undefined,undefined,undefined,'2.5.17')).updated,true); const s=await service.getBackupSnapshot('version');assert.equal(s.appImage.version,'2.5.17');assert.equal(s.appImage.wallets.w,'encrypted-wallet'); }); +test('revisioned incrementals reject a delayed old ciphertext and fence unversioned clients',async()=>{ + const p=await payload('incremental-cas');await service.repairAppBackup(p); + const first=(await service.getBackupSnapshot(p.appId)).revision; + const updated=await legacy.updateAppImage(p.appId,null,{w:'new-ciphertext'},null,null,null,null,false,first); + assert.equal(updated.updated,true); + assert.match(updated.newRevision,/^[a-f0-9]{64}$/); + assert.equal(updated.newRevision,(await service.getBackupSnapshot(p.appId)).revision); + await assert.rejects(legacy.updateAppImage(p.appId,null,{w:'old-ciphertext'},null,null,null,null,false,first),/Backup changed/); + assert.equal((await legacy.updateAppImage(p.appId,null,{w:'old-unversioned'})).updated,false); + const stored=await service.getBackupSnapshot(p.appId); + assert.equal(stored.appImage.wallets.w,'new-ciphertext'); + assert.equal(stored.appImage.incrementalRevisionRequired,true); + assert.notEqual(stored.revision,first); +}); +test('every v2 mutation returns its own committed revision',async()=>{ + const p=await payload('v2-response-revisions'); + const repair=await service.repairAppBackup({...p,signersObject:{old:'encrypted-signer'}},true); + assert.match(repair.newRevision,/^[a-f0-9]{64}$/); + assert.equal(repair.newRevision,(await service.getBackupSnapshot(p.appId)).revision); + const check=async mutate=>{ + const before=(await service.getBackupSnapshot(p.appId)).revision; + const result=await mutate(before); + assert.notEqual(result,false); + assert.notEqual(result.updated,false); + assert.match(result.newRevision,/^[a-f0-9]{64}$/); + assert.notEqual(result.newRevision,before); + assert.equal(result.newRevision,(await service.getBackupSnapshot(p.appId)).revision); + }; + await check(revision=>legacy.updateAppImage(p.appId,null,{w:'next-wallet'},null,null,null,null,false,revision)); + await check(revision=>legacy.migrateXfp(p.appId,[{oldSignerId:'old',newSignerId:'new',newSignerDetails:'next-signer'}],revision)); + await check(revision=>legacy.modifyLabels(p.appId,[{id:'new-label',content:'encrypted-label'}],[],revision)); + await check(revision=>legacy.addVaultImage(p.appId,null,'new-vault',null,'encrypted-vault',[],null,null,false,revision)); + await check(revision=>legacy.updateVault('new-vault','next-vault',p.appId,false,[],revision)); + await check(revision=>legacy.deleteVaults(p.appId,['new-vault'],revision)); + await check(revision=>legacy.deleteAppImageEntity(p.appId,['w'],['new'],revision)); + await check(revision=>legacy.deleteBackup(p.appId,revision)); +}); +test('paused old CAS transaction retries after a newer wallet commit and rejects stale ciphertext',async()=>{ + const p=await payload('incremental-inflight-race');await service.repairAppBackup(p); + const expectedRevision=(await service.getBackupSnapshot(p.appId)).revision; + const original=app.updateOne;let announce,release,paused=false,attempts=0; + const started=new Promise(resolve=>{announce=resolve;}); + const gate=new Promise(resolve=>{release=resolve;}); + app.updateOne=async function(filter,changes,options){ + if(changes.$set?.['wallets.w']==='old-ciphertext'){ + attempts++; + if(!paused){paused=true;announce();await gate;} + } + return original.call(this,filter,changes,options); + }; + let old; + try{ + old=legacy.updateAppImage(p.appId,null,{w:'old-ciphertext'},null,null,null,null,false,expectedRevision); + await started; + assert.equal((await legacy.updateAppImage(p.appId,null,{w:'new-ciphertext'},null,null,null,null,false,expectedRevision)).updated,true); + release(); + await assert.rejects(old,/Backup changed/); + assert.ok(attempts>=1); + assert.equal((await service.getBackupSnapshot(p.appId)).appImage.wallets.w,'new-ciphertext'); + }finally{release();app.updateOne=original;if(old)await Promise.allSettled([old]);} +}); +test('already-sent unversioned wallet transaction cannot land after the first CAS commit',async()=>{ + const p=await payload('incremental-legacy-inflight');await service.repairAppBackup(p); + const expectedRevision=(await service.getBackupSnapshot(p.appId)).revision; + const original=app.updateOne;let announce,release,paused=false; + const started=new Promise(resolve=>{announce=resolve;}); + const gate=new Promise(resolve=>{release=resolve;}); + app.updateOne=async function(filter,changes,options){ + if(changes.$set?.['wallets.w']==='old-unversioned' && !paused){paused=true;announce();await gate;} + return original.call(this,filter,changes,options); + }; + let old; + try{ + old=legacy.updateAppImage(p.appId,null,{w:'old-unversioned'}); + await started; + assert.equal((await legacy.updateAppImage(p.appId,null,{w:'new-revisioned'},null,null,null,null,false,expectedRevision)).updated,true); + release(); + assert.equal((await old).updated,false); + const stored=await service.getBackupSnapshot(p.appId); + assert.equal(stored.appImage.wallets.w,'new-revisioned'); + assert.equal(stored.appImage.incrementalRevisionRequired,true); + }finally{release();app.updateOne=original;if(old)await Promise.allSettled([old]);} +}); +for(const operation of ['update','delete']) test(`v2 repair fences an already-sent unversioned ${operation}`,async()=>{ + const p=await payload(`repair-fences-old-${operation}`);await service.repairAppBackup(p); + const expectedRevision=(await service.getBackupSnapshot(p.appId)).revision; + const original=app.updateOne;let announce,release,paused=false; + const started=new Promise(resolve=>{announce=resolve;}); + const gate=new Promise(resolve=>{release=resolve;}); + app.updateOne=async function(filter,changes,options){ + const isOld=operation==='update' + ? changes.$set?.['wallets.w']==='old-unversioned' + : Object.prototype.hasOwnProperty.call(changes.$unset || {},'wallets.w'); + if(isOld && !paused){paused=true;announce();await gate;} + return original.call(this,filter,changes,options); + }; + let old; + try{ + old=operation==='update' + ? legacy.updateAppImage(p.appId,null,{w:'old-unversioned'}) + : legacy.deleteAppImageEntity(p.appId,['w'],[]); + await started; + assert.equal((await service.repairAppBackup({...p,expectedRevision, + walletObject:{w:'new-repaired'}},true)).updated,true); + release(); + assert.equal((await old).updated,false); + const stored=await service.getBackupSnapshot(p.appId); + assert.equal(stored.appImage.wallets.w,'new-repaired'); + assert.equal(stored.appImage.incrementalRevisionRequired,true); + }finally{release();app.updateOne=original;if(old)await Promise.allSettled([old]);} +}); +test('all incremental delete, vault, label and signer paths reject stale revisions',async()=>{ + const p=await payload('incremental-paths'); + p.signersObject={key:'new-signer'}; + p.vaultObject={v:{vaultId:'v',vault:'new-vault',isArchived:false,signersData:[]}}; + p.labels=[{id:'tag',content:'new-label'}]; + await service.repairAppBackup(p); + const old=(await service.getBackupSnapshot(p.appId)).revision; + await legacy.updateAppImage(p.appId,null,{w:'new-wallet'},null,null,null,null,false,old); + await assert.rejects(legacy.deleteAppImageEntity(p.appId,['w'],[],old),/Backup changed/); + await assert.rejects(legacy.deleteVaults(p.appId,['v'],old),/Backup changed/); + await assert.rejects(legacy.deleteBackup(p.appId,old),/Backup changed/); + await assert.rejects(legacy.updateVault('v','old-vault',p.appId,false,[],old),/Backup changed/); + await assert.rejects(legacy.addVaultImage(p.appId,null,'v2',null,'old-vault',[],null,null,false,old),/Backup changed/); + await assert.rejects(legacy.modifyLabels(p.appId,[{id:'tag',content:'old-label'}],[],old),/Backup changed/); + await assert.rejects(legacy.migrateXfp(p.appId,[{oldSignerId:'key',newSignerId:'older',newSignerDetails:'old-signer'}],old),/Backup changed/); + const stored=await service.getBackupSnapshot(p.appId); + assert.equal(stored.appImage.wallets.w,'new-wallet'); + assert.equal(stored.appImage.signers.key,'new-signer'); + assert.deepEqual(stored.appImage.vaults,['v']); + assert.deepEqual(stored.appImage.labels,['tag']); + assert.equal(stored.allVaultImages[0].vault,'new-vault'); + assert.equal(stored.labels[0].content,'new-label'); +}); test('concurrent incremental updates preserve both wallets',async()=>{ const p=await payload('increments');await service.repairAppBackup(p); const results=await Promise.all([legacy.updateAppImage('increments',null,{a:'device-a'}),legacy.updateAppImage('increments',null,{b:'device-b'})]); @@ -222,6 +404,24 @@ test('label ownership collision fails without altering either backup',async()=>{ assert.equal((await service.getBackupSnapshot(owner.appId)).labels[0].content,'keep-this'); assert.equal((await service.getBackupSnapshot(other.appId)).exists,false); }); +test('incremental labels cannot overwrite or delete another account recovery record',async()=>{ + const first=await payload('incremental-label-owner'); + first.labels=[{id:'shared-label-id',content:'owner-ciphertext'}]; + await service.repairAppBackup(first); + const second=await payload('incremental-label-other');await service.repairAppBackup(second); + const before=(await service.getBackupSnapshot(second.appId)).revision; + await assert.rejects(legacy.modifyLabels(second.appId, + [{id:'shared-label-id',content:'attacker-ciphertext'}],[],before),/Backup changed/); + assert.equal((await legacy.modifyLabels(second.appId,[],['shared-label-id'],before)).updated,true); + assert.equal((await service.getBackupSnapshot(first.appId)).labels[0].content,'owner-ciphertext'); + // Older data can legitimately contain a shared reference. Unlinking one app + // must retain the label document until the final participant releases it. + await app.updateOne({appId:second.appId},{$addToSet:{labels:'shared-label-id'}}); + const sharedRevision=(await service.getBackupSnapshot(second.appId)).revision; + assert.equal((await legacy.modifyLabels(second.appId,[],['shared-label-id'],sharedRevision)).updated,true); + assert.equal((await service.getBackupSnapshot(first.appId)).labels[0].content,'owner-ciphertext'); + assert.deepEqual((await service.getBackupSnapshot(second.appId)).appImage.labels,[]); +}); test('incremental deletion acknowledges commit and leaves valid empty wallet/key maps',async()=>{ const p=await payload('entity-delete-empty');p.signersObject={key:'encrypted-key'}; @@ -330,7 +530,7 @@ function vaultUpdateRouteHandler(){ } visit(source);assert.ok(registration); vm.runInNewContext(ts.transpileModule(registration.getText(source),{compilerOptions:{target:ts.ScriptTarget.ES2020}}).outputText, - {bhr:legacy,console,router:{post:(_path,handler)=>{callback=handler;}}}); + {bhr:legacy,console,sendBackupMutationError:(res,error)=>res.status(503).json({updated:false,error:'BACKUP_NOT_UPDATED'}),router:{post:(_path,handler)=>{callback=handler;}}}); return callback; } diff --git a/tests/integration/automaticBackupHttp.test.cjs b/tests/integration/automaticBackupHttp.test.cjs new file mode 100644 index 0000000..6d4b56d --- /dev/null +++ b/tests/integration/automaticBackupHttp.test.cjs @@ -0,0 +1,277 @@ +const { test, before, after } = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const vm = require('node:vm'); +const { createRequire } = require('node:module'); +const ts = require('typescript'); +const { createFixture } = require('../support/automaticBackupHttpFixture.cjs'); + +// Use the exact mobile AES functions, including real salt/randomness. Extract +// only these pure exports; native RSA is unrelated to encrypted backups. +const keeperRoot = process.env.KEEPER_APP_SOURCE; +assert.ok(keeperRoot, 'Set KEEPER_APP_SOURCE to the compatible app checkout'); +const appRequire = createRequire(path.join(keeperRoot, 'package.json')); +const encryptionFile = path.join(keeperRoot, 'src/utils/service-utilities/encryption.ts'); +const encryptionText = fs.readFileSync(encryptionFile, 'utf8'); +const encryptionTree = ts.createSourceFile(encryptionFile, encryptionText, ts.ScriptTarget.Latest, true); +const declarations = encryptionTree.statements.filter(statement => ts.isVariableStatement(statement) && + statement.declarationList.declarations.some(declaration => ['encrypt', 'decrypt'].includes(declaration.name.getText(encryptionTree)))); +assert.equal(declarations.length, 2); +const cryptoModule = { exports: {} }; +const cryptoCode = ts.transpileModule(declarations.map(node => node.getText(encryptionTree)).join('\n'), { + compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2020 }, +}).outputText; +vm.runInThisContext(`(function(module,exports,cryptoJS){${cryptoCode}\n})`, { filename: encryptionFile }) + (cryptoModule, cryptoModule.exports, appRequire('crypto-js')); +const key = 'disposable-http-backup-key'; +const cipher = record => cryptoModule.exports.encrypt(key, JSON.stringify(record)); +const decode = encrypted => JSON.parse(cryptoModule.exports.decrypt(key, encrypted)); +const wallet = { id: 'wallet', name: 'before', networkType: 'TESTNET' }; +const signer = { masterFingerprint: 'aabbccdd', name: 'before' }; +const vault = { id: 'vault', name: 'before', archived: false }; +const label = { id: 'tag', label: 'before' }; +const changed = { name: 'after' }; +const operations = { + updateAppImage: () => ({ walletsObject: { wallet: cipher({ ...wallet, ...changed }) } }), + migrateXfps: () => ({ signerChanges: [{ oldSignerId: 'old-key', newSignerId: 'new-key', + newSignerDetails: cipher({ ...signer, ...changed }) }] }), + deleteAppImageEntity: () => ({ walletIds: ['wallet'], signers: [] }), + deleteVaults: () => ({ vaults: ['vault'] }), + updateVaultImage: () => ({ isUpdate: true, vaultId: 'vault', vault: cipher({ ...vault, ...changed }), + isArchived: false, signersData: [] }), + modifyLabels: () => ({ addLabels: [{ id: 'tag', content: cipher({ ...label, ...changed }) }], deleteLabels: [] }), + deleteBackup: () => ({}), +}; +let fixture; +before(async () => { fixture = await createFixture(); }); +after(async () => { if (fixture) await fixture.close(); }); +async function baseline(operation, suffix) { + await fixture.reset(); + const appId = `synthetic-${operation}-${suffix}`; + const snapshot = await fixture.seed({ appId, subscription: JSON.stringify({ level: 1, isCancelled: false }), + walletObject: { wallet: cipher(wallet), + remote: cipher({ id: 'remote', name: 'remote-only', networkType: 'TESTNET' }) }, + signersObject: { 'old-key': cipher(signer) }, + vaultObject: { vault: { vaultId: 'vault', vault: cipher(vault), isArchived: false, signersData: [] } }, + nodes: [], labels: [{ id: 'tag', content: cipher(label) }] }); + fixture.clearTrace(); + return { appId, snapshot, body: { appId, expectedRevision: snapshot.revision, + operation, ...operations[operation]() } }; +} +function checkEffect(operation, snapshot) { + if (operation === 'updateAppImage') assert.equal(decode(snapshot.appImage.wallets.wallet).name, 'after'); + if (operation === 'migrateXfps') { + assert.equal(snapshot.appImage.signers['old-key'], undefined); + assert.equal(decode(snapshot.appImage.signers['new-key']).name, 'after'); + } + if (operation === 'deleteAppImageEntity') assert.equal(snapshot.appImage.wallets.wallet, undefined); + if (operation === 'deleteVaults') assert.equal(snapshot.allVaultImages.length, 0); + if (operation === 'updateVaultImage') assert.equal(decode(snapshot.allVaultImages[0].vault).name, 'after'); + if (operation === 'modifyLabels') assert.equal(decode(snapshot.labels[0].content).name, 'after'); + if (operation === 'deleteBackup') { + assert.deepEqual(snapshot.appImage.wallets, {}); + assert.deepEqual(snapshot.appImage.signers, {}); + assert.equal(snapshot.allVaultImages.length, 0); + assert.equal(snapshot.labels.length, 0); + } else assert.equal(decode(snapshot.appImage.wallets.remote).name, 'remote-only'); +} + +for (const operation of Object.keys(operations)) { + test(`${operation}: real HTTP CAS acknowledges transaction and encrypted readback`, async () => { + const input = await baseline(operation, 'ack'); + const result = await fixture.post('revisionedBackupMutation', input.body); + assert.deepEqual(Object.keys(result.data).sort(), ['newRevision', 'preconditionApplied', 'updated']); + assert.equal(result.data.updated, true); + assert.equal(result.data.preconditionApplied, true); + assert.match(result.data.newRevision, /^[a-f0-9]{64}$/); + assert.notEqual(result.data.newRevision, input.snapshot.revision); + const snapshot = await fixture.snapshot(input.appId); + assert.equal(result.data.newRevision, snapshot.revision); + checkEffect(operation, snapshot); + await fixture.post('revisionedBackupMutation', input.body, 409); + assert.equal((await fixture.snapshot(input.appId)).revision, snapshot.revision); + }); + for (const mode of ['409', '404', 'malformed', 'equal-old-ack', 'lost-reply']) { + test(`${operation}: ${mode} uses actual transaction outcome without fake CAS success`, async () => { + const input = await baseline(operation, mode); + fixture.armFault({ mode, operation }); + if (mode === 'lost-reply') await assert.rejects(fixture.post('revisionedBackupMutation', input.body)); + else { + const result = await fixture.post('revisionedBackupMutation', input.body, + mode === '409' ? 409 : mode === '404' ? 404 : 200); + if (mode === '409') assert.equal(result.data.error, 'BACKUP_CHANGED'); + if (mode === 'malformed') assert.equal(result.data.newRevision, 'invalid'); + if (mode === 'equal-old-ack') assert.equal(result.data.newRevision, input.snapshot.revision); + } + const snapshot = await fixture.snapshot(input.appId); + const attempts = fixture.trace.filter(entry => entry.path === '/revisionedBackupMutation'); + assert.equal(attempts.length, 1); + if (mode === '404') { + assert.equal(attempts[0].routeInvoked, false); + assert.equal(snapshot.revision, input.snapshot.revision); + } else if (mode === '409') { + assert.equal(attempts[0].committed, false); + assert.notEqual(snapshot.revision, input.snapshot.revision); + assert.equal(snapshot.appImage.wallets.wallet, input.snapshot.appImage.wallets.wallet); + } else { + assert.equal(attempts[0].committed, true); + assert.notEqual(snapshot.revision, input.snapshot.revision); + checkEffect(operation, snapshot); + } + }); + } +} + +test('fanout fault skips the first commit and faults the next real acknowledgement', async () => { + const input = await baseline('updateAppImage', 'fanout'); + fixture.armFault({ mode: 'malformed', operation: 'updateAppImage', after: 1 }); + const first = await fixture.post('revisionedBackupMutation', input.body); + assert.match(first.data.newRevision, /^[a-f0-9]{64}$/); + const next = { ...input.body, expectedRevision: first.data.newRevision, + walletsObject: { wallet: cipher({ ...wallet, name: 'second' }) } }; + const second = await fixture.post('revisionedBackupMutation', next); + assert.equal(second.data.newRevision, 'invalid'); + const snapshot = await fixture.snapshot(input.appId); + assert.equal(decode(snapshot.appImage.wallets.wallet).name, 'second'); + assert.equal(fixture.trace.filter(entry => entry.committed).length, 2); +}); + +test('snapshot reply barrier permits a real peer transaction after read and before delivery', async () => { + const input = await baseline('updateAppImage', 'snapshot-barrier'); + let callbackCompleted = false; + fixture.armFault({ mode: 'before-snapshot-reply', beforeReply: async ({ body, service, db }) => { + await service.withBackupMutation(input.appId, async session => { + await db.getAppImageModel().updateOne({ appId: input.appId }, { + $set: { 'wallets.peer': cipher({ id: 'peer', name: 'concurrent-peer', networkType: 'TESTNET' }) }, + }, { session }); + return { updated: true }; + }, body.revision, true); + callbackCompleted = true; + } }); + const delayedSnapshot = await fixture.snapshot(input.appId); + assert.equal(callbackCompleted, true); + assert.equal(delayedSnapshot.revision, input.snapshot.revision); + const actual = await fixture.snapshot(input.appId); + assert.notEqual(actual.revision, delayedSnapshot.revision); + assert.equal(decode(actual.appImage.wallets.peer).name, 'concurrent-peer'); + await fixture.post('revisionedBackupMutation', input.body, 409); + assert.equal((await fixture.snapshot(input.appId)).revision, actual.revision); +}); + +test('seed preserves subscription metadata through the actual revisioned repair', async () => { + const input = await baseline('updateAppImage', 'metadata'); + assert.deepEqual(JSON.parse(input.snapshot.appImage.subscription), { level: 1, isCancelled: false }); + assert.equal(input.snapshot.appImage.publicId, `public-${input.appId}`); + assert.equal(input.snapshot.appImage.version, '2.6.4'); +}); + +test('fresh createNewApp image gains subscription during acknowledged revisioned repair', async () => { + await fixture.reset(); + const appId = 'synthetic-fresh-account-repair-metadata'; + const publicId = `public-${appId}`; + const created = await fixture.post('createNewApp', { appID: appId, publicId }, 200, { os: 'android' }); + assert.equal(created.data.created, true); + const before = await fixture.snapshot(appId); + assert.equal(before.exists, true); + assert.equal(before.appImage.subscription, undefined); + const subscription = JSON.stringify({ level: 3, isCancelled: false }); + const repair = { operation: 'repairAppBackup', appId, publicId, + expectedRevision: before.revision, replaceCurrentState: true, + version: '2.6.4', subscription, + walletObject: { wallet: cipher(wallet) }, signersObject: {}, + vaultObject: {}, nodes: [], labels: [] }; + const acknowledgement = await fixture.post('revisionedBackupMutation', repair); + assert.equal(acknowledgement.data.updated, true); + assert.equal(acknowledgement.data.preconditionApplied, true); + const readback = await fixture.snapshot(appId); + assert.equal(acknowledgement.data.newRevision, readback.revision); + assert.notEqual(readback.revision, before.revision); + assert.equal(readback.appImage.subscription, subscription); + assert.equal(readback.appImage.publicId, publicId); + assert.equal(readback.appImage.version, '2.6.4'); + assert.equal(decode(readback.appImage.wallets.wallet).name, 'before'); + await fixture.post('revisionedBackupMutation', repair, 409); + assert.equal((await fixture.snapshot(appId)).revision, readback.revision); +}); + +for (const [field, value] of Object.entries({ publicId: 'synthetic-other-public-id', + version: 'stale-fixture-version', subscription: '{"level":999}' })) { + test(`snapshot ${field} metadata fault changes only delivery after a real transactional read`, async () => { + const input = await baseline('updateAppImage', `malformed-${field}`); + fixture.armFault({ mode: 'malformed-metadata', metadata: { [field]: value } }); + const malformed = await fixture.snapshot(input.appId); + assert.equal(malformed.appImage[field], value); + assert.equal(malformed.revision, input.snapshot.revision); + const actual = await fixture.snapshot(input.appId); + assert.equal(actual.appImage[field], input.snapshot.appImage[field]); + assert.equal(actual.revision, input.snapshot.revision); + assert.equal(actual.appImage.wallets.wallet, input.snapshot.appImage.wallets.wallet); + }); +} + +test('actual legacy restore read preserves encrypted records and does not authorize a legacy write', async () => { + const input = await baseline('updateAppImage', 'clean-restore-read'); + await fixture.prepareRestore(input.appId); + fixture.clearTrace(); + const recovered = (await fixture.post('getAppImage', { appId: input.appId })).data; + assert.equal(recovered.appImage.appId, input.appId); + assert.equal(decode(recovered.appImage.wallets.wallet).name, 'before'); + assert.equal(decode(recovered.appImage.signers['old-key']).masterFingerprint, 'aabbccdd'); + assert.equal(decode(recovered.allVaultImages[0].vault).id, 'vault'); + assert.equal(decode(recovered.labels[0].content).id, 'tag'); + assert.equal(recovered.subscription.level, 1); + assert.equal((await fixture.snapshot(input.appId)).revision, input.snapshot.revision); + assert.equal(fixture.trace.some(entry => entry.path === '/revisionedBackupMutation'), false); + await fixture.post('getAppImage', { appId: input.appId }, 400, { appversion: '' }); +}); + +for (const fault of ['invalid-reference', 'unavailable', 'malformed-shape']) { + test(`snapshot ${fault} hook changes only wire data after actual transaction`, async () => { + const input = await baseline('updateAppImage', fault); + fixture.armFault({ mode: 'before-snapshot-reply', beforeReply: ({ body }) => { + if (fault === 'invalid-reference') body.appImage.vaults.push('synthetic-missing-vault'); + if (fault === 'unavailable') body.unavailableLabelIds = ['synthetic-unavailable-label']; + return fault === 'malformed-shape' ? 'synthetic-invalid-snapshot' : body; + } }); + const altered = await fixture.snapshot(input.appId); + if (fault === 'invalid-reference') assert.ok(altered.appImage.vaults.includes('synthetic-missing-vault')); + if (fault === 'unavailable') assert.equal(altered.unavailableLabelIds.length, 1); + if (fault === 'malformed-shape') assert.equal(typeof altered, 'string'); + const actual = await fixture.snapshot(input.appId); + assert.equal(actual.revision, input.snapshot.revision); + assert.deepEqual(actual.appImage.vaults, ['vault']); + assert.deepEqual(actual.unavailableLabelIds, []); + }); +} + +test('invalid JSON bytes fail at the response boundary after the actual snapshot read', async () => { + const input = await baseline('updateAppImage', 'malformed-json'); + fixture.armFault({ mode: 'malformed-json' }); + await assert.rejects(fixture.snapshot(input.appId), SyntaxError); + const actual = await fixture.snapshot(input.appId); + assert.equal(actual.revision, input.snapshot.revision); + assert.equal(actual.appImage.wallets.wallet, input.snapshot.appImage.wallets.wallet); +}); + +for (const [name, acknowledgement] of Object.entries({ + 'negative-updated': { updated: false }, + 'missing-precondition': { updated: true }, + 'missing-revision': { updated: true, preconditionApplied: true }, + 'null-body': null, +})) { + test(`malformed ${name} acknowledgement follows a real commit without replacing its trace`, async () => { + const input = await baseline('updateAppImage', name); + fixture.armFault({ mode: 'malformed', acknowledgement }); + const reply = await fixture.post('revisionedBackupMutation', input.body); + assert.deepEqual(reply.data, acknowledgement); + const actual = await fixture.snapshot(input.appId); + checkEffect('updateAppImage', actual); + assert.notEqual(actual.revision, input.snapshot.revision); + const trace = fixture.trace.find(entry => entry.path === '/revisionedBackupMutation'); + assert.equal(trace.committed, true); + assert.equal(trace.response.updated, true); + assert.equal(trace.response.preconditionApplied, true); + assert.equal(trace.response.newRevision, actual.revision); + }); +} diff --git a/tests/recoveryRoute.test.cjs b/tests/recoveryRoute.test.cjs index 0c5b0bd..464d12a 100644 --- a/tests/recoveryRoute.test.cjs +++ b/tests/recoveryRoute.test.cjs @@ -32,10 +32,55 @@ test('snapshot database failure terminates with a fixed retryable error',async() test('snapshot success preserves revision and encrypted records',async()=>{ const res=mockResponse(), snapshot={revision:'a'.repeat(64),appImage:{wallets:{w:'synthetic-ciphertext'}},labels:[],allVaultImages:[]}; await route({}, '/getBackupSnapshot', {getBackupSnapshot:async()=>snapshot})({body:{appId:'disposable'}},res); - assert.equal(res.result.status,200);assert.equal(res.result.body,snapshot); + assert.equal(res.result.status,200);assert.equal(res.result.body.revision,snapshot.revision); + assert.equal(res.result.body.appImage,snapshot.appImage); + assert.equal(res.result.body.revisionedMutations,true); }); class BackupConflict extends Error {} class InvalidBackupRequest extends Error {} +const mutationError=(res,error)=>res.status(error instanceof BackupConflict?409:error instanceof InvalidBackupRequest?400:503) + .json({updated:false,error:error instanceof BackupConflict?'BACKUP_CHANGED':'BACKUP_NOT_UPDATED'}); +test('revisioned mutation route requires a valid token before any backend call',async()=>{ + let calls=0;const res=mockResponse(); + await route({updateAppImage:async()=>{calls++;}},'/revisionedBackupMutation', + {sendBackupMutationError:mutationError,InvalidBackupRequest})({body:{operation:'updateAppImage',appId:'disposable'}},res); + assert.equal(res.result.status,400);assert.equal(calls,0); +}); +test('revisioned mutation route passes CAS token and acknowledges its application',async()=>{ + const token='a'.repeat(64),newRevision='b'.repeat(64),res=mockResponse();let received; + await route({updateAppImage:async(...args)=>{received=args;return {updated:true,newRevision};}},'/revisionedBackupMutation', + {sendBackupMutationError:mutationError,InvalidBackupRequest})({body:{operation:'updateAppImage',appId:'disposable',expectedRevision:token,walletsObject:{w:'ciphertext'}}},res); + assert.equal(res.result.status,200);assert.equal(res.result.body.updated,true);assert.equal(res.result.body.preconditionApplied,true); + assert.equal(res.result.body.newRevision,newRevision); + assert.equal(received[0],'disposable');assert.equal(received[2].w,'ciphertext');assert.equal(received[8],token); +}); +test('revisioned mutation route refuses a success without a transactional post-write revision',async()=>{ + const token='a'.repeat(64),res=mockResponse(); + await route({updateAppImage:async()=>({updated:true})},'/revisionedBackupMutation', + {sendBackupMutationError:mutationError,InvalidBackupRequest}) + ({body:{operation:'updateAppImage',appId:'disposable',expectedRevision:token}},res); + assert.equal(res.result.status,503); + assert.equal(res.result.body.updated,false); +}); +test('revisioned mutation route refuses unknown operations and reports conflicts',async()=>{ + const token='a'.repeat(64),invalid=mockResponse(),conflict=mockResponse(); + const handler=route({deleteBackup:async()=>{throw new BackupConflict();}},'/revisionedBackupMutation', + {sendBackupMutationError:mutationError,InvalidBackupRequest}); + await handler({body:{operation:'unknown',appId:'disposable',expectedRevision:token}},invalid); + await handler({body:{operation:'deleteBackup',appId:'disposable',expectedRevision:token}},conflict); + assert.equal(invalid.result.status,400);assert.equal(conflict.result.status,409); + assert.equal(conflict.result.body.error,'BACKUP_CHANGED'); +}); +test('v2 repair operation activates the incremental fence and acknowledges CAS',async()=>{ + const token='a'.repeat(64),newRevision='b'.repeat(64),res=mockResponse();let activated; + await route({},'/revisionedBackupMutation', + {sendBackupMutationError:mutationError,InvalidBackupRequest, + repairAppBackup:async(_body,fence)=>{activated=fence;return {updated:true,newRevision};}}) + ({body:{operation:'repairAppBackup',appId:'disposable',expectedRevision:token}},res); + assert.equal(res.result.status,200);assert.equal(res.result.body.preconditionApplied,true); + assert.equal(res.result.body.newRevision,newRevision); + assert.equal(activated,true); +}); for (const [error,status,message] of [[new BackupConflict(),409,'BACKUP_CHANGED'],[new InvalidBackupRequest(),400,'BACKUP_NOT_UPDATED'],[new Error('synthetic-private-canary'),503,'BACKUP_NOT_UPDATED']]) test(`repair failure returns stable code ${status}`,async()=>{ const res=mockResponse(); await route({}, '/repairAppBackup', {BackupConflict,InvalidBackupRequest,repairAppBackup:async()=>{throw error;}})({body:{appId:'disposable'}},res); diff --git a/tests/support/automaticBackupHttpFixture.cjs b/tests/support/automaticBackupHttpFixture.cjs new file mode 100644 index 0000000..415f1ed --- /dev/null +++ b/tests/support/automaticBackupHttpFixture.cjs @@ -0,0 +1,249 @@ +// Disposable HTTP fixture: exact production route handlers, service code, +// Mongoose schemas and transactions. No hosted bootstrapping or credentials. +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const vm = require('node:vm'); +const { createRequire } = require('node:module'); +const ts = require('typescript'); +const mongoose = require('mongoose'); +const express = require('express'); +const root = path.resolve(__dirname, '../..'); +let active = false; + +function sourceLoader(databaseUrl) { + const cache = new Map(); + const originalEnvironment = {}; + for (const [key, value] of Object.entries({ DATABASE_URL: databaseUrl, + LOCAL_DEV: 'true', ENVIRONMENT: 'DEVELOPMENT', BITCOIN_NETWORK: 'TESTNET' })) { + originalEnvironment[key] = process.env[key]; + process.env[key] = value; + } + const load = file => { + file = path.resolve(root, file); + if (cache.has(file)) return cache.get(file).exports; + const module = { exports: {} }; + cache.set(file, module); + const nativeRequire = createRequire(file); + const requireSource = name => { + // Configuration is provided above, never read a checkout/host .env. + if (name === 'dotenv') return { ...nativeRequire(name), config: () => ({ parsed: {} }) }; + if (name.startsWith('.')) { + const candidate = path.resolve(path.dirname(file), name); + const source = [candidate, `${candidate}.ts`, path.join(candidate, 'index.ts')] + .find(value => value.endsWith('.ts') && fs.existsSync(value)); + if (source) return load(source); + } + return nativeRequire(name); + }; + const javascript = ts.transpileModule(fs.readFileSync(file, 'utf8'), { + compilerOptions: { module: ts.ModuleKind.CommonJS, + target: ts.ScriptTarget.ES2020, esModuleInterop: true }, + }).outputText; + const wrapper = vm.runInThisContext(`(function(module,exports,require,__filename,__dirname){${javascript}\n})`, { filename: file }); + wrapper(module, module.exports, requireSource, file, path.dirname(file)); + return module.exports; + }; + return { load, restoreEnvironment() { + for (const [key, value] of Object.entries(originalEnvironment)) { + if (value === undefined) delete process.env[key]; else process.env[key] = value; + } + } }; +} + +function registerProductionRoutes(router, dependencies) { + const filename = path.join(root, 'src/routes/routes.ts'); + const text = fs.readFileSync(filename, 'utf8'); + const tree = ts.createSourceFile(filename, text, ts.ScriptTarget.Latest, true); + const selected = []; + const required = new Set(['/createNewApp', '/getBackupSnapshot', '/revisionedBackupMutation', '/getAppImage']); + function visit(node) { + if (ts.isFunctionDeclaration(node) && node.name?.text === 'sendBackupMutationError') selected.push(node.getText(tree)); + if (ts.isCallExpression(node) && node.expression.getText(tree) === 'router.post' && required.has(node.arguments[0]?.text)) { + required.delete(node.arguments[0].text); + selected.push(`${node.getText(tree)};`); + } + ts.forEachChild(node, visit); + } + visit(tree); + assert.equal(required.size, 0, 'Missing production backup route'); + const javascript = ts.transpileModule(selected.join('\n'), { + compilerOptions: { target: ts.ScriptTarget.ES2020 }, + }).outputText; + const names = Object.keys(dependencies); + vm.runInThisContext(`(function(router,${names.join(',')}){${javascript}\n})`, { filename }) + (router, ...names.map(name => dependencies[name])); +} + +async function createFixture(options = {}) { + assert.equal(active, false, 'Use one fixture per process and close it before another'); + const target = new URL(options.mongoUri || process.env.BACKUP_TEST_MONGO_URL || 'mongodb://127.0.0.1:27058'); + assert.equal(target.protocol, 'mongodb:'); + assert.ok(['127.0.0.1', 'localhost'].includes(target.hostname), 'Fixture Mongo must be loopback'); + assert.ok(!target.username && !target.password, 'Fixture never accepts credentials'); + const database = `keeper_auto_http_test_${process.pid}_${Date.now()}`; + target.pathname = `/${database}`; + target.search = '?directConnection=true&replicaSet=keeperQA'; + active = true; + const loader = sourceLoader(target.toString()); + let server; + try { + const db = loader.load('src/db.ts').default; + const service = loader.load('src/services/backupSnapshot.ts'); + const bhr = loader.load('src/services/bhr.ts'); + await mongoose.connection.asPromise(); + const models = [db.getAppImageModel(), db.getAppVaultImageModel(), + db.getVaultImageModel(), db.getLabelModel(), db.getVaultMapModel(), db.getAppModel()]; + await Promise.all(models.map(model => model.init())); + const trace = []; + const faults = []; + const replyTimers = new Set(); + const router = express.Router(); + router.use((req, res, next) => { + const entry = { path: req.path, operation: req.body?.operation, + appId: req.body?.appId, routeInvoked: false, committed: false }; + trace.push(entry); + const index = faults.findIndex(fault => { + const target = fault.path || (['before-snapshot-reply', 'malformed-metadata', 'malformed-json'].includes(fault.mode) + ? '/getBackupSnapshot' : '/revisionedBackupMutation'); + return target === req.path && (!fault.operation || fault.operation === entry.operation) + && (!fault.appId || fault.appId === entry.appId); + }); + let fault; + if (index >= 0) { + if (faults[index].after > 0) faults[index].after -= 1; + else fault = faults.splice(index, 1)[0]; + } + if (req.path === '/getBackupSnapshot') { + entry.routeInvoked = true; + if (fault?.mode === 'malformed-json') { + res.json = () => res.type('application/json').send('{'); + return next(); + } + if (fault?.mode === 'before-snapshot-reply' || fault?.mode === 'malformed-metadata') { + const originalJson = res.json.bind(res); + res.json = body => { + Promise.resolve().then(() => fault.beforeReply?.({ body, db, service, bhr })) + .then(changed => originalJson(fault.mode === 'malformed-metadata' + ? { ...body, appImage: { ...body.appImage, + ...(fault.metadata || { publicId: 'synthetic-metadata-mismatch' }) } } + : changed ?? body)).catch(next); + return res; + }; + } + return next(); + } + if (req.path !== '/revisionedBackupMutation') { entry.routeInvoked = true; return next(); } + if (fault?.mode === '404') { entry.status = 404; return res.status(404).json({ error: 'NOT_FOUND' }); } + const originalJson = res.json.bind(res); + res.json = body => { + entry.status = res.statusCode; + entry.response = body; + entry.committed = res.statusCode === 200 && body.updated === true; + if (entry.committed && fault?.mode === 'malformed') return originalJson( + Object.prototype.hasOwnProperty.call(fault, 'acknowledgement') + ? fault.acknowledgement + : { updated: true, preconditionApplied: true, newRevision: 'invalid' }); + if (entry.committed && fault?.mode === 'equal-old-ack') return originalJson({ updated: true, + preconditionApplied: true, newRevision: req.body.expectedRevision }); + if (entry.committed && fault?.mode === 'lost-reply') { req.socket.destroy(); return res; } + if (entry.committed && fault?.mode === 'delay-reply') { + const timer = setTimeout(() => { + replyTimers.delete(timer); originalJson(body); + }, fault.delayMs || 1000); + replyTimers.add(timer); return res; + } + return originalJson(body); + }; + const start = async () => { + if (fault?.beforeMutation) await fault.beforeMutation({ db, service, bhr, body: req.body }); + if (fault?.mode === '409') { + // A genuine independent transactional writer invalidates the supplied + // revision. The request then reaches the actual CAS and conflicts. + await service.withBackupMutation(req.body.appId, async session => { + await db.getAppImageModel().updateOne({ appId: req.body.appId }, + { $set: { version: 'synthetic-peer-write' } }, { session }); + return { updated: true }; + }, req.body.expectedRevision, true); + } + entry.routeInvoked = true; + next(); + }; + start().catch(next); + }); + const { createNewApp } = loader.load('src/services/app.ts'); + registerProductionRoutes(router, { bhr, createNewApp, ...service }); + const app = express(); + app.use(express.json({ limit: '50mb' })); + app.use(loader.load('src/localDevelopment.ts').localDevelopmentRoutes); + app.use(router); + app.use((_error, _req, res, _next) => res.status(503).json({ updated: false, error: 'FIXTURE_FAILED' })); + server = await new Promise(resolve => { const instance = app.listen(0, '127.0.0.1', () => resolve(instance)); }); + const url = `http://127.0.0.1:${server.address().port}/`; + const post = async (endpoint, body, expectedStatus = 200, headers = {}) => { + const response = await fetch(new URL(endpoint, url), { method: 'POST', + headers: { 'Content-Type': 'application/json', appversion: '2.6.4', ...headers }, body: JSON.stringify(body) }); + const data = await response.json(); + if (expectedStatus !== undefined) assert.equal(response.status, expectedStatus, + `Expected ${expectedStatus}, got ${response.status} on ${endpoint}: ${JSON.stringify(data)}`); + return { status: response.status, data }; + }; + const snapshot = async appId => (await post('getBackupSnapshot', { appId })).data; + return { + url, database, trace, db, service, bhr, post, snapshot, + armFault(fault) { + assert.ok(['404', '409', 'malformed', 'equal-old-ack', 'lost-reply', 'delay-reply', + 'before-snapshot-reply', 'malformed-metadata', 'malformed-json', undefined].includes(fault.mode)); + faults.push(fault); + }, + clearTrace() { trace.length = 0; }, + async reset() { faults.length = 0; trace.length = 0; await Promise.all(models.map(model => model.deleteMany({}))); }, + async prepareRestore(appId) { + const current = await snapshot(appId); + assert.equal(current.exists, true, 'Seed an encrypted backup before restore'); + // Actual free-tier App record makes production receipt validation take + // its normal no-provider branch. This is fixture setup, not a mocked + // subscription response or an automatic-write endpoint fallback. + const { PlebSubscription } = loader.load('src/interface.ts'); + await db.getAppModel().updateOne({ appId }, { $set: { + _id: `fixture-${appId}`, appId, publicId: current.appImage.publicId, + os: 'android', version: current.appImage.version, + subscription: PlebSubscription, + } }, { upsert: true, runValidators: true }); + }, + async seed(input) { + const image = input.appImage || input; + const appId = image.appId; + const current = await snapshot(appId); + const vaultObject = input.vaultObject || Object.fromEntries((input.allVaultImages || []).map(vault => + [vault.vaultId, { ...vault, signersData: vault.signersData || [] }])); + const labels = (input.labels || []).map(label => ({ id: label.id, + content: label.content ?? label.encryptedLabel })); + const repair = { appId, publicId: image.publicId || `public-${appId}`, + version: image.version || '2.6.4', subscription: image.subscription, + expectedRevision: current.revision, + walletObject: input.walletObject || image.wallets || {}, + signersObject: input.signersObject || image.signers || {}, + vaultObject, nodes: input.nodes || image.nodes || [], labels, + operation: 'repairAppBackup', replaceCurrentState: true }; + await post('revisionedBackupMutation', repair); + return snapshot(appId); + }, + async close() { + for (const timer of replyTimers) clearTimeout(timer); + if (server) { server.closeAllConnections(); await new Promise(resolve => server.close(resolve)); } + assert.equal(mongoose.connection.name, database, 'Refusing non-fixture cleanup'); + await mongoose.connection.dropDatabase(); + await mongoose.disconnect(); + for (const name of mongoose.modelNames()) mongoose.deleteModel(name); + loader.restoreEnvironment(); active = false; + }, + }; + } catch (error) { + if (server) server.close(); + await mongoose.disconnect(); loader.restoreEnvironment(); active = false; + throw error; + } +} + +module.exports = { createFixture };