[LeeroyHannigan]

What

Adds a Database Backup and Restore section to the admin guide (docs/manuals/05-admin-guide.md) with commands measured on SQLite and PostgreSQL, and points the upgrade manual, the security model and the deployment guide at it. The admin guide's Data Recovery entry becomes a pointer to the section.

• Stopping the server: extenddb stop exits 1 and stops nothing for a server started with serve --foreground, which writes no PID file, and when its --config path does not name the server's config file. The section says to stop the server through whatever runs it. Each procedure that needs the server stopped is one block in a subshell under set -e. The block first runs extenddb status --config /path/to/extenddb.toml | grep -q 'is not running', so it ends before it touches a file or a database while the server is up.
• SQLite: VACUUM INTO while the server runs, and a copy taken with the server stopped that copies the database file and whichever of its -wal and -shm files exist. The restore copies the saved files into a new work directory and runs PRAGMA integrity_check on them. It then moves the current files into the work directory and moves the saved files into place. The section says not to use the sqlite3 .backup command while the server takes writes.
• PostgreSQL: two pg_dump runs disagree when a table is created or deleted between them. The section gives a dump pair taken with the servers stopped and pg_basebackup with its prerequisites (replication entry in pg_hba.conf, wal_keep_size for -X fetch). The dump block also requires no session of the application role on either database. The restore renames databases that hold the target names to <name>_before_restore and restores the pair. It prints the catalog's data_database_name and data_database_connection_string rows. When they name another database or address, the section sets both to their full new values and reads them back. The renamed databases are removed only after the restored server passes its check, and a failed restore is undone by renaming.
• Every restore ends with a check that reads every table: aws dynamodb list-tables, then scan --select COUNT on each table.
• What a copy holds: the catalog keeps the AES-256-GCM key next to the encrypted secret access keys, the bcrypt password hashes, and on PostgreSQL the application role's password inside the data connection string. Copies made under umask 022 have mode 0644.
• Upgrade manual: step 2 makes plain-text dumps, which pg_restore refuses. Its new sentence now sends them to the Rollback Procedure, which restores them with psql -f. Deployment guide: archived WAL restores only on top of a base backup.

Why

The admin guide and the security model showed pg_dump extenddb_catalog and pg_dump extenddb_catalog_data. init creates extenddb_catalog and extenddb, so the second command names a database that does not exist. Both pages said ExtendDB stores all data in PostgreSQL, which leaves out SQLite. No page said how to copy a SQLite deployment, and none said whether two separate pg_dump runs give a pair that agrees. CreateBackup keeps its backups in the catalog on this release, so they are lost with the database. The off-host design (docs/design/15-backup-restore.md) keeps the database tools for whole-instance recovery.

Related: #624 (off-host backup and restore), #707 (keeping the encryption key outside the catalog).

Testing done

Measured on 1 November 2026 at main f49c5f823, with release builds of extenddb for SQLite and for PostgreSQL, sqlite3 3.46.0 (the SQLite version the server links) and PostgreSQL 16.15 (pgvector/pgvector:pg16). A client sent PutItem and UpdateItem one request at a time to two tables, one of them with a GSI whose key the UpdateItem changes, and logged each acknowledged write with its send and acknowledgement times. The copies taken while a server ran were taken under those writes. The copies taken with a server stopped were taken after its last acknowledged write. A table backup was taken before any copy. For each restored copy a server was started and checked:

• the copy held every write acknowledged before it began and none sent after it ended;
• the writes present formed a prefix of the log, so the copy describes one instant;
• a GSI query for each key value returned the base table's items with that value;
• the access key created before the copy authenticated, and a wrong secret got InvalidSignatureException;
• the table backup was listed AVAILABLE.

Each command block in the section was then run as printed, with only its placeholders replaced, against local servers; the PostgreSQL client tools ran inside the database container. Two blocks were not rerun: the DROP DATABASE of the renamed databases, and the tar extract of a base backup, which is unchanged.

SQLite:

Procedure	Result
VACUUM INTO, 1 MB database, under writes	0.05 to 0.07 s; every earlier write present; the last write present was acknowledged 20 to 21 ms after the command started
VACUUM INTO, 284 MB database, about 100 writes a second	2.0 s; 206 writes acknowledged during the copy; 34530 earlier writes present; checks pass
.backup, 284 MB database, about 100 writes a second	not finished after 300 s, and again after 120 s; 4.97 s with no client writing
A restore or a stopped-server copy started while the server ran without a PID file	extenddb stop exits 1 (wrote no PID file); the block ends at the extenddb status check; no file touched
-wal after extenddb stop	kept after seven of eight stops (up to 4.2 MB), removed after one
Stopped-server copy with -wal and -shm present, and with both absent (after the sqlite3 shell closed the database)	three files copied, then one; both restores served every item written before the copy
Restore into a directory with and without -wal and -shm	old files moved into the work directory; the copy served complete
Restore of a database file copied without its non-empty -wal	PRAGMA integrity_check: database disk image is malformed; the block ends before it moves anything
Copy put in place with the old -wal and -shm beside it	one run: ListTables answers, Scan is HTTP 500; another: the server cannot open the database and exits 34 s after the start, and the table-read check fails at its first request
File modes	copies 0600 under umask 077 and 0644 under umask 022; extenddb serve set a restored 0644 file to 0600
sqlite3 3.7.17	VACUUM INTO fails with near "INTO": syntax error; the schema cannot be read (malformed database schema (idx_login_attempts_source_ip_time))

PostgreSQL:

Procedure	Result
pg_dump of the catalog, then of the data database, under writes	0.15 s each; items consistent as of the data dump
One table deleted and one created between the two dumps	deleted table listed and ACTIVE, its item requests ResourceNotFoundException, DeleteTable removes it; created table missing, its 20 items left in a data table the catalog does not name
Stopped-server dump block while a server ran	ends at the extenddb status check; with a config whose port nothing listens on, ends at the session check (19 sessions); no dump written
Stopped-server dump block	3.7 s down, 3.0 s of it the server's start; restored pair holds all 5792 writes
Restore under new names on the same server	the restored catalog's rows name the original extenddb; with the rows unchanged the server served the original database's current items. After both rows were set: sessions on the restored databases, items written to the restored server absent from the original
Restore under the original names after renaming the current pair	restored server passes the checks; tables owned by the application role (28 and 9). The rename is refused while a server is connected (is being accessed by other users)
pg_restore of a truncated catalog dump	could not read from input file: end of file, exit 1; renaming the new pair aside and the old pair back restored service with every write present
New PostgreSQL server	pg_restore before the role exists stops at role "extenddb" does not exist; with roles from --no-role-passwords the server fails password authentication failed for user "extenddb" until the password is set; then checks pass
pg_basebackup -Ft -X fetch under writes	0.75 s for 59 MB and 3.0 s for 147 MB; the restored server logged consistent recovery state reached, and ExtendDB served it with the config file unchanged
pg_basebackup from a non-loopback address	no pg_hba.conf entry for replication connection; works after a host replication line and a reload
pg_dumpall --globals-only as a role with CREATEDB and CREATEROLE	permission denied for table pg_authid; with --no-role-passwords it succeeds
Upgrade manual step 2 dumps	pg_restore refuses them (input file appears to be a text format dump); the Rollback Procedure's CREATE DATABASE and psql -f lines restore them with every write present
extenddb settings set data_database_name	Setting 'data_database_name' is read-only and cannot be changed via this command.

Gates: cargo test --locked -p extenddb-storage-postgres --test doc_version_literals (3 passed). A test merge with #666, #682, #691, #228 and #668 is clean. #491 and #523 conflict on lines this PR replaces: they change the capitalization of extenddb stores all data in PostgreSQL in the admin guide and of extenddb stores all state in PostgreSQL in the security model, and this PR removes both lines.

Statements that depend on open PRs:

• fix: create SQLite database files 0600 and set wider ones to 0600 when opened #691 makes every command that opens a SQLite database set it and its -wal, -shm and -journal files to 0600. The section says extenddb serve does this, which is true on main; no statement needs fix: create SQLite database files 0600 and set wider ones to 0600 when opened #691.
• feat(migrations): one list of catalog migrations with a ledger in every store, for SQLite and PostgreSQL #666 has migrate connect as the application role, which must own every table of both PostgreSQL databases. The restore steps leave the tables owned by that role (measured). feat(migrations): one list of catalog migrations with a ledger in every store, for SQLite and PostgreSQL #666 also refuses a catalog whose two data database rows name different databases; the restore prints both rows, sets both to their full new values when either names another database or address, and reads them back.
• fix: create export files 0600 and the directories made for them 0700 #682 sets the modes of export files. The section does not cover export files.
• docs: state the catalog trust boundary for the credential encryption key #228 rewrites the last sentence of the security model's Backup and Recovery section. This PR leaves that sentence as it is.
• If Credential encryption key: keep it outside the catalog and add a rotation path #707 moves the encryption key out of the catalog, the section's paragraph on what a copy holds must change with it.

Follow-up: the section takes 05-admin-guide.md to 1210 lines. docs/design/15-backup-restore.md section 10 plans a page of its own for it, docs/manuals/14-backup-and-restore.md. Moving it there is left to a separate change, so that this one does not widen its conflicts with #491 and #523 in the admin guide.

Found while measuring, not changed here:

• extenddb catalog-check on a consistent restored PostgreSQL pair reports each data table as orphaned and each table as missing, and --fix would drop the tables it calls orphaned (catalog-check expects pre-id relation names; --fix drops live data tables on PostgreSQL #652). The section does not mention catalog-check.
• extenddb serve in daemon mode printed extenddb server started and exited 0, and the daemon then exited because it could not open a damaged SQLite file.

Checklist

• I have read CONTRIBUTING.md
• All tests pass (cargo test --workspace)
• Code is formatted (cargo fmt --check)
• Clippy is clean (cargo clippy -- -W clippy::pedantic)
• I have added or updated tests for new functionality
• I have updated documentation if behavior changed
• Breaking changes are noted below (if any)
• If this changes the wire protocol, Storage trait, auth model, on-disk
  format, or public CLI surface, an RFC has been accepted or is linked
  below. Otherwise, an ADR captures the decision (link below).

ADR / RFC: n/a

---

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.