fix(security): secure recovery authorization and deployment defaults

.dockerignore

@@ -4,6 +4,7 @@ npm-debug.log
 yarn-debug.log
 yarn-error.log
 .env
+deploy/
 *.md
 .DS_Store
 .vscode

.gitignore

@@ -5,6 +5,9 @@ coverage/
 logs/
 tsconfig.tsbuildinfo
 out/
+deploy/ca/
+deploy/certs/
+deploy/clients/
 .vscode/
 *.iml
 .nyc_output/

README.md

@@ -12,7 +12,7 @@ Key features include:
 - **Complete Infrastructure Control** - Host and manage all components in your own secure environment.
 - **KMS/HSM Integration** - Bring your own KMS or HSM by implementing the provided [advanced wallets key provider API interface specification](./key-provider-api-spec.yaml). Reference implementations available for [AWS HSM](./demo-key-provider-script/aws-interface.md) and [Dinamo HSM](./demo-key-provider-script/dinamo-interface.md).
 - **Network Isolation** - Advanced Wallet Manager operates in a completely isolated network segment with no external internet access.
-- **mTLS Security** - Optional mutual TLS with client certificate validation for secure inter-service communications.
+- **mTLS Security** - Mutual TLS with client certificate validation for secure inter-service communications (required for network-accessible recovery).
 - **Flexible Configuration** - Environment-based setup with file or variable-based certificates.
 
 ## Table of Contents
@@ -155,7 +155,7 @@ curl -X POST http://localhost:3081/advancedwallet/ping
 curl -X POST http://localhost:3081/ping/advancedWalletManager
 ```
 
-> **Note:** You should only use `TLS_MODE=disabled` for local development and testing. Always use mTLS in production environments. For information about configuring mTLS in production, see the [Production Setup](#production-setup) section.
+> **Note:** You should only use `TLS_MODE=disabled` for local development and testing. Never enable recovery on a non-local unauthenticated listener. Always use mTLS in production environments. For information about configuring mTLS in production, see the [Production Setup](#production-setup) section.
 
 ## Configuration
 
@@ -208,7 +208,8 @@ These settings are only required when you want to use a **separate AWM instance
 
 | Variable             | Description                                         | Default                | Applies To |
 | -------------------- | --------------------------------------------------- | ---------------------- | ---------- |
-| `RECOVERY_MODE`      | Enable recovery mode for wallet recovery operations | `false`                | Both       |
+| `RECOVERY_MODE`      | Enable recovery temporarily; disable immediately after use | `false`                | Both       |
+| `RECOVERY_AUTH_TOKEN` | Shared high-entropy secret (at least 32 bytes) required on both services while recovery is enabled | - | Both |
 | `HTTP_LOGFILE`       | Path to HTTP access log file                        | `logs/http-access.log` | Both       |
 | `KEEP_ALIVE_TIMEOUT` | Keep-alive timeout in milliseconds                  | -                      | Both       |
 | `HEADERS_TIMEOUT`    | Headers timeout in milliseconds                     | -                      | Both       |
@@ -363,7 +364,7 @@ The application includes a Docker Compose configuration that runs both Advanced
 The Docker Compose setup creates two isolated services:
 
 - **Advanced Wallet Manager (AWM)**: Runs in an isolated internal network with no external access for maximum security.
-- **Master BitGo Express (MBE)**: Connects to both internal network (for AWM communication) and public network (for external API access).
+- **Master BitGo Express (MBE)**: Connects to the internal network for AWM communication and an outbound network for BitGo API access. No ports are published to the host.
 - **Network Isolation**: AWM is completely isolated from external networks and only accessible through MBE.
 
 ### Network Configuration
@@ -377,33 +378,29 @@ The setup creates two distinct networks:
    - No external internet access for security
 
 2. **my-public-network**:
-   - Public bridge network
-   - Used for external access to MBE APIs
-   - Connected to host networking
+   - Outbound bridge network for MBE's BitGo API calls; it does not publish MBE to the host.
+   - Only attach trusted containers: containers on the same Docker network can reach each other's listeners.
 
 ### Prerequisites
 
-1. **Install Docker and Docker Compose**
-2. **Ensure your key provider API implementation is running** on your host machine (typically on port 3000)
-
-### Quick Start
-
-#### 1. Start Services
+1. Install Docker Compose and OpenSSL.
+2. Configure a reachable **HTTPS** key provider. Its CA certificate must be available as `deploy/certs/awm/key-provider-ca.pem`; the key provider must trust the generated AWM client certificate (or replace the bootstrap credentials with your own). Do not use the sample `demo.key`, `demo.crt`, or checked-in test certificates in a deployment.
+3. Generate distinct service and client certificates for **local evaluation only**. The bootstrap CA is a 30-day local CA: use your organization's PKI and a trusted key provider in production. Keep `deploy/ca` and `deploy/clients` private and off container volumes.
 
 ```bash
-# Navigate to project directory
-cd advanced-wallet
-
-# Start both services in background
-docker-compose up -d
+./scripts/bootstrap-compose-certs.sh
+# Install the CA cert used by the key provider to sign its HTTPS server certificate:
+cp /secure/path/to/key-provider-ca.pem deploy/certs/awm/key-provider-ca.pem
+export KEY_PROVIDER_URL=https://your-key-provider:3000
+# fingerprints.env pins the MBE client on AWM and your external client on MBE.
+docker compose --env-file deploy/certs/fingerprints.env up -d --build
 ```
 
-#### 2. Stop Services
+The compose file uses mTLS for MBE → AWM and for each inbound connection. It does **not** publish port 3081; to reach MBE, provision a separately secured client path and allowlisted mTLS client, or connect from a trusted private network. The generated `deploy/clients/mbe-client.{crt,key}` are for local evaluation only. `BIND=0.0.0.0` listens inside each container, **not** on a host port. Keep the internal network exclusive to trusted services, and never expose AWM directly. For production set `BITGO_ENV=prod` and replace all bootstrap certificates with certificates issued by your PKI.
 
-```bash
-# Stop and remove containers
-docker-compose down
-```
+**Recovery procedure:** generate a random secret (`openssl rand -hex 32`), provide it as `RECOVERY_AUTH_TOKEN` and set `RECOVERY_MODE=true` on **both** services for the recovery window only. Set `X-Recovery-Token: <secret>` on requests to `/advancedwallet/recovery` and `/advancedwallet/recoveryconsolidations`; MBE forwards the configured secret to AWM recovery endpoints. `Authorization: Bearer ...` is the separate BitGo API token and does not authorize recovery. The token is required even with mTLS; never send it over a network without TLS. Recovery requests can return fully signed transactions from xpubs/commonKeychain alone: treat those public key materials as sensitive and rotate the recovery secret after use. Disable `RECOVERY_MODE` immediately after recovery and restart both services. Recovery with TLS disabled is only allowed on a loopback listener.
+
+Stop the stack with `docker compose --env-file deploy/certs/fingerprints.env down`.
 
 ## API Endpoints
 
@@ -478,7 +475,7 @@ export KEY_PROVIDER_SERVER_CA_CERT_PATH=/secure/certs/key-provider-ca.crt
 # Security settings - production-grade
 export CLIENT_CERT_ALLOW_SELF_SIGNED=false
 export KEY_PROVIDER_SERVER_CERT_ALLOW_SELF_SIGNED=false
-export MTLS_ALLOWED_CLIENT_FINGERPRINTS=sha256:1a2b3c...,sha256:4d5e6f...
+export MTLS_ALLOWED_CLIENT_FINGERPRINTS=<uppercase-hex-sha256-fingerprint-of-MBE-client>
 export BITGO_ENV=prod
 npm start
 ```
@@ -503,7 +500,7 @@ export AWM_SERVER_CA_CERT_PATH=/secure/certs/awm-ca.crt
 # Security settings - production-grade
 export CLIENT_CERT_ALLOW_SELF_SIGNED=false
 export AWM_SERVER_CERT_ALLOW_SELF_SIGNED=false
-export MTLS_ALLOWED_CLIENT_FINGERPRINTS=sha256:7g8h9i...,sha256:0j1k2l...
+export MTLS_ALLOWED_CLIENT_FINGERPRINTS=<uppercase-hex-sha256-fingerprint-of-approved-client>
 npm start
 ```
 

docker-compose.yml

@@ -1,120 +1,75 @@
-version: '3.8'
-
+# Bootstrap certificates with scripts/bootstrap-compose-certs.sh before starting.
+# Supply KEY_PROVIDER_URL (HTTPS) and deploy/certs/awm/key-provider-ca.pem from your key provider.
+# Recovery is disabled by default; enable temporarily only with RECOVERY_AUTH_TOKEN set.
 services:
-  # Service for advanced-wallet-manager (AWM)
   advanced-wallet-manager:
-    build: . # Build from the Dockerfile inside the repo
-    container_name: advanced-wallet-manager
+    build: .
+    user: "${COMPOSE_UID:?Run scripts/bootstrap-compose-certs.sh}:${COMPOSE_GID:?Run scripts/bootstrap-compose-certs.sh}"
     networks:
-      - my-internal-network # Only part of the internal network
+      - my-internal-network
     environment:
-      # Application mode (required)
       - APP_MODE=advanced-wallet-manager
-
-      # Network settings
       - ADVANCED_WALLET_MANAGER_PORT=3080
-      - BIND=0.0.0.0
+      - BIND=0.0.0.0 # Container-only listener; no host port published
+      - TLS_MODE=mtls
+      - SERVER_TLS_KEY_PATH=/app/certs/awm-server.key
+      - SERVER_TLS_CERT_PATH=/app/certs/awm-server.crt
+      - MTLS_ALLOWED_CLIENT_FINGERPRINTS=${AWM_ALLOWED_CLIENT_FINGERPRINTS:?Run scripts/bootstrap-compose-certs.sh and supply --env-file deploy/certs/fingerprints.env}
+      - KEY_PROVIDER_URL=${KEY_PROVIDER_URL:?Set an HTTPS key provider URL}
+      - KEY_PROVIDER_SERVER_CA_CERT_PATH=/app/certs/key-provider-ca.pem
+      - KEY_PROVIDER_CLIENT_TLS_KEY_PATH=/app/certs/awm-key-provider-client.key
+      - KEY_PROVIDER_CLIENT_TLS_CERT_PATH=/app/certs/awm-key-provider-client.crt
       - TIMEOUT=305000
       - KEEP_ALIVE_TIMEOUT=65000
       - HEADERS_TIMEOUT=66000
 
-      # TLS settings
-      - TLS_MODE=disabled
-      - CLIENT_CERT_ALLOW_SELF_SIGNED=true
-
-      # Key provider settings (required)
-      - KEY_PROVIDER_URL=http://172.20.0.1:3000 # UPDATE TO YOUR OWN key provider URL
-      - ALLOW_PLAINTEXT_KEY_PROVIDER=true # Development only: permits http:// KEY_PROVIDER_URL with TLS_MODE=disabled
-      - KEY_PROVIDER_SERVER_CERT_ALLOW_SELF_SIGNED=true
-
-      # Optional key provider TLS settings (uncomment if using mTLS with key provider)
-      # - KEY_PROVIDER_SERVER_CA_CERT_PATH=/path/to/key-provider-ca-cert.pem
-      # - KEY_PROVIDER_CLIENT_TLS_KEY_PATH=/path/to/key-provider-client-key.pem
-      # - KEY_PROVIDER_CLIENT_TLS_CERT_PATH=/path/to/key-provider-client-cert.pem
-      # - KEY_PROVIDER_CLIENT_TLS_KEY=<key-content>
-      # - KEY_PROVIDER_CLIENT_TLS_CERT=<cert-content>
-
-      # Optional server TLS settings (uncomment if using mTLS)
-      # - SERVER_TLS_KEY_PATH=/path/to/server-key.pem
-      # - SERVER_TLS_CERT_PATH=/path/to/server-cert.pem
-      # - SERVER_TLS_KEY=<key-content>
-      # - SERVER_TLS_CERT=<cert-content>
-      # - MTLS_ALLOWED_CLIENT_FINGERPRINTS=ABC123,DEF456
-
-      # Logging and debug
       - HTTP_LOGFILE=logs/http-access.log
-      - RECOVERY_MODE=true
-      # Default to local signing mode 
+      - RECOVERY_MODE=${RECOVERY_MODE:-false}
+      - RECOVERY_AUTH_TOKEN=${RECOVERY_AUTH_TOKEN:-}
       - NODE_ENV=production
       - LOG_LEVEL=info
     restart: always
-    ports: [] # No public ports exposed
     volumes:
-      - ./logs:/app/logs # Mount logs directory
+      - ./logs:/app/logs
+      - ./deploy/certs/awm:/app/certs:ro
 
-  # Service for master-bitgo-express (MBE) - both internal and publicly accessible
   master-bitgo-express:
-    build: . # Build from the Dockerfile inside the repo
-    container_name: master-bitgo-express
+    build: .
+    user: "${COMPOSE_UID:?Run scripts/bootstrap-compose-certs.sh}:${COMPOSE_GID:?Run scripts/bootstrap-compose-certs.sh}"
     networks:
-      - my-internal-network # Connect to the internal network for internal communication
-      - my-public-network # Connect to the public network for external access
+      - my-internal-network
+      - my-public-network # Outbound BitGo API access; no host port published
     environment:
-      # Application mode (required)
       - APP_MODE=master-express
-
-      # Network settings
       - MASTER_EXPRESS_PORT=3081
-      - BIND=0.0.0.0
+      - BIND=0.0.0.0 # Container-only listener; no host port published
+      - TLS_MODE=mtls
+      - SERVER_TLS_KEY_PATH=/app/certs/mbe-server.key
+      - SERVER_TLS_CERT_PATH=/app/certs/mbe-server.crt
+      - MTLS_ALLOWED_CLIENT_FINGERPRINTS=${MBE_ALLOWED_CLIENT_FINGERPRINTS:?Run scripts/bootstrap-compose-certs.sh and supply --env-file deploy/certs/fingerprints.env}
+      - ADVANCED_WALLET_MANAGER_URL=https://advanced-wallet-manager:3080
+      - AWM_SERVER_CA_CERT_PATH=/app/certs/ca.crt
+      - AWM_CLIENT_TLS_KEY_PATH=/app/certs/mbe-awm-client.key
+      - AWM_CLIENT_TLS_CERT_PATH=/app/certs/mbe-awm-client.crt
+      - BITGO_ENV=test # Change to prod for production
+      - BITGO_DISABLE_ENV_CHECK=false
+      - BITGO_AUTH_VERSION=2
       - TIMEOUT=305000
       - KEEP_ALIVE_TIMEOUT=65000
       - HEADERS_TIMEOUT=66000
-
-      # BitGo API settings
-      - BITGO_ENV=test
-      - BITGO_DISABLE_ENV_CHECK=true
-      - BITGO_AUTH_VERSION=2
-      # - BITGO_CUSTOM_ROOT_URI=https://custom-bitgo-api.com
-      # - BITGO_CUSTOM_BITCOIN_NETWORK=testnet
-
-      # Advanced Wallet Manager connection (required)
-      - ADVANCED_WALLET_MANAGER_URL=http://advanced-wallet-manager:3080
-      - AWM_SERVER_CERT_ALLOW_SELF_SIGNED=true
-
-      # Optional AWM TLS settings (uncomment if using mTLS with AWM)
-      # - AWM_SERVER_CA_CERT_PATH=/path/to/awm-ca-cert.pem
-      # - AWM_CLIENT_TLS_KEY_PATH=/path/to/awm-client-key.pem
-      # - AWM_CLIENT_TLS_CERT_PATH=/path/to/awm-client-cert.pem
-      # - AWM_CLIENT_TLS_KEY=<key-content>
-      # - AWM_CLIENT_TLS_CERT=<cert-content>
-
-      # TLS settings
-      - TLS_MODE=disabled
-      - CLIENT_CERT_ALLOW_SELF_SIGNED=true
-
-      # Optional server TLS settings (uncomment if using mTLS)
-      # - SERVER_TLS_KEY_PATH=/path/to/server-key.pem
-      # - SERVER_TLS_CERT_PATH=/path/to/server-cert.pem
-      # - SERVER_TLS_KEY=<key-content>
-      # - SERVER_TLS_CERT=<cert-content>
-      # - MTLS_ALLOWED_CLIENT_FINGERPRINTS=ABC123,DEF456
-
-      # Logging and debug
       - HTTP_LOGFILE=logs/http-access.log
-      - RECOVERY_MODE=true
+      - RECOVERY_MODE=${RECOVERY_MODE:-false}
+      - RECOVERY_AUTH_TOKEN=${RECOVERY_AUTH_TOKEN:-}
       - NODE_ENV=production
       - LOG_LEVEL=info
     restart: always
-    ports:
-      - '3081:3081' # Expose MBE publicly on port 3081
     volumes:
-      - ./logs:/app/logs # Mount logs directory
+      - ./logs:/app/logs
+      - ./deploy/certs/mbe:/app/certs:ro
 
-# Networks section
 networks:
   my-internal-network:
-    driver: bridge # Internal communication network, no access to the internet
-    internal: true # Ensures this network is not accessible from outside
-
+    driver: bridge
+    internal: true
   my-public-network:
-    driver: bridge # Public network, allowing external access to MBE
+    driver: bridge

scripts/bootstrap-compose-certs.sh

@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+# Development bootstrap only: replace this CA and its certificates for production.
+set -euo pipefail
+umask 077
+cd "$(dirname "$0")/.."
+
+if [[ -e deploy/certs || -e deploy/ca || -e deploy/clients ]]; then
+  echo 'deploy credentials already exist; refusing to overwrite them' >&2
+  exit 1
+fi
+if [[ $(id -u) -eq 0 ]]; then
+  echo 'Run the bootstrap as an unprivileged user so containers can read its private keys without running as root' >&2
+  exit 1
+fi
+mkdir -p deploy logs
+chmod 700 logs
+work=$(mktemp -d deploy/.bootstrap.XXXXXX)
+trap 'rm -rf "$work"' EXIT
+mkdir -p "$work/ca" "$work/certs/awm" "$work/certs/mbe" "$work/clients"
+
+openssl req -x509 -newkey rsa:3072 -nodes -days 30 -sha256 \
+  -keyout "$work/ca/ca.key" -out "$work/ca/ca.crt" \
+  -subj '/CN=Advanced Wallets local development CA'
+
+issue_cert() {
+  local dir=$1 name=$2 cn=$3 usage=$4 san=$5
+  openssl req -newkey rsa:3072 -nodes -sha256 \
+    -keyout "$dir/$name.key" -out "$work/$name.csr" -subj "/CN=$cn"
+  printf 'subjectAltName=%s\nextendedKeyUsage=%s\n' "$san" "$usage" > "$work/$name.ext"
+  openssl x509 -req -in "$work/$name.csr" -CA "$work/ca/ca.crt" \
+    -CAkey "$work/ca/ca.key" -CAcreateserial -out "$dir/$name.crt" \
+    -days 30 -sha256 -extfile "$work/$name.ext"
+}
+
+issue_cert "$work/certs/awm" awm-server advanced-wallet-manager serverAuth 'DNS:advanced-wallet-manager'
+issue_cert "$work/certs/mbe" mbe-server localhost serverAuth 'DNS:localhost,IP:127.0.0.1'
+issue_cert "$work/certs/mbe" mbe-awm-client mbe-awm-client clientAuth 'DNS:mbe-awm-client'
+issue_cert "$work/certs/awm" awm-key-provider-client awm-key-provider-client clientAuth 'DNS:awm-key-provider-client'
+issue_cert "$work/clients" mbe-client mbe-client clientAuth 'DNS:mbe-client'
+cp "$work/ca/ca.crt" "$work/certs/awm/ca.crt"
+cp "$work/ca/ca.crt" "$work/certs/mbe/ca.crt"
+
+fingerprint() {
+  openssl x509 -in "$1" -noout -fingerprint -sha256 | cut -d= -f2 | tr -d ':'
+}
+{
+  printf 'AWM_ALLOWED_CLIENT_FINGERPRINTS=%s\n' "$(fingerprint "$work/certs/mbe/mbe-awm-client.crt")"
+  printf 'MBE_ALLOWED_CLIENT_FINGERPRINTS=%s\n' "$(fingerprint "$work/clients/mbe-client.crt")"
+  printf 'COMPOSE_UID=%s\nCOMPOSE_GID=%s\n' "$(id -u)" "$(id -g)"
+} > "$work/certs/fingerprints.env"
+
+mv "$work/ca" "$work/certs" "$work/clients" deploy/
+rm -rf "$work"
+trap - EXIT
+printf '%s\n' 'Generated 30-day local certificates under deploy/. Supply deploy/certs/awm/key-provider-ca.pem from your key provider before starting.'

src/__tests__/api/advancedWalletManager/recoveryMpc.test.ts

@@ -28,10 +28,12 @@ describe('recoveryMpc', () => {
       tlsMode: TlsMode.DISABLED,
 
       recoveryMode: true,
+      recoveryAuthToken: 'test-recovery-token-at-least-32-characters',
     };
 
     const app = expressApp(config);
     agent = request.agent(app);
+    agent.set('x-recovery-token', config.recoveryAuthToken!);
   });
 
   afterEach(() => {
@@ -175,10 +177,12 @@ describe('recoveryMpc', () => {
         httpLoggerFile: '',
         tlsMode: TlsMode.DISABLED,
         recoveryMode: true,
+        recoveryAuthToken: 'test-recovery-token-at-least-32-characters',
       };
 
       const dualApp = expressApp(dualCfg);
       const dualAgent = request.agent(dualApp);
+      dualAgent.set('x-recovery-token', dualCfg.recoveryAuthToken!);
 
       // User key served from primary KMS
       const userKmsNock = nock(primaryKmsUrl)