From 34a0dbfa6a1c2fbfb5d27c4eab7d4b88a5f5f82b Mon Sep 17 00:00:00 2001 From: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com> Date: Thu, 29 Oct 2026 23:46:18 +0000 Subject: [PATCH] docs: correct the quick start, monitoring commands, backend count and design index The README quick start ran aws dynamodb list-tables without credentials, and the AWS CLI stopped with "Unable to locate credentials". The quick start now reads the default account's ID with extenddb manage list-accounts, creates an IAM user, attaches a policy that allows DynamoDB, creates an access key, and exports the key before the request. init prints the admin password but not the account ID. A key this server did not issue gets UnrecognizedClientException, so configured AWS credentials do not help. /metrics requires admin HTTP Basic auth and answers 401 without it. The metrics commands in the README, the admin guide and the event ticketing demo now pass the admin credentials, and the deployment guide's monitoring checklist says the poller needs them. The demo's metrics and console URLs had **18443** as the port, which curl refuses, so the asterisks are removed. The storage design status line said PostgreSQL is the only supported backend. It now names the four backends in crates/: PostgreSQL, SQLite, MongoDB and Cassandra. #412 made the same correction to the README crate list and the architecture guide overview. The design index linked version 2.0 of the auth design, which 05-component-auth-v3.md supersedes. The index row now links version 3 and names version 2.0 as superseded. The Linux and macOS install manuals and the MongoDB setup page also run list-tables before any access key exists. #523 removes those pages. Refs #663 --- README.md | 22 ++++++++++++++++++++-- docs/design/04-component-storage.md | 4 ++-- docs/design/README.md | 2 +- docs/manuals/05-admin-guide.md | 3 ++- docs/manuals/11-deployment-guide.md | 2 +- docs/manuals/13-event-ticketing-demo.md | 8 ++++---- 6 files changed, 30 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 20e8a0cf7..7393c1c66 100755 --- a/README.md +++ b/README.md @@ -41,6 +41,24 @@ cargo build --release # Use with any AWS SDK (TLS with self-signed cert โ€” trust via AWS_CA_BUNDLE) export AWS_CA_BUNDLE=~/.extenddb/tls/cert.pem + +# Requests must be signed with an access key this server issued. Create an IAM +# user, attach a policy that allows DynamoDB, and create an access key for the +# user, with the admin password that init printed. list-accounts prints the +# account_id of the default account that init created. +./target/release/extenddb manage --user admin --password list-accounts +./target/release/extenddb manage --user admin --password \ + create-user --account-id --user-name quickstart +./target/release/extenddb manage --user admin --password \ + put-user-policy --account-id --user-name quickstart \ + --policy-name FullAccess \ + --policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"dynamodb:*","Resource":"*"}]}' +./target/release/extenddb manage --user admin --password \ + create-access-key --account-id --user-name quickstart + +# Set the access_key_id and secret_access_key that create-access-key printed +export AWS_ACCESS_KEY_ID= +export AWS_SECRET_ACCESS_KEY= aws dynamodb list-tables --endpoint-url https://127.0.0.1:18443 --region us-east-1 ``` @@ -171,8 +189,8 @@ extenddb init --catalog-db extenddb_catalog \ # Health check curl --cacert ~/.extenddb/tls/cert.pem https://127.0.0.1:18443/health -# JSON metrics (DynamoDB CloudWatch-style) -curl --cacert ~/.extenddb/tls/cert.pem https://127.0.0.1:18443/metrics +# JSON metrics (DynamoDB CloudWatch-style), admin credentials required +curl --cacert ~/.extenddb/tls/cert.pem -u admin: https://127.0.0.1:18443/metrics # Syslog (Linux) journalctl -t extenddb -f diff --git a/docs/design/04-component-storage.md b/docs/design/04-component-storage.md index dcfda3a3d..fb50916a6 100755 --- a/docs/design/04-component-storage.md +++ b/docs/design/04-component-storage.md @@ -15,8 +15,8 @@ The trait-based design allows new storage backends to be added by implementing t factory function, with no changes needed to the `engine` or `server` crates. The factory pattern enables runtime backend selection based on configuration. -**Current status**: PostgreSQL is the only supported backend. The trait architecture and plugin infrastructure provide -the foundation for future backend implementations. +**Current status**: four backends implement the storage traits, in `storage-postgres`, `storage-sqlite`, `storage-mongodb` +and `storage-cassandra`. Each binary is built with one of them. Backends accepted into the ExtendDB repository must satisfy the [Backend Acceptance Criteria](../rfcs/0003-backend-acceptance-criteria.md) (the DynamoDB-behavioral invariants for the storage layer) and the process defined in diff --git a/docs/design/README.md b/docs/design/README.md index f9806b336..cbd3db152 100755 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -10,7 +10,7 @@ This directory contains the authoritative design documents for ExtendDB (extendd | 02 | [High-Level Design](02-high-level-design.md) | Architecture overview, crate structure, request lifecycle, key design decisions, technology choices | | 03 | [Core](03-component-core.md) | Types, expression engine, validation, capacity calculation, errors (`extenddb-core` crate) | | 04 | [Storage](04-component-storage.md) | StorageEngine traits, input/output types, PostgreSQL backend, schema design, pagination, GSI consistency (`extenddb-storage`, `extenddb-storage-postgres` crates) | -| 05 | [Auth](05-component-auth.md) | AuthProvider trait, SigV4 validation, IAM policy engine, credential encryption (`extenddb-auth` crate) | +| 05 | [Auth](05-component-auth-v3.md) | AuthProvider trait, SigV4 validation, IAM policy engine, credential encryption (`extenddb-auth` crate). Supersedes [version 2.0](05-component-auth.md) | | 06 | [Server](06-component-server.md) | HTTP server, routing, middleware pipeline, response formatting, TLS, rate limiting, throughput tracking (`extenddb-server` crate) | | 07 | [Streams](07-component-streams.md) | DynamoDB Streams design space โ€” capture mechanism, shard management, retention (high-level; detailed design deferred) | | 08 | [Configuration](08-component-config.md) | TOML config, env vars, CLI, logging, metrics, health checks, deployment (VM, Kubernetes, Docker) | diff --git a/docs/manuals/05-admin-guide.md b/docs/manuals/05-admin-guide.md index a0cf56e7e..e390da6c8 100755 --- a/docs/manuals/05-admin-guide.md +++ b/docs/manuals/05-admin-guide.md @@ -864,7 +864,8 @@ Targets: `extenddb::audit::manage` (management ops), `extenddb::audit::settings` ### Metrics ```bash -curl --cacert ~/.extenddb/tls/cert.pem https://127.0.0.1:18443/metrics +# Requires an admin user's name and password (HTTP Basic auth); answers 401 without them +curl --cacert ~/.extenddb/tls/cert.pem -u admin: https://127.0.0.1:18443/metrics ``` JSON metrics endpoint with DynamoDB CloudWatch-style metric names and dimensions. The response shape is `{ metrics, buckets, segments, source }`. See `docs/design/06-component-server.md` ยง7.2 for the full schema and metric list. diff --git a/docs/manuals/11-deployment-guide.md b/docs/manuals/11-deployment-guide.md index 77624ce38..7c1060170 100755 --- a/docs/manuals/11-deployment-guide.md +++ b/docs/manuals/11-deployment-guide.md @@ -218,7 +218,7 @@ extenddb init \ ### Monitoring -- [ ] Poll `/metrics` for JSON snapshots and forward to your monitoring system (the response is custom JSON, not Prometheus exposition format; convert as needed) +- [ ] Poll `/metrics` with an admin user's credentials (HTTP Basic auth) for JSON snapshots and forward to your monitoring system (the response is custom JSON, not Prometheus exposition format; convert as needed) - [ ] Forward syslog to a log aggregation service - [ ] Set up alerts on health check failures (`/health`) - [ ] Monitor extenddb process with systemd, supervisord, or equivalent diff --git a/docs/manuals/13-event-ticketing-demo.md b/docs/manuals/13-event-ticketing-demo.md index ba29418f0..eed31bc86 100755 --- a/docs/manuals/13-event-ticketing-demo.md +++ b/docs/manuals/13-event-ticketing-demo.md @@ -510,11 +510,11 @@ event with `userIdentity.type: Service` โ€” proving the deletion was automatic. Verify that extenddb is recording operational metrics: ```bash -# Request counts and latencies -curl -s --cacert ~/.extenddb/tls/cert.pem https://127.0.0.1:**18443**/metrics | python3 -m json.tool +# Request counts and latencies (admin credentials required) +curl -s --cacert ~/.extenddb/tls/cert.pem -u "admin:$EXTENDDB_ADMIN_PASSWORD" https://127.0.0.1:18443/metrics | python3 -m json.tool # Management console metrics (open in browser) -echo "Open: https://127.0.0.1:**18443**/console/metrics" +echo "Open: https://127.0.0.1:18443/console/metrics" ``` You should see non-zero counts for: @@ -741,7 +741,7 @@ aws dynamodb describe-table --table-name Tickets | jq '{ ## 23. Final metrics check ```bash -curl -s --cacert ~/.extenddb/tls/cert.pem https://127.0.0.1:18443/metrics | python3 -c " +curl -s --cacert ~/.extenddb/tls/cert.pem -u "admin:$EXTENDDB_ADMIN_PASSWORD" https://127.0.0.1:18443/metrics | python3 -c " import json, sys m = json.load(sys.stdin) print('=== extenddb Metrics Summary ===')