From 7a64b739eb92c9530947ccb6ed71c005c64d2665 Mon Sep 17 00:00:00 2001 From: Scott Robinson Date: Tue, 3 Nov 2026 18:50:01 +0000 Subject: [PATCH] docs(security-model): describe the credential and policy caches The security model and the design guide said there was no in-process credential cache and that catalog state was never cached. Since the auth/authz cache landed, each server caches decrypted credentials, parsed policies, boundaries, tags, role-session data and table key definitions, on by default with a 60 s TTL, and drops entries only on the server that made the change. The security model's Credential Storage list now points to a new Credential and Policy Caching section. It says what is cached and what is not, lists the [auth.cache] settings and their limits, explains which changes take effect at once and which wait for the TTL (other servers on the same catalog, direct catalog writes), and gives the steps to make a revocation take effect immediately: change the catalog through the management API or console, then flush every other server with `extenddb manage cache invalidate` using the scope that matches the change, or restart those servers. It notes that AssumeRole session credentials cannot be deleted one at a time, and that deleting a role's inline policies is a way to cut off its sessions. The section lists the DynamoDB table operations that drop a table's cached key definitions and tags, names the console delete gap tracked in #787, and says which certificate names the CLI checks when it connects. Other corrections in the same documents: - Security model, Fail-Closed Design: policies are fetched from the catalog when they are not in the server's cache. - Security model, Logging: only some management operations are audit-logged at WARN (the list is now given), cache flushes are logged at INFO, and log records can contain access key IDs. - Design guide: credential lookups go through the credential cache. The Caching Design section's "What Is NOT Cached" part becomes "Risks of Caching Catalog State": it says the auth/authz caches hold some catalog state, that the TTL bounds how stale an entry from another server's change can be, and that no TTL is safe for a table recreated through another server, so several servers per catalog need the cross-instance invalidation in Appendix B of the cache design first. The sentences calling application-level caching unnecessary and catalog caching prohibited are removed. - Auth component design (05-component-auth-v3.md): its "No caching" note now points to the cache design. - Admin guide, getting-started guide and cache design document: fanout invalidations (DeleteAccount, DeleteRole, DeleteGroup) take effect when the call returns. The account and role sweeps use moka's invalidate_entries_if, which registers a predicate that get applies, so matching entries are not served afterwards; only their removal from moka's map happens later. The group fanout awaits each member; if the members cannot be listed, their entries wait for the TTL. max_entries is described as a maximum entry count rather than an LRU cap, since moka does not evict by LRU alone. The design document also said a forced flush was not supported; it now points to the manual invalidation API. Signed-off-by: Scott Robinson --- docs/design/05-component-auth-v3.md | 2 +- docs/design/12-auth-authz-cache.md | 10 ++--- docs/getting-started.md | 4 +- docs/manuals/02-design-guide.md | 20 ++++----- docs/manuals/05-admin-guide.md | 4 +- docs/manuals/10-security-model.md | 65 +++++++++++++++++++++++++++-- 6 files changed, 81 insertions(+), 24 deletions(-) diff --git a/docs/design/05-component-auth-v3.md b/docs/design/05-component-auth-v3.md index 7a920770b..287f797d0 100755 --- a/docs/design/05-component-auth-v3.md +++ b/docs/design/05-component-auth-v3.md @@ -12,7 +12,7 @@ The `auth` crate provides pluggable authentication and authorization for extendd The crate depends on `extenddb-core` (for types and errors) and `async_trait` (for object-safe async trait dispatch). It has no HTTP framework or storage dependencies. -> **No caching.** All credential lookups, policy fetches, and identity resolution read directly from Postgres on every request. The latency is sub-millisecond for indexed lookups, and this approach eliminates stale-cache bugs and cache invalidation complexity entirely. Multiple extenddb instances sharing the same catalog see consistent data without coordination. If profiling under production load shows database round-trips are a bottleneck, caching can be added as a transparent layer without changing any interfaces. +> **Caching.** Credential lookups and policy fetches go through in-memory caches with write-through invalidation and a TTL: the credential cache in this crate (`credential_cache.rs`) and the authorization caches in the server crate (`authz_cache.rs`). `docs/design/12-auth-authz-cache.md` describes that design and replaces the "no caching" decision this document originally recorded. ## 2. Key Design Decisions (from 2026-04-15 review session) diff --git a/docs/design/12-auth-authz-cache.md b/docs/design/12-auth-authz-cache.md index 93b8f1874..889720592 100644 --- a/docs/design/12-auth-authz-cache.md +++ b/docs/design/12-auth-authz-cache.md @@ -36,7 +36,7 @@ This document specifies an in-memory cache with stale-while-revalidate semantics - Eliminate the per-request catalog roundtrips for credentials, policies, boundaries, principal tags, resource tags, and TableKeyInfo on the steady-state hot path. - Eliminate the per-request `PolicyDocument::from_json` parse cost. - Refresh stale entries **without** making the user request wait (refresh-ahead, a.k.a. stale-while-revalidate). -- Bound memory with an LRU upper limit per cache. +- Bound memory with an upper limit on entries per cache. - Allow operators to disable the cache entirely as a kill switch. - Propagate self-induced changes (admin API mutations) instantly within a single process via write-through invalidation hooks. @@ -192,7 +192,7 @@ enabled = true ttl_seconds = 60 # hard TTL — entries beyond this are full misses soft_ttl_seconds = 30 # entries beyond soft_ttl trigger background refresh on access negative_ttl_seconds = 5 # how long to cache "not found" results -max_entries = 10000 # per-cache LRU size limit +max_entries = 10000 # maximum entries per cache ``` Default rationale: @@ -200,7 +200,7 @@ Default rationale: - **60 s hard TTL** mirrors AWS IAM's "policy propagation" expectation (~30 s–2 min). Long enough to absorb the perf bottleneck (60 s of caching → 99.99%+ hit rate per hot key under any meaningful load), short enough to bound the blast radius of off-instance credential revocation. - **30 s soft TTL** = TTL / 2 keeps refreshes from clustering at the boundary. - **5 s negative TTL** is short enough that newly-issued keys work quickly, long enough to absorb a dictionary attack against `lookup_credential`. -- **10 000 entries per cache** is a generous default for typical deployments; under memory pressure the LRU evicts cold entries. +- **10 000 entries per cache** is a generous default for typical deployments; beyond it, moka's TinyLFU policy drops the entries it rates least likely to be reused, which can be the new one. - **`enabled = false`** is a kill switch for incident response. The cache is configured statically (`extenddb.toml`). Runtime tunability is not added — cache TTL changes are rare and a server restart is acceptable. Adding it to the existing `settings` poller table is an option for a follow-up if operationally needed. @@ -416,12 +416,12 @@ Each phase is a separate PR. After each phase the test suite runs green and the > **Propagation timing for invalidations**: > > - **Single-key invalidations** (e.g. `DeleteAccessKey`, `PutUserPolicy`) bump an internal epoch counter and drop the moka entry immediately. The next request blocks on a fresh load. -> - **Fanout invalidations** (e.g. `DeleteAccount`, `DeleteRole`'s session sweep, `DeleteGroup`'s member fanout) use moka's `invalidate_entries_if`, which is **asynchronous** — predicate evaluation runs on moka's internal worker after the management call returns. There is a small window (~ms) between the API responding 200/204 and the matching entries actually being evicted from the underlying store. The accompanying epoch bump ensures any in-flight refresh on those keys is dropped on completion, but a request that reads the slot during this window can still see the pre-invalidation value. Operators should treat fanout invalidation as "eventually consistent within a few ms"; for hard cutover (e.g. revoking a compromised key), prefer the single-key invalidations (`DeleteAccessKey`) over the cascade ones. +> - **Fanout invalidations** (e.g. `DeleteAccount`, `DeleteRole`'s session sweep, `DeleteGroup`'s member fanout) take effect before the management call returns. The account and role sweeps use moka's `invalidate_entries_if`, which registers the predicate and returns; moka removes the entries from its map during later cache operations, but `get` applies registered predicates, so matching entries are not served once the call returns. The accompanying epoch bump drops any refresh in flight on that cache when it completes. `DeleteGroup` awaits the invalidation of each member. If a group's members cannot be listed, those members' entries age out at TTL. > - **Off-instance changes** (a separate process modifying the catalog directly, or a different instance in a multi-node deployment) take up to `auth.cache.ttl_seconds` to propagate. The cross-process invalidation channel described in Appendix B is not yet implemented. > > Self-induced single-key changes via the admin API or the web console propagate instantly on the local instance via write-through invalidation; both paths share the same `AuthCacheRegistry`. > -> To force-flush the cache without restart: not currently supported. To disable: set `auth.cache.enabled = false` in `extenddb.toml` and restart. +> To force-flush the cache without a restart, use the manual invalidation API (§6.1) on each instance. To disable: set `auth.cache.enabled = false` in `extenddb.toml` and restart. --- diff --git a/docs/getting-started.md b/docs/getting-started.md index 09d382c2f..ce9991c63 100755 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1426,12 +1426,12 @@ ttl_seconds = 60 # hard TTL — entries older than this are full miss soft_ttl_seconds = 30 # entries older than this serve cached AND trigger # a background refresh (request does not block) negative_ttl_seconds = 5 # how long "not found" results are remembered -max_entries = 10000 # per-cache LRU cap +max_entries = 10000 # maximum entries per cache ``` **Self-induced changes** (admin API and console mutations: `CreateAccessKey`, `PutUserPolicy`, `TagResource`, `CreateTable`, etc.) propagate instantly via write-through invalidation on the local instance. -**Fanout invalidations** (`DeleteAccount`, `DeleteRole` session sweep, `DeleteGroup` member fanout) complete asynchronously; there is a brief (~ms) window between the API responding success and the eviction landing. For hard cutover, prefer single-key invalidations (`DeleteAccessKey`) over cascades. +**Fanout invalidations** (`DeleteAccount`, `DeleteRole` session sweep, `DeleteGroup` member fanout) also take effect when the API returns. If a group change cannot list the group's members, those members' entries wait for `ttl_seconds`. **Off-instance changes** (multi-node deployments, or direct DB writes outside the management API) take up to `ttl_seconds` to propagate. diff --git a/docs/manuals/02-design-guide.md b/docs/manuals/02-design-guide.md index 26fac44cb..5d7b7ea2f 100755 --- a/docs/manuals/02-design-guide.md +++ b/docs/manuals/02-design-guide.md @@ -106,7 +106,7 @@ extenddb uses SigV4 signature verification with a local IAM credential store. Th Full SigV4 signature verification: 1. Extract `Authorization` header components (credential, signed headers, signature) -2. Look up access key in the credential store (database-backed, credential lookup per request; encryption key cached at startup) +2. Look up access key in the credential store (cached in memory, database on a miss; encryption key cached at startup) 3. Reconstruct the canonical request and string-to-sign 4. Derive the signing key: `HMAC-SHA256(HMAC-SHA256(HMAC-SHA256(HMAC-SHA256("AWS4" + secret, date), region), service), "aws4_request")` 5. Compare computed signature with the provided signature (constant-time comparison) @@ -128,7 +128,7 @@ Policy conditions support all IAM condition operators: `StringEquals`, `StringNo Access key secrets are encrypted at rest using AES-256-GCM. The encryption key is generated during `extenddb init` and stored in the `encryption_keys` table. Each access key record stores the encrypted secret and a unique nonce. -Credential lookups (access key → encrypted secret) read directly from the database on every request — there is no in-process cache for credentials. The encryption key used to decrypt secrets is cached at startup because it is immutable after `extenddb init` (see Caching Design below). +Credential lookups (access key ID → decrypted secret) go through each server's in-memory credential cache; a miss reads the database. See Credential and Policy Caching in the Security Model. The encryption key used to decrypt secrets is cached at startup because it is immutable after `extenddb init` (see Caching Design below). ## DynamoDB Streams Internals @@ -181,7 +181,7 @@ Item size includes attribute names and values, matching DynamoDB's size calculat ## Caching Design -extenddb caches a small set of operational settings in memory to avoid per-request database queries on hot paths. Catalog state (table metadata, auth policies, tags, GSI definitions) is never cached. +extenddb caches a small set of operational settings in memory to avoid per-request database queries on hot paths; this section covers those settings and the risks of caching catalog state. Credentials, policies, boundaries, tags and table key definitions are cached separately by the auth/authz caches (see Credential and Policy Caching in the Security Model, and `docs/design/12-auth-authz-cache.md`). ### What Is Cached @@ -194,18 +194,18 @@ extenddb caches a small set of operational settings in memory to avoid per-reque All cached values are operational tuning knobs where a briefly-stale value does not affect correctness. -### What Is NOT Cached (and Why) +### Risks of Caching Catalog State -Catalog state is never cached because correctness requires every request to see the current state: +The settings cache above holds no catalog state. The auth/authz caches hold some (credentials, IAM policies, boundaries, tags, and table key definitions), and a stale copy can be wrong in the ways listed below. On the server that makes a change, they drop the affected entries before the call returns. For a change made elsewhere, the TTL (`auth.cache.ttl_seconds`) bounds how long a stale entry can be used (see `docs/design/12-auth-authz-cache.md`). -- **Table metadata** (key schema, attribute definitions, status, billing mode): A stale cache could serve the wrong key schema after a table is deleted and recreated with the same name but different schema. The new table has a different `table_id`, different key schema, and different indexes — stale cache serves wrong schema, writes corrupt data, reads return garbage. +- **Table metadata** (table ID, key schema, attribute definitions, indexes, stream specification): A stale cache could serve the wrong key schema after a table is deleted and recreated with the same name but different schema. The new table has a different `table_id`, different key schema, and different indexes — stale cache serves wrong schema, writes corrupt data, reads return garbage. - **IAM policies and credentials**: A revoked Deny policy still cached as absent creates a security gap. A deleted access key still cached as valid allows unauthorized access. - **Tags**: Tag-based authorization (`aws:ResourceTag/*`) requires current tag values. - **GSI definitions**: Stale GSI metadata could route writes to wrong index tables. ### The Table-Name-Reuse Problem -The fundamental reason catalog state cannot be cached safely: +The main risk in caching catalog state: 1. Client calls `DeleteTable("Orders")` 2. Client immediately calls `CreateTable("Orders")` with a different key schema @@ -214,15 +214,15 @@ The fundamental reason catalog state cannot be cached safely: 5. Writes use wrong column layout → data corruption 6. Reads return items with wrong attribute interpretation → garbage -No safe TTL exists because delete-recreate can happen within milliseconds. Cross-instance invalidation (e.g., PostgreSQL LISTEN/NOTIFY) would be a prerequisite for any future catalog caching. +On the server that runs the table operation, `CreateTable`, `UpdateTable`, `DeleteTable`, `RestoreTableFromBackup` and `ImportTable` drop the table's cached key and index definitions before the call returns, and a cache refresh that started before the change discards its result (the epoch counter in `crates/cache/src/swr.rs`). Across servers no TTL is safe, because a delete and recreate can happen within milliseconds: after a table is recreated through one server, another server keeps the old `table_id` for up to `ttl_seconds`. Cross-instance invalidation (for example PostgreSQL LISTEN/NOTIFY, Appendix B of `docs/design/12-auth-authz-cache.md`) is a prerequisite for running several servers on one catalog. ### Multi-Instance Considerations -On SQLite, `extenddb serve` enforces one instance per database file with an exclusive lock on `.lock`. On PostgreSQL and MongoDB nothing enforces it, and running several instances against one catalog is not supported: the credential and policy caches are per instance with no cross-instance invalidation, and every instance runs every background worker (see the deployment guide). Any in-process cache of catalog state would likewise be invisible to other instances. PostgreSQL's own buffer pool provides memory-resident access to hot rows, making application-level caching unnecessary for most workloads. +On SQLite, `extenddb serve` enforces one instance per database file with an exclusive lock on `.lock`. On PostgreSQL and MongoDB nothing enforces it, and running several instances against one catalog is not supported: the credential and policy caches are per instance with no cross-instance invalidation, and every instance runs every background worker (see the deployment guide). ### Future Considerations -Caching of operational settings is currently unconditional. If issues arise (e.g., a setting change must take effect immediately for safety reasons), a runtime toggle (`extenddb settings set caching_enabled false`) should be added. Catalog caching remains prohibited without a cross-instance invalidation design and explicit human approval. +Caching of operational settings is currently unconditional. If issues arise (e.g., a setting change must take effect immediately for safety reasons), a runtime toggle (`extenddb settings set caching_enabled false`) should be added. Running several servers on one catalog needs the cross-instance invalidation in Appendix B of `docs/design/12-auth-authz-cache.md` first. --- diff --git a/docs/manuals/05-admin-guide.md b/docs/manuals/05-admin-guide.md index aff93b6f0..877b3b20f 100755 --- a/docs/manuals/05-admin-guide.md +++ b/docs/manuals/05-admin-guide.md @@ -133,14 +133,14 @@ In-memory stale-while-revalidate caches eliminate the per-request catalog roundt | `ttl_seconds` | `60` | Hard TTL — entries older than this trigger a fresh, request-blocking load. | | `soft_ttl_seconds` | `30` | Stale-but-usable threshold; older entries still serve immediately while a background task refreshes them. | | `negative_ttl_seconds` | `5` | Negative-cache TTL for "not found" results. | -| `max_entries` | `10000` | Per-cache LRU cap. | +| `max_entries` | `10000` | Maximum entries per cache. | Statistics are exposed at `/management/auth-cache-metrics` (JSON, admin-authenticated). The per-cache `pass_through` flag distinguishes "cache disabled" from "cache cold." The report includes every authorization sub-cache, among them the two principal creation-time caches (`user_created_at`, `role_created_at`) consulted on each user and role-session request to derive `aws:userid` and `aws:TokenIssueTime`. **Invalidation timing**: - **Single-key invalidations** (e.g. `DeleteAccessKey`, `PutUserPolicy`) drop the cached entry immediately; the next request sees the post-mutation state. -- **Fanout invalidations** (e.g. `DeleteAccount`, `DeleteRole` session sweep, `DeleteGroup` member fanout) are **asynchronous** — there is a small window (~ms) between the API returning success and the matching entries being evicted. For hard cutover (e.g. revoking a compromised key), prefer the single-key path (`DeleteAccessKey`) over the cascade. +- **Fanout invalidations** (e.g. `DeleteAccount`, `DeleteRole` session sweep, `DeleteGroup` member fanout) also take effect when the API returns. The account and role sweeps register a predicate with moka, and moka's `get` applies it, so matching entries are not served once the call returns; moka removes them from its map during later cache operations. The group fanout drops each member's entries in turn. If a group change cannot list the group's members, the server logs a warning and those members' entries wait for `ttl_seconds`. - **Off-instance changes** (multi-node deployments, or direct catalog writes) wait up to `ttl_seconds` to be observed locally. #### [server.tls] diff --git a/docs/manuals/10-security-model.md b/docs/manuals/10-security-model.md index bb7a1c75d..3f36845b1 100755 --- a/docs/manuals/10-security-model.md +++ b/docs/manuals/10-security-model.md @@ -56,7 +56,63 @@ extenddb uses SigV4 signature verification with a local credential store and IAM - Secret keys encrypted with AES-256-GCM using a per-catalog encryption key - Encryption key generated during `extenddb init` and stored in the catalog database - Console passwords hashed with bcrypt (cost factor 12) -- No in-process credential cache — every request reads directly from PostgreSQL +- Credentials and the inputs to authorization are cached in each server's memory; see Credential and Policy Caching below + +#### Credential and Policy Caching + +Each server keeps in-memory caches between the request path and the catalog database, so that a request does not read the catalog for every credential and policy lookup. The caches hold decrypted credentials (long-term access keys and AssumeRole session credentials), parsed user, group and role inline policies, permissions boundaries, principal tags, resource tags, role-session data, user and role creation times, and each table's key and index definitions. Authorization decisions themselves are not cached: policies are evaluated on every request. Item data is not cached. This section covers the credentials that sign API requests; web console logins are described under Web Console Security. + +The caches are on by default and are configured under `[auth.cache]` in `extenddb.toml` (see the Admin Guide): + +| Setting | Default | Effect | +|---------|---------|--------| +| `enabled` | `true` | `false` makes every lookup read the catalog | +| `ttl_seconds` | `60` | The oldest a cached entry can be when it is used. Must be greater than 0 | +| `soft_ttl_seconds` | `30` | After this age an entry is still used, and a background read replaces it. Must not exceed `ttl_seconds` | +| `negative_ttl_seconds` | `5` | How long an empty answer is cached: an unknown access key ID, a missing table, or a principal with no policies, boundary or tags. Must not exceed `ttl_seconds` | +| `max_entries` | `10000` | Entries per cache; beyond this, entries are evicted. Must be greater than 0 | + +These settings, whether set in the file or through `EXTENDDB__AUTH__CACHE__*` environment variables, are read at startup; changing them takes a restart. A server whose settings break one of the limits above refuses to start. + +**How a change reaches the cache.** A change made on a server through the management API, the web console, or one of the DynamoDB operations named below takes effect on that server immediately. Deleting an access key, user or role, changing an inline policy, permissions boundary, group membership or a user's or role's tags, and calling `TagResource` or `UntagResource` drop the cached entries they affect before the call returns. `CreateTable`, `UpdateTable`, `DeleteTable`, `RestoreTableFromBackup` and `ImportTable` drop the table's key and index definitions, and `CreateTable` and `DeleteTable` also drop its tags. There are two exceptions. If a group change cannot list the group's members, the server logs a warning and those members' entries wait for `ttl_seconds`. The console's Delete user and Delete role do not drop the principal's cached creation time, so a user or role created again with the same name within `ttl_seconds` resolves to the old `aws:userid`, and a condition on `aws:userid` matches the old id in that window (#787). Every other change is seen when the cached entry is next refreshed or expires, within `ttl_seconds`. That covers: + +- a change made on one server, as seen by the other servers that serve the same catalog; +- a change written to the catalog database directly rather than through a server. + +On those servers, a change to something that was cached as present (an access key, or a principal's existing policies, boundary or tags) takes up to `ttl_seconds` to arrive. A change to something that was cached as empty (a principal's first policy, boundary or tag) arrives within `negative_ttl_seconds`, because that is how long an empty answer is kept. Running more than one server on a catalog is not supported in this release (see the Deployment Guide); on SQLite, on Unix with the database on a local filesystem, a second server refuses to start. A request that has already been authorized when a credential is revoked is not stopped. + +**Revoking access at once.** To make a revocation take effect immediately, for example during incident response, take these steps in order. + +*Step 1: change the catalog.* Through the management API or console, delete the access key, the user, or the inline policy that grants the access, or remove the user from the group. The server that handles the call applies the change at once. + +AssumeRole session credentials (`ASIAEXTENDDB...`) cannot be deleted one at a time, and a session can last up to 12 hours. A session's permissions come from its role's inline policies, so deleting the role's inline policies stops the role's sessions. This affects every session of that role. + +*Step 2: flush the other servers.* On every other server that serves the same catalog, flush that server's cache. Run the command on that server's host with that server's config file, so that the CLI connects to that server and trusts its TLS certificate: + +```bash +read -rs EXTENDDB_PASSWORD && export EXTENDDB_PASSWORD # the admin password, not echoed +extenddb manage --config /path/to/extenddb.toml --user ADMIN_NAME cache invalidate SCOPE +unset EXTENDDB_PASSWORD +``` + +Replace `SCOPE` with the scope for what changed: + +| Change | Scope | +|--------|-------| +| A deleted access key | `credential --access-key-id ` | +| A deleted account | `account --account-id ` | +| A user's keys, inline policies, boundary or tags, or a deleted user | `user --account-id --user-name ` | +| A role's inline policies, boundary or tags, or a deleted role | `role --account-id --role-name ` | +| A group's inline policy or membership | `group-members --account-id --user-names ,` (every member), or `all --yes` | +| A table's tags (`TagResource`, `UntagResource`) | `resource-tags --arn ` | +| A deleted table, a table created again with the same name, or an updated table | `table-key-info --account-id --table-name `, and for a deleted or recreated table also `resource-tags --arn
` | +| Unsure | `all --yes` | + +Without `--endpoint`, the CLI connects to `server.bind_addr` and `server.port` from the config file (`127.0.0.1` when `bind_addr` is `0.0.0.0`). With `--endpoint https://:`, it connects to ``. When TLS is on (`server.tls.enabled`, or an `https://` endpoint), it trusts only the certificate at `server.tls.cert_path` in the config file it reads, and that certificate must cover the address the CLI connects to. The certificate `extenddb init` creates covers `localhost`, `127.0.0.1`, `bind_addr` when it is a specific address, and each name given with `--tls-san`. A CA-signed certificate usually covers none of the first three, so with one use `--endpoint https://:`, and make `cert_path` a PEM file that also holds the issuing CA's certificate. Do not use a load balancer's address: each call reaches one server behind it and reports success. The same flush is available to admins at `POST /management/cache/invalidate` and on the console's cache page, and each acts only on the server it reaches. + +Instead of flushing, you can restart each server, which starts with empty caches. A server started with `auth.cache.enabled = false` (or `EXTENDDB__AUTH__CACHE__ENABLED=false` in its environment) reads the catalog on every request, at the cost of several catalog queries per request. + +Flushing a cache does not revoke anything by itself: it only makes the server read the catalog again, so step 1 comes first. If step 2 is not done and the servers are not restarted, a change made in step 1 reaches the other servers within `ttl_seconds` (60 seconds by default). Lowering `ttl_seconds` shortens that window and adds catalog reads; lower `soft_ttl_seconds` and `negative_ttl_seconds` with it so that neither exceeds it. ## Authorization @@ -81,7 +137,7 @@ Policy sources collected for evaluation: - **Unparseable policies deny**: A stored policy that cannot be parsed results in access denied, not silent skip. A corrupted Deny policy still denies; a corrupted Allow policy is treated as absent. - **Auth before JSON parse**: SigV4 signature verification runs before the request body is parsed. Invalid signatures are rejected with constant-time comparison before any business logic executes. -- **Concurrent policy fetching**: Identity policies, group policies, and permissions boundaries are fetched concurrently from PostgreSQL. All must succeed for evaluation to proceed. +- **Concurrent policy fetching**: Identity policies, group policies, and permissions boundaries are fetched concurrently from the catalog when they are not in the server's cache (see Credential and Policy Caching). All must succeed for evaluation to proceed. - **Constant-time rejection for inactive keys**: Inactive or expired access keys are rejected without timing differences that could reveal key existence. - **Policy document validation on write**: Policy documents are validated for JSON structure and size-capped (6,144 bytes) when attached via the management API. Invalid documents are rejected before storage. - **Expression depth and token limits**: Expression parsing enforces configurable depth (default 150) and token limits (default 4,096) to prevent resource exhaustion. @@ -234,8 +290,9 @@ Import/export file paths are validated: ### Logging - All logging goes to syslog (no log files with sensitive data on disk) -- Management operations are audit-logged at WARN level -- Log messages do not contain credentials, access keys, or item data +- Some management operations are audit-logged at WARN, with target `extenddb::audit::manage` or `extenddb::audit::settings`: DeleteAdmin, user and role tag changes, AssumeRole (granted and denied), and settings changes. Other management changes, such as creating or deleting users, roles, access keys and policies, are not audit-logged +- Cache flushes (`cache invalidate`) are logged at INFO with the admin name, the scope and its selectors, so they are not recorded when the log level (`logging.level`, or the `log_level` setting, which overrides it) is `warn` or `error` +- Log messages do not contain secret keys, passwords, or item data. Access key IDs can appear, for example in cache flush records ### Default Configuration