diff --git a/platform/control-plane/access-control/index.md b/platform/control-plane/access-control/index.md index 271c58d..7fe76e1 100644 --- a/platform/control-plane/access-control/index.md +++ b/platform/control-plane/access-control/index.md @@ -39,7 +39,7 @@ Findings about the way access is set up. Each has a title, a sentence and a **Re ### Recent changes -The latest audit entries with the action (for example `perm_grant:create`), the resource type, the actor and the time. The actor is shown as a short user id (`by 8a66d837`), not a name. **View audit log** opens the full [Audit log](../audit-log.md). +The latest audit entries with the action (for example `perm_grant:create`), the resource type, the actor and the time. The actor is shown by email (`by admin@egret.local`), or by a short user id when the account no longer exists. **View audit log** opens the full [Audit log](../audit-log.md). ### How access works diff --git a/platform/control-plane/access-control/users.md b/platform/control-plane/access-control/users.md index 6210ecc..83fb49c 100644 --- a/platform/control-plane/access-control/users.md +++ b/platform/control-plane/access-control/users.md @@ -24,7 +24,7 @@ A summary strip counts **users**, **active** and **disabled**. Two filters narro | **User** | Avatar, display name and email. | | **Provider** | `local` for accounts created in egret, `oidc` for people who sign in through SSO. | | **Status** | **active** or **disabled**. A disabled user cannot sign in. | -| **Roles** | Platform roles the user holds, or a dash. | +| **Roles** | Platform roles granted to the user directly, or a dash. A role the user holds through a group is not shown (KI-260). | | **Groups** | How many groups the user belongs to. | The **⋯** menu on a row offers **View access** (the [Access Explorer](./access-explorer.md) for that person), **Assign roles**, **Edit**, **Impersonate** and **Delete**. @@ -45,7 +45,7 @@ Prefer SSO for people. A local account has its own password and does not follow ## Bulk select -**Bulk select** adds checkboxes. After you tick users, a bar shows *N selected* with **Disable**, **Enable**, **Assign role…** and **Clear**. **Done** leaves bulk mode. +**Bulk select** adds checkboxes. After you tick users, a bar shows *N selected* with **Disable**, **Enable**, **Assign role…** and **Clear**. **Done** leaves bulk mode. Bulk **Disable** acts at once, without the confirmation the user page asks for; the toast that reports it offers **Undo** (KI-262). ![Bulk select with two users ticked and the actions bar](/img/platform/admin/cp-users-bulk.png) @@ -61,7 +61,7 @@ Select a user to open their page. The header shows the avatar, name, status, pro ### Roles -*Platform-wide roles this user holds. Workspace roles are assigned per workspace.* +*Platform-wide roles this user holds. Workspace roles are assigned per workspace.* Only roles granted to the user directly are listed; roles granted to one of the user's groups are on the group's page. 1. Under **Scope**, choose **Platform** or **Workspace**. 2. For **Platform**, choose a **Role** (for example `data_steward` or `platform_admin`). For **Workspace**, pick the workspace first. @@ -75,7 +75,7 @@ The table lists **Role** and **Scope** with a remove button. With none it reads ## Disable and delete -**Disable** blocks sign-in without removing anything and signs the user out; it applies at once, without a confirmation. The status becomes **disabled** and **Enable** reverses it. **Delete** asks *Delete "\"?* (*This user, their sessions, group memberships, and role grants will be permanently removed. This cannot be undone.*) and needs you to type `delete`. Prefer disabling: grants and shares that name a deleted user become unmatched names (see [People & groups](../../data-governance/people-and-groups/index.md)). +**Disable** blocks sign-in without removing anything and signs the user out. It asks first: *Disable \?* (*They are signed out at once and cannot sign in until the account is enabled again. Their memberships and role grants are kept.*), with **Cancel** and **Disable user**. The status becomes **disabled**. **Enable** reverses it at once, without a confirmation. **Delete** asks *Delete "\"?* (*This user, their sessions, group memberships, and role grants will be permanently removed. This cannot be undone.*) and needs you to type `delete`. Prefer disabling: grants and shares that name a deleted user become unmatched names (see [People & groups](../../data-governance/people-and-groups/index.md)). ## Limits and known issues diff --git a/platform/control-plane/audit-log.md b/platform/control-plane/audit-log.md index a94084f..7ccaf57 100644 --- a/platform/control-plane/audit-log.md +++ b/platform/control-plane/audit-log.md @@ -6,6 +6,8 @@ sidebar_position: 7 The **audit log** is the append-only record of authorization decisions and changes on the platform: who did what, to which object, and whether it was allowed. **Control Plane → System → Audit log** reads it, newest first. It is visible to platform admins only. +The page opens on **Changes and denials**: every recorded change and every refused request, without the allowed decisions that each page view writes. Choose **All** under **Kind** to see those too. + ![The Audit log with its filters and rows](/img/platform/admin/cp-audit-log.png) ## Read the log @@ -15,7 +17,7 @@ The page shows a table with six columns: | Column | Meaning | |---|---| | **Time** | When it happened, relative. | -| **Actor** | The user's email, or a short id for an account the page cannot resolve (for example a service account). | +| **Actor** | The user's email, or a short id for an actor that is not a user who still exists (for example a deleted account or a service account). | | **Action** | What was attempted, in the form `thing:action`, for example `secret:read`, `member:add`, `perm_rule:create`. | | **Resource** | What it was about: the resource type and id, for example `workspace (e76a8865-…)` or `platform`. | | **Decision** | **allow** or **deny** for an authorization decision, a dash for a recorded change. | @@ -30,7 +32,7 @@ Every page loads 100 rows. **Load more** fetches the next 100 using a keyset cur | **Decision** | An authorization check: the actor asked to do something and the policy engine answered. | **allow** or **deny**, with the grants that decided. | `secret:read` on `platform`, **allow**, `grant-0` | | **Mutation** | A change that was made. | A dash. | `member:add`, `object_grant:put`, `perm_rule:create`, `tag:assign` | -Decisions are recorded for the platform's role-based checks. Impersonation is recorded as the mutations `impersonation.start` (with the reason) and `impersonation.end`, with the admin as the actor; the decisions made while impersonating are recorded under the impersonated user, with nothing that names the admin. Governance changes also appear on the Governance overview's **Recent changes**; the audit log keeps the raw entries. +Decisions are recorded for the platform's role-based checks. Impersonation is recorded as the mutations `impersonation.start` (with the reason) and `impersonation.end`, with the admin as the actor; the decisions made while impersonating are recorded under the impersonated user, with nothing that names the admin. Governance changes also appear on the Governance overview's **Recent changes**; the audit log keeps the raw entries. A workspace pre-registration is recorded as `member:invite` and `member:uninvite`, and its redemption at the person's first sign-in as `member:add` with the inviting admin as the actor and `"via": "pre-registration"` in the detail. ## Filter the log @@ -38,12 +40,12 @@ The filter bar sits above the table. | Filter | Options | |---|---| -| **Kind** | **All**, **Decision**, **Mutation**. | +| **Kind** | **Changes and denials** (the default: mutations and denied decisions), **All**, **Decision**, **Mutation**. | | **Decision** | **All**, **Allow**, **Deny**. | | **Action** | An exact action, for example `secret:read`. A partial word such as `secret` matches nothing. | | **From** and **To** | A date and time range (your browser's local time). | -The same filters work in the API: `GET /api/control-plane/audit` accepts `kind`, `decision`, `action`, `actor`, `workspace`, `from`, `to`, `cursor` and `limit`. +The same filters work in the API: `GET /api/control-plane/audit` accepts `kind`, `decision`, `hideAllowed`, `action`, `actor`, `workspace`, `from`, `to`, `cursor` and `limit`. `hideAllowed=true` is what **Changes and denials** sends. Each entry carries `actorEmail` next to `actorUserId`. ### Find out who did something diff --git a/platform/control-plane/query-engines/engine-detail.md b/platform/control-plane/query-engines/engine-detail.md index 0eeff89..5329e1a 100644 --- a/platform/control-plane/query-engines/engine-detail.md +++ b/platform/control-plane/query-engines/engine-detail.md @@ -8,15 +8,15 @@ Select an engine in **Control Plane → Engines** to open its page. The tabs dep ## Provisioned engine -![The Overview tab of the provisioned engine docs-trino: phase running, one pod, two catalogs, one workspace](/img/platform/admin/cp-engine-overview.png) +![The Overview tab of a provisioned engine on the Docker demo stack: phase running, one container, its catalogs and workspaces](/img/platform/admin/cp-engine-overview.png) -The header shows the engine name, its namespace (`egret-engines`), the image, the coordinator address, the authentication (*JWT auth*) and the Ranger plugin version, with the type and status badges. **Open in Engine & policy** opens the engine in Governance, **Restart** rolls the engine, and the **More engine actions** menu (**⋯**) holds **Delete engine**. The tabs are **Overview**, **Catalogs**, **Workspaces**, **Settings** and **Monitor**. +The header shows the engine name, its namespace (`egret-engines`), the image, the endpoint, the authentication (*JWT auth*) and the Ranger plugin version, with the type and status badges. The endpoint is the `host:port` a workspace connection to the engine dials, for example `docs-trino-trino.egret-engines.svc:8443` (port 8443 is the HTTPS listener of an engine with JWT delegation). **Open in Engine & policy** opens the engine in Governance, **Restart** restarts the engine, and the **More engine actions** menu (**⋯**) holds **Delete engine**. The tabs are **Overview**, **Catalogs**, **Workspaces**, **Settings** and **Monitor**. ### Overview -A strip shows the **phase**, **pods** (ready/total), **replicas** (for example *Coord 1 · Worker 0*), **catalogs**, **workspaces** and **auth** (*JWT auth*, impersonation, password auth or no delegation). For StarRocks, **FE replicas** and **CN replicas** replace *replicas*. +A strip shows the **phase**, **pods** (ready/total; **containers** on the Docker provisioner), **replicas** (for example *Coord 1 · Worker 0*, the counts the engine runs with), **catalogs**, **workspaces** and **auth** (*JWT auth*, impersonation, password auth or no delegation). For StarRocks, **FE replicas** and **CN replicas** replace *replicas*. -**Status** lists each pod with its **Phase** and **Ready** state and the **Logs** and **Exec** actions. With the Docker provisioner a "pod" is a container, for example `egret-engine-egret-engines-docs-trino-coordinator`. +**Status** lists each pod with its **Phase** and **Ready** state and the **Logs** and **Exec** actions. On the Docker provisioner the table lists containers, for example `egret-engine-egret-engines-docs-trino-coordinator`, without **Logs** and **Exec**, and a note says *Logs and a shell are not available here for an engine Docker runs: read a container's log with docker logs on the Docker host.* If the engine cannot start the page shows *The engine failed to start.* with the reason. While starting it reads *Starting up — the engine isn't reachable yet.* @@ -24,7 +24,7 @@ If the engine cannot start the page shows *The engine failed to start.* with the ![The Catalogs tab: demo_warehouse is primary and applied, tpch_bench is applied](/img/platform/admin/cp-engine-catalogs.png) -*Data sources this engine can query. Attaching or detaching a catalog applies it to the cluster and restarts the Trino coordinator, so queries in flight are lost.* Each catalog shows its **Kind**, **Primary**, the **Apply** state and when it was **Applied**, with **Test** (runs `SHOW SCHEMAS` on the engine) and **Detach**. The primary catalog cannot be detached: *demo_warehouse is this engine's primary catalog. Detach it by deleting the engine.* +*Data sources this engine can query. Attaching or detaching a catalog applies it to the cluster and restarts the Trino coordinator, so queries in flight are lost.* Each catalog shows its **Kind**, **Primary**, the **Apply** state and when it was **Applied**, with **Test** (runs `SHOW SCHEMAS` on the engine) and **Detach**. On an engine with JWT delegation the test runs as you, with the engine token Egret mints for you, so it fails if the engine's policy does not let you read the catalog. The primary catalog cannot be detached: *demo_warehouse is this engine's primary catalog. Detach it by deleting the engine.* To attach another, choose it in **Catalog to attach** and press **Attach**. With nothing left the form says *Every catalog egret can show you is already on this engine.* @@ -45,25 +45,37 @@ The engine then appears in that workspace's SQL Lab as a database named after th ![The Settings tab with the configuration sections; Worker and Security & Exposure carry values](/img/platform/admin/cp-engine-settings.png) -*Cluster-wide, coordinator, and worker Trino config plus security and exposure. Saving re-provisions the engine.* The sections are the ones of the create form (see [Provision an engine](./provision.md#the-configuration-sections-trino)). +*Cluster-wide, coordinator, and worker Trino config plus security and exposure. Saving re-provisions the engine.* (on the Docker provisioner: *Saving recreates the engine's containers.*) The sections are the ones of the create form (see [Provision an engine](./provision.md#the-configuration-sections-trino)). 1. Change a value, for example **Server → config.properties → Add property** `query.max-run-time` = `2h`. -2. Press **Review changes** to see the diff. The dialog says, for example, *2 fields will change.* with each field's old and new value, and *Saving re-provisions (rolls) the engine pods.* It only shows the diff; close it. -3. Press **Save changes**. There is no further confirmation. +2. Press **Save changes**. The **Review changes** dialog lists only what changes, for example *1 field will change.* with the old and new value, and says *Saving re-provisions (rolls) the engine pods; running queries are interrupted.* (on Docker: *Saving recreates the engine's containers; running queries are interrupted.*). +3. Press **Save changes** in the dialog to apply, or **Cancel** to save nothing. On the demo stack the save recreated the coordinator container within a minute while the header kept showing **running**, and the new value was in effect: `SHOW SESSION LIKE 'query_max_run_time'` reported the default `2.00h`. +With **Expose via ingress** on and no **Host**, the field says *Enter the host the ingress serves, or turn off Expose via ingress.* and **Save changes** stays disabled. The Docker provisioner refuses an ingress whatever the host (see [Provision an engine](./provision.md#kubernetes-and-docker)), although the section is still offered there (KI-264). + ### Monitor -![The Monitor tab on the demo stack: Engine not reachable, and no live queries](/img/platform/admin/cp-engine-monitor.png) +![The Monitor tab on the demo stack: the query counts, the node count and the live queries](/img/platform/admin/cp-engine-monitor.png) The tab has a strip with the **running**, **queued** and **blocked** query counts and the **nodes**, and the **Queries** table (*Live queries on the coordinator, refreshed every 3s*) with **Query**, **User**, **State**, **SQL**, **Progress**, **Elapsed**, **Memory** and **CPU**, a state filter (**All states**) and **Search user or SQL…**. -On Trino 483 engines the tab shows *Engine not reachable — is a coordinator running?* and *Coordinator unreachable — no live queries.* even while the engine is running and answering queries. This is a product issue, not a Docker limitation: the cluster summary asks Trino for `GET /v1/node`, which Trino 483 answers with 404, and an engine with JWT delegation gives egret's monitor account no sign-in. +How Egret signs in to the coordinator depends on the engine: + +| Engine | Egret asks as | What you see | +|---|---|---| +| JWT delegation (the default for a provisioned Trino engine) | **You**, with the engine token Egret mints for you | What the engine's own policy lets you see: other users' queries only if it lets you view them. Killing a query is decided the same way. | +| Password users or impersonation | The engine's `egret-monitor` account | The queries that account may see. | +| No authentication | `egret-monitor` | Every query. | + +The node list comes from Trino's `GET /v1/node` where the coordinator serves it. Trino 483 does not by default, and then Egret reads `system.runtime.nodes` instead. That query runs as the same sign-in, so on a JWT engine it appears in the **Queries** table under your name (on the demo stack: `SELECT http_uri FROM system.runtime.nodes`, user `admin`, **FINISHED**). If it is refused too, **nodes** shows `—`: the count is unknown, not zero, and the query counts are still shown. ### Restart -**Restart** asks *Restart engine?* (*This rolls the coordinator and worker pods; running queries are interrupted.*). On the demo stack the status went to **provisioning** and back to **running** in 38 seconds, and the container was recreated. +**Restart** asks *Restart engine?* (*This rolls the coordinator and worker pods; running queries are interrupted.* On the Docker provisioner: *This restarts the coordinator and worker containers; running queries are interrupted.*). For a Trino engine it adds *If the pods (containers) are gone, they are created again.* On the demo stack the status went to **provisioning** and back to **running** in 38 seconds, and the container was recreated. + +A Trino engine that has run before and now has no pod or container at all, for example after `just demo-down` and `just demo-up` on the demo stack, is created again from its stored settings when you press **Restart**. An engine that is still being created (status **provisioning**) is left to finish. ### Delete @@ -75,18 +87,16 @@ On Trino 483 engines the tab shows *Engine not reachable — is a coordinator ru - *Saved queries that use it lose their connection.* - *Data API endpoints (including trashed ones), alerts and datasets that use it block the delete — you will be shown where they are.* -Type the engine name and press **Delete**. You return to the Engines list. With the Docker provisioner the engine's containers are removed (measured: 7 seconds). The dialog says *This destroys the engine's Kubernetes cluster* whatever the provisioner is. The engine's policy profile is kept and appears under **Profiles without an engine** in Governance. +Type the engine name and press **Delete**. You return to the Engines list. The dialog says *This destroys the engine's Kubernetes cluster. It cannot be undone.*, or on the Docker provisioner *This removes the engine's containers. It cannot be undone.* (measured: 7 seconds). The engine's policy profile is kept and appears under **Profiles without an engine** in Governance. ### Kubernetes-only features -**Logs** and **Exec** work only with the Kubernetes provisioner: - -| Action | Kubernetes | Docker provisioner | -|---|---|---| -| **Logs** | Container logs for the pod, with **Last 1000 lines**, **Previous container**, **Follow** and **Refresh**. | The dialog opens and says *bad request: not available with the Docker engine provisioner*. | -| **Exec** | *Open shell into \?* opens an interactive shell (bash) inside the running pod. | The shell opens and closes at once with *[connection closed]*, without saying why. | +**Logs** and **Exec** work only with the Kubernetes provisioner, and are not shown for an engine Docker runs: -![The Logs dialog on the Docker demo stack](/img/platform/admin/cp-engine-logs.png) +| Action | Kubernetes | +|---|---| +| **Logs** | Container logs for the pod, with **Last 1000 lines**, **Previous container**, **Follow** and **Refresh**. | +| **Exec** | *Open shell into \?* opens an interactive shell (bash) inside the running pod. | With the Docker provisioner, read the log on the Docker host with `docker logs egret-engine-egret-engines--coordinator`. A failed engine's status message carries its last log lines. @@ -154,8 +164,9 @@ A Trino apply rebuilds the engine's entire catalog configuration. When one catal ## Limits and known issues - An external engine's page has no health status or restart. Egret cannot see the cluster. -- The monitor lists no queries on Trino 483 engines (see [Monitor](#monitor)). -- **Logs** and **Exec** are offered on Docker engines although they cannot work there. +- On the Docker provisioner, a save interrupted by the browser disconnecting (a closed tab, a dropped connection) can stop the old containers without creating new ones; the engine shows **failed** until the next save (KI-263). +- The Ingress section is offered on Docker engines, which refuse any ingress (KI-264). +- On an engine with impersonation and a Ranger policy, the monitor's `egret-monitor` account probably sees only its own queries (KI-265, not verified). - Delete is permanent. After a delete the Engines list can keep the row until you reload. ## Further reading diff --git a/platform/control-plane/query-engines/index.md b/platform/control-plane/query-engines/index.md index 0570996..0499c35 100644 --- a/platform/control-plane/query-engines/index.md +++ b/platform/control-plane/query-engines/index.md @@ -18,7 +18,7 @@ A summary strip counts **engines**, how many are **running** and **not running** |---|---| | **Name** | The engine name. It links to the engine page. | | **Type** | `trino` or `starrocks`. | -| **Endpoint** | Where it listens: `docs-trino-coordinator.egret-engines` for a provisioned engine, the host and port you entered (for example `trino-demo:8080`) for an external one. | +| **Endpoint** | The `host:port` a workspace connection to the engine dials: for example `docs-trino-trino.egret-engines.svc:8443` for a provisioned Trino engine with JWT delegation, the host and port you entered (for example `trino-demo:8080`) for an external one. | | **Status** | The lifecycle status of a provisioned engine (**provisioning**, **running**, **failed**). **External** for a registered one: egret does not monitor it. | | Actions | **Details** opens the engine. | @@ -34,7 +34,7 @@ Two buttons sit at the top right: Where a provisioned engine runs depends on how egret is deployed. On Kubernetes egret creates Deployments and Services in the engine namespace. With the Docker provisioner (`[engines] provisioner = "docker"`), egret starts each engine as Docker containers on the same Docker host. :::note Demo stack -The Docker demo stack uses the Docker provisioner: **Add engine** starts Trino as containers named `egret-engine-egret-engines--coordinator` (and `…-worker-`). The first engine pulls the `trinodb/trino` image; plan about 1.5 to 2 GB of memory per engine. StarRocks engines, pod logs and the shell are Kubernetes-only (see [Engine detail](./engine-detail.md#kubernetes-only-features)). +The Docker demo stack uses the Docker provisioner: **Add engine** starts Trino as containers named `egret-engine-egret-engines--coordinator` (and `…-worker-`). The first engine pulls the `trinodb/trino` image; plan about 1.5 to 2 GB of memory per engine. StarRocks engines, ingress, pod logs and the shell are Kubernetes-only (see [Engine detail](./engine-detail.md#kubernetes-only-features)). After `just demo-down` and `just demo-up` the engine containers are gone: press **Restart** on each engine to create them again. ::: ## Which one to choose diff --git a/platform/control-plane/query-engines/provision.md b/platform/control-plane/query-engines/provision.md index 1573657..59161c4 100644 --- a/platform/control-plane/query-engines/provision.md +++ b/platform/control-plane/query-engines/provision.md @@ -21,7 +21,7 @@ You need **engine:create** and **engine:provision**. Egret runs the engine where The engine page opens with the status **provisioning**. It turns **running** when the coordinator answers. Measured on the demo stack: 72 seconds for the first engine on a Docker host that already had the image, and 38 seconds for a restart. :::tip Single-node engine -For a small engine, open **Worker** and set **Replicas** to `0`: the coordinator then schedules the work itself. Left blank, the engine gets the default worker count. On the demo stack a blank value started one worker container next to the coordinator. +For a small engine, open **Worker** and set **Replicas** to `0`: the coordinator then schedules the work itself, and the Overview strip shows *Worker 0*. Left blank, the engine gets one worker: on the demo stack a blank value started one worker container next to the coordinator. ::: ## The Configuration sections (Trino) @@ -50,7 +50,7 @@ The form shows *Cluster-wide, coordinator, and worker Trino config. Leave fields | **Password users** | *Each user signs in with a username and the password held in the selected egret secret (hashed server-side).* For clients that cannot use JWT. | | **JWT authentication** | *Also accept egret-issued JWTs (e.g. from notebooks) as client credentials.* An engine needs JWT before it can be assigned to a workspace. | | **Per-user delegation** | How queries from SQL Lab, charts and dashboards run as the calling user (below). | -| **Ingress exposure** | *Create a Kubernetes Ingress for the coordinator. TLS terminates at the ingress.* With **Expose via ingress** on you set **Host**, **Ingress class**, **Ingress TLS cert** and **Annotations**. | +| **Ingress exposure** | *Create a Kubernetes Ingress for the coordinator. TLS terminates at the ingress.* With **Expose via ingress** on you set **Host**, **Ingress class**, **Ingress TLS cert** and **Annotations**. **Host** is required: without it the field says *Enter the host the ingress serves, or turn off Expose via ingress.* and **Create** stays disabled. | The delegation modes: @@ -87,7 +87,7 @@ The Docker provisioner runs the same Trino image with the same configuration fil | CPU and memory **requests**, health probes, placement | Yes | Accepted, no effect | | Ingress, autoscaling, ConfigMap and Secret mounts, init and sidecar containers, image pull secrets, additional exposed ports, `shareProcessNamespace` | Yes | Refused with an error that names the setting | | StarRocks engines | Yes | Refused | -| Pod logs, log follow and the shell (**Logs**, **Exec**) | Yes | Not available (see [Engine detail](./engine-detail.md#kubernetes-only-features)) | +| Pod logs, log follow and the shell (**Logs**, **Exec**) | Yes | Not shown; use `docker logs` (see [Engine detail](./engine-detail.md#kubernetes-only-features)) | :::note Demo stack The demo's Docker provisioner only starts images from `trinodb/trino`, with no published ports, no mounts and all capabilities dropped, on the network `egret-demo_default`. Read an engine's log with `docker logs egret-engine-egret-engines--coordinator`. @@ -96,8 +96,7 @@ The demo's Docker provisioner only starts images from `trinodb/trino`, with no p ## Limits and known issues - A Trino catalog attach or detach restarts the coordinator, so queries in flight are lost. -- Right after creation the Overview strip can show the default worker count (`Worker 2`) even when you set **Replicas** to `0`; it shows the real count after the first save of the settings. -- The engine monitor lists no queries on Trino 483 engines (see [Engine detail](./engine-detail.md#monitor)). +- The Ingress section is offered on the Docker provisioner, which refuses any ingress (KI-264). ## Further reading diff --git a/platform/data-governance/classifications/view-exposure.md b/platform/data-governance/classifications/view-exposure.md index f53203b..5baef9b 100644 --- a/platform/data-governance/classifications/view-exposure.md +++ b/platform/data-governance/classifications/view-exposure.md @@ -74,7 +74,7 @@ A view change on a catalog whose profile has a classified column is held until t ## Limits and known issues -- On the demo stack the view `demo_warehouse.analytics.revenue_by_country` (a DEFINER view over the tagged `sessions` table) is listed with *clearance unknown* and *may read classified data*, because egret could not compute its lineage: reading the view through the engine was refused (*Access Denied: Cannot access catalog demo_warehouse*). The row says *lineage could not be computed; egret has not judged this view. Run the lineage sweep, whose report names the reason.* The console has no button for the sweep; it is the API call `POST /api/control-plane/classifications/lineage-sweep`. +- A view whose lineage egret could not compute is listed with *clearance unknown* and *may read classified data*. The row says *lineage could not be computed; egret has not judged this view. Run the lineage sweep, whose report names the reason.* The console has no button for the sweep; it is the API call `POST /api/control-plane/classifications/lineage-sweep`. Lineage is computed only through a provisioned Trino engine in JWT delegation mode (see [Lineage](../lineage.md#how-egret-computes-lineage)); on the demo stack, the sweep through such an engine recorded the lineage of `demo_warehouse.analytics.revenue_by_country`. - A view created before its base table was tagged waited no time; the hold applies only once the catalog has a classified column. - The Trino list has a 15 second deadline. If egret's views cannot all be read, the page shows an error instead of a partial list. diff --git a/platform/data-governance/engines-and-policy/ranger-plugin.md b/platform/data-governance/engines-and-policy/ranger-plugin.md index ad8c794..ceb3dc4 100644 --- a/platform/data-governance/engines-and-policy/ranger-plugin.md +++ b/platform/data-governance/engines-and-policy/ranger-plugin.md @@ -38,8 +38,8 @@ The installation files are on the engine's **Plugin config** tab in the Control | Field | Meaning | |---|---| | **Ranger service name** | The profile name the plugin asks for, for example `docs-legacy-trino`. | -| **Policy download URL** | `/service/plugins/policies/download/`. | -| **Authentication** | Whether egret requires a bearer token on the download, for example *The policy download is anonymous: no token is needed.* Egret does not show or issue a token. | +| **Policy download URL** | `/ranger/service/plugins/policies/download/`. | +| **Authentication** | *The policy download needs this engine's Ranger credential. Its password was shown once, at registration or rotation; the files below carry a placeholder in its place.* When egret runs the policy service anonymously: *The policy download is anonymous: no credential is needed.* | | **Declared plugin version** | The Ranger plugin version recorded for the engine. | ### Warnings @@ -47,16 +47,17 @@ The installation files are on the engine's **Plugin config** tab in the Control The tab raises warnings when something will stop the download: - The download URL is a loopback, unspecified or in-cluster address, so an engine elsewhere cannot reach it. The URL comes from the `[catalog] public_url` setting. On the demo stack it is `http://egret:8080`: *The download URL (http://egret:8080) is an in-cluster address: an engine outside the cluster cannot reach it.* -- The download is anonymous: *anyone who can reach this URL can download every profile's policies. Restrict it at the network level.* -- Egret requires a bearer token on `/service`, and a stock Ranger plugin cannot send it, so the download is refused (401) unless something in front of egret adds the token or exempts the engine. +- The policy service runs anonymously: *egret runs the Ranger surface in anonymous mode: anyone who can reach the URL can download every profile's policies. Restrict it at the network level.* +- Ranger is off on this egret, so `/ranger` is not served. +- The declared plugin version is not one egret's evaluator models, so migrations into the profile are refused. ### You must supply -*egret renders no credential and invents none.* You provide: +You provide: - A network path from the engine to the download URL. -- The files shown, installed by you. Egret never connects to the engine. -- A way to present egret's bearer token on the download, for example a proxy that adds it. +- The files below, installed by you: egret never connects to this engine. +- The engine's Ranger credential (shown once, at registration or rotation) in the security XML, in place of the placeholder password. An engine registered before credentials existed has none: get one with **Rotate credential**. ### What egret assumes diff --git a/platform/data-governance/lineage.md b/platform/data-governance/lineage.md index 0672ce8..ddbe9ad 100644 --- a/platform/data-governance/lineage.md +++ b/platform/data-governance/lineage.md @@ -36,7 +36,17 @@ Lineage here is table-level, not column-level: it says which tables a view reads When a view is created in an egret catalog, egret uses its lineage to tag the view with the classifications of its base tables. If lineage fails to compute, the Governance overview raises a **view lineage failed** finding. -On the demo stack the view `demo_warehouse.analytics.revenue_by_country` reads `sessions`, but egret could not compute its lineage: reading the view through the provisioned engine was refused with *Access Denied: Cannot access catalog demo_warehouse*. So the **Lineage** tab of `sessions` shows the warning above instead of the view, and the view is listed on the tag's **View exposure** tab as *may read classified data*. See [View exposure](./classifications/view-exposure.md) and [Data](./data/index.md#lineage). +## How egret computes lineage + +Egret computes a view's lineage by sending `EXPLAIN` of the view to an engine that serves its catalog, as its own reader `egret-lineage`. Only egret mints that reader's token, and only `EXPLAIN` is sent as it. What the reader may do is added to the policies at download time, and only for the Ranger plugin of a **provisioned Trino engine in JWT delegation mode** bound to the profile, where egret signs every token (on Kubernetes or on the Docker provisioner). That plugin receives, for `egret-lineage`: + +- `use` and `select` on every catalog the profile's engines attach; +- `impersonate` of itself only, `execute` on query ids and `alter` of the session property `query_max_run_time`; +- a deny of every other access, and a leading exception in every mask, row filter and deny, so that a mask or filter cannot shorten the lineage it reads. + +Any other plugin (an external engine, or an engine in another delegation mode) is served the stored policies unchanged, so a lineage `EXPLAIN` there is refused and the view's lineage stays unknown: egret then treats the view as possibly reading classified data. + +On the demo stack the view `demo_warehouse.analytics.revenue_by_country` reads `sessions`. Once a provisioned engine served `demo_warehouse`, the lineage sweep recorded it (*computed: 1*). Right after the engine starts, the first sweeps can time out at the 5-second `EXPLAIN` limit; they pass once the engine is warm. Until lineage is computed, the **Lineage** tab of `sessions` shows the warning above instead of the view, and the view is listed on the tag's **View exposure** tab as *may read classified data*. See [View exposure](./classifications/view-exposure.md) and [Data](./data/index.md#lineage). ## Further reading diff --git a/platform/data-governance/overview.md b/platform/data-governance/overview.md index 9d8eae9..4601777 100644 --- a/platform/data-governance/overview.md +++ b/platform/data-governance/overview.md @@ -43,7 +43,7 @@ The checks the server runs are: On the demo stack, with the engine `docs-trino` serving `demo_warehouse`, the list held two findings: -- **Warning**: *egret could not compute the lineage of 1 view, so the tables it reads are not known.*, with the view and the engine's answer: `demo_warehouse.analytics.revenue_by_country — engine docs-trino: engine: Access Denied: Cannot access catalog demo_warehouse`. Egret reads view definitions through the engine, and the engine's policy did not let egret's own account into the catalog. +- **Warning**: *egret could not compute the lineage of 1 view, so the tables it reads are not known.*, with the view and the engine's answer: `demo_warehouse.analytics.revenue_by_country — engine docs-trino: engine: Access Denied: Cannot access catalog demo_warehouse`. Egret reads view definitions through the engine as its lineage reader. That answer came from a Docker engine before platform #728, when the engine's plugin was not recognised and so did not receive the reader's access; see [Lineage](./lineage.md#how-egret-computes-lineage) for which engines do. - **Info**: *3 classifications have no clearance: they gate nothing.*, listing the tags. Before an engine serves a catalog, the finding is that the catalog is attached to no engine. When everything is in order the list reads *Nothing needs attention.* diff --git a/platform/deploy-operate/authentication.md b/platform/deploy-operate/authentication.md index 5b92949..8454f0d 100644 --- a/platform/deploy-operate/authentication.md +++ b/platform/deploy-operate/authentication.md @@ -15,7 +15,8 @@ the realm file in the repository, and shows how groups become access. 2. The provider authenticates the person and redirects to `oidc.redirect_url` (`/api/auth/oidc/callback`). 3. Egret exchanges the code, validates the ID token and creates or updates the user. Users are matched by the token's `sub`; the email and name are refreshed at every sign-in. Group names from the `groups` claim are recorded and the user's - membership of them is synchronized. + membership of them is synchronized. If a workspace administrator added this email before the person's first sign-in, + those memberships are created now, but only when the ID token says `email_verified: true` (see [Members](../workspace-settings/members#add-someone-who-has-not-signed-in-yet)). 4. Egret sets a session cookie (`egret_session`, `HttpOnly`) and a CSRF cookie (`egret_csrf`), and redirects to `oidc.post_login_redirect` (`/`). @@ -32,7 +33,7 @@ request. The provider can end it earlier with OIDC back-channel logout; see belo | `oidc.client_secret` | `EGRET_OIDC__CLIENT_SECRET` | The client's secret. | | `oidc.redirect_url` | `EGRET_OIDC__REDIRECT_URL` | `https:///api/auth/oidc/callback`, exactly as registered at the provider. | | `oidc.groups_claim` | `EGRET_OIDC__GROUPS_CLAIM` | `groups` | -| `oidc.scopes` | `EGRET_OIDC__SCOPES` | Default `["openid","email","profile"]`. | +| `oidc.scopes` | `EGRET_OIDC__SCOPES` | Default `["openid","email","profile"]`. `openid` is always requested, once, whether or not you list it: the request carries `scope=openid email profile`. | | `security.secret_key` | `EGRET_SECURITY__SECRET_KEY` | Base64, at least 64 bytes. **Required** when OIDC is on. | | `session.secure` | `EGRET_SESSION__SECURE` | `true` when Egret is served over HTTPS. | @@ -119,8 +120,9 @@ EGRET_ADMIN_EMAIL=ops@example.com EGRET_ADMIN_PASSWORD='a long random password' password you supply is never printed: `init` prints `Password: taken from EGRET_ADMIN_PASSWORD (not shown)`. - `init` does nothing when a platform administrator already exists. `egret init --force` adds another one. - On the sign-in page the account is behind **Admin sign-in (break-glass)** when SSO is on. -- Local sign-in is rate limited (`rate_limit.login_*`: 5 attempts per 5 minutes, successful attempts included), which also - slows password guessing. +- Local sign-in is rate limited (`rate_limit.login_*`): 5 failed attempts within 5 minutes lock the address and email out + for 5 minutes. Successful attempts do not count, and a successful sign-in clears the failures before it. This slows + password guessing. See the [security checklist](./security-hardening) for how to protect and monitor it. diff --git a/platform/deploy-operate/engines-and-ranger.md b/platform/deploy-operate/engines-and-ranger.md index 26dd155..96ee8bb 100644 --- a/platform/deploy-operate/engines-and-ranger.md +++ b/platform/deploy-operate/engines-and-ranger.md @@ -29,6 +29,8 @@ jar out of each. | StarRocks | `starrocks/fe-ubuntu:4.1.4` | 2.8.0 | | StarRocks | `starrocks/fe-ubuntu:4.1.6` | 2.9.0 | +Plugin 2.5.0 cannot present the engine credential the policy service requires (see [Authentication of the plugin](#authentication-of-the-plugin)), so Egret refuses to create an engine on `trinodb/trino:470` or update one onto it (400), and refuses to register an external engine that declares a plugin older than 2.8.0. An existing Trino 470 engine must move to Trino 483. + New provisioned engines default to **Trino 483** and **StarRocks 4.1.6**, pinned by digest. `GET /api/control-plane/engines/modelled-ranger-plugins` lists the modelled pairs: Trino 2.5.0 and 2.8.0, StarRocks 2.8.0 and 2.9.0. An engine on a floating tag (`latest`, `4.1-latest`) or a rebuilt image is reported as **not modelled**: policies @@ -93,44 +95,57 @@ TLS on both before any real use. ## The policy service Turn it on with `ranger.enabled = true` (`EGRET_RANGER__ENABLED`). Egret then serves a read-only, Ranger-compatible API -under `/service`: +under `/ranger`, on the same server and port as `/api`. A plugin's `policy.rest.url` is `/ranger`, and the plugin +appends Ranger's own `/service/...` paths itself: | Route | Purpose | |---|---| -| `GET /service/plugins/policies/download/{service}?lastKnownVersion=N` | The policies of a policy profile. Answers `304` when `N` is the current version, `200` with the policy set otherwise, `404` for an unknown profile. | -| `GET /service/tags/download/{service}?lastKnownVersion=N` | The resource-to-tag mapping, from the classification tags assigned in Governance. | -| `GET /service/plugins/definitions`, `/definitions/name/{name}`, `/services`, `/policies/{id}` | Service definitions and listings. | +| `GET /ranger/service/plugins/secure/policies/download/{service}?lastKnownVersion=N` | The policies of a policy profile, for a plugin that sends a username and password (what Egret's engines do). The same answer is served at `/ranger/service/plugins/policies/download/{service}`. `304` when `N` is the current version, `200` with the policy set otherwise, `404` for an unknown profile. | +| `GET /ranger/service/tags/secure/download/{service}?lastKnownVersion=N` | The resource-to-tag mapping, from the classification tags assigned in Governance. Also served at `/ranger/service/tags/download/{service}`. | +| `GET /ranger/service/plugins/definitions`, `/definitions/name/{name}`, `/services`, `/policies/{id}` | Service definitions and listings, for the operator token only. | `{service}` is the name of a **policy profile**, created under **Governance → Engine & policy**. The plugin's -`service.name` must equal it. Egret has no write routes under `/service`: policies are authored through the authenticated -control-plane API. +`service.name` must equal it. Egret has no write routes under `/ranger`: policies are authored through the authenticated +control-plane API. The old `/service/...` paths at the server root answer `404`. Egret serves no Ranger roles or user +store: `/ranger/service/roles/secure/download/...` answers `404`, and the plugin carries on. ### Authentication of the plugin -`ranger.auth_token` decides who may call `/service`: +Every request under `/ranger/service/` is authenticated. `ranger.auth_mode` (`EGRET_RANGER__AUTH_MODE`, Helm +`ranger.authMode`) is `required` by default: -| `EGRET_RANGER__AUTH_TOKEN` | Behaviour | -|---|---| -| Empty | `/service` is open to anyone who can reach it. Stock plugins send no credentials, so they work. It is read-only. | -| Set | Every call needs `Authorization: Bearer `. Without it the answer is `401`. | +| Caller | Header | May read | +|---|---|---| +| An engine | `Authorization: Basic` with the engine's own credential, username `engine-` | The profile its engine is bound to and that profile's tag service. Anything else is `403`. | +| An operator | `Authorization: Bearer ` | Everything. An empty `auth_token` (the default) accepts no bearer. | +| Anyone else | none | Nothing: `401`, with the same body whatever was wrong. | -A **stock Ranger plugin cannot send a bearer token**, and the plugin configuration Egret renders contains no credential. So a -token set on Egret locks stock plugins out unless something in front of Egret adds the header. Egret says so itself: the -plugin-config endpoint for an external engine warns *egret requires a bearer token on /service, and a stock Ranger plugin -cannot send it*. The usual arrangement is an empty token plus a network policy or ingress rule that exposes `/service` only -to the engines; the Helm chart's ingress does not route `/service` at all. See the [security -checklist](./security-hardening). +`ranger.auth_token` (`EGRET_RANGER__AUTH_TOKEN`) is therefore only the operator's token for tooling. It does not lock the +plugins out: each engine presents its own credential, which needs a Ranger plugin 2.8.0 or later. -Checked on a running instance with the token set: +- **Engines Egret provisions** get their credential on the first apply of an engine bound to a policy profile. On + Kubernetes it reaches the pods in a Secret; on the Docker provisioner it is written into the containers. You never see + the password. +- **External engines** get theirs at registration: the register page shows the username, the password and a credential id + **once**, and no other response carries the password. The Plugin config tab renders the files with the username and the + placeholder `REPLACE_WITH_THE_ENGINE_PASSWORD`. **Rotate credential** on the engine's Overview tab issues a new one; the + previous one keeps working until you confirm the new one is installed (**Retire previous credential**). + +`ranger.auth_mode = "anonymous"` serves requests without a credential, and lets them read every profile. Use it only on a +throwaway local stack. The credential travels as HTTP Basic, so serve `/ranger` over TLS or keep it inside the cluster. + +Checked against an engine's credential: ```bash -curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8080/service/plugins/policies/download/docs-trino-policy -# 401 -curl -s -o /dev/null -w '%{http_code}\n' -H 'Authorization: Bearer ' \ - 'http://localhost:8080/service/plugins/policies/download/docs-trino-policy?lastKnownVersion=3' -# 304 (3 is the current policy version) +curl -s -o ranger-check.json -w '%{http_code} %{content_type}\n' \ + -u 'engine-:' \ + 'https://egret.example.com/ranger/service/plugins/secure/policies/download/' +# 200 application/json (a 200 with text/html is not Egret: route /ranger to Egret's backend) ``` +A running plugin that is refused keeps enforcing the policies it holds in memory and retries every poll interval; a plugin +that starts with no cached policies and is refused denies everything. + ## The plugin configuration Egret serves Egret renders the plugin files in one place, so the files it mounts into engines it provisions are the same files it gives @@ -157,7 +172,9 @@ ranger.plugin.config.resource=/etc/trino/ranger-trino-security.xml ranger.plugin.trino.service.namedocs-trino-policy -ranger.plugin.trino.policy.rest.urlhttp://egret:8080 +ranger.plugin.trino.policy.rest.urlhttp://egret:8080/ranger +ranger.plugin.trino.policy.rest.client.usernameengine-0b8f6c1e-… +ranger.plugin.trino.policy.rest.client.passwordREPLACE_WITH_THE_ENGINE_PASSWORD ranger.plugin.trino.policy.source.implorg.apache.ranger.admin.client.RangerAdminRESTClient ranger.plugin.trino.policy.cache.dir/tmp/ranger/docs-trino-policy/policycache ranger.plugin.trino.policy.pollIntervalMs30000 @@ -166,8 +183,9 @@ ranger.plugin.config.resource=/etc/trino/ranger-trino-security.xml ``` -`policy.rest.url` is Egret's base URL as the engine sees it (the plugin appends `/service/plugins/policies/download/...` -itself). The example above was returned by the API; on a fresh install it says `http://localhost:8080`, taken from +`policy.rest.url` is Egret's base URL as the engine sees it, followed by `/ranger` (the plugin appends +`/service/plugins/...` itself). The username and password lines carry the engine's credential; for an external engine the +password is a placeholder you replace with the one shown at registration. The example above follows what the API returns; on a fresh install it says `http://localhost:8080`, taken from `catalog.public_url`, which is not reachable from an engine in a cluster. Set `catalog.public_url` first. ### StarRocks diff --git a/platform/deploy-operate/kubernetes.md b/platform/deploy-operate/kubernetes.md index 21b27a3..722fd97 100644 --- a/platform/deploy-operate/kubernetes.md +++ b/platform/deploy-operate/kubernetes.md @@ -41,7 +41,7 @@ Optional objects appear with these values: | When | Extra objects | |---|---| | `oidc.enabled=true` and `oidc.devRealm=true` | `Deployment/keycloak`, `Service/keycloak`, `ConfigMap/keycloak-realm` (the realm file). | -| `ingress.enabled=true` | `Ingress/egret` (routes `/auth`, `/data-api`, `/oauth`, `/.well-known/*` and `/`) and `Ingress/egret-catalog`. `/service` (the policy API) is not routed. | +| `ingress.enabled=true` | `Ingress/egret` (routes `/auth`, `/data-api`, `/oauth`, `/ranger`, `/.well-known/*` and `/`) and `Ingress/egret-catalog`. `/ranger` (the policy API) is routed to Egret's backend so that engines outside the cluster can reach it. | | `sharedStore.enabled=true` | `Deployment/valkey` and `Service/valkey`. | | `workflow.containerRunner=kubernetes` | `Namespace/egret-tasks` (Pod Security `restricted`), `Role` and `RoleBinding` `egret-task-runner`, `NetworkPolicy/egret-task-isolation`. | | `apps.pod.launcher=kubernetes` | `Namespace/egret-apps` and `NetworkPolicy/egret-app-pod-isolation`. With `apps.origin` and `ingress.enabled` also set, `Ingress/egret-apps`. | diff --git a/platform/deploy-operate/security-hardening.md b/platform/deploy-operate/security-hardening.md index 24307b3..9e2d602 100644 --- a/platform/deploy-operate/security-hardening.md +++ b/platform/deploy-operate/security-hardening.md @@ -34,8 +34,8 @@ that means. Each item says how to check it. Tick them in order. (or let `init` generate one, printed once, and store it in a vault). Never keep the Helm default `Admin12345!` (`admin.password`). - [ ] **Move day-to-day administration to IdP groups.** Grant `platform_admin` to a group in **Control Plane → Groups**, and keep the local account for recovery. -- [ ] **Keep the local sign-in rate limit.** Five attempts per five minutes (`rate_limit.login_*`) is what stands in front of - password guessing, and successful attempts count. +- [ ] **Keep the local sign-in rate limit.** Five failed attempts per five minutes, then a five-minute lockout + (`rate_limit.login_*`), is what stands in front of password guessing. Only failures count. - [ ] **Know that local sign-in cannot be switched off.** With SSO on, the local form is only hidden behind **Admin sign-in (break-glass)**; the `POST /api/auth/login` route still answers. Restrict who can reach it at the network layer if that matters to you. @@ -71,14 +71,15 @@ Expose only what clients need. | `/` and `/api/*` | People | Behind TLS. | | `/data-api/*`, `/oauth/token`, `/.well-known/*` | Data API consumers and clients of Egret's machine identity provider | Rate limited (`rate_limit.token_*`). | | `/catalog/*` | Engines and Iceberg clients | Authenticated with Egret-issued tokens. | -| `/service/*` | **Engines only** | Read-only policy API; see below. | +| `/ranger/*` | **Engines only** | Read-only policy API, authenticated with each engine's own credential; see below. | | `/metrics`, `/healthz`, `/readyz` | **Your monitoring only** | **Unauthenticated.** `/metrics` answers `200` with Prometheus metrics to anyone who can reach it. The chart's ingress does not route them. | -- [ ] **Do not route `/service`, `/metrics`, `/healthz` or `/readyz` to the internet.** -- [ ] **Decide how the policy API is protected.** With `EGRET_RANGER__AUTH_TOKEN` empty, `/service` is open to anyone who can - reach it (read-only: policies and tags). With a token set, a stock Ranger plugin cannot send it and is refused with 401. - Prefer an empty token plus a network policy that admits only the engines. See [Engines and Ranger - setup](./engines-and-ranger). +- [ ] **Do not route `/ranger`, `/metrics`, `/healthz` or `/readyz` to the internet.** +- [ ] **Keep the policy API authenticated.** `ranger.auth_mode` is `required` by default: each engine presents its own + credential, and `ranger.auth_token` is only an operator token (leave it empty unless tooling needs it). Never run + `anonymous` outside a throwaway stack: it lets anyone who can reach `/ranger` download every profile's policies. Serve + `/ranger` over TLS or keep it inside the cluster, because the engine credential travels as HTTP Basic. See [Engines and + Ranger setup](./engines-and-ranger#authentication-of-the-plugin). - [ ] **Keep PostgreSQL and the object store off the public network.** In Kubernetes, the chart's task and app pod network policies exist to keep user code away from the metadata database: set `workflow.tasks.networkPolicy.podCidr`, `serviceCidr` and `apiServerCidrs` correctly, because wrong values silently re-open PostgreSQL on some network plugins. @@ -119,7 +120,7 @@ Expose only what clients need. ```bash curl -s -o /dev/null -w '%{http_code}\n' https://egret.example.com/metrics # 404 or 403 from the proxy, not 200 -curl -s -o /dev/null -w '%{http_code}\n' https://egret.example.com/service/plugins/services # not reachable from the internet +curl -s -o /dev/null -w '%{http_code}\n' https://egret.example.com/ranger/service/plugins/services # not reachable from the internet curl -sI https://egret.example.com/api/auth/config | grep -i set-cookie # no cookie yet; sign in and check Secure ``` diff --git a/platform/deploy-operate/troubleshooting/cannot-sign-in.md b/platform/deploy-operate/troubleshooting/cannot-sign-in.md index 34f9e74..bb57f95 100644 --- a/platform/deploy-operate/troubleshooting/cannot-sign-in.md +++ b/platform/deploy-operate/troubleshooting/cannot-sign-in.md @@ -33,16 +33,26 @@ when the server answers `401 {"error":"unauthorized"}`. Reusing the existing email fails with `conflict: user already exists: admin@egret.local`. Remove the recovery account when you no longer need it. -## *Too many attempts. Please wait a few minutes and try again.* +## *Too many failed sign-in attempts. Try again in 5 minutes.* -The server answered `429 {"error":"too many requests"}`. Local sign-in allows **5 attempts per 5 minutes** (`rate_limit.login_max_attempts` -and `login_window_secs`), counted per address and per email, and **successful attempts count**: reproduced with one successful -and four failed attempts, after which the next attempt was refused. Once exceeded, the key stays blocked for -`rate_limit.login_lockout_secs` (300). +Local sign-in allows **5 failed attempts within 5 minutes** (`rate_limit.login_max_attempts` and `login_window_secs`), +counted per address and email. The fifth failure still answers *Invalid email or password*; the attempt after it, even +with the right password, is refused for `rate_limit.login_lockout_secs` (300) with: + +```text +HTTP/1.1 429 Too Many Requests +retry-after: 300 + +{"error":"too many failed sign-in attempts; try again in 300 seconds","code":"rate_limited","retry_after_secs":300} +``` + +The sign-in page turns `retry_after_secs` into *Try again in N minutes* (or seconds, under a minute). Only failures count: +signing in successfully several times in a row never locks you out, and a successful sign-in clears the failures before +it. Attempts refused during the lockout do not extend it. **Fix** -- Wait. Do not keep retrying: a script that retries during the lockout can keep it open. +- Wait the time the message gives, then sign in with the right password. - Automation should not sign in with the local form at all. Use a Data API client or a catalog client for machine access, or keep one session and reuse its cookies. - In a throwaway instance, restarting the server clears the limiter, which is in memory. Do not do that on a shared one. @@ -72,13 +82,22 @@ csrf check failed [403] Read the CSRF cookie after sign-in and send it back in the header, as the web app does. `GET` requests do not need it. -## Signed in, but the app is empty or says *Forbidden* +## Signed in, but the app is empty, says *Forbidden* or shows *No workspaces yet* You are authenticated and have no permissions. An SSO user is created as a plain member; access comes from roles granted to -their **groups**. +their **groups**, or to them directly. Someone with no workspace and no Control Plane or Governance access sees a **No +workspaces yet** card with the email they signed in with, **Check again** and **Sign out**. Ask a workspace admin to add that +email, then press **Check again**. - Check the user under **Control Plane → Users** and the groups under **Control Plane → Groups**: is the group there, and does - it have a platform role or a workspace membership? + it have a platform role or a workspace membership? The Users list shows only roles granted to the user directly, so read + the group's own page for a role it holds. +- A group synced from the identity provider, such as `platform-admins`, holds no role by itself: an administrator grants it + one. The development stack's seed (`deploy/seed-sso-test-rbac.sql`) makes that grant; the Docker demo stack does not, so + `admin-sso` signs in there with no platform role. +- An email added to a workspace before the first sign-in binds only when the identity provider marks it verified + (`email_verified`). An unverified address lands on **No workspaces yet** and stays under **Waiting for first sign-in** on + the workspace's Members page. - No groups at all: the token does not carry them. See [SSO errors](./sso-errors). - Grant access with **Manage → Workspace settings → Members**, or `egret workspace add-group --workspace --group --role ws_editor`. diff --git a/platform/deploy-operate/troubleshooting/index.md b/platform/deploy-operate/troubleshooting/index.md index e4fbcd5..fff6587 100644 --- a/platform/deploy-operate/troubleshooting/index.md +++ b/platform/deploy-operate/troubleshooting/index.md @@ -36,7 +36,7 @@ docs. | Symptom | Cause and fix | Page | |---|---|---| | *502 Bad Gateway* creating a dataset or testing a database on Trino | Trino answered HTTP 406. Set `http-server.process-forwarded=true` on Trino. | [406 and 502](troubleshooting/engine-406-502) | -| *429 Too Many Requests* at sign-in | Five attempts per five minutes, successful ones included. Wait. | [Cannot sign in](troubleshooting/cannot-sign-in) | +| *429 Too Many Requests* at sign-in | Five failed attempts within five minutes lock you out for five minutes. Wait the time the page gives. | [Cannot sign in](troubleshooting/cannot-sign-in) | | A workflow execution stays in *Running* | The worker is off by default: `EGRET_WORKFLOW__WORKER_ENABLED=true`. | [Workflows stuck](troubleshooting/workflows-stuck) | | `Error: the apps API requires [storage] bucket, access_key, secret_key` at start | The object-store credentials are always required. | [Catalog and storage errors](troubleshooting/catalog-storage-errors) | | *secrets master_key is not configured* when saving a connection | Set `EGRET_SECRETS__MASTER_KEY`. | [Catalog and storage errors](troubleshooting/catalog-storage-errors) | diff --git a/platform/explore/alerts-and-reports/create-an-alert.md b/platform/explore/alerts-and-reports/create-an-alert.md index 0cbfb6a..df08476 100644 --- a/platform/explore/alerts-and-reports/create-an-alert.md +++ b/platform/explore/alerts-and-reports/create-an-alert.md @@ -21,7 +21,7 @@ The editor has four tabs, **General**, **Condition**, **Schedule** and **Notific ## General -![The General tab of an existing alert, with Recent runs underneath: On grace runs and a Success run with a webhook error](/img/platform/alerts/detail.png) +![The General tab of an existing alert, with Recent runs underneath](/img/platform/alerts/detail.png) | Setting | What it does | Default | |---|---|---| @@ -64,7 +64,7 @@ The alert **Pending orders above 100** uses these settings: | **Grace period** | 3600 seconds | | **Notification** | Email `ops@example.com` and a webhook to `https://example.com/egret-hook`, subject `[egret] {{name}} on {{date}}`, an opening and a closing note, and the **Order operations** dashboard attached as a PNG | -Pressing **Run now** from the list ran it once. The sample data has 363 partially shipped orders, so the value `363` passed `> 100`: the run was recorded as **Success**, meaning the alert fired. The email arrived (below). The webhook receiver answered `405`, so the run log's **Detail** reads `webhook returned 405`; see [Run history](./run-history). A second **Run now** a few minutes later, and the next scheduled run, were recorded as **On grace** and sent nothing. +Pressing **Run now** from the list ran it once. The sample data has 363 partially shipped orders, so the value `363` passed `> 100`: the alert fired. The email arrived (below). The webhook receiver answered `405`, so the run is recorded as **Error** and the run log's **Detail** reads `webhook example.com: webhook returned 405`; see [Run history](./run-history). Because the email reached someone, the grace period started anyway. A second **Run now** a few minutes later, and the next scheduled run, were recorded as **On grace** and sent nothing. A second alert, **Cancelled orders spike**, counts orders with status `X`. There are none, so the value is `0`, which does not pass `> 0`: each run is recorded as **Not triggered** and nobody is notified. @@ -79,7 +79,7 @@ The email has the alert name as a heading, the **Opening note**, the attached ch When a run's value meets the condition, Egret: 1. skips notification and records **On grace** if the alert already fired within its [grace period](./schedules#grace-period-seconds); -2. otherwise sends the message to every destination and records **Success**. +2. otherwise sends the message to every destination and records **Success**, or **Error** with each failed destination named if any destination did not take it. See [When delivery fails](./notifications#when-delivery-fails). The triggering rows are attached to the notification: as a CSV file on email, as JSON in the payload of a webhook. Slack does not receive them. See [Notifications](./notifications). diff --git a/platform/explore/alerts-and-reports/index.md b/platform/explore/alerts-and-reports/index.md index 1369385..e5b45d4 100644 --- a/platform/explore/alerts-and-reports/index.md +++ b/platform/explore/alerts-and-reports/index.md @@ -22,12 +22,12 @@ Open **Explore → Alerts & Reports**. The page has two tabs, **Alerts** (which |---|---| | **Name** | The name, a star to bookmark it, and its description. | | **Active** | A switch. Turn it off to pause the schedule without deleting it; a paused item shows **Paused** and keeps its history. | -| **Last run** | The result of the last *scheduled* run: **Success**, **Working**, **Error**, **Not triggered** or **On grace**. **Never run** until the scheduler has run it. See [Run history](./alerts-and-reports/run-history). | +| **Last run** | The result of the last run, scheduled or **Run now**: **Success**, **Working**, **Error**, **Not triggered** or **On grace**. A run whose message did not reach every destination is **Error**. **Never run** until it has run once. See [Run history](./alerts-and-reports/run-history). | | **Schedule** | A plain-language form of the cron schedule, with the timezone, for example `Every day at 08:00 Asia/Ho_Chi_Minh`. Hover for the raw cron expression. An hourly schedule (`0 * * * *`) and a custom expression such as `30 8 * * 1-5` are shown as the raw cron (KI-038). | | **Next run** | When the scheduler will run it next, for example `in 9 hours`. | | **Owners** | The people whose permissions the query runs with. | -Above the table: **Search** by name, description or owner; **Last run** filters by state (including **Never run**); the counters show how many are total, **active** and **paused**, and a **failing** shortcut appears when at least one item's last scheduled run was **Error** (for example "6 reports · 5 active · 1 paused · 1 failing"). The toolbar buttons are **Bookmarks**, **Bulk select**, **Trash** and **New folder**. Folders group items on the page. +Above the table: **Search** by name, description or owner; **Last run** filters by state (including **Never run**); the counters show how many are total, **active** and **paused**, and a **failing** shortcut appears when at least one item's last run was **Error** (for example "6 reports · 5 active · 1 paused · 1 failing"). The toolbar buttons are **Bookmarks**, **Bulk select**, **Trash** and **New folder**. Folders group items on the page. ### Row actions @@ -61,13 +61,12 @@ An alert **runs as its owner**. The query is executed with the owner's permissio | [Data API alerts](./alerts-and-reports/data-api-alerts) | Watching endpoint traffic. | :::note -Egret needs somewhere to send the message. Email needs workspace SMTP and Slack needs a workspace Slack app, both set under [Workspace settings → Notifications](../workspace-settings/notifications). Without them the alert still runs and records its result, but it tells nobody; the run log shows `smtp is not configured for this workspace` or `no slack bot token is configured for this destination`. On the Docker demo stack SMTP points at Mailpit, so every email lands in the inbox at `http://localhost:8025`. +Egret needs somewhere to send the message. Email needs workspace SMTP and Slack needs a workspace Slack app, both set under [Workspace settings → Notifications](../workspace-settings/notifications). Without them the alert still runs, but it tells nobody and the run is an **Error**; the run log shows `email: smtp is not configured for this workspace` or `slack #channel: no slack bot token is configured for this destination`. On the Docker demo stack SMTP points at Mailpit, so every email lands in the inbox at `http://localhost:8025`. ::: ## Limits and known issues -- **Last run** and the header badge on an item show the last *scheduled* run. A **Run now** result appears in the run log only, so a freshly created item you have only run manually still reads **Never run** (KI-032). -- PNG and PDF reports need the rendering service, and a multi-object PDF must render within 30 seconds; see [Reports](./alerts-and-reports/reports#png-and-pdf-need-the-renderer). +- PNG and PDF reports need the rendering service, and each object of a report must render within its time limit (30 seconds per object, plus 30); see [Reports](./alerts-and-reports/reports#png-and-pdf-need-the-renderer). ## Further reading diff --git a/platform/explore/alerts-and-reports/notifications.md b/platform/explore/alerts-and-reports/notifications.md index 716b75e..86382c0 100644 --- a/platform/explore/alerts-and-reports/notifications.md +++ b/platform/explore/alerts-and-reports/notifications.md @@ -18,7 +18,7 @@ Under **Deliver to**, press **Add delivery method** and choose **Email**, **Slac |---|---| | **Addresses** | Comma-separated recipients, for example `ops@example.com, data-team@example.com`. | -All addresses on one email destination receive a single message. For an alert the triggering rows are attached as a CSV file named after the alert, for example `urgent-open-orders-exist.csv`. Sending needs SMTP configured for the workspace; without it the run records the error `smtp is not configured for this workspace`. What an email looks like is shown in [Create an alert](./create-an-alert#what-the-recipient-gets) and [Reports](./reports#what-recipients-get). +All addresses on one email destination receive a single message. For an alert the triggering rows are attached as a CSV file named after the alert, for example `urgent-open-orders-exist.csv`. Sending needs SMTP configured for the workspace; without it the run is an **Error** with `email: smtp is not configured for this workspace` in its detail. What an email looks like is shown in [Create an alert](./create-an-alert#what-the-recipient-gets) and [Reports](./reports#what-recipients-get). ### Slack @@ -26,7 +26,7 @@ All addresses on one email destination receive a single message. For an alert th |---|---| | **Channel** | The channel to post to, for example `#data-alerts`. | -The credential lives in the workspace's Slack app, set in **Workspace settings → Notifications**. The bot needs the `chat:write` permission and an invite to the channel. Slack receives the message text only, not the rows. Without a configured app the run records `no slack bot token is configured for this destination`, and the run still counts as fired, so the grace period starts. +The credential lives in the workspace's Slack app, set in **Workspace settings → Notifications**. The bot needs the `chat:write` permission and an invite to the channel. Slack receives the message text only, not the rows. Without a configured app the run is an **Error** with `slack #channel: no slack bot token is configured for this destination`. If no other destination took the message, no grace period starts, so the next scheduled run tries again. :::note Demo stack The demo workspace has no Slack app, because a working one needs a real Slack workspace and bot token. A Slack destination there records the error above; posting to a channel could not be observed. @@ -65,13 +65,15 @@ Webhook delivery has guard rails: - The request times out after 10 seconds, and redirects are **not** followed, because a redirect would be a second destination that bypasses the checks. - A destination that resolves to a private, loopback or link-local address is refused before any request is sent: `webhook host resolves to a blocked address (127.0.0.1): loopback`, or for a container on the same Docker network `webhook host resolves to a blocked address (172.24.0.5): private`. An operator can allow specific in-cluster hosts with the `automation.webhook_allow_hosts` setting. -- A response that is not `2xx` is recorded with its status, for example `webhook returned 405`. +- A response that is not `2xx` is recorded with its status, for example `webhook example.com: webhook returned 405`. + +In the run log a webhook is named by its host only (`webhook example.com: …`), never by its full URL, because the path or query of a webhook URL often carries its secret. ## Message | Setting | What it does | Default | |---|---|---| -| **Email subject** | The subject line. Placeholders: `{{date}}`, `{{time}}`, `{{datetime}}`, `{{day}}`, `{{month}}`, `{{year}}`, `{{weekday}}`, `{{name}}`. Dates and times use the schedule's timezone. Under the box, **Sends as** previews the subject with today's values. | `[egret] name triggered` for an alert, `[egret] name report` for a report | +| **Email subject** | The subject line. Placeholders: `{{date}}`, `{{time}}`, `{{datetime}}`, `{{day}}`, `{{month}}`, `{{year}}`, `{{weekday}}`, `{{name}}`. Dates and times use the schedule's timezone. Under the box, **Sends as** previews the subject with today's values. | `[egret] name triggered` for an alert, `[egret] name report` for a report (`[egret] name` when the name already ends in the word *report*) | | **Opening note** | Plain text above the result, at the top of the notification. Say why the alert exists and who to ask. | Empty | | **Closing note** | Plain text below the result, before the **Open in egret** button. Say what to do about it. | Empty | @@ -100,13 +102,22 @@ For an alert, tick this to skip the notification when the query returned no rows ## When delivery fails -A failed delivery does not fail the run. The alert already fired, so the run is recorded as **Success** and the grace period starts, and the delivery error is shown in the run log's **Detail** column. When an item has several destinations, only the first error is recorded. See [Run history](./run-history). +Egret tries every destination of a run, even after one has failed. A run is **Success** only when every destination took the message; otherwise it is **Error**, the item counts as failing in the list, and the run log's **Detail** column names each destination that failed, for example `webhook 127.0.0.1: webhook host resolves to a blocked address (127.0.0.1): loopback; slack #ops: no slack bot token is configured for this destination`. + +What happens next depends on whether anyone received the message: + +| Outcome of the run | State | Grace period | +|---|---|---| +| Every destination took the message | **Success** | Starts | +| Some destinations took it, others failed | **Error** | Starts, so the recipients who got it are not sent the same message again on every run | +| Every destination failed | **Error** | Does not start: the next scheduled run tries again | + +An item with no destination at all counts as delivered. A note about a message that did go out, such as an attachment that was skipped, is shown in **Detail** but does not make the run an **Error**. See [Run history](./run-history). ## Limits and known issues - Webhooks to loopback and private addresses are blocked unless an operator allow-lists the host, so a local receiver cannot be used for testing without that setting (KI-035). The demo stack does not set it. - Slack messages do not carry the rows. Slack delivery was not observed on the demo stack (no Slack app). -- When several destinations fail, only the first error is recorded (KI-034). ## Further reading diff --git a/platform/explore/alerts-and-reports/reports.md b/platform/explore/alerts-and-reports/reports.md index ca31625..812e63c 100644 --- a/platform/explore/alerts-and-reports/reports.md +++ b/platform/explore/alerts-and-reports/reports.md @@ -37,13 +37,20 @@ The **General** tab is the same as for an [alert](./create-an-alert#general). A | **PNG image** | The picture inside the email body, not as a file. | Yes | | **PDF** | A file attached to the email and named after the report, for example `leadership-weekly-pack.pdf`. Several objects are always delivered as one PDF with one sheet per object, so the format switches to PDF when you add a second object. | Yes | | **CSV data** | The data behind the chart as an attached `.csv` file with a header row. | No | -| **Text** | The data as a plain table in the message body, one row per line, for example `F \| 1.0356810234900002e+9`. | No | +| **Text** | The data as a plain table in the message body, one row per line, for example `F \| 1,035,681,023.49`. | No | -CSV and text read the chart's query result directly. Numbers are written as the engine returned them, so a large decimal appears in scientific notation (`1.0356810234900002e+9`) instead of `1,035,681,023.49` (KI-169). +CSV and text read the chart's query result directly and write numbers the way the chart's table shows them by default: rounded to at most three decimals, with trailing zeros dropped. Integers keep every digit. + +| Format | Example | +|---|---| +| **Text** | `1,035,681,023.49`, digits grouped | +| **CSV data** | `1035681023.49`, not grouped, so a spreadsheet reads it as a number whatever its locale | + +A column's own format settings (decimals, currency, prefix and so on) are not applied; only the table's defaults are. Round or format the value in the chart's SQL if you need something else. ## What recipients get -A report sends one email to all the addresses of an email destination. The subject is your **Email subject**, or `[egret] report` when you leave it blank. The body has the report name as a heading, your **Opening note**, the result (the picture for PNG, a line saying the file is attached for PDF and CSV, the table for text), your **Closing note**, and an **Open in egret** button that links to the first object. +A report sends one email to all the addresses of an email destination. The subject is your **Email subject**, or `[egret] report` when you leave it blank. When the name already ends in the word *report*, the subject is `[egret] `, so `Weekly report` does not mail as `Weekly report report`. The body has the report name as a heading, your **Opening note**, the result (the picture for PNG, a line saying the file is attached for PDF and CSV, the table for text), your **Closing note**, and an **Open in egret** button that links to the first object. ![A PNG report of the Executive summary dashboard as it arrives, in the demo stack's Mailpit inbox](/img/platform/alerts/mail-report-png.png) @@ -59,7 +66,9 @@ Charts and dashboards are drawn in a browser, so PNG and PDF reports are produce The Docker demo stack runs the renderer and sends every email to Mailpit (`http://localhost:8025`), which is where the pictures on this page come from. Nothing leaves the machine. ::: -On the demo stack a one-dashboard PNG took about 10 seconds, and the six-card **Sales performance** dashboard about 20 seconds as a PDF. Egret waits at most **30 seconds** for the whole render, and the renderer draws the objects of a multi-object PDF one after the other. A report whose objects together take longer fails with the same `render failed: error sending request …` message although the renderer is up (KI-167). +On the demo stack a one-dashboard PNG took about 10 seconds, and the six-card **Sales performance** dashboard about 20 seconds as a PDF. The renderer draws the objects of a multi-object PDF one after the other and waits up to 20 seconds for each page to finish drawing, so Egret's time limit grows with the number of objects: **30 seconds per object, plus 30 seconds** (60 seconds for one object, 90 for two, 180 for five). A five-dashboard PDF that took 36.9 seconds was delivered. + +A render that runs past the limit fails with `render failed: the renderer did not finish within 90s for 2 page(s) (timeout)`. A page that never signals it is ready fails earlier with the renderer's own message, for example `print page … did not signal readiness within 20000ms (timeout)`: a dashboard with many cards can hit this, so split it or deliver fewer cards. ## A worked example @@ -71,12 +80,10 @@ These reports were created on the demo workspace and run with **Run now** and on | **Revenue by status (text)** | Chart **Revenue by status** | **Text** | **Success**. The three rows were in the message body. | | **Executive summary snapshot** | Dashboard **Executive summary** | **PNG image** | **Success**. The picture was in the message body. | | **Leadership weekly pack** | Dashboards **Executive summary** and **Weekly sales review** | **PDF** | **Success**. A two-page PDF, one dashboard per page, rendered in about 26 seconds. | -| **Sales performance weekly digest** | Dashboards **Sales performance** and **Executive summary** | **PDF** | **Error** on every run: `render failed: error sending request for url (http://renderer:8080/render)`, the 30-second limit above. | +| **Sales performance weekly digest** | Dashboards **Sales performance** and **Executive summary** | **PDF** | **Error** on every run before platform #726, when the time limit was a fixed 30 seconds. | A successful run shows **Notification sent** in the run log's **Detail** column; a failed one shows the error. See [Run history](./run-history). -![The run log of a two-dashboard PDF report that exceeds the render time limit](/img/platform/alerts/run-log-report-error.png) - ## Reports and grace Reports ignore the grace period. A report that runs on its schedule always delivers. @@ -84,8 +91,7 @@ Reports ignore the grace period. A report that runs on its schedule always deliv ## Limits and known issues - The rendered pages show chart cards only, each under its chart's own name. Text and heading cards, card title overrides and tabs are not reproduced; see [Export, reports and print](../dashboards/export-and-reports#scheduled-report-rendering) (KI-073). -- A multi-object PDF fails when its renders take longer than 30 seconds together (KI-167). -- CSV and text reports write unformatted numbers (KI-169). +- CSV and text reports apply the table's default number format, not a column's own format settings (KI-266). - The opening and closing notes appear in the HTML part of the email only; the plain-text part has the result line alone (KI-170). - Report subjects accept the same placeholders as alerts, for example `Weekly sales overview - {{date}}`. diff --git a/platform/explore/alerts-and-reports/run-history.md b/platform/explore/alerts-and-reports/run-history.md index 4dd6b57..2fadff4 100644 --- a/platform/explore/alerts-and-reports/run-history.md +++ b/platform/explore/alerts-and-reports/run-history.md @@ -6,7 +6,7 @@ sidebar_position: 6 Every run of an alert, Data API alert or report writes a row to its **run log**, whether it fired or not. The run log is the answer to "why didn't I get a message?": a schedule that wrote nothing is indistinguishable from a scheduler that never ran, so every outcome is recorded. -![The run log dialog for an alert, showing a Success run with a delivery error](/img/platform/alerts/run-log.png) +![The run log of an alert with email, webhook and Slack destinations: an Error run whose Detail names the two failed destinations (the email went out), followed by an On grace run; the two older runs date from before platform #726, when a failed delivery was still a Success](/img/platform/alerts/run-log.png) ## Open the run log @@ -22,41 +22,43 @@ Each row has four columns. | **State** | The outcome of the run. | | **Started** | When the run began, for example `8 minutes ago`. | | **Value** | The number the run measured: the first column of the first row for an alert, the metric for a Data API alert. Empty for a report. | -| **Detail** | An error or delivery message, shown in red. **Notification sent** when the message went out without an error, and `—` when nothing was sent. | +| **Detail** | An error or delivery message, shown in red. **Notification sent** when every destination took the message, and `—` when nothing was sent. | ## Run states | State | Meaning | Notification sent | |---|---|---| -| **Success** | The run completed and the condition was met (for a report, it ran). If delivery then failed, the reason appears in **Detail**. | Yes, unless delivery failed | +| **Success** | The run completed, the condition was met (for a report, it ran), and every destination took the message. | Yes | | **Not triggered** | The run completed and the condition was not met, or the query returned no rows or a null, or **Don't send when there are no rows** applied. | No | | **On grace** | The condition was met, but the item already fired within its [grace period](./schedules#grace-period-seconds). | No | -| **Error** | The run could not complete: the query failed, the first column was not a number, or a report could not be rendered. **Detail** has the reason. | No | +| **Error** | The run could not complete (the query failed, the first column was not a number, or a report could not be rendered), or it fired and at least one destination did not take the message. **Detail** has the reason, one entry per failed destination. | Only to the destinations that took it | | **Working** | A run is in progress, or a worker died mid-run and the run has not yet been reclaimed after the working timeout. | No | The state labels in the API are `success`, `noop`, `grace`, `error` and `working`. :::note -**Success** means the item fired, not that the message arrived. An alert whose condition was met but whose email server is missing is **Success**, with the failure in **Detail**. Read **Detail** before assuming a message was sent. +A run that fired but could not deliver to every destination is **Error**, so it counts in the list's **failing** total and shows a red badge. When some destinations took the message and others did not, **Detail** names each failure and the destinations that worked still received it. ::: ## Examples from a real run log -These runs were captured on the Docker demo stack, where email goes to Mailpit and the renderer is running: +These runs were captured on the Docker demo stack, where email goes to Mailpit and the renderer is running. States and **Detail** are shown as Egret records them since platform #726: | Item | State | Value | Detail | |---|---|---|---| -| Pending orders above 100 (condition `> 100`, email and a webhook to a public host) | **Success** | 363 | `webhook returned 405` (the email arrived) | +| Pending orders above 100 (condition `> 100`, email and a webhook to a public host) | **Error** | 363 | `webhook : webhook returned 405` (the email arrived) | | Pending orders above 100, run again within the grace period | **On grace** | 363 | — | | Cancelled orders spike (`> 0`) | **Not triggered** | 0 | — | | Urgent open orders exist (value is not null and not zero) | **Success** | 1488 | **Notification sent** | | Orders by status error rate, a Data API alert (`> 0.05`) | **Success** | 0.27 | **Notification sent** | -| Late orders to Slack, with no workspace Slack app | **Success** | 64 | `no slack bot token is configured for this destination` | -| Webhook to an internal host (`http://mailpit:8025/hook`) | **Success** | 7304 | `webhook host resolves to a blocked address (172.24.0.5): private` | +| Late orders to Slack, with no workspace Slack app | **Error** | 64 | `slack #: no slack bot token is configured for this destination` | +| Webhook to an internal host (`http://mailpit:8025/hook`) | **Error** | 7304 | `webhook mailpit: webhook host resolves to a blocked address (172.24.0.5): private` | | Executive summary snapshot, a PNG report | **Success** | — | **Notification sent** | -| Sales performance weekly digest, a two-dashboard PDF report | **Error** | — | `render failed: error sending request for url (http://renderer:8080/render)` | +| A five-dashboard PDF report (36.9 seconds to render) | **Success** | — | **Notification sent** | -Captured earlier on a workspace without SMTP or renderer, the other messages you may see are `smtp is not configured for this workspace` (with **Success**), `query failed: engine: line 2:8: Column 'nope' cannot be resolved` and `alert query must return a number or NULL in its first column, got "O"` (both **Error**). +Other messages you may see, all with **Error**: `email: smtp is not configured for this workspace`, `query failed: engine: line 2:8: Column 'nope' cannot be resolved` and `alert query must return a number or NULL in its first column, got "O"`. + +When several destinations fail, **Detail** lists every one, separated by `; `, each prefixed with the destination: `email: …`, `slack #channel: …` or `webhook : …`. A webhook is named by its host only, because the path and query of a webhook URL often carry its secret. A note about a delivery that did go out, such as a skipped attachment, follows the failures; it does not make the run an **Error**. ## Why didn't I get a message? @@ -66,14 +68,13 @@ Work down this list. 2. **What did the last run record?** Open the run log. 3. **Not triggered**: the condition was not met. Check the **Value** against your threshold, and check that the query returns a number in its first column. 4. **On grace**: it fired recently. Wait out the grace period, or lower it. -5. **Error**: read **Detail**. The query, the first column, the renderer or the engine failed. `render failed: error sending request …` means the renderer could not be reached or did not answer within 30 seconds; see [Reports](./reports#png-and-pdf-need-the-renderer). -6. **Success with a Detail message**: the condition was met and delivery failed. A missing SMTP server, a missing Slack app, a webhook to a blocked or failing address, or a non-`2xx` response is recorded here. +5. **Error**: read **Detail**. Either the query, the first column, the renderer or the engine failed, or the condition was met and a destination refused the message: a missing SMTP server, a missing Slack app, a webhook to a blocked or failing address, or a non-`2xx` response. Each failed destination is named. `render failed: … (timeout)` means the renderer did not finish within its time limit; `render failed: error sending request …` means it could not be reached. See [Reports](./reports#png-and-pdf-need-the-renderer). ## Run now -**Run now** (in the row menu) runs the item once and shows the state in a message, for example *Ran now: Success*. It writes a run-log row like any run. +**Run now** (in the row menu) runs the item once and shows the state in a message, for example *Ran now: Success*. It writes a run-log row like any run, and updates the **Last run** column in the list and the status badge on the item's page. It does not move **Next run**. -A manual run is recorded in the run log, but the **Last run** column in the list and the status badge on the item's page show only the result of the last **scheduled** run. After **Run now** alone they can still read **Never run**; open the run log to see the manual run. +If a scheduled run of the same item is in progress (**Working**) when you press **Run now**, the manual run is still recorded in the run log, and the scheduled run writes **Last run** when it ends. ## Retention @@ -81,9 +82,7 @@ Finished runs are kept for the item's **Run log retention (days)**, 90 by defaul ## Limits and known issues -- When several destinations fail, only the first failure is recorded in **Detail**. - **Value** is empty for reports, and in the run log dialog of a report the **Value** and **Detail** headings run together (KI-171). -- The list's **Last run** and the header badge ignore manual runs (KI-032), so test an item with **Run now** and read the run log, not the list. ## Further reading diff --git a/platform/explore/alerts-and-reports/schedules.md b/platform/explore/alerts-and-reports/schedules.md index 489ed4c..9853099 100644 --- a/platform/explore/alerts-and-reports/schedules.md +++ b/platform/explore/alerts-and-reports/schedules.md @@ -33,7 +33,7 @@ The link switches to a **Cron expression** box. It takes five fields, as in cron The list shows each item's schedule in words (`Every 15 minutes`, `Every day at 08:00`) and its **Next run**. :::note -The scheduler checks for due items on a fixed interval, 30 seconds by default (`automation.tick_interval_secs`). An item runs at its next tick after it becomes due, so a run can start up to one interval late. Items that fall due together run one after another: on the demo stack twelve items due at 12:10:00 started between 12:10:18 and 12:11:21, the slow PNG and PDF renders holding up the rest. While an item waits its turn the list shows **Working**. +The scheduler checks for due items on a fixed interval, 30 seconds by default (`automation.tick_interval_secs`). An item runs at its next tick after it becomes due, so a run can start up to one interval late. Items that fall due together run side by side, up to four at a time, so a slow PNG or PDF render no longer holds up the other alerts: on the demo stack five items due at 02:37:00 were picked up together, the three quick ones finished within half a second and the two PDFs after 28.7 and 36.9 seconds. A tick picks up at most 20 due items; the rest wait for the next tick. While an item runs or waits its turn the list shows **Working**. ::: ### Timezone @@ -48,7 +48,7 @@ After an alert fires, it stays quiet for this long so a condition that stays tru The default in the editor is 14,400 seconds (four hours). Set `0` to notify on every run that meets the condition. Reports ignore the grace period. -A run counts as having fired once the condition was met, even if delivery then failed. A destination that was down therefore still starts the grace period. +The grace period starts from the last run that reached at least one destination. A run whose every destination failed starts none, so the next run tries again; a run that reached some destinations and failed on others starts it, so the recipients who got the message are not sent it again on every run. Both are recorded as **Error**; see [Notifications](./notifications#when-delivery-fails). ### Working timeout (seconds) @@ -60,7 +60,7 @@ How long a finished run stays in the [run log](./run-history). Default 90 days. ## What a manual run does -**Run now** in the list runs the item immediately through the same code the scheduler uses. It is subject to the same condition test and the same grace period. +**Run now** in the list runs the item immediately through the same code the scheduler uses. It is subject to the same condition test and the same grace period. Its result updates **Last run** and the status badge, but not **Next run**. ## Limits and known issues diff --git a/platform/explore/dashboards/export-and-reports.md b/platform/explore/dashboards/export-and-reports.md index 9c69bc8..3a6ba73 100644 --- a/platform/explore/dashboards/export-and-reports.md +++ b/platform/explore/dashboards/export-and-reports.md @@ -73,7 +73,7 @@ This is what a PDF of **Sales performance** (two tabs, a heading, a text card an - There is no whole-dashboard export button; reports (PNG, PDF) are the way to get the dashboard as one file. - Rendered reports ignore card title and visualization overrides, text cards and tabs. - The download popover's row limit text does not match the real limit (10,000 rows). -- PNG and PDF report rendering needs the rendering service. A large dashboard takes about 20 seconds to render, and a multi-dashboard PDF fails when the renders together exceed 30 seconds (KI-167). +- PNG and PDF report rendering needs the rendering service. A large dashboard takes about 20 seconds to render. Egret allows 30 seconds per dashboard plus 30 seconds for the whole render, and a page that does not finish drawing within 20 seconds fails. - A table card overflows its box on the rendered page (KI-168). ## Further reading diff --git a/platform/faq.md b/platform/faq.md index f98789a..076c99d 100644 --- a/platform/faq.md +++ b/platform/faq.md @@ -84,7 +84,8 @@ Under **Manage → Workspace settings → Engines**, not under Explore. See [You ### Why do I get *429 Too Many Requests* at sign-in? -Local sign-in allows 5 attempts per 5 minutes, and successful attempts count. Wait for the window to pass. See [Cannot sign +Five failed local sign-ins within 5 minutes lock that address and email out for 5 minutes. The sign-in page says how long +to wait, and the response carries `Retry-After`. Successful sign-ins do not count. See [Cannot sign in](./deploy-operate/troubleshooting/cannot-sign-in). ### I signed in with SSO and can see nothing. Why? diff --git a/platform/get-started/first-login-tour.md b/platform/get-started/first-login-tour.md index fb3cef2..142b049 100644 --- a/platform/get-started/first-login-tour.md +++ b/platform/get-started/first-login-tour.md @@ -39,8 +39,9 @@ The sign-in page changes with the server configuration. Egret creates the account as a plain member. What a person can do comes from roles granted to their **groups**, and those grants are made by an administrator: assign a platform role to the group under **Control Plane → Groups**, or add the group to a workspace under **Manage → Workspace settings → Members**. From a terminal, `egret workspace add-group --workspace - --group --role ws_editor` does the second one. Until a grant exists, the person can sign in and sees -only *No workspaces yet. Ask an admin to add you to one.* + --group --role ws_editor` does the second one. A workspace administrator can also add the person by +email before their first sign-in. Until a grant exists, the person can sign in and sees only a **No workspaces yet** card +with their email, **Check again** and **Sign out**. ::: ### Break-glass administrator @@ -64,7 +65,7 @@ covers securing it. | Message | Meaning | |---|---| | *Invalid email or password* | The email and password do not match a local account. | -| *Too many attempts. Please wait a few minutes and try again.* | The local sign-in limit (5 attempts per 5 minutes) was exceeded. Successful attempts count too. | +| *Too many failed sign-in attempts. Try again in 5 minutes.* | Five failed attempts within 5 minutes. Wait the time the message gives; successful attempts do not count. | For SSO errors see [SSO errors](../deploy-operate/troubleshooting/sso-errors). diff --git a/platform/get-started/quick-start.md b/platform/get-started/quick-start.md index 6f32fd8..a24c6aa 100644 --- a/platform/get-started/quick-start.md +++ b/platform/get-started/quick-start.md @@ -115,8 +115,8 @@ user starts with no access: the group-to-role grants are made by an administrato tour](./first-login-tour) explains both. :::caution Sign-in rate limit -Local sign-in allows 5 attempts per 5 minutes, successful ones included. Further attempts answer *429 Too Many Requests* -until the window passes. +Five failed local sign-ins within 5 minutes lock that address and email out for 5 minutes: further attempts answer +*429 Too Many Requests* with a `Retry-After` header. Successful sign-ins do not count. ::: ## Connect Trino diff --git a/platform/get-started/run-the-demo-stack.md b/platform/get-started/run-the-demo-stack.md index b5f83f3..a24219a 100644 --- a/platform/get-started/run-the-demo-stack.md +++ b/platform/get-started/run-the-demo-stack.md @@ -64,12 +64,13 @@ Open [http://egret.localtest.me:8080](http://egret.localtest.me:8080). | SSO member | **Sign in with SSO** | `member-sso` / `Member12345!` | Use the local administrator first: it is the account the seed uses, and it sees the `default` workspace and everything in it. An -SSO user has no workspace until an administrator adds them, and Egret only knows an SSO user after their first sign-in. Sign in -once with the SSO user (the first screen says *No workspaces yet. Ask an admin to add you to one.*), then add them to a workspace -as the local administrator under **Manage → Workspace settings → Members**. The [First login tour](./first-login-tour) explains both +SSO user has no workspace until an administrator adds them. As the local administrator, add the SSO user's email under +**Manage → Workspace settings → Members** before or after their first sign-in: an email nobody has signed in with yet is +listed under **Waiting for first sign-in** and becomes a membership when that user first signs in (the demo realm's users +have verified emails). An SSO user with no workspace sees a **No workspaces yet** card. The [First login tour](./first-login-tour) explains both kinds of sign-in. -Local sign-in allows 5 attempts per 5 minutes for each address and email, successful ones included. +Five failed local sign-ins within 5 minutes lock that address and email out for 5 minutes; successful ones do not count. ## What runs and where @@ -178,7 +179,7 @@ Everything in the user docs except the items under [Not available on Docker](#no | Feature | Why | Instead | |---|---|---| -| An engine's **Logs** and **Exec** buttons | They need Kubernetes; they answer *not available with the Docker engine provisioner*. | `docker logs egret-engine-egret-engines--coordinator` | +| An engine's **Logs** and **Exec** buttons | They need Kubernetes, so the engine page does not show them for an engine Docker runs. | `docker logs egret-engine-egret-engines--coordinator` | | StarRocks and RisingWave engines, ingress, autoscaling, extra mounts or containers, image pull secrets, extra ports | The Docker provisioner refuses them with an error that names the option. | A Kubernetes install; see [Kubernetes](../deploy-operate/kubernetes). | | CPU and memory requests, probes, placement | They have no Docker meaning. CPU and memory *limits* become container limits. | | | Workflow tasks that run containers or scripts | No container runner is configured; plain tasks such as `Log` run. | A Kubernetes install. | @@ -190,8 +191,9 @@ Everything in the user docs except the items under [Not available on Docker](#no | `just demo-down` | Stops and removes the containers, including the engines Egret started (containers labelled `egret.managed=true` and `egret.project=egret-demo`). **Keeps the data volumes**, so `just demo-up` brings your content back. Mail in Mailpit is lost. | | `just demo-reset` | The same, and **deletes the data volumes** (PostgreSQL, the object store, the Data Apps files). The next `just demo-up` starts from an empty, freshly seeded stack. | -An engine that was running before `just demo-down` stays listed as `provisioning` after `just demo-up`, with no containers behind -it. Delete it and add it again. +An engine that was running before `just demo-down` is listed as `provisioning` after `just demo-up`, with no containers behind +it. Open it and press **Restart**: Egret creates its containers again from the stored settings, and the engine returns to +`running`. ## Security @@ -218,10 +220,10 @@ Production installs use the Kubernetes launcher for apps and the Kubernetes engi | SSO fails with an issuer mismatch or a redirect error | The browser address is not exactly `DEMO_URL`: use `egret.localtest.me`, not `localhost`, on the same port. After changing the port, run `just demo-reset && just demo-up`. | | `egret.localtest.me` or `apps.localtest.me` does not resolve | You are offline or a DNS filter blocks the name. Add the `/etc/hosts` line from [Host names](#host-names). | | The page does not load, or `wait.sh` times out | Check the services with `docker compose … ps` (above) and read the log of the unhealthy one with `just demo-logs `. `curl http://egret.localtest.me:8080/readyz` does not help here: the edge sends that path to the web app, which answers with its HTML page. | -| The sign-in form answers *Too many attempts* | 5 attempts per 5 minutes per address and email. Wait. | -| An SSO user sees *No workspaces yet* | Expected for a new SSO user. Add them to a workspace as the local administrator. | +| The sign-in form answers *Too many failed sign-in attempts* | 5 failed attempts within 5 minutes per address and email. Wait the time it gives. | +| An SSO user sees *No workspaces yet* | Expected for a new SSO user nobody has added. Add their email to a workspace as the local administrator, then press **Check again**. | | The app editor is blank or says *Not connected to a runtime* | Wait for the first start (about 30 seconds), then **Reload editor**. Check that you are on `egret.localtest.me`, then read `just demo-logs egret`. | -| An engine stays `provisioning` | `docker ps -a --filter label=egret.engine=` lists its containers and `docker logs egret-engine-egret-engines--coordinator` shows why. A container that exited is often short of memory: give Docker 1.5 to 2 GB per engine and prefer single-node engines. | +| An engine stays `provisioning` | After `just demo-down` and `just demo-up`, press **Restart** on the engine. Otherwise `docker ps -a --filter label=egret.engine=` lists its containers and `docker logs egret-engine-egret-engines--coordinator` shows why. A container that exited is often short of memory: give Docker 1.5 to 2 GB per engine and prefer single-node engines. | | No mail arrives | Open Mailpit at `http://localhost:8025`. The seed points the `default` workspace's mail server at it; another workspace needs its own **Notifications** settings. | ## Further reading diff --git a/platform/known-issues.md b/platform/known-issues.md index baada9d..e6089ba 100644 --- a/platform/known-issues.md +++ b/platform/known-issues.md @@ -13,7 +13,7 @@ and missing pieces. Entries are **recorded, not fixed**; they feed the product r | ID | Area | Type | Sev | Description | |---|---|---|---|---| | KI-001 | Databases | UX | S2 | Registering a dataset against a Trino without `http-server.process-forwarded=true` fails with a generic *502 Bad Gateway*. Egret always sends `X-Forwarded-Proto` and Trino answers HTTP 406. The UI should say what is wrong. | -| KI-002 | Auth | UX | S3 | Local sign-in is limited to 5 attempts per 5 minutes and rejected (429) attempts keep the window open. There is no message telling the user when they can retry. | +| KI-002 | Auth | UX | S3 | Local sign-in is limited to 5 attempts per 5 minutes and rejected (429) attempts keep the window open. There is no message telling the user when they can retry. *Status:* fixed in platform #727 (October 2026): a refused sign-in answers 429 with `Retry-After` and `{"code":"rate_limited","retry_after_secs":…}`, and the page says *Too many failed sign-in attempts. Try again in 5 minutes.* Refused attempts do not extend the lockout. | | KI-003 | Dashboards | UX | S2 | A chart that is not fully configured (no X-axis) renders as an **empty card** on a dashboard, with no hint, while the chart editor says *Set up your Bar*. | | KI-004 | SQL Lab | UX | S3 | Floating-point results are shown with representation noise (`63339475.32000001`). Provide display rounding or a number format in results. *Status:* fixed in platform #721 (October 2026): `double`, `real` and `float` cells are shown rounded to 15 significant digits (`63339475.32000001` reads `63339475.32`, `0.1 + 0.2` reads `0.3`). Copy, export, CSV and the expanded cell keep the exact value; integers and decimals are not changed. | | KI-005 | Workspace settings | UX | S3 | The Workspace settings page shows two **Collapse** buttons (the app sidebar and the settings sidebar). | @@ -28,16 +28,16 @@ and missing pieces. Entries are **recorded, not fixed**; they feed the product r | KI-014 | Workflows | UX | S3 | A new flow shows *Version —* in its header, and *Run* says it runs "the saved draft"; it is unclear what a version is until **Create version** is pressed. | | KI-015 | Governance | Missing | S2 | **Data Lineage** is in the sidebar but is a placeholder page with no functionality. | | KI-016 | Control Plane | Defect | S3 | `/control-plane/config` renders a completely blank page inside the shell. *Docker demo stack (Oct 2026):* no longer blank; it now redirects without end to `/control-plane/config/workspaces/workspaces/workspaces/…`. | -| KI-017 | Audit log | UX | S2 | Every page view writes `*:read allow` rows, so the audit log is dominated by routine reads and hides grant changes. Offer a default filter or separate reads from changes. | -| KI-018 | Access overview | UX | S3 | *Recent changes* identifies the actor by an id prefix (`by 6d60b4c4`) instead of a name. | -| KI-019 | Roles | Consistency | S3 | Control Plane → Roles lists four workspace roles (including `data_api_invoker`); Workspace settings → Roles & permissions lists three. Verify whether the fourth is hidden on purpose. | +| KI-017 | Audit log | UX | S2 | Every page view writes `*:read allow` rows, so the audit log is dominated by routine reads and hides grant changes. Offer a default filter or separate reads from changes. *Status:* fixed in platform #727 (October 2026): the log opens on **Kind → Changes and denials** (`hideAllowed=true`), which leaves out allowed decisions; **All** still shows them. | +| KI-018 | Access overview | UX | S3 | *Recent changes* identifies the actor by an id prefix (`by 6d60b4c4`) instead of a name. *Status:* fixed in platform #727 (October 2026): the audit API carries `actorEmail`, and *Recent changes* and the audit log show `by admin@egret.local`. A deleted account keeps the id prefix. | +| KI-019 | Roles | Consistency | S3 | Control Plane → Roles lists four workspace roles (including `data_api_invoker`); Workspace settings → Roles & permissions lists three. Verify whether the fourth is hidden on purpose. *Status:* fixed in platform #727 (October 2026): `data_api_invoker` is hidden on purpose (it is granted on one endpoint, never as a membership), and the workspace page now says so. The Members and Access requests role pickers read the workspace's own role list, so they no longer offer it. A direct API call still accepts it (KI-261). | | KI-020 | People & groups | UX | S3 | Every user shows *No username yet*, even users who signed in through SSO. *Status:* closed on the Docker demo stack (Oct 2026): People & groups → Users shows usernames for everyone who signed in or was granted (`member-sso`, `alice-nguyen`, …); only `admin-sso`, never granted, shows *No username yet*. | -| KI-021 | Control Plane | Defect? | S3 | verify: a group (`platform-admins`) mapped from the identity provider gives its member no platform role (Users shows no role for `admin-sso`). Check whether group-to-role mapping is expected to be configured separately. | +| KI-021 | Control Plane | Defect? | S3 | verify: a group (`platform-admins`) mapped from the identity provider gives its member no platform role (Users shows no role for `admin-sso`). Check whether group-to-role mapping is expected to be configured separately. *Status:* closed, by design (October 2026, platform #727): the IdP group is synced, but a group holds a platform role only through an administrator's grant to that group. The development stack's seed (`deploy/seed-sso-test-rbac.sql`) grants `platform-admins` the role `platform_admin`; the Docker demo stack has no such grant, so `admin-sso` signs in there with no platform role. The Users list shows only directly granted roles (KI-260). | | KI-022 | Engines | Limitation | S2 | Engine provisioning (**Add engine**) needs Kubernetes. On a Docker-only setup the only way to attach an engine is **Register external engine**, and the UI does not say so. *Status:* closed on the Docker demo stack (Oct 2026): **Add engine** works with the Docker provisioner (`EGRET_ENGINES__PROVISIONER=docker`, Trino only): `docs-trino` ran in 72 s, restart 38 s, delete 7 s. StarRocks and RisingWave engines, ingress, autoscaling, pod logs and exec still need Kubernetes. See KI-195, KI-196 and KI-213. | ## Findings from the detailed pages -Collected while writing the detailed pages for each area (October 2026), checked against the running demo instance and the product source. KI-154 to KI-213 come from re-verifying every area on the Docker demo stack (`just demo-up`, October 2026), which also closed or updated older rows (marked *Status* or *Docker demo stack*). KI-214 to KI-235 come from a whole-app UI consistency sweep of 78 screens (admin, light theme, 1440×900, same stack), then the dark theme, 768 and 390 px widths, the editors and a non-admin member. KI-236 to KI-246 come from the Wave 1 fixes of the Known-issues fix programme (October 2026), which also closed the rows marked *fixed in platform #…*. KI-247 to KI-259 come from the Wave 2 fixes (SQL Lab, charts, datasets and the Data API: platform #720, #721, #724 and #725), which closed or partly closed the rows they name. Same legend as above. **Open S1 items:** none (KI-013, KI-040, KI-059, KI-060, KI-112, KI-136, KI-159, KI-175 and KI-187 are closed). +Collected while writing the detailed pages for each area (October 2026), checked against the running demo instance and the product source. KI-154 to KI-213 come from re-verifying every area on the Docker demo stack (`just demo-up`, October 2026), which also closed or updated older rows (marked *Status* or *Docker demo stack*). KI-214 to KI-235 come from a whole-app UI consistency sweep of 78 screens (admin, light theme, 1440×900, same stack), then the dark theme, 768 and 390 px widths, the editors and a non-admin member. KI-236 to KI-246 come from the Wave 1 fixes of the Known-issues fix programme (October 2026), which also closed the rows marked *fixed in platform #…*. KI-247 to KI-259 come from the Wave 2 fixes (SQL Lab, charts, datasets and the Data API: platform #720, #721, #724 and #725), which closed or partly closed the rows they name. KI-260 to KI-266 come from the Wave 2b fixes (alerts and reports delivery, authentication and roles, engines and monitor: platform #726, #727 and #728), which closed the rows they name; KI-021 was closed as by design and KI-148 as no longer applying. Same legend as above. **Open S1 items:** none (KI-013, KI-040, KI-059, KI-060, KI-112, KI-136, KI-159, KI-175 and KI-187 are closed). ### Data API, Alerts and Apps @@ -52,9 +52,9 @@ Collected while writing the detailed pages for each area (October 2026), checked | KI-029 | Data API | Gap | S3 | `GET /api/ws/{ws}/data-api/activity/summary` returns `total.endpoint_name` set to one arbitrary endpoint's name ("Yearly revenue summary") although `total` covers all endpoints. The UI does not show it. *Steps:* `GET .../activity/summary?range=24h`. | | KI-030 | Data API | Gap | S3 | A `data_access_denied` (403) could not be reproduced: no Ranger policy exists for the demo connection, so the page documents it from the OpenAPI text only. The governed-connection refusals (publish refused, `503 governance_stale`) were not provoked either, because changing the shared connection was out of bounds. *Steps:* `platform/data-api/governed-connections`. | | KI-031 | Data API | UX | S3 | **Database and parameters** is collapsed by default on the Query tab, so the parameters table and database picker are hidden until opened; the count in the summary is the only hint. *Steps:* Query tab of any endpoint. | -| KI-032 | Alerts | Bug | S2 | **Run now** writes a run-log row but does not update the item's last state: the list **Last run** and the header badge stay **Never run**, while **Recent runs** on the same page shows the runs. Scheduled runs do update it. *Steps:* Create an alert, press **Run now**: `GET /api/ws/default/alerts` still has `last_state: null`; `/alerts/{id}` shows badge "Never run" above two recent runs. | -| KI-033 | Alerts | UX | S2 | A run is **Success** (and starts the grace period) even when delivery failed; the failure is only in the run-log **Detail**. Nothing in the list marks an alert whose messages never arrive. *Steps:* Alert **Pending orders above 100**: run log `Success` with `webhook host resolves to a blocked address (127.0.0.1): loopback`; **Urgent open orders exist**: `Success` with `smtp is not configured for this workspace`. *Docker demo stack (Oct 2026):* still true; the failed delivery also starts the grace period: "Late orders to Slack" (no Slack app) and "Webhook to an internal host" (blocked) show **On grace** after the next scheduled run. | -| KI-034 | Alerts | Bug | S3 | With several destinations only the first failure is recorded. An alert with an email and a webhook destination recorded only the webhook error; the missing-SMTP error for the email destination was not shown. *Steps:* Alert **Pending orders above 100** (email + webhook), run log. | +| KI-032 | Alerts | Bug | S2 | **Run now** writes a run-log row but does not update the item's last state: the list **Last run** and the header badge stay **Never run**, while **Recent runs** on the same page shows the runs. Scheduled runs do update it. *Steps:* Create an alert, press **Run now**: `GET /api/ws/default/alerts` still has `last_state: null`; `/alerts/{id}` shows badge "Never run" above two recent runs. *Status:* fixed in platform #726 (October 2026): **Run now** writes its outcome onto the item, so **Last run** in the list and the header badge show it (*Not triggered · a few seconds ago* on the demo stack). **Next run** is not moved, and a row a scheduled run is working on is left to that run. | +| KI-033 | Alerts | UX | S2 | A run is **Success** (and starts the grace period) even when delivery failed; the failure is only in the run-log **Detail**. Nothing in the list marks an alert whose messages never arrive. *Steps:* Alert **Pending orders above 100**: run log `Success` with `webhook host resolves to a blocked address (127.0.0.1): loopback`; **Urgent open orders exist**: `Success` with `smtp is not configured for this workspace`. *Docker demo stack (Oct 2026):* still true; the failed delivery also starts the grace period: "Late orders to Slack" (no Slack app) and "Webhook to an internal host" (blocked) show **On grace** after the next scheduled run. *Status:* fixed in platform #726 (October 2026): a firing is **Success** only when every destination took it, otherwise **Error**, so the list's **failing** count shows it. The grace period starts from the last run that reached at least one destination: when every destination failed none starts and the next run tries again; when some failed it starts, so the recipients who got it are not sent it again. | +| KI-034 | Alerts | Bug | S3 | With several destinations only the first failure is recorded. An alert with an email and a webhook destination recorded only the webhook error; the missing-SMTP error for the email destination was not shown. *Steps:* Alert **Pending orders above 100** (email + webhook), run log. *Status:* fixed in platform #726 (October 2026): every destination is tried and every failure recorded, separated by `; ` and named `email: …`, `webhook : …` or `slack #channel: …`. A webhook is named by its host only, because its path or query can carry its secret. | | KI-035 | Alerts | Gap | S3 | Webhook destinations that resolve to loopback or private addresses are refused and there is no UI hint or setting; `automation.webhook_allow_hosts` is operator config, so a local receiver cannot be used to test webhooks. *Steps:* Run now on an alert with `http://127.0.0.1:19099/egret`. *Docker demo stack (Oct 2026):* Docker service names resolve to private addresses (`webhook host resolves to a blocked address (172.24.0.5): private`), so the demo stack cannot receive webhooks locally; a public endpoint was reached (`webhook returned 405`). | | KI-036 | Alerts | Env | S2 | PNG and PDF reports fail because the rendering service is not deployed: `render failed: error sending request for url (http://egret-renderer:8080/render)`. Only CSV and text formats could be run. *Steps:* Report **Sales overview weekly digest** (PDF), **Run now**. *Status:* closed on the Docker demo stack (Oct 2026): with `EGRET_AUTOMATION__RENDERER_URL=http://renderer:8080`, PNG reports, single and two-dashboard PDFs and an alert's PNG attachment were delivered to Mailpit (a PNG dashboard report arrived in 23 s). The remaining failure mode is KI-167. | | KI-037 | Alerts | Gap | S3 | The editor offers six comparison operators; the API also accepts `between` and `not between`. *Steps:* `validator_config` handling in `egret-automation` vs the Condition tab. | @@ -64,13 +64,13 @@ Collected while writing the detailed pages for each area (October 2026), checked | KI-041 | Apps | Gap | S3 | The Create app dialog asks only for a name. The API accepts `description` and `database`, and the template pre-fills a connection cell from `database`, but the UI sends neither, so every new app starts without a connection and the description can only be set later through **Rename…**. *Steps:* `POST /api/ws/{ws}/apps` body; `Apps.tsx` `NewAppDialog`. | | KI-042 | Apps | UX | S3 | `PUT .../draft` without `If-Match` returns `If-Match with the draft revision is required`; an API client that only knows the draft body gets no hint of the revision. Publishing before saving silently publishes the starter notebook. *Steps:* `POST /apps` then `POST /apps/{id}/publish` with `{}` publishes v1 of the template. | | KI-156 | Apps | Gap | S2 | The marimo editor has no autocomplete or hover help: the apps proxy refuses `/api/lsp/health` (403, console "Language server initialization failed"), and also `POST /api/packages/sandbox`, `POST /api/datasources/discover` and `GET /api/usage`, so the package and data-source panels do not work and every editor load logs 5 or 6 console errors. Refusing the LSP is by design (demo guide), but the editor does not tell the author. The first load also shows *Not connected to a runtime* for about 30 s before *Reconnected*, and opening the editor alone bumps the draft revision (1 → 2). *Steps:* Apps → **+ App** → Create, watch the console and network log. (Reported by the start and apps areas.) | -| KI-157 | Reports | UX | S3 | A report's mail subject is always `[egret] report`, so a report whose name ends in "report" mails "… report report". *Steps:* report "start: first dashboard report" → Run now; Mailpit subject `[egret] start: first dashboard report report`. | -| KI-167 | Reports | Bug | S2 | A multi-object PDF report fails with `render failed: error sending request for url (http://renderer:8080/render)` although the renderer is up. The render client has a fixed 30 s timeout (`render/client.rs` `CLIENT_TIMEOUT`) while the renderer captures the objects one after another with up to 20 s each, so two medium dashboards exceed it; the error does not say "timeout". *Workaround:* split the report, keep each under about 25 s. *Steps:* **Sales performance weekly digest** (PDF: Sales performance + Executive summary) → Run now: error after 30.5 s, three times. Sales performance alone renders in about 20 s; Executive summary + Weekly sales review (about 26 s) succeeds. | -| KI-169 | Reports | Bug | S2 | CSV and Text reports write numbers unformatted, as `f64` debug text: `"F","1.0356810234900002e+9"` and `F \| 1.0356810234900002e+9`, so spreadsheet users see scientific notation and float noise. *Workaround:* round or cast in the chart's dataset SQL. *Steps:* **Revenue by status CSV** and **Revenue by status (text)** → Run now → Mailpit. | +| KI-157 | Reports | UX | S3 | A report's mail subject is always `[egret] report`, so a report whose name ends in "report" mails "… report report". *Steps:* report "start: first dashboard report" → Run now; Mailpit subject `[egret] start: first dashboard report report`. *Status:* fixed in platform #726 (October 2026): the default subject drops ` report` when the name already ends in the word *report* (`[egret] b4-revenue by status csv report` on the demo stack). | +| KI-167 | Reports | Bug | S2 | A multi-object PDF report fails with `render failed: error sending request for url (http://renderer:8080/render)` although the renderer is up. The render client has a fixed 30 s timeout (`render/client.rs` `CLIENT_TIMEOUT`) while the renderer captures the objects one after another with up to 20 s each, so two medium dashboards exceed it; the error does not say "timeout". *Workaround:* split the report, keep each under about 25 s. *Steps:* **Sales performance weekly digest** (PDF: Sales performance + Executive summary) → Run now: error after 30.5 s, three times. Sales performance alone renders in about 20 s; Executive summary + Weekly sales review (about 26 s) succeeds. *Status:* fixed in platform #726 (October 2026): the render time limit is 30 seconds per object plus 30 seconds, and a timeout says so (`the renderer did not finish within 90s for 2 page(s) (timeout)`). A five-dashboard PDF that took 36.9 s was delivered. A page that does not signal it is ready within 20 s still fails, with the renderer's own message. | +| KI-169 | Reports | Bug | S2 | CSV and Text reports write numbers unformatted, as `f64` debug text: `"F","1.0356810234900002e+9"` and `F \| 1.0356810234900002e+9`, so spreadsheet users see scientific notation and float noise. *Workaround:* round or cast in the chart's dataset SQL. *Steps:* **Revenue by status CSV** and **Revenue by status (text)** → Run now → Mailpit. *Status:* fixed in platform #726 (October 2026): numbers are written as the table shows them by default, at most three decimals and trailing zeros dropped: `F | 1,035,681,023.49` in text, `"F","1035681023.49"` in CSV (not grouped). A column's own format settings are not applied (KI-266). | | KI-170 | Alerts / Notifications | Gap | S3 | The **Opening note** and **Closing note** appear only in the HTML part of the email; the text/plain part has only "The alert … fired. Observed value: …" or "The report … is attached." *Steps:* alert **Pending orders above 100** (both notes set) → Mailpit → Text tab. | | KI-171 | Alerts / Run log | UX | S3 | In a report's run log dialog the **Value** column is empty and too narrow: its heading runs into **Detail** ("ValueDetail") and **Started** wraps. *Steps:* Reports tab → Run log of **Sales performance weekly digest** (screenshot `alerts/run-log-report-error`). | | KI-172 | Alerts API | Docs | S3 | For a report, a non-empty `targets` silently replaces `chart_id`/`dashboard_id`: `POST /alerts` with `dashboard_id` = Sales performance and `targets` = [chart Top clerks] delivered a PDF of the chart alone and recorded **Success**. The editor always sends every object (primary first) in `targets`, so only API clients hit this, and the API reference does not say so. *Steps:* `runner.rs` `render_image_report` uses `targets_for` when non-empty. | -| KI-173 | Alerts / Scheduler | UX | S3 | Items due in the same tick run one after another, so a slow PNG/PDF render delays every other alert in the workspace: twelve items due at 12:10:00 started between 12:10:18 and 12:11:21, all showing **Working** meanwhile; an alert with a dashboard attachment takes about 25 s per run. *Steps:* set twelve items to `*/2 * * * *`, watch `GET /alerts` `last_state`. | +| KI-173 | Alerts / Scheduler | UX | S3 | Items due in the same tick run one after another, so a slow PNG/PDF render delays every other alert in the workspace: twelve items due at 12:10:00 started between 12:10:18 and 12:11:21, all showing **Working** meanwhile; an alert with a dashboard attachment takes about 25 s per run. *Steps:* set twelve items to `*/2 * * * *`, watch `GET /alerts` `last_state`. *Status:* fixed in platform #726 (October 2026): items due in one tick run side by side, at most four at a time. Five items due at 02:37:00: the three quick ones ended within 0.5 s of being picked up, the two PDFs at 28.7 s and 36.9 s. | | KI-175 | Apps | Bug | S1 | On the Docker demo stack published apps stop opening a few minutes after the first view: every later **Open** stays on *Starting the app…* until Egret is restarted. The egret image has no `kill` binary, so `SubprocessPodLauncher::kill_group` (`crates/egret-apps/src/subprocess_pod.rs`) cannot signal the process group; when the idle reaper calls `stop_run`, it waits forever on `child.wait()` while holding the `running_run` mutex (the guard of `self.running_run.lock().await.remove(&key)` lives through the block), and every `ensure_run` blocks behind it. Edit pods use a separate lock and keep working, but `stop_edit` (idle 30 min) likely has the same problem. *Workaround:* restart the egret container. *Fix idea:* install procps or use `killpg`, and do not hold the mutex across `kill_group`. *Steps:* open a published app as a viewer; more than 5 minutes later open another (`Customer lookup`): `POST …/ticket` 200, `GET http://apps.localtest.me:18080/r//` never answers, the first run's processes are still alive and the log has no "stopped an idle app run pod". Seen twice. *Status:* fixed in platform #707 (`eae470b6`, October 2026): the launcher signals the group with `killpg` and waits at most 5 s after SIGKILL. Checked on a rebuilt demo image: the idle reaper logged *stopped an idle app run pod* about 5 minutes after the first view, and the next **Open** started a new pod in 3 s. `stop_run` still holds the `running_run` lock while it waits, now for 5 s at most (10 s in the worst case), and `stop_edit` shares the fix. | | KI-176 | Apps | Bug | S3 | The **Versions** dialog is stale after a publish (still *Nothing has been published yet.*) until the page is reloaded: `usePublishApp` (`frontend/src/api/apps.ts:119`) invalidates the app and the list but not `getAppsVersionsQueryKey`, while `useRestoreVersion` does. *Steps:* new app → Versions → Ctrl+S → **Publish draft rev 2** → toast *Published version 1* → Versions still empty; reload → `v1`. | | KI-177 | Apps | UX | S2 | **Publish draft rev N** pressed right after Ctrl+S answers `409 {"error":"the draft changed since it was read","current_revision":2}` (toast *The draft changed while you were publishing…*), because the header re-reads the draft revision only every 5 s (`useApp(..., {refetchMs})`). *Workaround:* wait until the button shows the new revision, then publish. *Steps:* new app → cell `1 + 1` → Ctrl+S → **Publish draft rev 1** within 0.3 s → 409; about 7 s later **Publish draft rev 2** works. | @@ -211,7 +211,7 @@ Collected while writing the detailed pages for each area (October 2026), checked |---|---|---|---|---| | KI-125 | Governance / Engines | Gap | S2 | A policy profile created with **New profile** (Trino) can never be bound to an engine: binding fails with "has no usable tag service". The console cannot set `tagService`, so an external Trino engine can only be bound by creating the `tag` + `trino` profile pair through the API. *Steps:* Engine & policy → New profile → docs-trino-profile; Control Plane → Engines → docs-trino → Settings → Policy profile → Save changes (400) *Docker demo stack (Oct 2026):* still true, and the **Policy profile** pickers (Register external engine, engine Settings) list tag-less profiles such as `docs-external-policy` without marking them unbindable. | | KI-126 | Workspace settings / Notifications | Bug | S2 | Saving SMTP settings returns HTTP 500 "internal error" on this instance (the Notifications form shows "internal error"), so the saved state, "Send test message" and the Slack save could not be verified. *Steps:* `PUT /api/ws/docs-sandbox/notifications/smtp` with host smtp.example.com, port 587, from alerts@example.com *Status:* closed on the Docker demo stack (Oct 2026): with `EGRET_SECRETS__MASTER_KEY` set, SMTP to Mailpit (`mailpit:1025`, no encryption) saves, **Send test message** answers *Test message sent…* and mail arrives; the Slack save answers *Slack settings saved.* | -| KI-127 | Workspace settings / Roles | UX | S3 | A custom role that inherits shows only its own permissions ("2 permissions") on the workspace Roles & permissions page, while the Control Plane role page counts 8 (including inherited). `data_api_invoker` is missing from the workspace list but is offered in the Members role picker. *Steps:* /explore/docs-sandbox/settings/roles vs /control-plane/roles/<finance_reporter> | +| KI-127 | Workspace settings / Roles | UX | S3 | A custom role that inherits shows only its own permissions ("2 permissions") on the workspace Roles & permissions page, while the Control Plane role page counts 8 (including inherited). `data_api_invoker` is missing from the workspace list but is offered in the Members role picker. *Steps:* /explore/docs-sandbox/settings/roles vs /control-plane/roles/<finance_reporter> *Status:* fixed in platform #727 (October 2026): the count includes inherited permissions (*9 permissions (7 inherited)* for a role with two of its own over `ws_viewer`), and the inherited ones are listed under *Inherited from \*. The `data_api_invoker` half is KI-019. | | KI-128 | Workspace settings / Members | UX | S3 | Typing a partial email ("member-sso") in Add member gives no suggestions and the add silently does nothing visible; only the exact email works. *Steps:* /explore/docs-sandbox/settings/members → Add member *Status:* closed on the Docker demo stack (Oct 2026): typing `emma` in Add member suggests *Emma Schmidt emma.schmidt@egret.local*. | | KI-129 | Control plane / Storage | Bug (unverified) | S3 | On a connection created with Key prefix `sandbox`, the Browse tab lists the bucket root, not the prefix. *Steps:* Storage → sandbox-lake → Browse *Docker demo stack (Oct 2026):* still true (`sandbox-lake`, prefix `sandbox/`). Related: KI-208. *Status:* fixed in platform #715 (October 2026): Browse and Download keys are relative to the connection's Key prefix, and the root crumb shows `bucket/prefix` (`warehouse/sandbox`). | | KI-130 | Governance / Overview | Doc | S3 | The count tiles say "classifications" and the tag page "0 tagged objects" counts BI objects only; a tag on a table column does not raise the count. *Steps:* /governance/classifications/<tag> after tagging `user_id` *Status:* closed on the Docker demo stack (Oct 2026): the Classifications list has a **Data** column (*0 tables · 4 columns*), and the tag page's **Where used** lists the tables and columns. | @@ -220,14 +220,14 @@ Collected while writing the detailed pages for each area (October 2026), checked | KI-133 | Control plane / Workspaces | Gap | S3 | No way to delete a workspace, only archive; archived workspaces still count in the summary strip. *Steps:* /control-plane/workspaces | | KI-134 | Control plane / Audit log | Gap | S3 | The Action filter is an exact match (`secret` matches nothing; `secret:read` does). Creating a secret, workspace, catalog or engine registration did not show as a mutation entry when filtered by kind. *Steps:* /control-plane/audit-log, API `kind=mutation` *Docker demo stack (Oct 2026):* wider than written: Kind = Mutation lists no entry for creating users, groups, roles, workspaces, secrets, storage connections, catalogs or tags, for provisioning, restarting, reconfiguring or deleting an engine, for declaring a catalog or for saving SMTP settings. Action is still an exact match. | | KI-135 | Governance / Engines | Gap | S3 | "Seed default policies" on an empty manual profile showed no change and no message. *Docker demo stack (Oct 2026):* it does add 13 policies (`{"created":13}`), shown on **Policies** as egret-seeded, but Advanced still says *This profile has no hand-written policy.* and no message is shown. *Steps:* Profile page → Advanced → Seed default policies | -| KI-194 | Engines / Monitor | Bug | S2 | The **Monitor** tab of a running Trino 483 engine says *Engine not reachable — is a coordinator running?* and *Coordinator unreachable — no live queries.*: `/cluster` and `/queries` answer 502 because the summary calls `GET /v1/node` (404 on Trino 483) and a JWT-delegation engine gives the `egret-monitor` account no sign-in. Not Docker-specific. *Workaround:* use `system.runtime.queries` in SQL Lab. *Steps:* docs-trino → Monitor while a query runs. | -| KI-195 | Engines | UX | S3 | On a Docker-provisioned engine the **Logs** and **Exec** buttons stay visible although they cannot work: Logs shows the raw *bad request: not available with the Docker engine provisioner*; Exec opens a shell that closes at once with *[connection closed]* (the WebSocket answers 400). Hide them or say why. | -| KI-196 | Engines | Consistency | S3 | Engine dialogs speak Kubernetes on a Docker engine: Delete says *This destroys the engine's Kubernetes cluster*, Restart says it *rolls the coordinator and worker pods*, Review changes says *re-provisions (rolls) the engine pods*. | -| KI-197 | Engines | Bug | S3 | After **Add engine** with Worker → Replicas `0`, the stored spec says `workerReplicas: 2` and Overview shows *Coord 1 · Worker 2* while only the coordinator container exists; the first **Save changes** on Settings corrects it. With the field blank, one worker container starts and the strip still says 2. | -| KI-198 | Engines / Settings | UX | S3 | **Review changes** lists an unrelated `auth.passwordUsers [] Added` next to the real change and has only **Close**; **Save changes** then applies without confirmation while the header stays **running** although the coordinator container is recreated. *Steps:* docs-trino → Settings → Server → add `query.max-run-time` = 2h → Review changes. | -| KI-199 | Engines | UX | S3 | The engine header and list show the endpoint `docs-trino-coordinator.egret-engines:8080`, which does not resolve on the Docker network; the container answers as `docs-trino-trino(.egret-engines(.svc))` and the workspace connection uses `docs-trino-trino.egret-engines.svc:8443`. | -| KI-200 | Engines / Add engine | UX | S3 | **Expose via ingress** without a **Host** is silently dropped: the engine is created with no ingress and no message. With a host the Docker provisioner refuses (*not supported by the Docker engine provisioner: ingress*), as expected. | -| KI-201 | Governance / Lineage | Bug | S2 | View lineage fails on a provisioned engine: *engine docs-trino: engine: Access Denied: Cannot access catalog demo_warehouse*, because the engine's own policy does not let Egret's lineage reader into the catalog. The overview shows a **Warning**, the table's Lineage tab *No reader is known*, View exposure lists the view as *clearance unknown*; `POST /classifications/lineage-sweep` fails the same way, and the console has no control for the sweep the UI asks you to run. *Steps:* view `demo_warehouse.analytics.revenue_by_country` over `sessions`. | +| KI-194 | Engines / Monitor | Bug | S2 | The **Monitor** tab of a running Trino 483 engine says *Engine not reachable — is a coordinator running?* and *Coordinator unreachable — no live queries.*: `/cluster` and `/queries` answer 502 because the summary calls `GET /v1/node` (404 on Trino 483) and a JWT-delegation engine gives the `egret-monitor` account no sign-in. Not Docker-specific. *Workaround:* use `system.runtime.queries` in SQL Lab. *Steps:* docs-trino → Monitor while a query runs. *Status:* fixed in platform #728 (October 2026): when `/v1/node` answers 404 the nodes come from `system.runtime.nodes`, and if that is refused too the node count shows `—`. On a JWT-delegation engine the monitor signs in as the viewer with their engine token; impersonation and password engines keep `egret-monitor` (see KI-265). | +| KI-195 | Engines | UX | S3 | On a Docker-provisioned engine the **Logs** and **Exec** buttons stay visible although they cannot work: Logs shows the raw *bad request: not available with the Docker engine provisioner*; Exec opens a shell that closes at once with *[connection closed]* (the WebSocket answers 400). Hide them or say why. *Status:* fixed in platform #728 (October 2026): on a Docker engine **Logs** and **Exec** are not shown, and a note points to `docker logs`. | +| KI-196 | Engines | Consistency | S3 | Engine dialogs speak Kubernetes on a Docker engine: Delete says *This destroys the engine's Kubernetes cluster*, Restart says it *rolls the coordinator and worker pods*, Review changes says *re-provisions (rolls) the engine pods*. *Status:* fixed in platform #728 (October 2026): the strip, the status table, Restart, Delete, Settings and Review changes say *containers* on a Docker engine (*This removes the engine's containers.*, *Saving recreates the engine's containers*). | +| KI-197 | Engines | Bug | S3 | After **Add engine** with Worker → Replicas `0`, the stored spec says `workerReplicas: 2` and Overview shows *Coord 1 · Worker 2* while only the coordinator container exists; the first **Save changes** on Settings corrects it. With the field blank, one worker container starts and the strip still says 2. *Status:* fixed in platform #728 (October 2026): the stored counts follow the per-role settings the engine runs with, on create and on every save; the demo engine shows *Worker 0* with one container. | +| KI-198 | Engines / Settings | UX | S3 | **Review changes** lists an unrelated `auth.passwordUsers [] Added` next to the real change and has only **Close**; **Save changes** then applies without confirmation while the header stays **running** although the coordinator container is recreated. *Steps:* docs-trino → Settings → Server → add `query.max-run-time` = 2h → Review changes. *Status:* fixed in platform #728 (October 2026): the empty `passwordUsers` is no longer listed, and **Save changes** opens **Review changes**, which saves only on its own **Save changes** (**Cancel** saves nothing). | +| KI-199 | Engines | UX | S3 | The engine header and list show the endpoint `docs-trino-coordinator.egret-engines:8080`, which does not resolve on the Docker network; the container answers as `docs-trino-trino(.egret-engines(.svc))` and the workspace connection uses `docs-trino-trino.egret-engines.svc:8443`. *Status:* fixed in platform #728 (October 2026): the header and list show the address a workspace connection dials, for example `b6-trino-trino.egret-engines.svc:8443`. | +| KI-200 | Engines / Add engine | UX | S3 | **Expose via ingress** without a **Host** is silently dropped: the engine is created with no ingress and no message. With a host the Docker provisioner refuses (*not supported by the Docker engine provisioner: ingress*), as expected. *Status:* fixed in platform #728 (October 2026): with **Expose via ingress** on, **Host** is required: the field says *Enter the host the ingress serves, or turn off Expose via ingress.* and Save and Create wait for it. The section is still offered on Docker engines (KI-264). | +| KI-201 | Governance / Lineage | Bug | S2 | View lineage fails on a provisioned engine: *engine docs-trino: engine: Access Denied: Cannot access catalog demo_warehouse*, because the engine's own policy does not let Egret's lineage reader into the catalog. The overview shows a **Warning**, the table's Lineage tab *No reader is known*, View exposure lists the view as *clearance unknown*; `POST /classifications/lineage-sweep` fails the same way, and the console has no control for the sweep the UI asks you to run. *Steps:* view `demo_warehouse.analytics.revenue_by_country` over `sessions`. *Status:* fixed in platform #728 (October 2026): Docker containers get pod-shaped hostnames, so their plugin receives the `egret-lineage` access a provisioned JWT-mode Trino engine gets; the lineage sweep went from *Access Denied: Cannot access catalog demo_warehouse* to *computed: 1*. The console still has no control for the sweep. | | KI-202 | Governance / Migrate | Gap | S3 | A one-policy migration cannot be proved when its catalog is also served by a provisioned engine: *proving this migration needs 48735 probe evaluations across 2 profile(s), 2697570 units of work … over the limit of 1300000*. On a catalog only the external engine serves, the same policy proved with 1575 probes. *Steps:* `select` on `tpch_bench.*.*.*` in `docs-legacy-trino` (also served by docs-trino, 28 policies) → Preview migration. | | KI-203 | Governance / Engine permissions | UX | S3 | In **Grant engine permission** the user search list stays open after a pick and covers the **Grant** button until you click elsewhere. | | KI-204 | Control plane / Engines | UX | S3 | After **Register engine** without a profile you land on Plugin config with the raw API text *conflict: engine trino-demo-ext is bound to no policy profile: bind a profile first Bind a policy profile on the Settings tab, then come back.* (prefix kept, sentences run together). | @@ -235,10 +235,10 @@ Collected while writing the detailed pages for each area (October 2026), checked | KI-206 | Control plane / Storage | Security | S2 | **Storage → \ → Browse** lists `query-results/*.jsonl`, the stored SQL Lab results of every user, and `GET /storage/{id}/download?key=…` returns them to anyone with `storage:read`, which the built-in `data_steward` role holds. These are each user's post-mask results. *Status:* fixed in platform #715 (October 2026): in egret's own bucket the prefixes egret writes (`engine.results_prefix`, `apps.storage.prefix`, `workflow.storage.prefix`) are closed to everyone except a platform admin: Browse hides them and says so, listing or downloading under them answers 403 with the reason, and path tricks answer 400. The raw warehouse data files are still readable: KI-236. | | KI-207 | Control plane / Secrets | UX | S3 | System-managed secrets (`engine-internal-*`, `engine-monitor-*`, `engine-tls-*`, `catalog-client-*`) are listed with user secrets, can be rotated or deleted, and are offered in the catalog **Password secret** picker. The **Used by** and **Created** columns are cut off at 1440 px. *Status:* fixed in platform #715 (October 2026): secrets egret creates for engines and catalog clients carry `system_managed`, show a **System** badge, cannot be renamed, rotated or deleted by hand (409), and are left out of every secret picker. The cut-off columns did not reproduce at 1440 px. | | KI-208 | Control plane / Catalogs | Gap | S3 | A second egret warehouse on the same bucket is refused (*Storage profile overlaps with existing warehouse demo-warehouse*), also through a connection with another key prefix (`sandbox/`); the New catalog form does not say a warehouse needs its own bucket. Related: KI-129. | -| KI-209 | Control plane / Users | UX | S3 | **Disable** on a user page acts at once and signs the user out without confirmation, while **Delete** asks for a typed confirmation. | +| KI-209 | Control plane / Users | UX | S3 | **Disable** on a user page acts at once and signs the user out without confirmation, while **Delete** asks for a typed confirmation. *Status:* fixed in platform #727 (October 2026): **Disable** on a user page asks *Disable \?* first; **Enable** acts at once. Bulk **Disable** on the Users list still acts at once (KI-262). | | KI-210 | Access requests | Gap | S3 | No mail is sent when someone requests access or when a request is approved or denied, although the workspace has working SMTP (Mailpit received nothing); admins only see the queue when they open it. | | KI-212 | Workspace settings / Notifications | UX | S3 | Before settings exist the page loads `GET /notifications/smtp` and `/notifications/slack`, which answer 404 and show as console errors. | -| KI-213 | Engines / Demo stack | Bug | S2 | After `just demo-down` and `just demo-up`, engines added with **Add engine** stay `provisioning`: their containers are gone and nothing recreates them, and **Restart** now tells you to delete and re-add the engine. *Workaround:* delete the engine and add it again (its policy profile stays under *Profiles without an engine*). | +| KI-213 | Engines / Demo stack | Bug | S2 | After `just demo-down` and `just demo-up`, engines added with **Add engine** stay `provisioning`: their containers are gone and nothing recreates them, and **Restart** now tells you to delete and re-add the engine. *Workaround:* delete the engine and add it again (its policy profile stays under *Profiles without an engine*). *Status:* fixed in platform #728 (October 2026): **Restart** on a Trino engine that ran before and has no pod or container left creates them again from its stored settings; after `just demo-down` and `just demo-up` the demo engine returned to `running`. | ### Install, configuration, security and operations @@ -250,20 +250,20 @@ Collected while writing the detailed pages for each area (October 2026), checked | KI-139 | Configuration | Gap | S3 | The server never reads `.env`; `cargo run -p egret-bin -- init` therefore ignores it, and there is no `just init` recipe. Only `just` recipes see `.env`. *Steps:* `crates/egret-bin` has no dotenv use; `just --list`. | | KI-140 | Configuration | UX | S3 | Unknown keys and misspelled `EGRET_*` variables are silently ignored at the top level and in most sections (only four sub-tables use `deny_unknown_fields`). *Steps:* Source: `crates/egret-config/src/lib.rs`. *Status:* fixed in platform #718 (October 2026): an unknown `EGRET_*` variable or configuration-file key logs a `WARN` at start with a did-you-mean hint, and never stops the start; variables read outside the configuration and the test-harness prefixes are allow-listed. | | KI-141 | Configuration | UX | S3 | Changing `server.port` leaves `catalog.public_url`, and with it the machine IdP `issuer` and token endpoint, at `http://localhost:8080`. *Steps:* Server on 18080: `/.well-known/openid-configuration` says `"issuer":"http://localhost:8080"`. *Status:* fixed in platform #718 (October 2026): an unset `catalog.public_url` defaults to `http://localhost:`. `oidc.redirect_url` does not follow the port yet: KI-240. | -| KI-142 | Authentication | UX | S3 | The authorization request carries `scope=openid+openid+email+profile`: the default scopes already contain `openid` and the client adds it again. *Steps:* `curl -D - localhost:8080/api/auth/oidc/login`, `location` header. | +| KI-142 | Authentication | UX | S3 | The authorization request carries `scope=openid+openid+email+profile`: the default scopes already contain `openid` and the client adds it again. *Steps:* `curl -D - localhost:8080/api/auth/oidc/login`, `location` header. *Status:* fixed in platform #727 (October 2026): the request carries `scope=openid+email+profile`. | | KI-143 | Authentication | Gap | S3 | Local sign-in cannot be switched off when SSO is on; the form is only hidden behind **Admin sign-in (break-glass)**, and `POST /api/auth/login` still answers. *Steps:* `frontend/src/auth/LoginPage.tsx`; sign-in works with SSO on. | -| KI-144 | Authentication | UX | S3 | The sign-in rate limit counts successful attempts (1 success + 4 failures, then 429), which surprises people who sign in a few times in a row. *Steps:* Throwaway instance, 7 consecutive attempts. | -| KI-145 | Authentication | Bug | S3 | `POST /api/auth/login` returns `platform_roles: []` and `permissions: []` for the bootstrap `platform_admin`, while `GET /api/auth/me` returns the roles. *Steps:* Throwaway instance after `egret init`. | +| KI-144 | Authentication | UX | S3 | The sign-in rate limit counts successful attempts (1 success + 4 failures, then 429), which surprises people who sign in a few times in a row. *Steps:* Throwaway instance, 7 consecutive attempts. *Status:* fixed in platform #727 (October 2026): only failed attempts count, and a successful sign-in clears them; success, four failures, success answered 200. | +| KI-145 | Authentication | Bug | S3 | `POST /api/auth/login` returns `platform_roles: []` and `permissions: []` for the bootstrap `platform_admin`, while `GET /api/auth/me` returns the roles. *Steps:* Throwaway instance after `egret init`. *Status:* fixed in platform #727 (October 2026): the login response carries the same roles and permissions as `GET /api/auth/me` (`platform_roles: ["data_steward"]` for a steward). | | KI-146 | CLI | Security | S3 | `egret init` prints the password even when it was supplied in `EGRET_ADMIN_PASSWORD`, so it lands in terminal history and CI logs. *Steps:* `EGRET_ADMIN_PASSWORD=... egret init --force` prints `Password: ...`. *Status:* fixed in platform #719 (October 2026): `egret init` prints `Password: taken from EGRET_ADMIN_PASSWORD (not shown)` for a supplied password, and an empty `EGRET_ADMIN_PASSWORD` now generates one. | | KI-147 | Security hardening | Gap | S2 | `GET /metrics` answers 200 with Prometheus metrics, `/healthz` and `/readyz` too, with no authentication. Only the Helm ingress happens not to route them. *Steps:* `curl localhost:8080/metrics` returned 204 lines. | -| KI-148 | Ranger | UX | S2 | `ranger.auth_token` locks out stock Ranger plugins (they cannot send a bearer token; the rendered plugin config has no credential) while the setting reads like a hardening switch. The plugin-config endpoint warns about it; the configuration docs do not. *Steps:* `/service/plugins/policies/download/...` 401 without header; `GET /api/control-plane/engines/{id}/ranger-plugin-config` warning. | +| KI-148 | Ranger | UX | S2 | `ranger.auth_token` locks out stock Ranger plugins (they cannot send a bearer token; the rendered plugin config has no credential) while the setting reads like a hardening switch. The plugin-config endpoint warns about it; the configuration docs do not. *Steps:* `/service/plugins/policies/download/...` 401 without header; `GET /api/control-plane/engines/{id}/ranger-plugin-config` warning. *Status:* closed, no longer applies (October 2026): since platform #713 and #714 the policy service is under `/ranger` and each engine presents its own credential (HTTP Basic, `engine-`); `ranger.auth_token` is only the operator's bearer token and locks no plugin out. See [Engines and Ranger setup](./deploy-operate/engines-and-ranger#authentication-of-the-plugin). | | KI-149 | Ranger | Gap | S3 | `GET .../policies/services/{service}/trino-config` renders `policy.rest.url` from `catalog.public_url`, so a fresh install hands an engine in a cluster `http://localhost:8080`. The endpoint for external engines warns; the Trino one does not. *Steps:* Response for `docs-trino-policy`: `http://localhost:8080`. | | KI-150 | Helm | Gap | S2 | The chart has no values for an external PostgreSQL or S3: the ConfigMap hardcodes `postgres:5432` and `http://rustfs:9000`, and storage credentials come from `deps.rustfs.*`. The README tells you to set `deps.enabled=false` and "point the EGRET_* values at your services". *Steps:* `templates/configmap-env.yaml`, `templates/secret.yaml`; `helm template --set deps.enabled=false`. | | KI-151 | Helm | Docs | S3 | Chart README is stale: says OIDC is off "in this milestone" and builds only backend and frontend images, while the chart has `oidc.enabled` and `just images-build` builds four images. Chart `description` still says "local kind deployment". *Steps:* `deploy/helm/egret/README.md`, `Chart.yaml`. | | KI-152 | Helm | Security | S3 | The bundled Keycloak (with `oidc.enabled`) imports the dev realm: `sslRequired: none`, `*` redirect URI, password grant, known user passwords and the client secret `egret-secret`. *Steps:* `deploy/helm/egret/files/egret-realm.json`. *Status:* fixed in platform #719 (October 2026): the Helm chart refuses `oidc.enabled` unless `oidc.devRealm: true` (default `false`) is also set, and then prints a warning in the install notes; the kind recipes opt in. | | KI-153 | OpenAPI | Gap | S2 | Only the Data API bearer scheme is declared; session cookie and CSRF are not, so generated clients cannot authenticate. Most operations have no summary (only an `operationId`), and many descriptions are internal doc comments (long, referencing internal types and task numbers). *Steps:* `openapi.json`; `scripts/gen-api-reference.mjs` has to truncate descriptions. | | KI-155 | Demo stack | Docs | S3 | The demo edge does not route `/readyz` and `/healthz` to Egret: `curl http://egret.localtest.me:18080/readyz` answers 200 with the web app's HTML, which looks healthy whatever the backend state, so the generic first checks of the troubleshooting index do not work on the demo stack. | -| KI-185 | Authentication | UX | S2 | An SSO user lands on *No workspaces yet* after the first sign-in and can do nothing until a workspace admin adds them; Egret only knows the user after that first sign-in, so the admin cannot add them beforehand. The demo guide documents this. *Workaround:* sign in once, then have an admin add the user. Seen by the demo-stack guide author, not re-observed. | +| KI-185 | Authentication | UX | S2 | An SSO user lands on *No workspaces yet* after the first sign-in and can do nothing until a workspace admin adds them; Egret only knows the user after that first sign-in, so the admin cannot add them beforehand. The demo guide documents this. *Workaround:* sign in once, then have an admin add the user. Seen by the demo-stack guide author, not re-observed. *Status:* fixed in platform #727 (October 2026): with SSO on, a workspace admin can add an email nobody has signed in with; it is listed under *Waiting for first sign-in* (with **Withdraw**) and becomes a membership at that person's first sign-in, only if the IdP says the email is verified and only with a workspace role. Someone with no workspace lands on a *No workspaces yet* card with their email, **Check again** and **Sign out**. | | KI-214 | UI consistency | Consistency | S3 | Page titles come in five sizes: 13 px (SQL Lab, the chart editor, an endpoint's detail page), 16 px (an app's viewer), 18 px (every Explore and Governance list), 20 px (a dashboard and a flow) and 24 px (the alert and report editor, Workspace settings, every Control Plane page, Governance → Data). Measured on the `h1` of 78 screens at 1440×900. Pick one title size per page type (list, detail, editor, settings) and put it in `ListPageShell` and a shared detail header. | | KI-215 | UI consistency | Consistency | S3 | There is no shared page width. Lists and Control Plane pages run edge to edge (content from x=248 to x=1416); Workspace settings starts at x=256; the alert and report editor is a centred 896 px column (x=384…1280); **Choose a visualization** is a centred 768 px column (x=448…1216). Moving between pages shifts the left edge of the content by up to 200 px. The source has no page container: `max-w-3xl` appears 29 times, `max-w-5xl` 9, `max-w-2xl` 9, `max-w-4xl` and `max-w-6xl` twice each. | | KI-216 | UI consistency | Consistency | S3 | Five tab styles for the same job. Dataset detail and People & groups use a grey pill strip (40 px high, 6 px radius); the chart editor's **Data / Display** and an endpoint's **Query / Config / …** use an underline; the flow editor (**Edit / Revisions / Executions / Triggers**) uses a small pill strip, the alert editor and the catalog detail (8 tabs) use a pill strip with an icon touching its label, with no gap; Control Plane → Engines (**Trino / StarRocks / RisingWave**) and Access Explorer use a smaller pill; Storage and Secrets use a pill strip with tiny icons. FE-PAGE-2 says detail sections use `ui/tabs`; 24 files do, and four hand-roll a tab list (`DashboardTabBar`, `PanelTabStrip`, `NamespaceFiles`, the Data API `Query.tsx`). | @@ -310,18 +310,25 @@ Collected while writing the detailed pages for each area (October 2026), checked | KI-257 | Charts / Box plot | UX | S3 | With **Values to display** = **All**, the value labels of a box can overlap: overlap hiding is switched off on purpose for that mode. (Found while fixing KI-165.) | | KI-258 | Charts / Funnel | Gap | S3 | The default (trapezoid) funnel is not offered `.png`: it is not drawn as one chart image. Only **Bar — a column per step** can be downloaded as `.png`. Before #724 the trapezoid funnel was offered `.png`, but the file held only the first band. (Found while fixing KI-043.) | | KI-259 | Charts / Map | UX | S3 | When rows exist but none matches a region, the region map still draws its colour legend, running from 0 to 1, next to an unshaded map and the *Showing 0 of N rows…* note. An empty result shows *No data* and no legend. (Found while fixing KI-046.) | +| KI-260 | Control plane / Users | UX | S3 | The **Roles** column of **Control Plane → Users** and the **Roles** section of a user's page list only platform roles granted to the user directly. A role the user holds through a group (for example `platform_admin` through `platform-admins`) shows as a dash, although the user has it. *Workaround:* read the group's page, or **View access** (Access Explorer). (Found while closing KI-021; read from the code: the list reads `roles_of("user", id)`.) | +| KI-261 | Workspace settings / Members | Consistency | S3 | `POST /api/ws/{workspace}/members` still accepts the role `data_api_invoker`, from a direct API call or for an SSO pre-registration, although the Members and Access requests pickers no longer offer it because it is meant to be granted on one Data API endpoint only. (Found while fixing KI-019.) | +| KI-262 | Control plane / Users | UX | S3 | Bulk **Disable** on the Users list signs the selected users out at once, with only an **Undo** in the toast, while **Disable** on a single user's page now asks first (KI-209). (Found while fixing KI-209.) | +| KI-263 | Engines / Docker | Bug | S2 | On the Docker provisioner, a settings save whose browser disconnects mid-way (a closed tab, a dropped connection) is cancelled between removing the old containers and starting the new ones: the engine is left with no container and shows **failed** until the next save. *Workaround:* save the settings again. (Found while fixing KI-198; read from the code: the update removes, then starts, inside the request.) | +| KI-264 | Engines / Docker | UX | S3 | The **Ingress exposure** section of Add engine and Settings is offered on the Docker provisioner, which refuses any ingress (*not supported by the Docker engine provisioner: ingress*). Since KI-200 a host is required, so the refusal now comes only after you fill one in. (Found while fixing KI-200.) | +| KI-265 | Engines / Monitor | Bug | S3 | verify: on an engine with impersonation (not JWT delegation) bound to a Ranger policy, the monitor signs in as `egret-monitor`, which holds no `impersonate` grant in the engine's policy, so it probably sees only its own queries, not other users'. Not reproduced: no such engine ran on the demo stack. (Found while fixing KI-194.) | +| KI-266 | Reports | Limitation | S3 | CSV and Text reports format numbers with the chart table's defaults (at most three decimals; digit grouping in Text only), not with a column's own format settings (decimals, currency, prefix, suffix). *Workaround:* round or format the value in the chart's SQL. (Decision in platform #726, KI-169.) | ## Not verified The Docker demo stack (`just demo-up`) now exercises what the earlier demo instance could not: **Add engine** with the Docker provisioner (Trino), the Data Apps editor and viewer, PNG and PDF reports through the renderer, and SMTP delivery to Mailpit. These are still not exercised there, so the matching pages describe them from source and say so: -- **Kubernetes-only features:** StarRocks and RisingWave engines, ingress, autoscaling, pod logs and exec on engines, the Kubernetes kernel and app launchers. +- **Kubernetes-only features:** StarRocks and RisingWave engines, ingress, autoscaling, pod logs and exec on engines, the Kubernetes kernel and app launchers. For a Kubernetes engine, the monitor's viewer sign-in and node fallback (KI-194) and **Restart** recreating a removed engine (KI-213) are covered only by tests with fakes. - **Ranger enforcement on an external engine:** the demo Trino has no Ranger plugin. Rule overlaps that already exist cannot be shown, because Egret refuses to create them. - **Governed connections:** `data_access_denied` and `governance_stale` responses, and different rows per viewer in an app (`trino-demo` runs every query as `egret`). - **Non-admin use:** a delegated (per-user) query by a non-admin through the UI, SQL Lab and dataset behaviour for a viewer, and the permissions needed for Tags. These were blocked by KI-159, which #712 fixed; they have not been re-checked for these docs. - **Delivery:** Slack (no Slack app on the stack) and webhook request bodies (private and Docker hosts are blocked, KI-035; payload shape comes from source). - **Workflows:** script tasks and `WorkingDirectory`, Kestra JDBC plugin tasks (no container runner or plugin host, refused with `task_type_not_served`), `egret.sql`, `egret.dataapi.call`, `egret.dataset.refresh`, `egret.report.send`, `core.http` and `core.slack` (no handler on any deployment), mail from a workflow (`MailSend` is unknown) and webhook calls (off on the stack, see KI-188). - **SQL Lab errors:** `connect:`, `timeout` and authentication errors (the only connection is healthy and not delegated). -- **Demo stack lifecycle:** `just demo-up` from an empty machine and its build time, `demo-reset` and `demo-seed`, and the demo-stack upgrade path. Engine state after `demo-down`/`demo-up` is KI-213. +- **Demo stack lifecycle:** `just demo-up` from an empty machine and its build time, `demo-reset` and `demo-seed`, and the demo-stack upgrade path. After `demo-down`/`demo-up`, **Restart** recreates an engine's containers (KI-213, fixed in #728). - **Elsewhere:** tagging BI objects, and "Limit this filter's choices" on dashboards. - **UI consistency sweep (KI-214 to KI-235):** 78 admin screens in the light theme at 1440×900, the same 52 pages in the dark theme and at 768 and 390 px wide, the editors (SQL Lab with results, dashboard edit and filter settings, chart editor, flow editor, endpoint tabs, app editor) and a non-admin SSO member (`member-sso`, workspace *data-science-lab*). **Dark theme:** no defect found; a script found no light-coloured block on any of the 52 pages and 12 of them were checked by eye, and the native selects (KI-217) follow the dark palette. Not swept: the dashboard and chart editing panels beyond the screens above, the flow **Topology** and **Logs** views (this build has no such tabs), the app editor once started, and a non-admin on pages that need a connection (KI-159). diff --git a/platform/reference/configuration-keys.md b/platform/reference/configuration-keys.md index 000bb62..78e375a 100644 --- a/platform/reference/configuration-keys.md +++ b/platform/reference/configuration-keys.md @@ -74,9 +74,9 @@ Two limiters, both in memory per replica. The window and the lockout are in seco | Key | Environment variable | Default | Meaning | |---|---|---|---| -| `login_max_attempts` | `EGRET_RATE_LIMIT__LOGIN_MAX_ATTEMPTS` | `5` | Local sign-in attempts allowed per window, per address and per email. Successful and failed attempts both count. | +| `login_max_attempts` | `EGRET_RATE_LIMIT__LOGIN_MAX_ATTEMPTS` | `5` | Failed local sign-in attempts allowed per window, per address and email. Successful attempts do not count, and a success clears the failures. | | `login_window_secs` | `EGRET_RATE_LIMIT__LOGIN_WINDOW_SECS` | `300` | Length of the sign-in window. | -| `login_lockout_secs` | `EGRET_RATE_LIMIT__LOGIN_LOCKOUT_SECS` | `300` | How long a key stays blocked (HTTP 429) once it exceeds the limit. | +| `login_lockout_secs` | `EGRET_RATE_LIMIT__LOGIN_LOCKOUT_SECS` | `300` | How long a key stays blocked (HTTP 429 with `Retry-After`) once it reaches the limit. | | `token_max_attempts` | `EGRET_RATE_LIMIT__TOKEN_MAX_ATTEMPTS` | `60` | Requests to the machine identity provider's `POST /oauth/token` (client credentials) and to the token exchange per window, keyed on address and client id. Requests that repeat an already verified secret are not counted. | | `token_window_secs` | `EGRET_RATE_LIMIT__TOKEN_WINDOW_SECS` | `60` | Length of the token window. | | `token_lockout_secs` | `EGRET_RATE_LIMIT__TOKEN_LOCKOUT_SECS` | `60` | How long a key stays blocked after exceeding the token limit. | @@ -148,7 +148,8 @@ The S3-compatible object store used for the catalog warehouse, query results, wo | Key | Environment variable | Default | Meaning | |---|---|---|---| | `enabled` | `EGRET_RANGER__ENABLED` | `false` | Serve the Ranger-compatible policy service that engine plugins poll, and turn on policy administration. See [Engines and Ranger setup](../deploy-operate/engines-and-ranger). | -| `auth_token` | `EGRET_RANGER__AUTH_TOKEN` | empty | Token the Ranger plugins present to the policy service. | +| `auth_mode` | `EGRET_RANGER__AUTH_MODE` | `required` | `required`: every request under `/ranger` needs an engine credential or the operator token. `anonymous`: requests without a credential are served, for a throwaway local stack only. | +| `auth_token` | `EGRET_RANGER__AUTH_TOKEN` | empty | The operator's bearer token for the policy service. Empty accepts no bearer. Engines do not use it: each presents its own credential. | ## engine diff --git a/platform/reference/rest-api/auth.md b/platform/reference/rest-api/auth.md index 5f051b5..52ce52f 100644 --- a/platform/reference/rest-api/auth.md +++ b/platform/reference/rest-api/auth.md @@ -96,7 +96,7 @@ Operation id: `login`. Auth: None (public). |---|---| | 200 | Logged in; sets session + CSRF cookies | | 401 | Invalid email or password | -| 429 | Too many login attempts | +| 429 | Too many failed attempts; `Retry-After` says when to try again | | 500 | Internal error | ## POST `/api/auth/logout` diff --git a/platform/reference/rest-api/control-plane-audit.md b/platform/reference/rest-api/control-plane-audit.md index b0f2d3f..f950243 100644 --- a/platform/reference/rest-api/control-plane-audit.md +++ b/platform/reference/rest-api/control-plane-audit.md @@ -24,6 +24,7 @@ Operation id: `list_audit`. Auth: Session cookie. Forbidden (403): Requires plat | `action` | query | string | no | | | `kind` | query | string | no | | | `decision` | query | string | no | | +| `hideAllowed` | query | boolean | no | `true` leaves out authorization decisions that allowed the call (routine reads): what is left is changes and denials. | | `workspace` | query | string (uuid) | no | | | `from` | query | string (date-time) | no | | | `to` | query | string (date-time) | no | | diff --git a/platform/reference/rest-api/control-plane-engines.md b/platform/reference/rest-api/control-plane-engines.md index f820ecf..e2930f6 100644 --- a/platform/reference/rest-api/control-plane-engines.md +++ b/platform/reference/rest-api/control-plane-engines.md @@ -128,6 +128,8 @@ Operation id: `list_engines`. Auth: Session cookie. Forbidden (403): Missing eng | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | | `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | | `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | +| `endpoint` | string | yes | `host:port` clients reach the engine on, as a workspace connection to it dials it (`Engine::client_endpoint`). External: the declared address. | +| `runtime` | EngineRuntime | no | | | Status | Meaning | |---|---| @@ -178,6 +180,8 @@ Operation id: `create_engine`. Auth: Session cookie and `X-Egret-CSRF` header. F | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | | `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | | `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | +| `endpoint` | string | yes | `host:port` clients reach the engine on, as a workspace connection to it dials it (`Engine::client_endpoint`). External: the declared address. | +| `runtime` | EngineRuntime | no | | | `catalogAttachErrors` | CatalogAttachErrorDto[] | yes | Catalogs from the body's `catalogs` that could not be attached. Empty on success. The engine EXISTS and is provisioned; these catalogs simply are not on it, and each can be retried with `POST /api/control-plane/engines/\{id\}/catalogs/\{cid\}`.... | | Status | Meaning | @@ -224,6 +228,8 @@ Operation id: `register_external_engine`. Auth: Session cookie and `X-Egret-CSRF | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | | `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | | `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | +| `endpoint` | string | yes | `host:port` clients reach the engine on, as a workspace connection to it dials it (`Engine::client_endpoint`). External: the declared address. | +| `runtime` | EngineRuntime | no | | | `rangerCredential` | RangerCredentialDto | yes | The credential the engine's Ranger plugin presents to egret's `/ranger` surface (HTTP Basic). **Shown once**: egret never returns this password again; a lost one is replaced with `POST... | | Status | Meaning | @@ -280,6 +286,8 @@ Operation id: `get_engine`. Auth: Session cookie. Forbidden (403): Missing engin | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | | `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | | `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | +| `endpoint` | string | yes | `host:port` clients reach the engine on, as a workspace connection to it dials it (`Engine::client_endpoint`). External: the declared address. | +| `runtime` | EngineRuntime | no | | | Status | Meaning | |---|---| @@ -335,6 +343,8 @@ Operation id: `update_engine`. Auth: Session cookie and `X-Egret-CSRF` header. F | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | | `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | | `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | +| `endpoint` | string | yes | `host:port` clients reach the engine on, as a workspace connection to it dials it (`Engine::client_endpoint`). External: the declared address. | +| `runtime` | EngineRuntime | no | | | Status | Meaning | |---|---| @@ -411,7 +421,7 @@ Operation id: `get_cluster`. Auth: Session cookie. Forbidden (403): Missing engi | `running` | integer (int32) | yes | | | `queued` | integer (int32) | yes | | | `blocked` | integer (int32) | yes | | -| `nodes` | NodeDto[] | yes | | +| `nodes` | NodeDto[], nullable | no | `None` when the coordinator would not list its nodes: the count is unknown, not zero ([`cluster_nodes`]). | | Status | Meaning | |---|---| @@ -461,6 +471,8 @@ Operation id: `update_external_engine`. Auth: Session cookie and `X-Egret-CSRF` | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | | `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | | `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | +| `endpoint` | string | yes | `host:port` clients reach the engine on, as a workspace connection to it dials it (`Engine::client_endpoint`). External: the declared address. | +| `runtime` | EngineRuntime | no | | | Status | Meaning | |---|---| @@ -674,13 +686,14 @@ Operation id: `restart_engine`. Auth: Session cookie and `X-Egret-CSRF` header. | Status | Meaning | |---|---| -| 200 | Coordinator + worker pods rolled (rollout restart) | +| 200 | Coordinator + worker pods (or containers) rolled; a Trino engine that ran before and now has none (removed outside egret) is created again from its stored spec | | 400 | Kubernetes not configured or restart failed | | 401 | Not authenticated | | 403 | Missing engine:provision permission | | 404 | No such engine | -| 409 | The engine is external (`code` = `engine_is_external`): egret does not restart it | +| 409 | The engine is external (`code` = `engine_is_external`): egret does not restart it; or, when it is created again, the engine changed concurrently (retry) | | 500 | Internal error | +| 502 | Creating the engine again was refused by the cluster | ## POST `/api/control-plane/engines/{id}/retire-previous-ranger-credential` diff --git a/platform/reference/rest-api/index.md b/platform/reference/rest-api/index.md index b5d3cab..a7a75d7 100644 --- a/platform/reference/rest-api/index.md +++ b/platform/reference/rest-api/index.md @@ -112,7 +112,7 @@ described by their auth line here. | Authentication and identity | [Current user](rest-api/me) | 1 | | Workspaces and members | [Workspace administration](rest-api/admin-workspaces) | 10 | | Workspaces and members | [User administration](rest-api/admin-users) | 3 | -| Workspaces and members | [Workspace members](rest-api/workspace-members) | 9 | +| Workspaces and members | [Workspace members](rest-api/workspace-members) | 11 | | Workspaces and members | [Access requests](rest-api/access-requests) | 4 | | Workspaces and members | [Sharing](rest-api/shares) | 6 | | Explore (BI) | [Databases](rest-api/databases) | 9 | diff --git a/platform/reference/rest-api/workspace-members.md b/platform/reference/rest-api/workspace-members.md index b82e0e1..b6833fe 100644 --- a/platform/reference/rest-api/workspace-members.md +++ b/platform/reference/rest-api/workspace-members.md @@ -8,7 +8,7 @@ description: REST endpoints in the workspace-members group. Workspace membership (add/remove principals). -This page lists 9 endpoints (OpenAPI tag `workspace-members`). It is generated from `openapi.json`; see the [REST API overview](../rest-api) for authentication, CSRF and error conventions. +This page lists 11 endpoints (OpenAPI tag `workspace-members`). It is generated from `openapi.json`; see the [REST API overview](../rest-api) for authentication, CSRF and error conventions. | Method | Path | Operation id | Notes | |---|---|---|---| @@ -16,6 +16,8 @@ This page lists 9 endpoints (OpenAPI tag `workspace-members`). It is generated f | POST | `/api/ws/{workspace}/access-requests/{id}/approve` | `approve_access_request` | `POST /ws/\{workspace\}/access-requests/\{id\}/approve` — approve a pending request: add the requester as a member with the chosen role + optional expiry via the... | | POST | `/api/ws/{workspace}/access-requests/{id}/deny` | `deny_access_request` | `POST /ws/\{workspace\}/access-requests/\{id\}/deny` — deny a pending request with an optional note. | | POST | `/api/ws/{workspace}/impersonate` | `ws_impersonate_start` | `POST /api/ws/\{workspace\}/impersonate` — a `ws_admin` of `\{workspace\}` (or a platform_admin) starts impersonating a MEMBER of that workspace. | +| GET | `/api/ws/{workspace}/invites` | `list_invites` | | +| DELETE | `/api/ws/{workspace}/invites/{id}` | `delete_invite` | | | GET | `/api/ws/{workspace}/members` | `list_members` | | | POST | `/api/ws/{workspace}/members` | `add_member` | | | DELETE | `/api/ws/{workspace}/members/{ptype}/{pid}` | `remove_member` | | @@ -153,6 +155,55 @@ Guards run in a deliberate order so a caller learns nothing before its own autho | 409 | OIDC caller: re-authenticate at `authorize_url`, then the callback creates the overlay | | 500 | Internal error | +## GET `/api/ws/{workspace}/invites` + +Operation id: `list_invites`. Auth: Session cookie. Forbidden (403): Missing workspace permission. + +| Parameter | In | Type | Required | Description | +|---|---|---|---|---| +| `workspace` | path | string | yes | Workspace slug | + +**Response 200 (application/json)**: array of `InviteDto` + +| Field | Type | Required | Description | +|---|---|---|---| +| `id` | string | yes | | +| `email` | string | yes | | +| `role` | string | yes | | +| `invited_by_name` | string, nullable | no | | +| `created_at` | string (date-time) | yes | | + +| Status | Meaning | +|---|---| +| 200 | Emails pre-registered for their first SSO sign-in | +| 401 | Not authenticated | +| 403 | Missing workspace permission | +| 500 | Internal error | + +## DELETE `/api/ws/{workspace}/invites/{id}` + +Operation id: `delete_invite`. Auth: Session cookie and `X-Egret-CSRF` header. Forbidden (403): Missing workspace permission. + +| Parameter | In | Type | Required | Description | +|---|---|---|---|---| +| `workspace` | path | string | yes | Workspace slug | +| `id` | path | string (uuid) | yes | Pre-registration id | +| `workspace` | path | string | yes | | + +**Response 200 (application/json)**: `Ack` + +| Field | Type | Required | Description | +|---|---|---|---| +| `ok` | boolean | yes | | + +| Status | Meaning | +|---|---| +| 200 | Pre-registration withdrawn | +| 401 | Not authenticated | +| 403 | Missing workspace permission | +| 404 | No such pre-registration in this workspace | +| 500 | Internal error | + ## GET `/api/ws/{workspace}/members` Operation id: `list_members`. Auth: Session cookie. Forbidden (403): Missing workspace permission. @@ -196,15 +247,17 @@ Operation id: `add_member`. Auth: Session cookie and `X-Egret-CSRF` header. Forb | `principal_id` | string | yes | | | `role` | string | yes | | -**Response 200 (application/json)**: `Ack` +**Response 200 (application/json)**: `AddMemberResult` | Field | Type | Required | Description | |---|---|---|---| | `ok` | boolean | yes | | +| `invited` | boolean | yes | `true` when the email belongs to nobody yet and was pre-registered instead: the membership binds at that SSO user's first sign-in (KI-185). | | Status | Meaning | |---|---| -| 200 | Member added | +| 200 | Member added, or (`invited`) an unknown email pre-registered for its first SSO sign-in | +| 400 | Unknown principal or role | | 401 | Not authenticated | | 403 | Missing workspace permission | | 500 | Internal error | @@ -251,7 +304,8 @@ Operation id: `list_workspace_roles`. Auth: Session cookie. Forbidden (403): Mis | `name` | string | yes | | | `description` | string | yes | | | `parent_role_id` | string, nullable | no | | -| `permissions` | WsPermDto[] | yes | | +| `permissions` | WsPermDto[] | yes | The role's own permissions. | +| `inherited_permissions` | WsPermDto[] | yes | Permissions held through the parent chain and not held directly; a holder of the role has them too. | | Status | Meaning | |---|---| diff --git a/platform/workspace-settings/access-requests.md b/platform/workspace-settings/access-requests.md index 7507ad0..6b0ab3b 100644 --- a/platform/workspace-settings/access-requests.md +++ b/platform/workspace-settings/access-requests.md @@ -32,7 +32,7 @@ Each request shows the requester's email, the requested role, how long ago they ### Approve 1. Press **Approve**. The dialog **Approve request** says *Add \ to the workspace.* -2. Keep the requested **Role** or choose another. +2. Keep the requested **Role** or choose another. The picker lists the workspace's roles, as on [Roles & permissions](./roles-and-permissions.md). 3. Optionally set **Access expires**. *Optional — leave blank for no expiry.* 4. Press **Confirm**. A toast says *Request approved*. diff --git a/platform/workspace-settings/members.md b/platform/workspace-settings/members.md index f95a485..9fc06a2 100644 --- a/platform/workspace-settings/members.md +++ b/platform/workspace-settings/members.md @@ -28,7 +28,7 @@ A note under the title says *Workspace roles apply only inside \ — 1. Press **Add member**. 2. Under **Type**, choose **User** or **Group**. -3. Type the person's email (*User email…*) or the group's name (*Group name…*). A few letters are enough: matching people are suggested with their name and email (typing `emma` suggests *Emma Schmidt emma.schmidt@egret.local*); pick one. The person must already exist in the platform: an SSO user has signed in once, or a local account has been created. +3. Type the person's email (*User email…*) or the group's name (*Group name…*). A few letters are enough: matching people are suggested with their name and email (typing `emma` suggests *Emma Schmidt emma.schmidt@egret.local*); pick one. The person must already exist in the platform (an SSO user has signed in once, or a local account has been created), unless single sign-on is on: then you can type the email of someone who has not signed in yet (see [below](#add-someone-who-has-not-signed-in-yet)). 4. Choose the **Role**. The default is `ws_viewer`. 5. Press **Add member**. A toast says *Member added*. @@ -42,7 +42,18 @@ Prefer adding groups: when someone joins the group in your identity provider, th ### The roles you can choose -The role picker lists every workspace role defined on the platform: `ws_admin`, `ws_editor`, `ws_viewer`, `data_api_invoker` and any custom role (such as `finance_reporter`). See [Roles & permissions](./roles-and-permissions.md) for what each grants and [Roles](../control-plane/access-control/roles.md) to create a custom one. +The role picker lists the workspace's roles, the same list as [Roles & permissions](./roles-and-permissions.md): `ws_admin`, `ws_editor`, `ws_viewer` and any custom role (such as `finance_reporter`). `data_api_invoker` is not offered: it is granted on one Data API endpoint to a service account, never as a membership. See [Roles](../control-plane/access-control/roles.md) to create a custom role. + +### Add someone who has not signed in yet + +Egret learns about an SSO user at their first sign-in. With single sign-on on, you can still add that person beforehand: + +1. Press **Add member**, keep **User**, and type the full email address. The dialog says *Someone who has not signed in yet can be added by email. They join when they first sign in with SSO.* +2. Choose a workspace role and press **Add member**. The toast says *\ will join as \ at their first sign-in*. + +The email is listed under **Waiting for first sign-in** below the member table, with its role, when it was added and by whom. When that person first signs in with SSO, the entry becomes a normal membership and leaves the list. It binds only when the identity provider says the email is verified (`email_verified: true` in the ID token); otherwise the person lands on **No workspaces yet** and the entry stays. Only a workspace role can be given this way, never a platform role. If the person is already a member by then, their current role is kept. + +**Withdraw** on an entry asks *Withdraw the invitation?* (*\ will not join \ when they first sign in.*) and removes it. Without single sign-on, an email that matches no account is refused (`unknown user: `). ## Change a role @@ -85,6 +96,7 @@ People who are not yet members can ask to join (see [Access requests](./access-r - A workspace can have no admin when it is created in the Control Plane. Add a member with `ws_admin` first. - Platform admins have access to every workspace without being members. +- A direct `POST /api/ws/{workspace}/members` call still accepts `data_api_invoker`, which the role picker no longer offers (KI-261). ## Further reading diff --git a/platform/workspace-settings/roles-and-permissions.md b/platform/workspace-settings/roles-and-permissions.md index 751b67a..51f809f 100644 --- a/platform/workspace-settings/roles-and-permissions.md +++ b/platform/workspace-settings/roles-and-permissions.md @@ -21,7 +21,7 @@ A strip counts the **roles**, how many are **built-in** and how many are **custo ## The roles -Each role is a row with its name, a **built-in** label for the three platform defaults, and its permission count. +Each role is a row with its name, a **built-in** label for the three platform defaults, and its permission count. The count includes what the role inherits from its parent, for example **9 permissions (7 inherited)**. | Role | Permissions | Intended for | |---|---|---| @@ -29,7 +29,7 @@ Each role is a row with its name, a **built-in** label for the three platform de | `ws_editor` | 17 | Building and editing content. | | `ws_viewer` | 7 | Reading content. | -Custom roles, for example `finance_reporter`, are listed after the built-in ones with an **Inherits from \** label when they have a parent. The summary counts them as **custom**. The page lists only these roles; `data_api_invoker`, which only calls published Data API endpoints, is not listed here although the **Members** role picker offers it. +Custom roles, for example `finance_reporter`, are listed after the built-in ones with an **Inherits from \** label when they have a parent. The summary counts them as **custom**. The page lists only these roles. A note under the title explains why `data_api_invoker`, listed in the Control Plane, is missing: it is granted on one Data API endpoint, never as a workspace role. The **Members** and **Access requests** role pickers offer the same roles as this page. ## See what a role grants @@ -47,7 +47,7 @@ Select a row to expand it. The permissions are grouped by area, each with its de ![The custom role finance_reporter expanded: it inherits from ws_viewer and adds chart and dashboard management](/img/platform/admin/ws-roles-expanded.png) -A role that inherits from another shows only the permissions it adds itself. `finance_reporter`, which inherits from `ws_viewer` and adds `chart:manage` and `dashboard:manage`, shows **2 permissions** here, while the Control Plane role page counts 8 because it includes the seven inherited reads. Read the parent's row to see the inherited ones. +A role that inherits from another lists its own permissions first, then the ones it inherits under **Inherited from \**. A role that inherits from `ws_viewer` (seven reads) and adds two permissions of its own, such as `finance_reporter` with `chart:manage` and `dashboard:manage`, shows **9 permissions (7 inherited)**. A permission the role also holds directly is listed once, under its own permissions. ## Choose a role for someone diff --git a/scripts/shots/admin.mjs b/scripts/shots/admin.mjs index 53cbe95..c142b15 100644 --- a/scripts/shots/admin.mjs +++ b/scripts/shots/admin.mjs @@ -24,6 +24,8 @@ const seq = (...fns) => async (page) => { }; // Open an engine of the Control Plane list by name (the list links each name). const engine = (name) => seq(link(name), async (page) => page.waitForTimeout(1500)); +// The provisioned engine the cp-engine-* shots open (DOCS_ENGINE overrides it for a pass that adds its own). +const DOCS_ENGINE = process.env.DOCS_ENGINE ?? 'docs-trino'; export const shots = [ // ---- Governance @@ -295,17 +297,17 @@ export const shots = [ await page.getByText('Ingress exposure', {exact: true}).scrollIntoViewIfNeeded(); }, }, - {id: 'admin/cp-engine-overview', path: `${cp}/engines`, settle: 2000, prepare: engine('docs-trino')}, - {id: 'admin/cp-engine-catalogs', path: `${cp}/engines`, settle: 2000, prepare: seq(engine('docs-trino'), tab('Catalogs'))}, - {id: 'admin/cp-engine-workspaces', path: `${cp}/engines`, settle: 2000, prepare: seq(engine('docs-trino'), tab('Workspaces'))}, - {id: 'admin/cp-engine-settings', path: `${cp}/engines`, settle: 2000, prepare: seq(engine('docs-trino'), tab('Settings'))}, - {id: 'admin/cp-engine-monitor', path: `${cp}/engines`, settle: 4000, prepare: seq(engine('docs-trino'), tab('Monitor'))}, + {id: 'admin/cp-engine-overview', path: `${cp}/engines`, settle: 2000, prepare: engine(DOCS_ENGINE)}, + {id: 'admin/cp-engine-catalogs', path: `${cp}/engines`, settle: 2000, prepare: seq(engine(DOCS_ENGINE), tab('Catalogs'))}, + {id: 'admin/cp-engine-workspaces', path: `${cp}/engines`, settle: 2000, prepare: seq(engine(DOCS_ENGINE), tab('Workspaces'))}, + {id: 'admin/cp-engine-settings', path: `${cp}/engines`, settle: 2000, prepare: seq(engine(DOCS_ENGINE), tab('Settings'))}, + {id: 'admin/cp-engine-monitor', path: `${cp}/engines`, settle: 4000, prepare: seq(engine(DOCS_ENGINE), tab('Monitor'))}, { id: 'admin/cp-engine-logs', path: `${cp}/engines`, settle: 2500, prepare: async (page) => { - await engine('docs-trino')(page); + await engine(DOCS_ENGINE)(page); await page.getByRole('button', {name: 'Logs'}).first().click(); }, }, @@ -314,7 +316,7 @@ export const shots = [ path: `${cp}/engines`, settle: 800, prepare: async (page) => { - await engine('docs-trino')(page); + await engine(DOCS_ENGINE)(page); await page.getByRole('button', {name: 'More engine actions'}).click(); await page.getByRole('menuitem', {name: 'Delete engine'}).click(); }, diff --git a/scripts/shots/alerts.mjs b/scripts/shots/alerts.mjs index 4de6f00..2070718 100644 --- a/scripts/shots/alerts.mjs +++ b/scripts/shots/alerts.mjs @@ -3,6 +3,8 @@ // and the Mailpit inbox have content. Mail shots read Mailpit (MAILPIT, default http://localhost:8025): // the demo stack delivers every mail there. const ws = '/explore/default'; +// The alert whose run log alerts/run-log shows (RUN_LOG_ALERT overrides it). +const RUN_LOG_ALERT = process.env.RUN_LOG_ALERT ?? 'Pending orders above 100'; const MAILPIT = process.env.MAILPIT ?? 'http://localhost:8025'; // Every shot sets its own height, so a tall shot never leaks its viewport into the next one. const vp = (page, h = 900) => page.setViewportSize({width: 1440, height: h}); @@ -58,7 +60,7 @@ export const shots = [ path: `${ws}/alerts`, prepare: async (page) => { await vp(page); - await page.getByRole('button', {name: 'Run log for Pending orders above 100'}).click(); + await page.getByRole('button', {name: `Run log for ${RUN_LOG_ALERT}`}).click(); await page.waitForTimeout(1500); }, settle: 500, diff --git a/static/img/platform/admin/cp-engine-delete.png b/static/img/platform/admin/cp-engine-delete.png index ffdf9c4..f3817c2 100644 Binary files a/static/img/platform/admin/cp-engine-delete.png and b/static/img/platform/admin/cp-engine-delete.png differ diff --git a/static/img/platform/admin/cp-engine-monitor.png b/static/img/platform/admin/cp-engine-monitor.png index d135c8f..b5c449e 100644 Binary files a/static/img/platform/admin/cp-engine-monitor.png and b/static/img/platform/admin/cp-engine-monitor.png differ diff --git a/static/img/platform/admin/cp-engine-overview.png b/static/img/platform/admin/cp-engine-overview.png index e2fed8a..90e55a1 100644 Binary files a/static/img/platform/admin/cp-engine-overview.png and b/static/img/platform/admin/cp-engine-overview.png differ diff --git a/static/img/platform/admin/cp-engine-settings.png b/static/img/platform/admin/cp-engine-settings.png index e035ebf..d49beef 100644 Binary files a/static/img/platform/admin/cp-engine-settings.png and b/static/img/platform/admin/cp-engine-settings.png differ diff --git a/static/img/platform/alerts/run-log.png b/static/img/platform/alerts/run-log.png index ce27642..9f543da 100644 Binary files a/static/img/platform/alerts/run-log.png and b/static/img/platform/alerts/run-log.png differ diff --git a/static/img/platform/ops/get-started/sql-lab-run.png b/static/img/platform/ops/get-started/sql-lab-run.png index c17fbf5..0e899b0 100644 Binary files a/static/img/platform/ops/get-started/sql-lab-run.png and b/static/img/platform/ops/get-started/sql-lab-run.png differ diff --git a/static/img/platform/sql-datasets/sqllab-error.png b/static/img/platform/sql-datasets/sqllab-error.png index f2429a6..8ef1b52 100644 Binary files a/static/img/platform/sql-datasets/sqllab-error.png and b/static/img/platform/sql-datasets/sqllab-error.png differ