diff --git a/platform/control-plane/catalogs/create-catalog.md b/platform/control-plane/catalogs/create-catalog.md index 088017a..6dae5a3 100644 --- a/platform/control-plane/catalogs/create-catalog.md +++ b/platform/control-plane/catalogs/create-catalog.md @@ -38,7 +38,7 @@ The catalog's page opens. Continue with [Catalog detail](./catalog-detail.md) to | **TPC-DS** | Generated benchmark data. | **Scale**. | | **Generic** | Whatever properties you write in **Advanced properties**. | The engines it is valid for (**trino** needs `connector.name`; **starrocks** needs `type`) and the property pairs. | -The **Storage connection** list is the platform's [storage connections](../storage/index.md). **Password secret** lists [secrets](../secrets.md), so a credential is never typed into the catalog. The list shows every secret, including the ones egret creates for engines and catalog clients (`engine-internal-…`, `engine-monitor-…`, `engine-tls-…`, `catalog-client-…`); pick one of your own. +The **Storage connection** list is the platform's [storage connections](../storage/index.md). **Password secret** lists [secrets](../secrets.md), so a credential is never typed into the catalog. The list leaves out the **System** secrets egret creates for engines and catalog clients (`engine-internal-…`, `engine-monitor-…`, `engine-tls-…`, `catalog-client-…`). ## Advanced properties diff --git a/platform/control-plane/secrets.md b/platform/control-plane/secrets.md index 1e1a13d..15683a8 100644 --- a/platform/control-plane/secrets.md +++ b/platform/control-plane/secrets.md @@ -14,7 +14,15 @@ You need **secret:read** to open the page; creating, rotating and deleting need A strip counts the **secrets**. -The list also shows the secrets egret creates and manages itself: `engine-internal--…` (the Trino internal-communication secret), `engine-monitor-…` (the egret-monitor password for the native Trino monitor), `engine-tls-…` (the coordinator keystore) for every provisioned engine, and `catalog-client-…` (a Catalog Client credential an engine presents to egret's catalog). Leave them alone: rotating or deleting one breaks the engine or client that uses it. On a 1440-pixel-wide screen the **Used by** and **Created** columns are cut off at the right; scroll the table sideways. +The list also shows the secrets egret creates and manages itself: `engine-internal--…` (the Trino internal-communication secret), `engine-monitor-…` (the egret-monitor password for the native Trino monitor), `engine-tls-…` (the coordinator keystore) for every provisioned engine, the StarRocks root and Trino delegator passwords, and `catalog-client-…` (a Catalog Client credential an engine presents to egret's catalog). The page says *Secrets marked System are created by egret for engines and catalogs and are managed with them.* + +### System secrets + +A secret egret manages carries a **System** badge next to its name, in the list and on its page. Egret rotates and deletes it together with the engine or catalog that created it, so you cannot change it by hand: + +- **Edit**, **Rotate value** and **Delete** are disabled, with the reason *Managed by egret: it is rotated and deleted with the engine or catalog that created it.* +- The API refuses a rename, a rotation or a delete with 409: *«name» is managed by egret: it is rotated and deleted with the engine or catalog that created it, not here*. The secret's `system_managed` field is `true`. +- No secret picker offers it: the **Secret** fields of storage and database connections, the catalog **Password secret** pickers and the engine create and security forms leave it out. | Column | Meaning | |---|---| @@ -67,7 +75,7 @@ Rotate when a credential changes, or on a schedule. Everything that references t ## Delete a secret -Press **Delete** on the list or the secret page. A secret that is in use cannot be deleted: the **Used by** count must be zero. Detach it from every connection first. +Press **Delete** on the list or the secret page. A secret that is in use cannot be deleted: the **Used by** count must be zero. Detach it from every connection first. A **System** secret cannot be deleted here at all ([System secrets](#system-secrets)). :::warning A secret is a platform-wide object. Rotating it changes the credential for every connection that references it. @@ -83,6 +91,7 @@ A secret is a platform-wide object. Rotating it changes the credential for every - Values cannot be read back, only replaced. - Names are global across the platform. +- Secrets that existed before the **System** flag was introduced are marked from the engines and catalogs that point at them, and from the name and description of a minted secret. A minted secret that nothing points at any more and whose name or description was changed by hand is not marked. ## Further reading diff --git a/platform/control-plane/storage/index.md b/platform/control-plane/storage/index.md index bb086b2..335129d 100644 --- a/platform/control-plane/storage/index.md +++ b/platform/control-plane/storage/index.md @@ -61,9 +61,20 @@ The header shows the name, provider and the **BUCKET** and **ENDPOINT**, with ** ## Browse a connection -The **Browse** tab lists the bucket's folders and files, so you can confirm a connection works and see what egret stored: for example folders such as `apps`, `query-results`, `workflow` and the warehouse folders named by id. Select a folder to open it; files show their size. An empty folder says *This folder is empty.* +The **Browse** tab lists the connection's folders and files, so you can confirm a connection works and see what egret stored: for example folders such as `apps`, `query-results`, `workflow` and the warehouse folders named by id. Select a folder to open it; files show their size and can be downloaded. An empty folder says *This folder is empty.* -The demo bucket is shared by everything egret stores, so `query-results` holds the stored result files of every user's SQL Lab queries. Anyone with **storage:read** can list and download them, including the built-in `data_steward` role; these files are what each person was allowed to see, after their masks and row filters. Keep **storage:read** for administrators. +Browse and download start at the connection's **Key prefix**, not at the bucket root. Keys are relative to that prefix, and the first crumb of the path shows `bucket/prefix`: a connection on bucket `warehouse` with Key prefix `sandbox` opens at `warehouse/sandbox`. A connection without a Key prefix opens at the bucket root and its crumb shows the bucket name. + +### Folders only a platform admin can open + +In egret's own bucket (the `[storage] bucket` of the server configuration), egret writes other users' data under three prefixes: stored SQL Lab results (`engine.results_prefix`, `query-results` by default), Data App sources (`apps.storage.prefix`) and workflow files (`workflow.storage.prefix`). A stored SQL Lab result is what one person was allowed to see, after their masks and row filters. These prefixes are closed to everyone who is not a platform admin, whatever storage permission they hold: + +- Browse leaves them out and says so under the list, for example *apps, query-results, workflow are not shown: egret stores its users' SQL Lab results, Data App sources and workflow files there. Only a platform admin can open them.* +- Listing or downloading a key under them answers 403 with the reason: *… holds data egret writes for its users (stored SQL Lab results, Data App sources, workflow files), so it cannot be browsed or downloaded with storage:read; only a platform admin can open it*. The match ignores case and also applies after percent-decoding the key. + +A platform admin sees and downloads them as before. + +Keys that try to leave the folder are refused with 400 for everyone, platform admins included: a `.` or `..` segment, an empty segment (a leading `/`, a trailing `/` on a file, or `//`), a backslash or a control character, also once percent-decoded. ![The Browse tab listing the top-level folders of the warehouse bucket](/img/platform/admin/cp-storage-browse.png) @@ -74,7 +85,7 @@ Press **Delete**. The dialog *Delete «\»?* says *This storage con ## Limits and known issues - Only S3-compatible stores work; Google Cloud Storage and Azure Blob are not available yet. -- The **Browse** tab lists the bucket root even for a connection with a **Key prefix** (checked with `sandbox-lake`, prefix `sandbox/`). +- **storage:read** still lets a holder (for example the built-in `data_steward` role) download the raw Iceberg data files of every warehouse in the bucket, which bypasses column masks and row filters (KI-236). Keep **storage:read** for administrators. - A key prefix does not keep two egret warehouses apart: a warehouse on `sandbox-lake` was refused with *Storage profile overlaps with existing warehouse demo-warehouse*. Use a separate bucket for each warehouse. ## Further reading diff --git a/platform/deploy-operate/authentication.md b/platform/deploy-operate/authentication.md index ed3207a..5b92949 100644 --- a/platform/deploy-operate/authentication.md +++ b/platform/deploy-operate/authentication.md @@ -61,7 +61,8 @@ The decoded access token for `tester` in this realm contains `"groups": ["data-t :::danger The realm file is for development only It disables TLS (`sslRequired: none`), accepts `*` as a redirect URI, enables the password grant, contains a client secret -and three users with known passwords. Do not import it into a shared Keycloak. Create your own realm. +and three users with known passwords. Do not import it into a shared Keycloak. Create your own realm. The Helm chart +bundles this realm with `oidc.enabled=true` only when you also set `oidc.devRealm=true` (see [Kubernetes](./kubernetes)). ::: ### Set up your own realm @@ -114,7 +115,8 @@ misconfigured, or has not been connected yet. EGRET_ADMIN_EMAIL=ops@example.com EGRET_ADMIN_PASSWORD='a long random password' egret init ``` -- Without `EGRET_ADMIN_PASSWORD`, `init` generates a password and prints it once. Store it in a vault. +- Without `EGRET_ADMIN_PASSWORD` (or with it empty), `init` generates a password and prints it once. Store it in a vault. A + password you supply is never printed: `init` prints `Password: taken from EGRET_ADMIN_PASSWORD (not shown)`. - `init` does nothing when a platform administrator already exists. `egret init --force` adds another one. - On the sign-in page the account is behind **Admin sign-in (break-glass)** when SSO is on. - Local sign-in is rate limited (`rate_limit.login_*`: 5 attempts per 5 minutes, successful attempts included), which also diff --git a/platform/deploy-operate/configuration.md b/platform/deploy-operate/configuration.md index 66c4b9b..86938d8 100644 --- a/platform/deploy-operate/configuration.md +++ b/platform/deploy-operate/configuration.md @@ -35,7 +35,9 @@ does not: run `set -a; source .env; set +a` first. The server does not read `.en The server validates the result before it listens. A missing or malformed required value stops the process with one line naming the key; see [Startup checks](../reference/configuration-keys#startup-checks). Unknown keys are ignored, so a -misspelled variable leaves the default in place without any message. +misspelled variable leaves the default in place, but the server logs a `WARN` for each one at start, with the closest real +key (`ignored environment variable EGRET_SERVER__PORTT: … (did you mean EGRET_SERVER__PORT?)`). The warning never stops the +start. ## The keys you set first diff --git a/platform/deploy-operate/kubernetes.md b/platform/deploy-operate/kubernetes.md index 5999e13..21b27a3 100644 --- a/platform/deploy-operate/kubernetes.md +++ b/platform/deploy-operate/kubernetes.md @@ -40,7 +40,7 @@ Optional objects appear with these values: | When | Extra objects | |---|---| -| `oidc.enabled=true` | `Deployment/keycloak`, `Service/keycloak`, `ConfigMap/keycloak-realm` (the realm file). | +| `oidc.enabled=true` and `oidc.devRealm=true` | `Deployment/keycloak`, `Service/keycloak`, `ConfigMap/keycloak-realm` (the realm file). | | `ingress.enabled=true` | `Ingress/egret` (routes `/auth`, `/data-api`, `/oauth`, `/.well-known/*` and `/`) and `Ingress/egret-catalog`. `/service` (the policy API) is not routed. | | `sharedStore.enabled=true` | `Deployment/valkey` and `Service/valkey`. | | `workflow.containerRunner=kubernetes` | `Namespace/egret-tasks` (Pod Security `restricted`), `Role` and `RoleBinding` `egret-task-runner`, `NetworkPolicy/egret-task-isolation`. | @@ -61,7 +61,8 @@ just deploy-kind # builds egret-backend, -frontend, -renderer and -apps imag 10m` and then restarts the pods, because a rebuilt `:dev` image with an unchanged tag is otherwise ignored. The web app is at [http://localhost:30080](http://localhost:30080); sign in as `admin@egret.local` with the password from `values.yaml` (`admin.password`). -For single sign-on and an ingress, `just deploy-kind-oidc` adds `--set oidc.enabled=true --set ingress.enabled=true`. It +For single sign-on and an ingress, `just deploy-kind-oidc` adds `--set oidc.enabled=true --set oidc.devRealm=true --set +ingress.enabled=true` (`just deploy-kind-governance` sets the same two OIDC values). It needs the ingress controller and the DNS rewrite recipes (`just ingress-install`, `just coredns-rewrite`) first. Tear down with `just kind-undeploy` (the release only) or `just k8s-down` (the cluster). @@ -128,7 +129,8 @@ with more than one replica and no shared store, so the guard holds outside the c | Value | Default | Meaning | |---|---|---| | `admin.email`, `admin.password` | `admin@egret.local`, `Admin12345!` | The break-glass administrator `egret init` creates. **Change the password.** | -| `oidc.enabled` | `false` | Turn on SSO against the **bundled Keycloak** (`oidc.realm`, `clientId`, `clientSecret`, `keycloakImage`, `keycloakAdmin`, `keycloakAdminPassword`). The bundled Keycloak runs in development mode with default credentials. | +| `oidc.enabled` | `false` | Turn on SSO against the **bundled Keycloak** (`oidc.realm`, `clientId`, `clientSecret`, `keycloakImage`, `keycloakAdmin`, `keycloakAdminPassword`). The bundled Keycloak runs in development mode with default credentials. Requires `oidc.devRealm=true`. | +| `oidc.devRealm` | `false` | Confirms that you want the bundled Keycloak with the only realm the chart ships, the **development** realm (`files/egret-realm.json`: no TLS requirement, a `*` redirect URI, the password grant, users with published passwords, client secret `egret-secret`). With `oidc.enabled=true` and this left `false`, the chart refuses to render and names those settings. With it on, the install notes print a warning that anyone who reaches the Keycloak can sign in as a platform admin. Never set it on a cluster other people can reach. | To use your own identity provider, leave `oidc.enabled=false` (so no Keycloak is deployed) and set the `EGRET_OIDC__*` variables on the Deployment yourself, as in [Authentication setup](./authentication). The chart has no values for an diff --git a/platform/deploy-operate/local.md b/platform/deploy-operate/local.md index a0f00bd..5f5df8d 100644 --- a/platform/deploy-operate/local.md +++ b/platform/deploy-operate/local.md @@ -49,11 +49,11 @@ keys](../reference/configuration-keys); the ones that matter for the compose sta | Need | Variable | Value for the compose stack | |---|---|---| | Metadata database | `EGRET_DATABASE__URL` | `postgres://egret:egret@localhost:5432/egret` | -| Object storage | `EGRET_STORAGE__ENDPOINT`, `__BUCKET`, `__ACCESS_KEY`, `__SECRET_KEY` | `http://localhost:9000`, a bucket you create, `egret`, `egretsecret` | +| Object storage | `EGRET_STORAGE__ENDPOINT`, `__BUCKET`, `__ACCESS_KEY`, `__SECRET_KEY` | `http://localhost:9000`, `warehouse` (create it yourself), `egret`, `egretsecret` | | Single sign-on | `EGRET_OIDC__ISSUER_URL`, `__CLIENT_ID`, `__CLIENT_SECRET`, `__REDIRECT_URL` | `http://localhost:8081/realms/egret`, `egret`, `egret-secret`, `http://localhost:5173/api/auth/oidc/callback` | -`.env.example` does not include the `EGRET_STORAGE__*` variables, and the server will not start without them; the Quick -start shows the lines to add. +`.env.example` sets all of these, and a test in the repository keeps it passing the server's startup checks. The bucket +itself is not created for you; the Quick start shows how. The OIDC redirect URL must be on the origin the browser uses. With the Vite dev server that is port 5173, so the session cookie is set for the app and not for port 8080. @@ -73,9 +73,9 @@ EGRET_STORAGE__ACCESS_KEY=egret EGRET_STORAGE__SECRET_KEY=egretsecret \ ``` If you also enable Data Apps (`apps.origin`), give each instance its own `apps.listen`: that listener binds -`0.0.0.0:8081` by default. Changing the port does not change the catalog's advertised address: with the catalog on, also -set `EGRET_CATALOG__PUBLIC_URL` to the new origin, or the token issuer in `/.well-known/openid-configuration` keeps -saying `http://localhost:8080` (observed on an instance started on port 18080). +`0.0.0.0:8081` by default. When `EGRET_CATALOG__PUBLIC_URL` is unset, the catalog's advertised address (and with it the token issuer in +`/.well-known/openid-configuration`) follows the port: `http://localhost:18080` here. Set it yourself when clients reach the +instance under another name. ## StarRocks and Ranger in the compose stack diff --git a/platform/deploy-operate/security-hardening.md b/platform/deploy-operate/security-hardening.md index 94d1057..24307b3 100644 --- a/platform/deploy-operate/security-hardening.md +++ b/platform/deploy-operate/security-hardening.md @@ -30,8 +30,8 @@ that means. Each item says how to check it. Tick them in order. ## 2. The break-glass administrator -- [ ] **Set the password yourself** with `EGRET_ADMIN_PASSWORD` at `egret init` (or let `init` generate one and store it in a - vault). Never keep the Helm default `Admin12345!` (`admin.password`). +- [ ] **Set the password yourself** with `EGRET_ADMIN_PASSWORD` at `egret init`; `init` does not print a supplied password + (or let `init` generate one, printed once, and store it in a vault). Never keep the Helm default `Admin12345!` (`admin.password`). - [ ] **Move day-to-day administration to IdP groups.** Grant `platform_admin` to a group in **Control Plane → Groups**, and keep the local account for recovery. - [ ] **Keep the local sign-in rate limit.** Five attempts per five minutes (`rate_limit.login_*`) is what stands in front of @@ -54,6 +54,8 @@ Egret speaks plain HTTP; terminate TLS in front of it. which is the safe direction. - [ ] **Use HTTPS to the identity provider**, and register only the exact redirect URI. The development realm uses `sslRequired: none` and `*` as a redirect URI. +- [ ] **Do not set `oidc.devRealm=true` in the Helm chart** outside a cluster only you can reach. It is the opt-in that lets + `oidc.enabled=true` install the bundled Keycloak with the development realm; without it the chart refuses to render. - [ ] **Enable TLS on the engines.** Trino enforces JWT only over HTTPS (over HTTP it accepts a valid token but also trusts a bare `X-Trino-User`). StarRocks' per-user JWT login sends the token in clear text with `mysql_clear_password` unless the front end has TLS. diff --git a/platform/deploy-operate/storage-and-catalog.md b/platform/deploy-operate/storage-and-catalog.md index 4d823a6..4a3775a 100644 --- a/platform/deploy-operate/storage-and-catalog.md +++ b/platform/deploy-operate/storage-and-catalog.md @@ -19,6 +19,8 @@ The `[storage]` section is the object store the server itself uses. One bucket h | Data Apps drafts and versions | `apps.storage.prefix` | `apps/` | | The default catalog warehouse | `catalog.warehouse` (`s3://`) | `s3://warehouse` | +The first three prefixes hold other users' data, so **Control Plane → Storage → Browse** and its download close them to everyone except a platform admin ([Storage](../control-plane/storage/index.md#folders-only-a-platform-admin-can-open)). + ```bash EGRET_STORAGE__ENDPOINT=http://localhost:9000 # omit for AWS S3 EGRET_STORAGE__REGION=us-east-1 @@ -59,7 +61,7 @@ EGRET_CATALOG__PUBLIC_URL=http://egret.example.internal:8080 | `catalog.enabled` | Off by default. When on, `warehouse`, `auth_token` and the storage credentials are required. The server logs lakekeeper's own migrations on first start, in the same database. | | `catalog.auth_token` | A static token for catalog administration. Treat it as a secret. | | `catalog.warehouse` | The default catalog's location. | -| `catalog.public_url` | The address clients and engines use to reach this server, without `/catalog`. The default is `http://localhost:8080`, **whatever `server.port` is**: change it when the server is reachable under another name, port or scheme. It is also where the machine identity provider's issuer falls back to. | +| `catalog.public_url` | The address clients and engines use to reach this server, without `/catalog`. Unset, it is `http://localhost:`: change it when the server is reachable under another name or scheme. It is also where the machine identity provider's issuer falls back to. | | `catalog.system_user` | The stable name (`egret-system`) of the principal that bootstraps the catalog's default project. Keep it stable across deploys. | ### What an engine sees diff --git a/platform/deploy-operate/troubleshooting/cannot-sign-in.md b/platform/deploy-operate/troubleshooting/cannot-sign-in.md index 516b5f2..34f9e74 100644 --- a/platform/deploy-operate/troubleshooting/cannot-sign-in.md +++ b/platform/deploy-operate/troubleshooting/cannot-sign-in.md @@ -15,7 +15,7 @@ when the server answers `401 {"error":"unauthorized"}`. **Causes** - The email or password is wrong. The break-glass account is the one `egret init` created; `.env.example` and the Helm chart - default it to `admin@egret.local`, but its password is the one `init` printed, or `EGRET_ADMIN_PASSWORD`, or the chart's + default it to `admin@egret.local`, but its password is the one `init` generated and printed, or the `EGRET_ADMIN_PASSWORD` it was given (never printed), or the chart's `admin.password`. - The account was never created: a fresh database has no administrator until `egret init` runs. In Kubernetes the `egret-init` hook job creates it after install; check `kubectl get job -n egret-system egret-init`. diff --git a/platform/deploy-operate/troubleshooting/catalog-storage-errors.md b/platform/deploy-operate/troubleshooting/catalog-storage-errors.md index 4f4c9e1..2db4ec0 100644 --- a/platform/deploy-operate/troubleshooting/catalog-storage-errors.md +++ b/platform/deploy-operate/troubleshooting/catalog-storage-errors.md @@ -13,7 +13,7 @@ catalog: | Message | Cause and fix | |---|---| -| `Error: the apps API requires [storage] bucket, access_key, secret_key (it stores draft/version content there regardless of apps.pod.launcher)` | The object-store credentials are always required. A copy of `.env.example` does not set them. Set `EGRET_STORAGE__BUCKET`, `EGRET_STORAGE__ACCESS_KEY` and `EGRET_STORAGE__SECRET_KEY`. | +| `Error: the apps API requires [storage] bucket, access_key, secret_key (it stores draft/version content there regardless of apps.pod.launcher)` | The object-store credentials are always required. `.env.example` sets them; check that they were not removed or misspelled (an unknown name logs a `WARN`). Set `EGRET_STORAGE__BUCKET`, `EGRET_STORAGE__ACCESS_KEY` and `EGRET_STORAGE__SECRET_KEY`. | | `Error: catalog.enabled requires warehouse and auth_token` | Set `EGRET_CATALOG__AUTH_TOKEN` (and `EGRET_CATALOG__WAREHOUSE`). | | `Error: catalog.enabled requires storage access_key/secret_key/bucket` | The catalog needs the same storage settings. | | `Error: secrets.master_key must be base64 of >=32 bytes` | `EGRET_SECRETS__MASTER_KEY` is set but does not decode to 32 bytes or more. Generate one with `head -c 32 /dev/urandom \| base64`. | @@ -58,8 +58,8 @@ of a real failure: curl -s http:///.well-known/openid-configuration ``` -If `issuer` or `token_endpoint` names an address clients cannot reach (`http://localhost:8080` on a server that is not on -localhost, or on a different port: reproduced on port 18080), set `EGRET_CATALOG__PUBLIC_URL` to the address engines use, and +If `issuer` or `token_endpoint` names an address clients cannot reach (`http://localhost:`, the default when +`catalog.public_url` is unset, on a server that is not reached as localhost), set `EGRET_CATALOG__PUBLIC_URL` to the address engines use, and `EGRET_IDP__ISSUER` to the address of clients outside the cluster. **Changing the issuer strands StarRocks engines already provisioned**, because their security integration compares the old issuer byte for byte; re-provision them. See [Configuration](../configuration#address-settings-that-people-confuse). diff --git a/platform/deploy-operate/troubleshooting/workflows-stuck.md b/platform/deploy-operate/troubleshooting/workflows-stuck.md index 656f85e..4c16d28 100644 --- a/platform/deploy-operate/troubleshooting/workflows-stuck.md +++ b/platform/deploy-operate/troubleshooting/workflows-stuck.md @@ -11,8 +11,9 @@ sidebar_position: 4 ### Cause: the workflow worker is off The worker that claims and runs tasks is **off by default** (`workflow.worker_enabled = false`). With it off, **Run** works: -Egret creates the execution, shows it as *Running*, and nothing ever advances it. There is no error and the flow editor shows -no warning. +Egret creates the execution, shows it as *Running*, and nothing ever advances it. There is no error, but the flow page, the +**Run** dialog, the **Executions** list and the execution show the notice *The workflow worker is off on this server, so executions are created but do not run. An administrator turns it on with workflow.workerEnabled (EGRET_WORKFLOW__WORKER_ENABLED=true).* The answer to **Run** carries +`"worker_enabled": false`. Reproduced on a server started with the defaults: diff --git a/platform/explore/dashboards/cards.md b/platform/explore/dashboards/cards.md index c275fdf..ed818bc 100644 --- a/platform/explore/dashboards/cards.md +++ b/platform/explore/dashboards/cards.md @@ -70,7 +70,7 @@ A heading is one line of large bold text, drawn without a card background (switc ## Text cards -A text card shows **GitHub-flavoured Markdown**: headings, lists, bold and italic, links, tables, code. Raw HTML is not rendered. In edit mode the card is a plain text box that shows the Markdown source; the rendered result appears in view mode. +A text card shows **GitHub-flavoured Markdown**: headings, lists, bold and italic, links, tables, code. Raw HTML is not rendered. In edit mode the card is a plain text box that shows the Markdown source; the rendered result appears in view mode. Spaces and Enter typed in the box go into the text (Enter starts a new line); Enter or Space pressed on the selected card itself, outside the box, opens its settings. ### Show a filter value in text @@ -89,7 +89,8 @@ How values are written into the text: | No value | Nothing (an empty string) | | One value | The value | | Several values | The values joined with a comma and a space, for example `O, P` | -| A date filter of type `all-options` | `[object Object]` (a known defect; use a different date type if you need the value in text) | +| A date filter | The value as its chip reads it, for example `1995-01-01 – 1996-12-31`, `After 1997-06-01`, `Past 3 months` or `Q2 1997` | +| A number `between` | `min – max`, for example `10 – 20` | An unknown slug is left as typed. Text cards cannot be connected to a column like chart cards; they only read filter values. @@ -130,9 +131,7 @@ Iframe cards have no title, no description, no auto height and cannot host filte ## Limits and known issues -- A space or Enter typed into a heading or text card in edit mode is not inserted; paste the text instead. See [Edit a dashboard](./editing#limits-and-known-issues). - The card description supports plain text only. -- Text cards show `[object Object]` for `all-options` date filters. ## Further reading diff --git a/platform/explore/dashboards/editing.md b/platform/explore/dashboards/editing.md index 9bfd8ba..4673e28 100644 --- a/platform/explore/dashboards/editing.md +++ b/platform/explore/dashboards/editing.md @@ -146,7 +146,6 @@ Shortcuts work when focus is not in a text field, and not with Ctrl, Cmd or Alt ## Limits and known issues -- In edit mode, a space or Enter typed into a **heading** or **text** card is not inserted, because the card treats those keys as "open settings". Paste text that already contains spaces and line breaks. This is recorded in the project backlog. - Scalar cards (a single number) and any chart whose columns are only metrics cannot be connected to a filter in the editor. See [Dashboard filters](./filters#connect-a-filter-to-cards). ## Further reading diff --git a/platform/explore/dashboards/filters.md b/platform/explore/dashboards/filters.md index 428b0cf..7fd2305 100644 --- a/platform/explore/dashboards/filters.md +++ b/platform/explore/dashboards/filters.md @@ -64,13 +64,11 @@ Shown for `text`, `number` and `date` filters. It decides how the value becomes | `date` `relative` | the last 7, 30, 90 or 365 days up to today | | `date` `month-year` | the whole chosen month | | `date` `quarter-year` | the whole chosen quarter of a year | -| `date` `all-options` | a relative window (past, next or current n days, weeks, months, quarters, years) or a specific date (on, before, after, between). See the warning below. | +| `date` `all-options` | a relative window (past, next or current n days, weeks, months, quarters, years) or a specific date (on, before, after, between). | Dates are compared as calendar dates in UTC. -:::warning -A viewer's choice in an `all-options` date filter is not kept: the picker writes it to the link as a JSON object, and Egret reads that back as plain text, so the chip shows no value and the cards are not filtered. Only a **default value** set by the editor works. Use `range`, `relative`, `single`, `month-year` or `quarter-year` for dates that viewers change. -::: `id` and `category` filters match any of the chosen values (`in`); an `id` filter with one value matches that value exactly. A `boolean` filter offers **Any**, **True** and **False**. A `location` filter with `is-not` excludes the chosen values. +`id` and `category` filters match any of the chosen values (`in`); an `id` filter with one value matches that value exactly. A `boolean` filter offers **Any**, **True** and **False**. A `location` filter with `is-not` excludes the chosen values. A `temporal_unit` filter does not filter rows. It changes how a date column on the connected card is grouped, for example from month to quarter. It works only on charts whose date column is already grouped by a time unit. @@ -109,7 +107,7 @@ Shown for `category`, `location` and `id` filters, except when the widget is **I The value the filter has when a viewer opens the dashboard without a value in the link. Set it with the same widget viewers use; **Clear** removes the default. -A viewer cannot clear a filter that has a default. Pressing the **✕** on its chip removes the value from the link, and the default comes back at once. Change the value instead, or ask an editor to remove the default. +A viewer can clear a filter that has a default with the **✕** on its chip; the filter then stays empty after a reload and in a shared link. A **required** filter cannot be emptied: its **✕** returns it to the default. **Reset all filters** puts every filter back to its default ([Use filters and URL parameters](./using-filters#reset-all-filters)). ### Limit this filter's choices @@ -176,8 +174,6 @@ If the header is too narrow for the title and filters, the filters collapse into - The **Search box** widget is the same as **Dropdown**. - The `text` and `number` types accept a single value only; a "Multiple values" choice is not offered. -- A filter with a default cannot be cleared by a viewer ([above](#default-value)). -- `all-options` date filters do not keep a value chosen by a viewer, and their `{{slug}}` prints `[object Object]` in text cards. - Limiting a filter's choices is described from the product source. It has not been verified end to end on the demo data, where every priority occurs with every status. ## Further reading diff --git a/platform/explore/dashboards/using-filters.md b/platform/explore/dashboards/using-filters.md index 156aed9..cc4bfb7 100644 --- a/platform/explore/dashboards/using-filters.md +++ b/platform/explore/dashboards/using-filters.md @@ -14,7 +14,7 @@ When you open a dashboard that has filters, you see them as chips above the card 2. Choose a value. With **Auto-apply filters** on, the cards reload at once. 3. Click outside the popover, or press Esc, to close it. -A chip with a value is shaded and shows `Name: value`; a chip without a value shows only its name. A required filter has an asterisk. The **✕** at the right of a chip clears its value. +A chip with a value is shaded and shows `Name: value`; a chip without a value shows only its name. A required filter has an asterisk. The **✕** at the right of a chip clears its value. Clearing a filter that has a default leaves it empty, and it stays empty after a reload or in a shared link; clearing a required filter returns it to its default, because its cards cannot run without a value. The widget depends on the filter's type. @@ -56,7 +56,7 @@ The **Auto-apply filters** switch at the right of the filter bar decides when a ## Reset all filters -Open **⋯** in the dashboard header and choose **Reset all filters**. It removes the values from the link. The item is shown only when the link carries at least one value; a filter that shows only its default value does not count. +Open **⋯** in the dashboard header and choose **Reset all filters**. It puts every filter back to its default value (and empties the filters that have none) in one step, and also drops changes you have not applied yet. The item is shown only when at least one filter differs from its default, which includes a default you cleared. A link that names exactly the default value does not count as a change. ## URL parameters @@ -72,11 +72,13 @@ Egret writes the address with the usual percent-encoding; the example is decoded |---|---| | One value | The text itself: `priority=1-URGENT` | | Several values, or a pair (number `between`, date `range`) | A JSON array: `status=["O","P"]` | -| No value | The parameter is absent | +| A value chosen in an `all-options` date filter | A JSON object, as the date picker produces it | +| A cleared default | The slug with an empty value: `order_date=` | +| No value, on a filter without a default | The parameter is absent | The rules: -- A parameter in the link beats the filter's default. A filter without a parameter takes its default value, if it has one. +- A parameter in the link beats the filter's default, and an empty parameter (`order_date=`) means the default was cleared. A filter without a parameter takes its default value, if it has one. - Changing a filter updates the address without adding a history entry, so the browser's Back button leaves the dashboard. - Unknown parameters are ignored. A value that is not valid for the type produces an empty or failing condition, not an error message. - Two other parameters control viewing: `refresh=` and `fullscreen=true` ([Fullscreen and auto refresh](./view-options)). @@ -85,8 +87,7 @@ The rules: ## Limits and known issues -- A filter that has a default cannot be cleared with its **✕**; the default returns. -- A viewer's value in an `all-options` date filter is lost ([Dashboard filters](./filters#operator)). +- A required filter cannot be emptied; its **✕** returns it to the default. - Auto-apply is stored in the browser, so it is not shared and is lost when you clear site data. ## Further reading diff --git a/platform/explore/sql-lab/index.md b/platform/explore/sql-lab/index.md index 5612254..85db5e6 100644 --- a/platform/explore/sql-lab/index.md +++ b/platform/explore/sql-lab/index.md @@ -92,7 +92,7 @@ Because tabs live in the browser, clearing site data or switching browsers gives ## Limits and known issues -- SQL Lab needs at least one database connection granted to the workspace. Without one the database picker is empty and **Run** reports *Pick a database*. +- SQL Lab needs at least one database connection granted to the workspace. Without one the database picker is empty, the Explorer says *No connection is granted to this workspace. Ask a workspace admin.*, and **Run** reports *Pick a database*. - The row limit, the 60-second query timeout and the 100,000-row ceiling are server settings; see [Query results](./results.md#limits). - Cancelling a query leaves the Results panel empty without saying the query was cancelled. See the [editor page](./editor.md#run-and-cancel). - On Trino, an `ORDER BY` without a `LIMIT` can be lost, because Egret wraps the query to apply the row limit. See [Row limit](./editor.md#row-limit). diff --git a/platform/explore/sql-lab/permissions.md b/platform/explore/sql-lab/permissions.md index 2d2be21..dc9f776 100644 --- a/platform/explore/sql-lab/permissions.md +++ b/platform/explore/sql-lab/permissions.md @@ -45,9 +45,17 @@ What this means in practice: The database picker lists only the connections that are granted to the workspace; see [Databases](../../workspace-settings/databases). A request for any other database is refused with *database not granted to workspace*. -:::warning Only workspace admins see the picker's list -The picker is filled from `GET /api/ws/{ws}/databases`, which answers 403 to everyone who is not a workspace admin. A member with `ws_editor`, `ws_viewer` or a custom role that has `query:run` therefore sees an empty picker ("Select a database") even though the connection is granted, and nothing says why. Queries sent through the API with a `database_id` do run. See [Known issues](../../known-issues) (KI-159). -::: +The picker is filled from `GET /api/ws/{ws}/databases`. What that list holds depends on your role in the workspace: + +| You hold | You get | +|---|---| +| Workspace admin (`workspace:manage_members`) | Every granted connection with its full settings | +| `query:run` or `dataset:read` (for example `ws_editor`, `ws_viewer` or a custom role) | Every granted connection, with only its id, name, engine and whether it uses per-user delegation. Host, port, options, secret and status are left out. | +| Neither | A 403 | + +Creating, editing, deleting, testing and opening one connection stay with workspace admins. + +When no connection is granted to the workspace, the Explorer says *No connection is granted to this workspace. Ask a workspace admin.* If the list is refused (403), it says *You can't list this workspace's connections. Ask a workspace admin for a role that can run queries.* ## What the engine allows diff --git a/platform/faq.md b/platform/faq.md index ce05841..f98789a 100644 --- a/platform/faq.md +++ b/platform/faq.md @@ -60,13 +60,14 @@ with a dataset, charts and a dashboard on Trino's `tpch` sample data. Sign in at ### Why does the server not start after I copied `.env.example`? -`.env.example` does not set the object-store credentials, and the server requires them even if you never use object storage: -`the apps API requires [storage] bucket, access_key, secret_key`. Add `EGRET_STORAGE__BUCKET`, `EGRET_STORAGE__ACCESS_KEY` and -`EGRET_STORAGE__SECRET_KEY`. See the [Quick start](./get-started/quick-start). +A copy of `.env.example` passes the server's startup checks (a test in the repository keeps it so). If the server still +refuses with `the apps API requires [storage] bucket, access_key, secret_key`, a variable was removed or misspelled: the log +shows a `WARN` with *ignored environment variable …* for a name it does not know. Also check that the dependencies run +(`just up`) and that the `warehouse` bucket exists. See the [Quick start](./get-started/quick-start). ### What is the administrator password? -The one `egret init` printed, or the one you set with `EGRET_ADMIN_PASSWORD`. The Helm chart's default is `Admin12345!` and +The one `egret init` generated and printed, or the one you set with `EGRET_ADMIN_PASSWORD` (`init` does not print that one). The Helm chart's default is `Admin12345!` and you should change it. On the demo stack it is `Admin12345!` for `admin@egret.local`. See [First login tour](./get-started/first-login-tour). ### Where do I register a database? @@ -124,8 +125,8 @@ as `SUM(revenue)`. See [Your first dashboard](./get-started/first-dashboard#4-ad ### My workflow execution stays in *Running* forever. Why? -The workflow worker is off by default. Set `EGRET_WORKFLOW__WORKER_ENABLED=true`, and `EGRET_WORKFLOW__SCHEDULER_ENABLED=true` -for schedules and triggers. See [Workflows stuck](./deploy-operate/troubleshooting/workflows-stuck). +The workflow worker is off by default; the flow page, the Run dialog and the execution then say *The workflow worker is off on +this server*. Set `EGRET_WORKFLOW__WORKER_ENABLED=true`, and `EGRET_WORKFLOW__SCHEDULER_ENABLED=true` for schedules and triggers. See [Workflows stuck](./deploy-operate/troubleshooting/workflows-stuck). ## Operating diff --git a/platform/get-started/quick-start.md b/platform/get-started/quick-start.md index c829f4e..6f32fd8 100644 --- a/platform/get-started/quick-start.md +++ b/platform/get-started/quick-start.md @@ -43,28 +43,15 @@ Docker Compose](../deploy-operate/local) page describes each service. Never reuse it anywhere else; see the [security checklist](../deploy-operate/security-hardening). ::: -## Add the settings `.env.example` leaves out +## Create the bucket -The server refuses to start unless the object-store credentials are set, even if you never use object storage: +The server refuses to start unless the object-store credentials are set, even if you never use object storage. +`.env.example` sets them (`EGRET_STORAGE__BUCKET=warehouse` and the credentials the compose file gives RustFS), and a test +in the repository keeps the file passing the server's startup checks. It also sets `EGRET_SECRETS__MASTER_KEY`, which +encrypts the credentials you store (for example a database password). Like the secret key above, that value is public: +replace it with `head -c 32 /dev/urandom | base64` for anything but your laptop. -```text -the apps API requires [storage] bucket, access_key, secret_key (it stores draft/version content there ...) -``` - -`.env.example` sets `AWS_*` variables but not the `EGRET_STORAGE__*` ones the server reads. Append these to `.env`. The -credentials are the ones the compose file gives RustFS. - -```bash -{ - echo "EGRET_STORAGE__BUCKET=warehouse" - echo "EGRET_STORAGE__ACCESS_KEY=egret" - echo "EGRET_STORAGE__SECRET_KEY=egretsecret" - echo "EGRET_SECRETS__MASTER_KEY=$(head -c 32 /dev/urandom | base64)" -} >> .env -``` - -`EGRET_SECRETS__MASTER_KEY` encrypts the credentials you store (for example a database password). Without it, saving a -connection **with a secret** fails. The bucket does not exist yet: nothing creates it. Create it in the RustFS console at +The bucket does not exist yet: nothing creates it. Create it in the RustFS console at [http://localhost:9001](http://localhost:9001) (user `egret`, password `egretsecret`), or with any S3 client pointed at `http://localhost:9000`: ```bash @@ -88,10 +75,12 @@ cargo run -p egret-bin -- init ```text Bootstrap admin: admin@egret.local Password: +(generated; shown only this once) Default workspace: default ``` -Copy the password. To choose it yourself, set `EGRET_ADMIN_PASSWORD` (and `EGRET_ADMIN_EMAIL`) before running `init`. +Copy the password. To choose it yourself, set `EGRET_ADMIN_PASSWORD` (and `EGRET_ADMIN_EMAIL`) before running `init`; `init` +then prints `Password: taken from EGRET_ADMIN_PASSWORD (not shown)` instead of the value. `init` does nothing if a platform administrator already exists; add `--force` to create another one. ```bash diff --git a/platform/known-issues.md b/platform/known-issues.md index c0e79a0..8f55acf 100644 --- a/platform/known-issues.md +++ b/platform/known-issues.md @@ -24,7 +24,7 @@ and missing pieces. Entries are **recorded, not fixed**; they feed the product r | KI-010 | Dashboards | Docs | S3 | The dashboard grid is 24 columns wide; this is not stated anywhere in the UI or API reference. | | KI-011 | Databases | UX | S3 | A newly created connection shows status *Unknown* until **Test** is pressed, even though creation already validated reachability. *Docker demo stack (Oct 2026):* after **Test** succeeds the row shows a green **OK**, but the **Status** column stays *Unknown* and the counter *0 ok* until the page is reloaded, although the API already answers `status: ok`; the counter also reads "1 connections". *Steps:* Workspace settings → Engines → **Test** on `trino-demo`. (Reported by the start area.) | | KI-012 | Workspace settings | UX | S3 | The Notifications page does not follow the layout of the other settings tabs: the heading is offset, and Encryption is an unstyled native select. | -| KI-013 | Workflows | UX | S1 | With the default config the workflow worker is **off**. *Run* succeeds and the execution sits in *Running* indefinitely (one waited 53 s until the worker was enabled). Only the Triggers page warns; Run, Flow and Executions give no hint. *Docker demo stack (Oct 2026):* still true for the default configuration; the demo stack turns the worker and scheduler on (`EGRET_WORKFLOW__WORKER_ENABLED=true`, `EGRET_WORKFLOW__SCHEDULER_ENABLED=true`), so the symptom is not seen there. | +| KI-013 | Workflows | UX | S1 | With the default config the workflow worker is **off**. *Run* succeeds and the execution sits in *Running* indefinitely (one waited 53 s until the worker was enabled). Only the Triggers page warns; Run, Flow and Executions give no hint. *Docker demo stack (Oct 2026):* still true for the default configuration; the demo stack turns the worker and scheduler on (`EGRET_WORKFLOW__WORKER_ENABLED=true`, `EGRET_WORKFLOW__SCHEDULER_ENABLED=true`), so the symptom is not seen there. *Status:* fixed in platform #716 (October 2026): with the worker off, the flow page, the Run dialog, the Executions list and an unfinished execution show a notice naming `workflow.workerEnabled` / `EGRET_WORKFLOW__WORKER_ENABLED=true`, and the Run answer, the flow and the executions carry `worker_enabled`. | | KI-014 | Workflows | UX | S3 | A new flow shows *Version —* in its header, and *Run* says it runs "the saved draft"; it is unclear what a version is until **Create version** is pressed. | | KI-015 | Governance | Missing | S2 | **Data Lineage** is in the sidebar but is a placeholder page with no functionality. | | KI-016 | Control Plane | Defect | S3 | `/control-plane/config` renders a completely blank page inside the shell. *Docker demo stack (Oct 2026):* no longer blank; it now redirects without end to `/control-plane/config/workspaces/workspaces/workspaces/…`. | @@ -37,7 +37,7 @@ and missing pieces. Entries are **recorded, not fixed**; they feed the product r ## Findings from the detailed pages -Collected while writing the detailed pages for each area (October 2026), checked against the running demo instance and the product source. KI-154 to KI-213 come from re-verifying every area on the Docker demo stack (`just demo-up`, October 2026), which also closed or updated older rows (marked *Status* or *Docker demo stack*). KI-214 to KI-235 come from a whole-app UI consistency sweep of 78 screens (admin, light theme, 1440×900, same stack), then the dark theme, 768 and 390 px widths, the editors and a non-admin member. Same legend as above. **Open S1 items:** KI-059, KI-060, KI-112, KI-136, KI-159, KI-187 (KI-040 and KI-175 are closed). +Collected while writing the detailed pages for each area (October 2026), checked against the running demo instance and the product source. KI-154 to KI-213 come from re-verifying every area on the Docker demo stack (`just demo-up`, October 2026), which also closed or updated older rows (marked *Status* or *Docker demo stack*). KI-214 to KI-235 come from a whole-app UI consistency sweep of 78 screens (admin, light theme, 1440×900, same stack), then the dark theme, 768 and 390 px widths, the editors and a non-admin member. KI-236 to KI-246 come from the Wave 1 fixes of the Known-issues fix programme (October 2026), which also closed the rows marked *fixed in platform #…*. Same legend as above. **Open S1 items:** none (KI-013, KI-040, KI-059, KI-060, KI-112, KI-136, KI-159, KI-175 and KI-187 are closed). ### Data API, Alerts and Apps @@ -111,11 +111,11 @@ Collected while writing the detailed pages for each area (October 2026), checked | ID | Area | Type | Sev | Description | |---|---|---|---|---| -| KI-059 | dashboards / edit | Bug | S1 | A space or Enter typed into a heading or text card in edit mode is not inserted. The card wrapper (`role="button"`) handles Enter and Space with `preventDefault` and the keydown bubbles up from the input/textarea, so "Weekly sales review" becomes "Weeklysalesreview" and a two-line text becomes one word. Paste works. *Steps:* Dashboard → Edit → text or heading menu → type "a b". Source: `DashboardView.tsx` dashcard-body `onKeyDown`. | -| KI-060 | dashboards / filters | Bug | S1 | A value chosen in a `date` / `all-options` filter is lost. The picker emits an object, the URL stores it as JSON, and `useDashboardParams` reads it back as a plain string (only arrays and `between`/`range` are parsed), so the chip shows no value and the cards are not filtered. Only the editor's default works. *Steps:* Sales performance on a dashboard with an all-options date filter: open the chip → Specific → After 1997-06-01; chip stays "Order date", reload does not restore it. | -| KI-061 | dashboards / filters | Bug | S2 | A filter that has a default value cannot be cleared by a viewer: the chip's X removes the URL parameter, then the default is applied again. "Reset all filters" is offered only when the URL has a parameter, so defaults never count. *Steps:* Sales performance → X on "Order date: 1995-01-01 – 1996-12-31" → chip returns. | +| KI-059 | dashboards / edit | Bug | S1 | A space or Enter typed into a heading or text card in edit mode is not inserted. The card wrapper (`role="button"`) handles Enter and Space with `preventDefault` and the keydown bubbles up from the input/textarea, so "Weekly sales review" becomes "Weeklysalesreview" and a two-line text becomes one word. Paste works. *Steps:* Dashboard → Edit → text or heading menu → type "a b". Source: `DashboardView.tsx` dashcard-body `onKeyDown`. *Status:* fixed in platform #711 (October 2026): spaces and Enter typed into a heading or text card in edit mode stay in the text; Enter or Space pressed on the card itself still opens its settings. | +| KI-060 | dashboards / filters | Bug | S1 | A value chosen in a `date` / `all-options` filter is lost. The picker emits an object, the URL stores it as JSON, and `useDashboardParams` reads it back as a plain string (only arrays and `between`/`range` are parsed), so the chip shows no value and the cards are not filtered. Only the editor's default works. *Steps:* Sales performance on a dashboard with an all-options date filter: open the chip → Specific → After 1997-06-01; chip stays "Order date", reload does not restore it. *Status:* fixed in platform #711 (October 2026): a value chosen in an `all-options` date filter is read back from the link, so the chip and the cards keep it after a reload and in a shared URL. | +| KI-061 | dashboards / filters | Bug | S2 | A filter that has a default value cannot be cleared by a viewer: the chip's X removes the URL parameter, then the default is applied again. "Reset all filters" is offered only when the URL has a parameter, so defaults never count. *Steps:* Sales performance → X on "Order date: 1995-01-01 – 1996-12-31" → chip returns. *Status:* fixed in platform #711 (October 2026): a viewer can clear a filter's default, the clear is written as `?slug=` with an empty value and survives a reload, a required filter cannot be emptied, and **Reset all filters** is offered whenever a filter differs from its default and returns to the defaults. | | KI-062 | dashboards / filters | Gap | S2 | The editor can only connect a filter to a chart's own dimensions and plain columns (`columnsForParam`). Single-number (scalar), gauge and other metric-only cards offer "This card doesn't have any fields that can be mapped", and a time-series cannot be filtered by a column that is not one of its dimensions. The server accepts such mappings (API-created ones work). The column list is also not narrowed by filter type. *Steps:* Edit Sales performance → select a filter → KPI cards show the "no fields" message. | -| KI-063 | dashboards / cards | Bug | S3 | Text and heading cards print `[object Object]` for a `{{slug}}` bound to an all-options date filter (`interpolateParams` stringifies the object). *Steps:* Add `{{order_date}}` to a text card on a dashboard whose date filter is all-options. | +| KI-063 | dashboards / cards | Bug | S3 | Text and heading cards print `[object Object]` for a `{{slug}}` bound to an all-options date filter (`interpolateParams` stringifies the object). *Steps:* Add `{{order_date}}` to a text card on a dashboard whose date filter is all-options. *Status:* fixed in platform #711 (October 2026): text and heading cards print date values the way the filter chip reads them (`After 1997-06-01`, `1995-01-01 – 1996-12-31`); no value shape prints `[object Object]`. | | KI-064 | dashboards / cards | UX | S3 | The card **Title** box says "Blank = chart name", but after typing and clearing it the card stores `title: ""`, which hides the caption, the description icon and the title link. *Steps:* Edit → click a chart card → type "x" in Title, clear it → caption disappears. | | KI-065 | dashboards / filters | UX | S3 | A filter's slug (used in URLs and `{{slug}}`) is assigned once as `filter`, `filter_2`, … and is neither shown nor editable; renaming the filter does not change it. *Steps:* Add three filters and rename them; URL parameters stay `filter`, `filter_2`, `filter_3`. | | KI-066 | dashboards / filters | UX | S3 | The **Search box** widget option has no effect: `FilterControl` renders the same list with a search input for Dropdown and Search box. *Steps:* Filter settings → How should people filter on this? | @@ -173,7 +173,7 @@ Collected while writing the detailed pages for each area (October 2026), checked | KI-111 | Datasets | UX | S3 | Trashing a dataset hides it but its charts keep running; nothing tells the owner of those charts. *Steps:* Trash a dataset with a chart; the chart still returns rows. | | KI-154 | SQL Lab | Bug | S2 | `ORDER BY` is ignored on Trino: Egret wraps a single SELECT as `SELECT * FROM () AS _egret_lim LIMIT n`, and Trino drops the ordering of a subquery, so rows come back in a different, unsorted order on each run. Hits the first-dashboard tutorial query. *Workaround:* add your own `LIMIT` after `ORDER BY` (then the order holds). *Fix idea:* append the limit instead of wrapping, or carry the `ORDER BY` outside. *Steps:* `SELECT orderstatus, count(*) AS orders, round(sum(totalprice), 2) AS revenue FROM tpch.tiny.orders GROUP BY 1 ORDER BY 3 DESC` five times: P,F,O / F,P,O / P,O,F; with `LIMIT 1000` appended, F,O,P every time. `system.runtime.queries` shows the wrapped text. (Reported by the start and sql-charts areas.) | | KI-158 | Datasets | UX | S3 | The dataset header says "1 metrics" (no singular). *Steps:* dataset "Order priority summary" after adding one metric. | -| KI-159 | SQL Lab / Workspace settings | Bug | S1 | Only workspace admins can list a workspace's connections. Members with `ws_editor` (direct or through a group), `ws_viewer` or a custom role with `query:run` get 403 on `GET /api/ws/{ws}/databases` (`registry/routes.rs:109`), so their SQL Lab database picker is empty ("Select a database"), also for a provisioned engine assigned to the workspace, and nothing tells them why. Queries sent through the API with a `database_id` do run. As a result a delegated (per-user) query by a non-admin through the UI is still unproven. *Workaround:* use the API with `database_id`, or make the user a workspace admin. *Steps:* `bao.tran` (ws_editor) or `member-sso` in Docs sandbox → SQL Lab → Select a database: no options, console 403; ws_admin `alice.nguyen` sees two. (Reported by the start and admin areas.) | +| KI-159 | SQL Lab / Workspace settings | Bug | S1 | Only workspace admins can list a workspace's connections. Members with `ws_editor` (direct or through a group), `ws_viewer` or a custom role with `query:run` get 403 on `GET /api/ws/{ws}/databases` (`registry/routes.rs:109`), so their SQL Lab database picker is empty ("Select a database"), also for a provisioned engine assigned to the workspace, and nothing tells them why. Queries sent through the API with a `database_id` do run. As a result a delegated (per-user) query by a non-admin through the UI is still unproven. *Workaround:* use the API with `database_id`, or make the user a workspace admin. *Steps:* `bao.tran` (ws_editor) or `member-sso` in Docs sandbox → SQL Lab → Select a database: no options, console 403; ws_admin `alice.nguyen` sees two. (Reported by the start and admin areas.) *Status:* fixed in platform #712 (October 2026): a member holding `query:run` or `dataset:read` gets the workspace's granted connections from `GET /api/ws/{ws}/databases`, cut down to id, name, engine and delegated; admins keep the full rows, everyone else gets 403, and create, edit, delete, detail and test stay admin only. SQL Lab says *No connection is granted to this workspace. Ask a workspace admin.* for an empty list. Follow-ups: KI-243, KI-244, KI-245. | | KI-160 | SQL Lab | UX | S3 | Explorer search lists same-named tables from different schemas in no fixed order (`tpch.sf1000`, `tpch.sf300`, `tpch.sf100000`, `tpch.sf1`, …, `tpch.tiny`), so finding the `tiny` copy means reading every row. Sort by catalog.schema after the name. *Steps:* Explorer → Search tables… → `customer`. | | KI-161 | SQL Lab | UX | S3 | Singular is not handled: the status badge and counter read *Done in 1.1 s · 1 rows* and *1 rows*. *Steps:* `SELECT count(*) FROM tpch.tiny.orders`, or Explain query. | | KI-162 | SQL Lab | UX | S3 | The quick **Chart** view of a result uses fractional ticks for counts (`1,833.3`, `3,666.5`, …) and rotates one-letter X labels by 45°. *Steps:* run the order-status query of KI-154, press **Chart**. | @@ -184,7 +184,7 @@ Collected while writing the detailed pages for each area (October 2026), checked | ID | Area | Type | Sev | Description | |---|---|---|---|---| -| KI-112 | workflows | Bug | S1 | SECRET inputs are stored and returned in clear text (execution API, Overview tab). *Steps:* Run a flow with a SECRET input via API; open Overview: value visible. | +| KI-112 | workflows | Bug | S1 | SECRET inputs are stored and returned in clear text (execution API, Overview tab). *Steps:* Run a flow with a SECRET input via API; open Overview: value visible. *Status:* fixed in platform #719 (October 2026): a `SECRET` input is masked (`******`) in every API and UI read and in the run logs, stored encrypted with `secrets.master_key` (a start without a master key is refused with 400), and replay uses the stored value. Task outputs and rendered values are not masked: KI-246. | | KI-113 | workflows | Gap | S2 | Flow-level `finally:` and `afterExecution:` are accepted without warning but never run (topology shows "Finally · Not run"). *Steps:* docs.demo/flaky-import execution. | | KI-114 | workflows | Gap | S2 | `concurrency`, `checks`, `sla`, `quotas` only warn; two runs of a `limit: 1` flow ran together. *Steps:* docs.demo/one-at-a-time. | | KI-115 | workflows | Gap | S2 | `trigger.*` (date, body) unreadable in task expressions; `{{ trigger.date }}` in a scheduled flow fails with "undefined value". *Steps:* nightly-heartbeat first runs. | @@ -197,7 +197,7 @@ Collected while writing the detailed pages for each area (October 2026), checked | KI-122 | workflows | UX | S3 | Schedule `inputs` with `{{ }}` and bad cron only fail at the scheduler (Last error), not on save. *Steps:* Triggers drawer. | | KI-123 | workflows | UX | S3 | Flow header shows "Active" for a never-published draft. *Steps:* Flow page. | | KI-124 | workflows | Gap | S3 | `errorLogs()`, `render()`, `max()`, `min()`, urlencode/base64 filters missing. *Steps:* expression-probe flow. | -| KI-187 | workflows | Bug | S1 | One long task stalls every workflow on the replica: the claim loop runs the task runs it claimed one after another (`tick_with` awaits `run_one` per run) and only then ticks again, so a `Sleep` or any slow task holds up every other execution until it ends. Kestra runs worker tasks concurrently. *Steps:* run `docs.ops/slow-export` (Sleep) and in the same second a backfill of `docs.demo.sales/daily-revenue`: the four `Write` tasks claimed in the same batch stayed *Running* for 2 min 13 s and new `Log` tasks were not claimed until the Sleep was killed. | +| KI-187 | workflows | Bug | S1 | One long task stalls every workflow on the replica: the claim loop runs the task runs it claimed one after another (`tick_with` awaits `run_one` per run) and only then ticks again, so a `Sleep` or any slow task holds up every other execution until it ends. Kestra runs worker tasks concurrently. *Steps:* run `docs.ops/slow-export` (Sleep) and in the same second a backfill of `docs.demo.sales/daily-revenue`: the four `Write` tasks claimed in the same batch stayed *Running* for 2 min 13 s and new `Log` tasks were not claimed until the Sleep was killed. *Status:* fixed in platform #716 (October 2026): the worker runs the task runs it claims concurrently, up to `workflow.max_concurrent_runs` (default 8), so one long `Sleep` no longer holds up other executions. Related: KI-237. | | KI-188 | workflows | Gap | S3 | The webhook route (`/workflows/v1/{ws}/executions/webhook/...`) is not forwarded by the shipped entry points: the frontend nginx has no `/workflows/` location and the Helm ingress sends `/` to the frontend, so with webhooks enabled a call reaches the SPA (`GET` 200 HTML, `POST` 405 from nginx). The drawer does not tell the operator to add the route. *Steps:* `curl -X POST http://egret.localtest.me:8080/workflows/v1/default/executions/webhook/docs.demo/order-webhook/orders-demo-key` → 405. | | KI-189 | workflows | UX | S3 | A schedule that does not fire still shows a **Next run**: a switched-off one shows a time in the past ("6 minutes ago"), one with `disabled: true` in its YAML shows "in 5 days". It should show a dash. *Steps:* Triggers page, `every_minute` (docs.demo/nightly-heartbeat, off) and `weekly` (docs.ops/purge-old-logs). | | KI-190 | workflows | UX | S3 | The triggers table is wider than the content area at 1440 px: on the **Triggers** page **Backfill** and **Details** are off-screen (sideways scroll, no hint), on a flow's **Triggers** tab **Details** is cut to "Deta", and the *Disabled in the flow's YAML* note is clipped. *Steps:* Workflows → Triggers at 1440×900. | @@ -213,7 +213,7 @@ Collected while writing the detailed pages for each area (October 2026), checked | KI-126 | Workspace settings / Notifications | Bug | S2 | Saving SMTP settings returns HTTP 500 "internal error" on this instance (the Notifications form shows "internal error"), so the saved state, "Send test message" and the Slack save could not be verified. *Steps:* `PUT /api/ws/docs-sandbox/notifications/smtp` with host smtp.example.com, port 587, from alerts@example.com *Status:* closed on the Docker demo stack (Oct 2026): with `EGRET_SECRETS__MASTER_KEY` set, SMTP to Mailpit (`mailpit:1025`, no encryption) saves, **Send test message** answers *Test message sent…* and mail arrives; the Slack save answers *Slack settings saved.* | | KI-127 | Workspace settings / Roles | UX | S3 | A custom role that inherits shows only its own permissions ("2 permissions") on the workspace Roles & permissions page, while the Control Plane role page counts 8 (including inherited). `data_api_invoker` is missing from the workspace list but is offered in the Members role picker. *Steps:* /explore/docs-sandbox/settings/roles vs /control-plane/roles/<finance_reporter> | | KI-128 | Workspace settings / Members | UX | S3 | Typing a partial email ("member-sso") in Add member gives no suggestions and the add silently does nothing visible; only the exact email works. *Steps:* /explore/docs-sandbox/settings/members → Add member *Status:* closed on the Docker demo stack (Oct 2026): typing `emma` in Add member suggests *Emma Schmidt emma.schmidt@egret.local*. | -| KI-129 | Control plane / Storage | Bug (unverified) | S3 | On a connection created with Key prefix `sandbox`, the Browse tab lists the bucket root, not the prefix. *Steps:* Storage → sandbox-lake → Browse *Docker demo stack (Oct 2026):* still true (`sandbox-lake`, prefix `sandbox/`). Related: KI-208. | +| KI-129 | Control plane / Storage | Bug (unverified) | S3 | On a connection created with Key prefix `sandbox`, the Browse tab lists the bucket root, not the prefix. *Steps:* Storage → sandbox-lake → Browse *Docker demo stack (Oct 2026):* still true (`sandbox-lake`, prefix `sandbox/`). Related: KI-208. *Status:* fixed in platform #715 (October 2026): Browse and Download keys are relative to the connection's Key prefix, and the root crumb shows `bucket/prefix` (`warehouse/sandbox`). | | KI-130 | Governance / Overview | Doc | S3 | The count tiles say "classifications" and the tag page "0 tagged objects" counts BI objects only; a tag on a table column does not raise the count. *Steps:* /governance/classifications/<tag> after tagging `user_id` *Status:* closed on the Docker demo stack (Oct 2026): the Classifications list has a **Data** column (*0 tables · 4 columns*), and the tag page's **Where used** lists the tables and columns. | | KI-131 | Governance / Data | UX | S3 | Grant and rule row actions are icon-only (pencil, bin) and the row text is not a link. The single grant/rule page (`/governance/grants/`) has no visible link from the row. *Steps:* Data → Permissions tab | | KI-132 | Governance / Data | UX | S3 | The New grant dialog taller than a 900 px viewport scrolls inside the dialog and the Preview result sits below the fold. *Steps:* Data → Grant access → Preview | @@ -232,8 +232,8 @@ Collected while writing the detailed pages for each area (October 2026), checked | KI-203 | Governance / Engine permissions | UX | S3 | In **Grant engine permission** the user search list stays open after a pick and covers the **Grant** button until you click elsewhere. | | KI-204 | Control plane / Engines | UX | S3 | After **Register engine** without a profile you land on Plugin config with the raw API text *conflict: engine trino-demo-ext is bound to no policy profile: bind a profile first Bind a policy profile on the Settings tab, then come back.* (prefix kept, sentences run together). | | KI-205 | Audit log | Gap | S2 | Decisions made while impersonating are recorded under the impersonated user with an empty `detail`; only `impersonation.start`/`impersonation.end` name the admin, so the workspace Activity tab shows `member-sso` as the actor of what `ops.admin` did. | -| KI-206 | Control plane / Storage | Security | S2 | **Storage → \ → Browse** lists `query-results/*.jsonl`, the stored SQL Lab results of every user, and `GET /storage/{id}/download?key=…` returns them to anyone with `storage:read`, which the built-in `data_steward` role holds. These are each user's post-mask results. | -| KI-207 | Control plane / Secrets | UX | S3 | System-managed secrets (`engine-internal-*`, `engine-monitor-*`, `engine-tls-*`, `catalog-client-*`) are listed with user secrets, can be rotated or deleted, and are offered in the catalog **Password secret** picker. The **Used by** and **Created** columns are cut off at 1440 px. | +| KI-206 | Control plane / Storage | Security | S2 | **Storage → \ → Browse** lists `query-results/*.jsonl`, the stored SQL Lab results of every user, and `GET /storage/{id}/download?key=…` returns them to anyone with `storage:read`, which the built-in `data_steward` role holds. These are each user's post-mask results. *Status:* fixed in platform #715 (October 2026): in egret's own bucket the prefixes egret writes (`engine.results_prefix`, `apps.storage.prefix`, `workflow.storage.prefix`) are closed to everyone except a platform admin: Browse hides them and says so, listing or downloading under them answers 403 with the reason, and path tricks answer 400. The raw warehouse data files are still readable: KI-236. | +| KI-207 | Control plane / Secrets | UX | S3 | System-managed secrets (`engine-internal-*`, `engine-monitor-*`, `engine-tls-*`, `catalog-client-*`) are listed with user secrets, can be rotated or deleted, and are offered in the catalog **Password secret** picker. The **Used by** and **Created** columns are cut off at 1440 px. *Status:* fixed in platform #715 (October 2026): secrets egret creates for engines and catalog clients carry `system_managed`, show a **System** badge, cannot be renamed, rotated or deleted by hand (409), and are left out of every secret picker. The cut-off columns did not reproduce at 1440 px. | | KI-208 | Control plane / Catalogs | Gap | S3 | A second egret warehouse on the same bucket is refused (*Storage profile overlaps with existing warehouse demo-warehouse*), also through a connection with another key prefix (`sandbox/`); the New catalog form does not say a warehouse needs its own bucket. Related: KI-129. | | KI-209 | Control plane / Users | UX | S3 | **Disable** on a user page acts at once and signs the user out without confirmation, while **Delete** asks for a typed confirmation. | | KI-210 | Access requests | Gap | S3 | No mail is sent when someone requests access or when a request is approved or denied, although the workspace has working SMTP (Mailpit received nothing); admins only see the queue when they open it. | @@ -244,23 +244,23 @@ Collected while writing the detailed pages for each area (October 2026), checked | ID | Area | Type | Sev | Description | |---|---|---|---|---| -| KI-136 | Quick start | Bug | S1 | A clean `cp .env.example .env; just up; just serve` cannot start. `.env.example` sets `AWS_*` but not `EGRET_STORAGE__BUCKET/ACCESS_KEY/SECRET_KEY`, and `Config::validate` always requires them ("the apps API requires [storage] bucket, access_key, secret_key"). `migrate` and `init` fail the same way. README/CONTRIBUTING repeat the three commands. *Steps:* Ran `egret migrate` with only `EGRET_DATABASE__URL` set: `Error: the apps API requires [storage] ...`. | +| KI-136 | Quick start | Bug | S1 | A clean `cp .env.example .env; just up; just serve` cannot start. `.env.example` sets `AWS_*` but not `EGRET_STORAGE__BUCKET/ACCESS_KEY/SECRET_KEY`, and `Config::validate` always requires them ("the apps API requires [storage] bucket, access_key, secret_key"). `migrate` and `init` fail the same way. README/CONTRIBUTING repeat the three commands. *Steps:* Ran `egret migrate` with only `EGRET_DATABASE__URL` set: `Error: the apps API requires [storage] ...`. *Status:* fixed in platform #704 (October 2026), kept by a test added in #718: `.env.example` sets the `EGRET_STORAGE__*` variables and the master key, and a test loads it through `Config::load` and `Config::validate` so it cannot fall behind again. KI-137 (the bucket) and KI-138 (Trino catalogs) stay open. | | KI-137 | Quick start | Gap | S2 | Nothing creates the object-store bucket (compose RustFS has none; no `warehouse` bucket existed). The server starts, and the first write to the bucket fails with `500 {"error":"internal error"}` (log: `s3 error: service error`). *Steps:* `POST /api/ws/default/apps` with a nonexistent bucket: 500. | | KI-138 | Quick start | Gap | S2 | The compose Trino has no catalogs (`deploy/trino/catalog/` holds only `.gitkeep`) and enforces JWT over TLS, so no repository path gives a first-time user a Trino with sample data for the first-dashboard tutorial. *Steps:* `deploy/trino/catalog`, `deploy/trino/etc/config.properties`. | | KI-139 | Configuration | Gap | S3 | The server never reads `.env`; `cargo run -p egret-bin -- init` therefore ignores it, and there is no `just init` recipe. Only `just` recipes see `.env`. *Steps:* `crates/egret-bin` has no dotenv use; `just --list`. | -| KI-140 | Configuration | UX | S3 | Unknown keys and misspelled `EGRET_*` variables are silently ignored at the top level and in most sections (only four sub-tables use `deny_unknown_fields`). *Steps:* Source: `crates/egret-config/src/lib.rs`. | -| KI-141 | Configuration | UX | S3 | Changing `server.port` leaves `catalog.public_url`, and with it the machine IdP `issuer` and token endpoint, at `http://localhost:8080`. *Steps:* Server on 18080: `/.well-known/openid-configuration` says `"issuer":"http://localhost:8080"`. | +| KI-140 | Configuration | UX | S3 | Unknown keys and misspelled `EGRET_*` variables are silently ignored at the top level and in most sections (only four sub-tables use `deny_unknown_fields`). *Steps:* Source: `crates/egret-config/src/lib.rs`. *Status:* fixed in platform #718 (October 2026): an unknown `EGRET_*` variable or configuration-file key logs a `WARN` at start with a did-you-mean hint, and never stops the start; variables read outside the configuration and the test-harness prefixes are allow-listed. | +| KI-141 | Configuration | UX | S3 | Changing `server.port` leaves `catalog.public_url`, and with it the machine IdP `issuer` and token endpoint, at `http://localhost:8080`. *Steps:* Server on 18080: `/.well-known/openid-configuration` says `"issuer":"http://localhost:8080"`. *Status:* fixed in platform #718 (October 2026): an unset `catalog.public_url` defaults to `http://localhost:`. `oidc.redirect_url` does not follow the port yet: KI-240. | | KI-142 | Authentication | UX | S3 | The authorization request carries `scope=openid+openid+email+profile`: the default scopes already contain `openid` and the client adds it again. *Steps:* `curl -D - localhost:8080/api/auth/oidc/login`, `location` header. | | KI-143 | Authentication | Gap | S3 | Local sign-in cannot be switched off when SSO is on; the form is only hidden behind **Admin sign-in (break-glass)**, and `POST /api/auth/login` still answers. *Steps:* `frontend/src/auth/LoginPage.tsx`; sign-in works with SSO on. | | KI-144 | Authentication | UX | S3 | The sign-in rate limit counts successful attempts (1 success + 4 failures, then 429), which surprises people who sign in a few times in a row. *Steps:* Throwaway instance, 7 consecutive attempts. | | KI-145 | Authentication | Bug | S3 | `POST /api/auth/login` returns `platform_roles: []` and `permissions: []` for the bootstrap `platform_admin`, while `GET /api/auth/me` returns the roles. *Steps:* Throwaway instance after `egret init`. | -| KI-146 | CLI | Security | S3 | `egret init` prints the password even when it was supplied in `EGRET_ADMIN_PASSWORD`, so it lands in terminal history and CI logs. *Steps:* `EGRET_ADMIN_PASSWORD=... egret init --force` prints `Password: ...`. | +| KI-146 | CLI | Security | S3 | `egret init` prints the password even when it was supplied in `EGRET_ADMIN_PASSWORD`, so it lands in terminal history and CI logs. *Steps:* `EGRET_ADMIN_PASSWORD=... egret init --force` prints `Password: ...`. *Status:* fixed in platform #719 (October 2026): `egret init` prints `Password: taken from EGRET_ADMIN_PASSWORD (not shown)` for a supplied password, and an empty `EGRET_ADMIN_PASSWORD` now generates one. | | KI-147 | Security hardening | Gap | S2 | `GET /metrics` answers 200 with Prometheus metrics, `/healthz` and `/readyz` too, with no authentication. Only the Helm ingress happens not to route them. *Steps:* `curl localhost:8080/metrics` returned 204 lines. | | KI-148 | Ranger | UX | S2 | `ranger.auth_token` locks out stock Ranger plugins (they cannot send a bearer token; the rendered plugin config has no credential) while the setting reads like a hardening switch. The plugin-config endpoint warns about it; the configuration docs do not. *Steps:* `/service/plugins/policies/download/...` 401 without header; `GET /api/control-plane/engines/{id}/ranger-plugin-config` warning. | | KI-149 | Ranger | Gap | S3 | `GET .../policies/services/{service}/trino-config` renders `policy.rest.url` from `catalog.public_url`, so a fresh install hands an engine in a cluster `http://localhost:8080`. The endpoint for external engines warns; the Trino one does not. *Steps:* Response for `docs-trino-policy`: `http://localhost:8080`. | | KI-150 | Helm | Gap | S2 | The chart has no values for an external PostgreSQL or S3: the ConfigMap hardcodes `postgres:5432` and `http://rustfs:9000`, and storage credentials come from `deps.rustfs.*`. The README tells you to set `deps.enabled=false` and "point the EGRET_* values at your services". *Steps:* `templates/configmap-env.yaml`, `templates/secret.yaml`; `helm template --set deps.enabled=false`. | | KI-151 | Helm | Docs | S3 | Chart README is stale: says OIDC is off "in this milestone" and builds only backend and frontend images, while the chart has `oidc.enabled` and `just images-build` builds four images. Chart `description` still says "local kind deployment". *Steps:* `deploy/helm/egret/README.md`, `Chart.yaml`. | -| KI-152 | Helm | Security | S3 | The bundled Keycloak (with `oidc.enabled`) imports the dev realm: `sslRequired: none`, `*` redirect URI, password grant, known user passwords and the client secret `egret-secret`. *Steps:* `deploy/helm/egret/files/egret-realm.json`. | +| KI-152 | Helm | Security | S3 | The bundled Keycloak (with `oidc.enabled`) imports the dev realm: `sslRequired: none`, `*` redirect URI, password grant, known user passwords and the client secret `egret-secret`. *Steps:* `deploy/helm/egret/files/egret-realm.json`. *Status:* fixed in platform #719 (October 2026): the Helm chart refuses `oidc.enabled` unless `oidc.devRealm: true` (default `false`) is also set, and then prints a warning in the install notes; the kind recipes opt in. | | KI-153 | OpenAPI | Gap | S2 | Only the Data API bearer scheme is declared; session cookie and CSRF are not, so generated clients cannot authenticate. Most operations have no summary (only an `operationId`), and many descriptions are internal doc comments (long, referencing internal types and task numbers). *Steps:* `openapi.json`; `scripts/gen-api-reference.mjs` has to truncate descriptions. | | KI-155 | Demo stack | Docs | S3 | The demo edge does not route `/readyz` and `/healthz` to Egret: `curl http://egret.localtest.me:18080/readyz` answers 200 with the web app's HTML, which looks healthy whatever the backend state, so the generic first checks of the troubleshooting index do not work on the demo stack. | | KI-185 | Authentication | UX | S2 | An SSO user lands on *No workspaces yet* after the first sign-in and can do nothing until a workspace admin adds them; Egret only knows the user after that first sign-in, so the admin cannot add them beforehand. The demo guide documents this. *Workaround:* sign in once, then have an admin add the user. Seen by the demo-stack guide author, not re-observed. | @@ -286,6 +286,17 @@ Collected while writing the detailed pages for each area (October 2026), checked | KI-233 | Responsive | UX | S3 | At 390 px, content is clipped at the right edge with no visible cue instead of scrolling or wrapping: the dashboards list loses its **Updated** column, the alert editor's **Details** text is cut mid-word, the tab strips of the alert editor (*Notification* is out of sight) and of Workspace settings (*Roles & permissions* is cut) show no arrow or fade, the title *New Data API alert* is truncated to *New Data API a…*, and the workspace picker shows *De* for *Default*. No page scrolls sideways at the document level. | | KI-234 | Responsive | Bug | S3 | At 768 px the app sidebar keeps its full 224 px, so tables lose their width: on **Datasets** the **Database** column wraps *trino-demo (trino)* onto three lines and its heading runs into **Source** (*DatabaseSource*, the same collision as KI-171). | | KI-235 | Dashboards | UX | S3 | The dashboard **Filter settings** side panel (340 px, see KI-068) is built from bare browser selects and a free **Default value** box, unlike the card settings panel that opens in the same place (grouped rows, switches, a styled picker). *Column to filter on* appears three times on the canvas behind it as the same grey placeholder. | +| KI-236 | Control plane / Storage | Security | S2 | `storage:read` (held by the built-in `data_steward` role) still lets a user download the raw Iceberg data files of every warehouse in the bucket (for example `.../data/*.parquet`), which bypasses column masks and row filters. Same kind of leak as KI-206, for table data instead of egret's own folders. (Found while fixing KI-206.) | +| KI-237 | Workflows | Bug | S3 | Restarting Egret while a long task runs leaves the run stuck in Running until the lease reaper takes it, after about an hour: Docker's 10 s stop grace kills a long `Sleep` before the graceful wait ends, and nothing hands the run back at start. (Found while fixing KI-187.) | +| KI-238 | Development | Bug | S3 | The `egret` Docker build shares one cargo cache between worktrees and can treat a changed crate as up to date, so an image built from a branch fails with a missing field or runs old code until the files are touched inside the build. (Found while fixing KI-187.) | +| KI-239 | CI | Bug | S3 | The test `worker::claim::tests::a_discard_only_block_claim_survives_renewal_and_the_poll` fails now and then on unrelated branches (it failed on #716 and #718 and passed on re-run); it depends on timing. | +| KI-240 | Configuration | Bug | S3 | The default of `oidc.redirect_url` is the literal `http://localhost:8080/...` and does not follow `server.port`, unlike `catalog.public_url` since #718. (Found while fixing KI-141.) | +| KI-241 | Configuration | Bug | S3 | The default `apps.listen` (`0.0.0.0:8081`) is the host port the dev compose gives Keycloak; the apps host only binds when `apps.origin` is set, and then it would clash. (Found while fixing KI-136.) | +| KI-242 | CLI | Bug | S3 | Two `egret init` runs at the same time race to create the `default` workspace and the loser fails. (Found while fixing KI-146.) | +| KI-243 | Workspace settings | UX | S3 | On Workspace settings → Engines an empty list looks the same to an admin and to a non-admin, so a non-admin in a workspace with no connection sees the empty state and its Add button answers 403. (Found while fixing KI-159.) | +| KI-244 | Alerts | UX | S3 | The alert schedule editor's hint for an empty connection list tells non-admins to add a connection, which only an admin can do. (Found while fixing KI-159.) | +| KI-245 | Audit log | UX | S3 | Every connection listing by a non-admin writes a denied `workspace:manage_members` decision to the audit log, because the endpoint asks the admin question first. (Found while fixing KI-159.) | +| KI-246 | Workflows | Limitation | S3 | The outputs and the rendered values of tasks that use a SECRET input are not masked (the same rule as for `secret()` values); only the inputs and the run logs are. (Design choice of #719.) | ## Not verified @@ -294,7 +305,7 @@ The Docker demo stack (`just demo-up`) now exercises what the earlier demo insta - **Kubernetes-only features:** StarRocks and RisingWave engines, ingress, autoscaling, pod logs and exec on engines, the Kubernetes kernel and app launchers. - **Ranger enforcement on an external engine:** the demo Trino has no Ranger plugin. Rule overlaps that already exist cannot be shown, because Egret refuses to create them. - **Governed connections:** `data_access_denied` and `governance_stale` responses, and different rows per viewer in an app (`trino-demo` runs every query as `egret`). -- **Non-admin use:** a delegated (per-user) query by a non-admin through the UI, SQL Lab and dataset behaviour for a viewer, and the permissions needed for Tags (blocked by KI-159). +- **Non-admin use:** a delegated (per-user) query by a non-admin through the UI, SQL Lab and dataset behaviour for a viewer, and the permissions needed for Tags. These were blocked by KI-159, which #712 fixed; they have not been re-checked for these docs. - **Delivery:** Slack (no Slack app on the stack) and webhook request bodies (private and Docker hosts are blocked, KI-035; payload shape comes from source). - **Workflows:** script tasks and `WorkingDirectory`, Kestra JDBC plugin tasks (no container runner or plugin host, refused with `task_type_not_served`), `egret.sql`, `egret.dataapi.call`, `egret.dataset.refresh`, `egret.report.send`, `core.http` and `core.slack` (no handler on any deployment), mail from a workflow (`MailSend` is unknown) and webhook calls (off on the stack, see KI-188). - **SQL Lab errors:** `connect:`, `timeout` and authentication errors (the only connection is healthy and not delegated). diff --git a/platform/reference/cli.md b/platform/reference/cli.md index f669cc6..49ec567 100644 --- a/platform/reference/cli.md +++ b/platform/reference/cli.md @@ -58,17 +58,21 @@ and makes the administrator its `ws_admin`. It runs migrations first. | Input | Meaning | |---|---| | `EGRET_ADMIN_EMAIL` | The administrator's email. Default `admin@egret.local`. | -| `EGRET_ADMIN_PASSWORD` | The password. If unset, a random one is generated. | +| `EGRET_ADMIN_PASSWORD` | The password. If unset or empty, a random one is generated. | | `--force` | Create **another** administrator even if one exists. Without it, `init` logs `a platform_admin already exists` and does nothing. | -The email and password are printed once: +The email and a generated password are printed once: ```text Bootstrap admin: admin@egret.local Password: +(generated; shown only this once) Default workspace: default ``` +A password you supplied in `EGRET_ADMIN_PASSWORD` is not printed, so it does not land in terminal history or CI logs. The +second line then reads `Password: taken from EGRET_ADMIN_PASSWORD (not shown)`. + An email that already exists fails with `conflict: user already exists: `, so a second administrator needs a different email. See [Cannot sign in](../deploy-operate/troubleshooting/cannot-sign-in). diff --git a/platform/reference/configuration-keys.md b/platform/reference/configuration-keys.md index 01962af..000bb62 100644 --- a/platform/reference/configuration-keys.md +++ b/platform/reference/configuration-keys.md @@ -21,9 +21,20 @@ The environment wins. - Lists and tables take TOML syntax in the variable: `EGRET_OIDC__SCOPES='["openid","email","profile"]'`. - Keys marked **required** in the Default column have an empty default and a boot-time check; the server refuses to start without them (see [Startup checks](#startup-checks)). -- A key the server does not know is silently ignored at the top level and in most sections. A typo in a variable name - therefore leaves the default in place. The sub-tables `workflow.storage`, `workflow.kubernetes`, - `workflow.plugin_host`, `apps.storage` and the entries of `workflow.plugin_configurations` reject unknown keys. +- A key the server does not know is ignored at the top level and in most sections, so a typo leaves the default in + place, but every subcommand logs a `WARN` for it right after the configuration has passed the + [startup checks](#startup-checks), with the closest real key when one is within two edits: + + ```text + ignored environment variable EGRET_SERVER__PORTT: no configuration key has this name (did you mean EGRET_SERVER__PORT?) + ignored key server.portt in /egret.toml: no configuration key has this name (did you mean server.port?) + ``` + + The warning never stops the start. It covers every `EGRET_*` variable and every key of the configuration file, except + the [variables read outside the configuration](#variables-outside-the-configuration-file) and the test-harness prefixes + `EGRET_TEST_`, `EGRET_E2E_` and `EGRET_ALLOW_`. The sub-tables `workflow.storage`, `workflow.kubernetes`, + `workflow.plugin_host`, `apps.storage` and the entries of `workflow.plugin_configurations` go further and reject + unknown keys. ## server @@ -105,7 +116,7 @@ This is a different issuer from `oidc`, which describes the external provider pe | Key | Environment variable | Default | Meaning | |---|---|---|---| -| `master_key` | `EGRET_SECRETS__MASTER_KEY` | empty | Base64 key of at least 32 bytes used for AES-256-GCM encryption of stored credentials. Empty disables sealing: registering a database or storage connection **with a secret** then fails. If set, it must decode to 32 bytes or more. | +| `master_key` | `EGRET_SECRETS__MASTER_KEY` | empty | Base64 key of at least 32 bytes used for AES-256-GCM encryption of stored credentials. Empty disables sealing: registering a database or storage connection **with a secret** then fails, and so does starting a workflow that declares a `SECRET` input (400). If set, it must decode to 32 bytes or more. | ## catalog @@ -116,7 +127,7 @@ The embedded Iceberg REST catalog. See [Storage and catalog setup](../deploy-ope | `enabled` | `EGRET_CATALOG__ENABLED` | `false` | Run the catalog in the server process. Requires `warehouse`, `auth_token` and the `storage` credentials. | | `warehouse` | `EGRET_CATALOG__WAREHOUSE` | `s3://warehouse` | Warehouse location of the default catalog. | | `auth_token` | `EGRET_CATALOG__AUTH_TOKEN` | empty (required with `enabled`) | Static token for catalog administration. | -| `public_url` | `EGRET_CATALOG__PUBLIC_URL` | `http://localhost:8080` | Base origin of this server, without `/catalog`, advertised to catalog clients. Engines in a cluster resolve it, so in Kubernetes it is a Service DNS name. It is also the fallback for `idp.issuer`. | +| `public_url` | `EGRET_CATALOG__PUBLIC_URL` | `http://localhost:` (an unset or blank value follows `server.port`) | Base origin of this server, without `/catalog`, advertised to catalog clients. Engines in a cluster resolve it, so in Kubernetes it is a Service DNS name. It is also the fallback for `idp.issuer`. | | `system_user` | `EGRET_CATALOG__SYSTEM_USER` | `egret-system` | Stable name of the in-process principal that bootstraps the catalog's default project. Keep it stable across deploys; changing it creates a different system administrator. | ## storage @@ -206,6 +217,7 @@ Switches and limits for the workflow engine. Both loops default to **off**. | `scheduler_enabled` | `EGRET_WORKFLOW__SCHEDULER_ENABLED` | `false` | Start the trigger loop, so `cron:` schedule and flow triggers fire. It needs the worker on to be useful. | | `tick_interval_secs` | `EGRET_WORKFLOW__TICK_INTERVAL_SECS` | `1` | Seconds between claim ticks. | | `batch` | `EGRET_WORKFLOW__BATCH` | `8` | Most task runs one claim tick takes. | +| `max_concurrent_runs` | `EGRET_WORKFLOW__MAX_CONCURRENT_RUNS` | `8` | Most claimed task runs one replica runs at the same time. A tick claims at most the free slots, so a slow task holds one slot and not the whole replica. `0` is refused at start. | | `reaper_interval_secs` | `EGRET_WORKFLOW__REAPER_INTERVAL_SECS` | `60` | Seconds between reaper passes. | | `working_timeout_s` | `EGRET_WORKFLOW__WORKING_TIMEOUT_S` | `3600` | How long a lease may go unreleased before the reaper hands the run back. Too short and a slow live task is run twice; too long and a task of a dead replica stays stuck. | | `container_default_timeout_s` | `EGRET_WORKFLOW__CONTAINER_DEFAULT_TIMEOUT_S` | `3600` | Timeout of a script or container task that declares none. Other task types run unbounded. | @@ -310,8 +322,9 @@ These are read directly from the environment. | Variable | Used by | Meaning | |---|---|---| | `EGRET_ADMIN_EMAIL` | `egret init`, `egret workspace add-group` | Email of the bootstrap administrator. Default `admin@egret.local`. | -| `EGRET_ADMIN_PASSWORD` | `egret init` | Password of the bootstrap administrator. If unset, a random one is generated and printed once. | +| `EGRET_ADMIN_PASSWORD` | `egret init` | Password of the bootstrap administrator. It is not printed (`init` says *taken from EGRET_ADMIN_PASSWORD (not shown)*). If unset or empty, a random one is generated and printed once. | | `EGRET_ENGINE_TLS_INSECURE` | Query connectors and the engine monitor | `1`, `true` or `yes` skip TLS certificate verification when connecting to engines. Development only. | +| `EGRET_WRITE_PROPERTY_TREATMENT` | The workflow test-corpus generator | A development switch, not used by a running server. | | `DATABASE_URL` | `sqlx` tooling and the test suite | Used by compile-time query checking and tests, not by the server (which reads `EGRET_DATABASE__URL`). `.env.example` sets both. | ## Startup checks @@ -329,6 +342,7 @@ same check. A failure prints one line and exits. These are the rules: | Always: `storage.bucket`, `access_key` and `secret_key` | `the apps API requires [storage] bucket, access_key, secret_key ...` | | `engine.max_rows` and `query_timeout_secs` are at least 1; `preview_rows` is at most `max_rows` | `engine.max_rows and engine.query_timeout_secs must be >= 1` | | `workflow.kill_poll_ms` is at least 100 | `workflow.kill_poll_ms: ... is below the minimum 100` | +| `workflow.max_concurrent_runs` is at least 1 | `workflow.max_concurrent_runs must be >= 1 (EGRET_WORKFLOW__MAX_CONCURRENT_RUNS)` | | `workflow.container_runner` other than `none` needs the storage credentials and `plugin_host.image` | `workflow.container_runner other than none requires [storage] ...` | | `apps.origin` set: both origins are `scheme://host[:port]`, differ from each other, and the TTLs are at least 1 | `apps.origin must differ from apps.egret_origin ...` | | `apps.pod.launcher` is `none`, `subprocess` or `kubernetes`, with the keys each needs | `apps.pod.launcher must be none, subprocess or kubernetes ...` | @@ -336,8 +350,8 @@ same check. A failure prints one line and exits. These are the rules: :::warning The storage credentials are required even if you never use object storage The apps API stores draft and version content in the storage bucket regardless of the pod launcher, so the check applies -on every start, including `egret migrate` and `egret init`. A copy of `.env.example` alone does not satisfy it. See the -[Quick start](../get-started/quick-start). +on every start, including `egret migrate` and `egret init`. The repository's `.env.example` sets them to the dev compose's +RustFS credentials, and a test keeps it passing these checks. See the [Quick start](../get-started/quick-start). ::: ## Further reading diff --git a/platform/reference/rest-api/control-plane-engines.md b/platform/reference/rest-api/control-plane-engines.md index 69ee9c3..f820ecf 100644 --- a/platform/reference/rest-api/control-plane-engines.md +++ b/platform/reference/rest-api/control-plane-engines.md @@ -8,7 +8,7 @@ description: REST endpoints in the control-plane-engines group. Engine provisioning: Trino-on-Kubernetes clusters, their catalogs, native query monitor, and pod logs/exec. -This page lists 24 endpoints (OpenAPI tag `control-plane-engines`). It is generated from `openapi.json`; see the [REST API overview](../rest-api) for authentication, CSRF and error conventions. +This page lists 26 endpoints (OpenAPI tag `control-plane-engines`). It is generated from `openapi.json`; see the [REST API overview](../rest-api) for authentication, CSRF and error conventions. | Method | Path | Operation id | Notes | |---|---|---|---| @@ -33,6 +33,8 @@ This page lists 24 endpoints (OpenAPI tag `control-plane-engines`). It is genera | DELETE | `/api/control-plane/engines/{id}/queries/{qid}` | `kill_query` | | | GET | `/api/control-plane/engines/{id}/ranger-plugin-config` | `ranger_plugin_config` | `GET /api/control-plane/engines/\{id\}/ranger-plugin-config`. | | POST | `/api/control-plane/engines/{id}/restart` | `restart_engine` | `POST /engines/\{id\}/restart` — roll the engine's pods without changing config (like `kubectl rollout restart`). | +| POST | `/api/control-plane/engines/{id}/retire-previous-ranger-credential` | `retire_previous_ranger_credential` | `POST /engines/\{id\}/retire-previous-ranger-credential` — the operator's confirmation that an EXTERNAL engine's plugin now runs with the credential named `ins... | +| POST | `/api/control-plane/engines/{id}/rotate-ranger-credential` | `rotate_ranger_credential` | `POST /engines/\{id\}/rotate-ranger-credential` — mint a new Ranger plugin credential. | | POST | `/api/control-plane/engines/{id}/rotate-secrets` | `rotate_engine_secrets` | `POST /engines/\{id\}/rotate-secrets` — mint a new Trino internal-communication secret and `egret-monitor` password and re-apply the engine (pods roll). | | GET | `/api/control-plane/engines/{id}/workspaces` | `list_engine_workspaces` | | | PUT | `/api/control-plane/engines/{id}/workspaces` | `set_engine_workspaces` | | @@ -124,6 +126,8 @@ Operation id: `list_engines`. Auth: Session cookie. Forbidden (403): Missing eng | `rangerPluginVersion` | string, nullable | no | The Ranger plugin version egret reads this engine at. External: the version the operator declared. Provisioned: the version its exact image ships (`egret_perm::plugin_version_for_image`); absent when the image is not one egret has measured. | | `rangerPluginVersionModelled` | boolean, nullable | no | Whether egret's Ranger evaluator has been read against this engine's (type, plugin version) PAIR: only the pairs in `egret_perm::MODELLED_RANGER_PLUGINS` are. When `false`, a migration to the profile bound to this engine is refused (never... | | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | +| `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | +| `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | | Status | Meaning | |---|---| @@ -172,12 +176,14 @@ Operation id: `create_engine`. Auth: Session cookie and `X-Egret-CSRF` header. F | `rangerPluginVersion` | string, nullable | no | The Ranger plugin version egret reads this engine at. External: the version the operator declared. Provisioned: the version its exact image ships (`egret_perm::plugin_version_for_image`); absent when the image is not one egret has measured. | | `rangerPluginVersionModelled` | boolean, nullable | no | Whether egret's Ranger evaluator has been read against this engine's (type, plugin version) PAIR: only the pairs in `egret_perm::MODELLED_RANGER_PLUGINS` are. When `false`, a migration to the profile bound to this engine is refused (never... | | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | +| `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | +| `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | | `catalogAttachErrors` | CatalogAttachErrorDto[] | yes | Catalogs from the body's `catalogs` that could not be attached. Empty on success. The engine EXISTS and is provisioned; these catalogs simply are not on it, and each can be retried with `POST /api/control-plane/engines/\{id\}/catalogs/\{cid\}`.... | | Status | Meaning | |---|---| | 201 | Engine created and provisioned; the primary catalog is attached, and `catalogAttachErrors` lists any of the request's other `catalogs` the engine came up without | -| 400 | Unknown/unprovisioned warehouse, invalid input, or provisioning failed | +| 400 | Unknown/unprovisioned warehouse, invalid input (including a StarRocks engine name longer than 48 characters: the operator derives 63-character pod volume names from it), or provisioning failed | | 401 | Not authenticated | | 403 | Missing engine:create permission | | 409 | An engine with this name already exists, or its same-name policy profile is bound to another engine | @@ -199,7 +205,7 @@ Operation id: `register_external_engine`. Auth: Session cookie and `X-Egret-CSRF | `authzServiceId` | integer (int64), nullable | no | A policy profile to bind. Its type must equal `type`, no other engine may bind it, and a Trino profile needs a usable `tagService`. | | `catalogs` | string (uuid)[] | no | Registry catalogs to declare attached. Nothing is applied to the engine. | -**Response 201 (application/json)**: `EngineDto` +**Response 201 (application/json)**: `RegisteredExternalEngineDto` | Field | Type | Required | Description | |---|---|---|---| @@ -216,6 +222,9 @@ Operation id: `register_external_engine`. Auth: Session cookie and `X-Egret-CSRF | `rangerPluginVersion` | string, nullable | no | The Ranger plugin version egret reads this engine at. External: the version the operator declared. Provisioned: the version its exact image ships (`egret_perm::plugin_version_for_image`); absent when the image is not one egret has measured. | | `rangerPluginVersionModelled` | boolean, nullable | no | Whether egret's Ranger evaluator has been read against this engine's (type, plugin version) PAIR: only the pairs in `egret_perm::MODELLED_RANGER_PLUGINS` are. When `false`, a migration to the profile bound to this engine is refused (never... | | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | +| `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | +| `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | +| `rangerCredential` | RangerCredentialDto | yes | The credential the engine's Ranger plugin presents to egret's `/ranger` surface (HTTP Basic). **Shown once**: egret never returns this password again; a lost one is replaced with `POST... | | Status | Meaning | |---|---| @@ -269,10 +278,12 @@ Operation id: `get_engine`. Auth: Session cookie. Forbidden (403): Missing engin | `rangerPluginVersion` | string, nullable | no | The Ranger plugin version egret reads this engine at. External: the version the operator declared. Provisioned: the version its exact image ships (`egret_perm::plugin_version_for_image`); absent when the image is not one egret has measured. | | `rangerPluginVersionModelled` | boolean, nullable | no | Whether egret's Ranger evaluator has been read against this engine's (type, plugin version) PAIR: only the pairs in `egret_perm::MODELLED_RANGER_PLUGINS` are. When `false`, a migration to the profile bound to this engine is refused (never... | | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | +| `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | +| `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | | Status | Meaning | |---|---| -| 200 | The engine row flattened with live Kubernetes status | +| 200 | The engine row flattened with live Kubernetes status. Side effect: when a provisioned engine's previous Ranger credential is pending and the rollout of the current one is observed complete, this read retires the previous one | | 401 | Not authenticated | | 403 | Missing engine:read permission | | 404 | No such engine | @@ -322,6 +333,8 @@ Operation id: `update_engine`. Auth: Session cookie and `X-Egret-CSRF` header. F | `rangerPluginVersion` | string, nullable | no | The Ranger plugin version egret reads this engine at. External: the version the operator declared. Provisioned: the version its exact image ships (`egret_perm::plugin_version_for_image`); absent when the image is not one egret has measured. | | `rangerPluginVersionModelled` | boolean, nullable | no | Whether egret's Ranger evaluator has been read against this engine's (type, plugin version) PAIR: only the pairs in `egret_perm::MODELLED_RANGER_PLUGINS` are. When `false`, a migration to the profile bound to this engine is refused (never... | | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | +| `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | +| `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | | Status | Meaning | |---|---| @@ -446,6 +459,8 @@ Operation id: `update_external_engine`. Auth: Session cookie and `X-Egret-CSRF` | `rangerPluginVersion` | string, nullable | no | The Ranger plugin version egret reads this engine at. External: the version the operator declared. Provisioned: the version its exact image ships (`egret_perm::plugin_version_for_image`); absent when the image is not one egret has measured. | | `rangerPluginVersionModelled` | boolean, nullable | no | Whether egret's Ranger evaluator has been read against this engine's (type, plugin version) PAIR: only the pairs in `egret_perm::MODELLED_RANGER_PLUGINS` are. When `false`, a migration to the profile bound to this engine is refused (never... | | `rangerPluginVersionNote` | string, nullable | no | Why the engine is not modelled, and how to fix it (for a provisioned engine: pin its image). Absent when modelled. | +| `rangerCredentialPending` | boolean | yes | Whether a PREVIOUS Ranger plugin credential is still valid beside the current one (spec 2026-10-20-ranger-surface-auth A10/A11): a rotation is pending. Provisioned: until the rollout of the current credential is observed complete, when... | +| `rangerCredentialId` | string (uuid), nullable | no | The id of the engine's CURRENT Ranger plugin credential: the `credentialId` registration or the last rotation returned with it. An opaque id that grants nothing (never the password); an external engine's operator sends it as `installed` to... | | Status | Meaning | |---|---| @@ -629,18 +644,18 @@ Operation id: `ranger_plugin_config`. Auth: Session cookie. Forbidden (403): Mis | Field | Type | Required | Description | |---|---|---|---| | `serviceName` | string | yes | The bound policy profile: the Ranger service name the plugin asks for. | -| `downloadUrl` | string | yes | `\{public_url\}/service/plugins/policies/download/\{serviceName\}`: where the plugin's download lands. The files below carry `public_url` itself (the plugin appends the path). | -| `authMode` | string | yes | `anonymous`, or `bearer-configured` (egret requires a bearer token on `/service`; the token is never shown here). | +| `downloadUrl` | string | yes | `\{public_url\}/ranger/service/plugins/policies/download/\{serviceName\}`: where the plugin's download lands. The files below carry `public_url` itself (the plugin appends the path). | +| `authMode` | string | yes | `engine-credential` (egret's `/ranger` surface requires the engine's own Ranger credential, shown once at registration or rotation; the files carry a placeholder in place of its password) or `anonymous` (egret serves the surface to anyone... | | `rangerPluginVersion` | string, nullable | no | The plugin version the engine was DECLARED to run. | | `rangerPluginVersionModelled` | boolean | yes | Whether egret's evaluator models the (type, version) pair. When `false`, migrations into this profile are refused. | | `warnings` | string[] | yes | Things that make the install not work, or not safe, as configured. | | `assumptions` | string[] | yes | What egret's `explain` and migration gate assume about the plugin and cannot check on an external engine. | -| `adminMustSupply` | string[] | yes | What the admin must supply themselves. egret renders no credential and invents none. | +| `adminMustSupply` | string[] | yes | What the admin must supply themselves, the credential's password among them: egret shows it only once and renders a placeholder here. | | `files` | RangerPluginFileDto[] | yes | | | Status | Meaning | |---|---| -| 200 | The Ranger plugin files to install on this external engine, rendered by the same code that renders the files egret mounts into its own engines. No credential is included or minted | +| 200 | The Ranger plugin files to install on this external engine, rendered by the same code that renders the files egret mounts into its own engines. The security XML carries the engine's username and a placeholder in place of its password: the... | | 401 | Not authenticated | | 403 | Missing engine:read permission | | 404 | No such engine | @@ -667,6 +682,66 @@ Operation id: `restart_engine`. Auth: Session cookie and `X-Egret-CSRF` header. | 409 | The engine is external (`code` = `engine_is_external`): egret does not restart it | | 500 | Internal error | +## POST `/api/control-plane/engines/{id}/retire-previous-ranger-credential` + +Operation id: `retire_previous_ranger_credential`. Auth: Session cookie and `X-Egret-CSRF` header. Forbidden (403): Missing engine:update permission. + +`POST /engines/\{id\}/retire-previous-ranger-credential` — the operator's confirmation that an EXTERNAL engine's plugin now runs with the credential named `installed` (the `credentialId` registration or the last rotation returned): the previous one is deleted by its exact id and stops authenticating. Standing rule: egret never deletes a credential the cluster may still hold, and for an external engine only its operator can say it no longer does — about a SPECIFIC credential, so a stale or duplicated confirmation sent before a later rotation retires nothing (409). + +The check is one compare-and-set on both columns (current = `installed`, previous = the id read from the row). Idempotent: with nothing pending and a matching `installed` it answers 204 and deletes nothing. A provisioned engine's previous credential is retired by egret once the rollout of the current one is observed complete, so it is refused (409). + +| Parameter | In | Type | Required | Description | +|---|---|---|---|---| +| `id` | path | string (uuid) | yes | Engine id | + +**Request body (application/json)**: `RetirePreviousRangerCredential` + +| Field | Type | Required | Description | +|---|---|---|---| +| `installed` | string (uuid) | yes | The `credentialId` that registration or the last rotation returned with the credential now installed on the engine. | + +| Status | Meaning | +|---|---| +| 204 | `installed` is the current credential: the previous Ranger plugin credential is retired (it no longer authenticates), or none was pending. The current one is untouched, and a further rotation is allowed again | +| 400 | The body is missing or malformed | +| 401 | Not authenticated | +| 403 | Missing engine:update permission | +| 404 | No such engine | +| 409 | Nothing was retired: the engine is provisioned (egret retires its previous credential itself once the rollout of the current one is observed complete), or `installed` is not the engine's current credential (a confirmation sent before a... | +| 500 | Internal error | + +## POST `/api/control-plane/engines/{id}/rotate-ranger-credential` + +Operation id: `rotate_ranger_credential`. Auth: Session cookie and `X-Egret-CSRF` header. Forbidden (403): Missing engine:update permission. + +`POST /engines/\{id\}/rotate-ranger-credential` — mint a new Ranger plugin credential. Spec 2026-10-20-ranger-surface-auth A10/A11: the new credential is committed and the old one demoted to previous; both authenticate until the previous one is retired. A rotation is refused (409) while a previous one is still pending, so a rotation never deletes a credential. + +**Provisioned engine** (Trino or StarRocks): the new credential is applied and the route answers 204 once Kubernetes ACCEPTS the apply; the pods then roll. The previous credential stays valid until that rollout is OBSERVED complete — every pod that runs the plugin (Trino: coordinator and workers; StarRocks: the FE) runs the new credential and none of an older template is left — and is then deleted by its exact id, by a periodic sweep or the next read of the engine ([`crate::ranger_credential::RangerCredentialStore::retire_previous_if_rolled_out`]). A stalled rollout keeps it valid for as long as it stalls. A failed apply deletes nothing; the previous one is retired once a later apply has... + +| Parameter | In | Type | Required | Description | +|---|---|---|---|---| +| `id` | path | string (uuid) | yes | Engine id | + +**Response 200 (application/json)**: `RangerCredentialDto` + +| Field | Type | Required | Description | +|---|---|---|---| +| `username` | string | yes | `engine-<engine id>`. | +| `password` | string | yes | The password. egret does not show it again. | +| `credentialId` | string (uuid) | yes | Names THIS credential. Send it as `installed` to `POST /api/control-plane/engines/\{id\}/retire-previous-ranger-credential` once the engine runs with it; a confirmation naming any other credential retires nothing. An opaque id: it grants... | + +| Status | Meaning | +|---|---| +| 200 | External engine: the new Ranger plugin credential, shown only here (`Cache-Control: no-store`). The previous credential still authenticates until the operator, once the new one is installed, sends its `credentialId` as `installed` to `POST... | +| 204 | Provisioned engine: Kubernetes accepted the apply of a new Ranger plugin credential; the engine's pods restart. The previous credential remains valid until the rollout is observed complete (every pod that runs the Ranger plugin runs the new... | +| 400 | Provisioned engine bound to no policy profile (it has no Ranger credential), or Kubernetes is not configured | +| 401 | Not authenticated | +| 403 | Missing engine:update permission | +| 404 | No such engine | +| 409 | Nothing was rotated, for one of two reasons: a previous rotation is still pending (provisioned: the rollout of the current credential has not been observed complete yet, so the previous one is still valid; retry once the rollout has... | +| 500 | Internal error | +| 502 | Provisioned engine: Kubernetes rejected the apply; the engine row now names the new credential and keeps the previous one — both still authenticate — until a later apply of the new one has been observed to roll out | + ## POST `/api/control-plane/engines/{id}/rotate-secrets` Operation id: `rotate_engine_secrets`. Auth: Session cookie and `X-Egret-CSRF` header. Forbidden (403): Missing engine:update permission. diff --git a/platform/reference/rest-api/control-plane-governance.md b/platform/reference/rest-api/control-plane-governance.md index fd665bf..c3da60a 100644 --- a/platform/reference/rest-api/control-plane-governance.md +++ b/platform/reference/rest-api/control-plane-governance.md @@ -8,18 +8,80 @@ description: REST endpoints in the control-plane-governance group. Governance Data: browse a catalog and read an object's lineage (DG frontend redesign #1). -This page lists 7 endpoints (OpenAPI tag `control-plane-governance`). It is generated from `openapi.json`; see the [REST API overview](../rest-api) for authentication, CSRF and error conventions. +This page lists 11 endpoints (OpenAPI tag `control-plane-governance`). It is generated from `openapi.json`; see the [REST API overview](../rest-api) for authentication, CSRF and error conventions. | Method | Path | Operation id | Notes | |---|---|---|---| +| GET | `/api/control-plane/classifications/{id}/handwritten` | `tag_handwritten` | Hand-written tag policies of a classification. | +| GET | `/api/control-plane/classifications/{id}/usage` | `tag_usage` | Where a classification is used. | | GET | `/api/control-plane/governance/attention` | `attention` | What needs attention in governance, per the reader's permissions. | | GET | `/api/control-plane/governance/browse/{catalog_id}` | `browse` | | | GET | `/api/control-plane/governance/catalogs` | `list_catalogs` | `GET /governance/catalogs`: every registry catalog, by name, for the Governance Data tree. | | GET | `/api/control-plane/governance/changes` | `changes` | Recent governance changes, newest first. | | GET | `/api/control-plane/governance/engines/{id}/plugin` | `engine_plugin` | What the Ranger plugins of an engine's profile report downloading. | | GET | `/api/control-plane/governance/lineage/{catalog_id}` | `object_lineage` | | +| GET | `/api/control-plane/governance/objects/{catalog_id}/handwritten` | `object_handwritten` | Hand-written policies that reach a Data object. | +| GET | `/api/control-plane/governance/principals/{kind}/{id}/handwritten` | `principal_handwritten` | Hand-written policies that name a principal. | | GET | `/api/control-plane/governance/tags/{catalog_id}` | `object_tags` | | +## GET `/api/control-plane/classifications/{id}/handwritten` + +Operation id: `tag_handwritten`. Auth: Session cookie. Forbidden (403): Lacks policy:read. + +Hand-written tag policies of a classification. + +Every hand-written (operator) policy of every usable tag service whose tag resource matches this classification's NAME through the tag service-def's matcher (wildcards yes, case folding no), with the Trino profiles and engines that download it; `sameName` lists other tags with this name, which those policies reach too. `no_tag_service` when no Trino profile has a tag service. Requires policy:read. + +| Parameter | In | Type | Required | Description | +|---|---|---|---|---| +| `id` | path | string (uuid) | yes | Tag id | + +**Response 200 (application/json)**: `TagHandWritten` + +| Field | Type | Required | Description | +|---|---|---|---| +| `tag` | TagRef | yes | | +| `state` | TagPolicyState | yes | | +| `policies` | TagPolicyRow[] | yes | | +| `sameName` | SameNameTag[] | yes | Other tags with this name: engine tag policies on the name apply to them too (CL10). | +| `unreadable` | UnreadablePolicy[] | yes | Hand-written tag policies whose body could not be read: they may name this tag, so the list above may be incomplete. | + +| Status | Meaning | +|---|---| +| 200 | The hand-written tag policies naming the classification | +| 401 | Not authenticated | +| 403 | Lacks policy:read | +| 404 | No tag with this id | +| 500 | Internal error | + +## GET `/api/control-plane/classifications/{id}/usage` + +Operation id: `tag_usage`. Auth: Session cookie. Forbidden (403): Lacks both tag:read and policy:read. + +Where a classification is used. + +The tables and columns of egret catalogs carrying the classification (grouped by table; assignments in a warehouse no catalog names are counted as `orphaned`), and the BI objects carrying it: each is checked with the caller's read permission, in workspace, kind, id order, up to 500 per request; readable ones are listed, unreadable ones only counted (no name, no workspace), the rest reported as `remaining`, and assignments to deleted objects counted. Requires tag:read or policy:read. + +| Parameter | In | Type | Required | Description | +|---|---|---|---|---| +| `id` | path | string (uuid) | yes | Tag id | + +**Response 200 (application/json)**: `TagUsage` + +| Field | Type | Required | Description | +|---|---|---|---| +| `tag` | UsageTag | yes | | +| `data` | DataUsage | yes | | +| `bi` | BiUsage | yes | | + +| Status | Meaning | +|---|---| +| 200 | Where the classification is used | +| 401 | Not authenticated | +| 403 | Lacks both tag:read and policy:read | +| 404 | No tag with this id | +| 500 | Internal error | + ## GET `/api/control-plane/governance/attention` Operation id: `attention`. Auth: Session cookie. Forbidden (403): Holds neither policy:read nor tag:read. @@ -190,6 +252,74 @@ Operation id: `object_lineage`. Auth: Session cookie. Forbidden (403): Missing p | 503 | (egret catalog) egret's warehouse could not be read to tell a table from a view | | 504 | (egret catalog) egret's warehouse did not answer within the deadline | +## GET `/api/control-plane/governance/objects/{catalog_id}/handwritten` + +Operation id: `object_handwritten`. Auth: Session cookie. Forbidden (403): Lacks policy:read. + +Hand-written policies that reach a Data object. + +Every hand-written (operator) Ranger policy of the profiles whose engines reach the catalog, and of their tag services, that reaches the object: `applies` (the policy names this object, or it and everything below it), `part` (it names something below), `parent` (it names a parent only, which engines do not apply to this object's data) or `unknown` (the body cannot be read). A resource match, not an access decision. egret-seeded policies are counted per profile. External engines are read from the database only. Requires policy:read. + +| Parameter | In | Type | Required | Description | +|---|---|---|---|---| +| `catalog_id` | path | string (uuid) | yes | Registry catalog id | +| `schema` | query | string | no | The schema (StarRocks: database); needed for `table`. | +| `table` | query | string | no | The table or view; needs `schema`, needed for `column`. | +| `column` | query | string | no | The column; needs `table`. | + +**Response 200 (application/json)**: `ObjectHandWritten` + +| Field | Type | Required | Description | +|---|---|---|---| +| `catalog` | HandWrittenCatalog | yes | | +| `target` | HandWrittenTarget | yes | | +| `state` | ListedState | yes | | +| `reason` | NotDeterminedReason | no | | +| `profiles` | HandWrittenProfile[] | yes | | +| `policies` | HandWrittenPolicy[] | yes | | +| `seeded` | SeededCount[] | yes | egret-seeded (`infra`) policies that reach the object, counted per profile and never listed. | + +| Status | Meaning | +|---|---| +| 200 | The hand-written policies reaching the object | +| 400 | A level named without its parent, or empty or `*` | +| 401 | Not authenticated | +| 403 | Lacks policy:read | +| 404 | No catalog with this id | +| 500 | Internal error | + +## GET `/api/control-plane/governance/principals/{kind}/{id}/handwritten` + +Operation id: `principal_handwritten`. Auth: Session cookie. Forbidden (403): Lacks policy:read. + +Hand-written policies that name a principal. + +Every hand-written (operator) Ranger policy, in every profile and tag service, with an item naming the principal: a user's username (`direct`), or a group or role the user holds (`group` / `role`, with `onEngines: false`: engines resolve no groups or roles, so such an item reaches nobody there). Items naming `public` or `\{USER\}` are reported under `everyone`, never attributed. A user with no username is `no_username` with only the `everyone` policies. Requires policy:read. + +| Parameter | In | Type | Required | Description | +|---|---|---|---|---| +| `kind` | path | string | yes | `user`, `group` or `role` | +| `id` | path | string | yes | A user's id, or a group's or role's name | + +**Response 200 (application/json)**: `PrincipalHandWritten` + +| Field | Type | Required | Description | +|---|---|---|---| +| `principal` | PrincipalRef | yes | | +| `state` | PrincipalState | yes | | +| `policies` | PrincipalPolicy[] | yes | | +| `everyone` | EveryoneSummary | yes | Policies with an item naming `public` or `\{USER\}`: they apply to everyone, never attributed to this principal. | +| `unreadable` | UnreadablePolicy[] | yes | Hand-written policies whose body could not be read: they may name this principal, so the list above may be incomplete. | + +| Status | Meaning | +|---|---| +| 200 | The hand-written policies naming the principal | +| 400 | An unknown kind | +| 401 | Not authenticated | +| 403 | Lacks policy:read | +| 404 | No such user, group or role | +| 500 | Internal error | + ## GET `/api/control-plane/governance/tags/{catalog_id}` Operation id: `object_tags`. Auth: Session cookie. Forbidden (403): Missing policy:read and tag:read permission. diff --git a/platform/reference/rest-api/control-plane-secrets.md b/platform/reference/rest-api/control-plane-secrets.md index 0f1c8fe..494aba7 100644 --- a/platform/reference/rest-api/control-plane-secrets.md +++ b/platform/reference/rest-api/control-plane-secrets.md @@ -32,6 +32,7 @@ Operation id: `list`. Auth: Session cookie. Forbidden (403): Missing secret:read | `used_by` | integer (int64) | yes | | | `type` | string | yes | | | `keys` | string[] | yes | | +| `system_managed` | boolean | yes | Minted and managed by egret for an engine or a catalog: it cannot be renamed, rotated or deleted here, and pickers leave it out (KI-207). | | Status | Meaning | |---|---| @@ -65,6 +66,7 @@ Operation id: `create`. Auth: Session cookie and `X-Egret-CSRF` header. Forbidde | `used_by` | integer (int64) | yes | | | `type` | string | yes | | | `keys` | string[] | yes | | +| `system_managed` | boolean | yes | Minted and managed by egret for an engine or a catalog: it cannot be renamed, rotated or deleted here, and pickers leave it out (KI-207). | | Status | Meaning | |---|---| @@ -104,6 +106,7 @@ Operation id: `update`. Auth: Session cookie and `X-Egret-CSRF` header. Forbidde | `used_by` | integer (int64) | yes | | | `type` | string | yes | | | `keys` | string[] | yes | | +| `system_managed` | boolean | yes | Minted and managed by egret for an engine or a catalog: it cannot be renamed, rotated or deleted here, and pickers leave it out (KI-207). | | Status | Meaning | |---|---| @@ -112,7 +115,7 @@ Operation id: `update`. Auth: Session cookie and `X-Egret-CSRF` header. Forbidde | 401 | Not authenticated | | 403 | Missing secret:update permission | | 404 | No such secret | -| 409 | The new name clashes with an existing secret | +| 409 | The new name clashes with an existing secret, or the secret is managed by egret | | 500 | Internal error | ## DELETE `/api/control-plane/secrets/{id}` @@ -135,6 +138,6 @@ Operation id: `remove`. Auth: Session cookie and `X-Egret-CSRF` header. Forbidde | 401 | Not authenticated | | 403 | Missing secret:delete permission | | 404 | No such secret | -| 409 | Secret is still referenced by one or more connections | +| 409 | Secret is still referenced by one or more connections, or is managed by egret | | 500 | Internal error | diff --git a/platform/reference/rest-api/control-plane-storage.md b/platform/reference/rest-api/control-plane-storage.md index 5196710..71dde41 100644 --- a/platform/reference/rest-api/control-plane-storage.md +++ b/platform/reference/rest-api/control-plane-storage.md @@ -202,48 +202,50 @@ Operation id: `remove`. Auth: Session cookie and `X-Egret-CSRF` header. Forbidde ## GET `/api/control-plane/storage/{id}/download` -Operation id: `download_object`. Auth: Session cookie. Forbidden (403): Missing storage:read permission. +Operation id: `download_object`. Auth: Session cookie. Forbidden (403): Missing storage:read permission, or the key is under a prefix egret writes for its users and the caller is not a platform admin. Stream a single object from a storage connection's bucket as an attachment. | Parameter | In | Type | Required | Description | |---|---|---|---|---| | `id` | path | string (uuid) | yes | Storage connection id | -| `key` | query | string | yes | Object key to download | +| `key` | query | string | yes | Object key to download, relative to the connection's key prefix | | Status | Meaning | |---|---| | 200 | Object bytes streamed as an attachment | -| 400 | Invalid request or object could not be read | +| 400 | Invalid key (a `.`/`..` or empty segment, a backslash or a control character, also once percent-decoded), or the object could not be read | | 401 | Not authenticated | -| 403 | Missing storage:read permission | +| 403 | Missing storage:read permission, or the key is under a prefix egret writes for its users and the caller is not a platform admin | | 404 | No such storage connection | | 500 | Internal error | ## GET `/api/control-plane/storage/{id}/objects` -Operation id: `browse_objects`. Auth: Session cookie. Forbidden (403): Missing storage:read permission. +Operation id: `browse_objects`. Auth: Session cookie. Forbidden (403): Missing storage:read permission, or the prefix is one egret writes for its users and the caller is not a platform admin. Browse the objects/folders in a storage connection's bucket (folder-style). | Parameter | In | Type | Required | Description | |---|---|---|---|---| | `id` | path | string (uuid) | yes | Storage connection id | -| `prefix` | query | string | no | Folder prefix to list under | +| `prefix` | query | string | no | Folder prefix to list under, relative to the connection's key prefix | **Response 200 (application/json)**: `ListingDto` | Field | Type | Required | Description | |---|---|---|---| +| `root` | string | yes | The connection's key prefix (`""` when it has none). Every key in this listing, and every `prefix`/`key` the routes take, is relative to it. | | `folders` | string[] | yes | | | `objects` | ObjectDto[] | yes | | +| `hiddenSystemFolders` | string[] | yes | Folders at this level left out because egret writes its users' data there (stored SQL Lab results, Data App sources, workflow files) and the caller is not a platform admin. | | Status | Meaning | |---|---| | 200 | Folders and objects under the prefix | -| 400 | Invalid request or bucket could not be listed | +| 400 | Invalid prefix, or the bucket could not be listed | | 401 | Not authenticated | -| 403 | Missing storage:read permission | +| 403 | Missing storage:read permission, or the prefix is one egret writes for its users and the caller is not a platform admin | | 404 | No such storage connection | | 500 | Internal error | diff --git a/platform/reference/rest-api/databases.md b/platform/reference/rest-api/databases.md index 1c8be2d..4cdd59d 100644 --- a/platform/reference/rest-api/databases.md +++ b/platform/reference/rest-api/databases.md @@ -30,25 +30,11 @@ Operation id: `list`. Auth: Session cookie. Forbidden (403): Missing workspace p |---|---|---|---|---| | `workspace` | path | string | yes | Workspace slug | -**Response 200 (application/json)**: array of `DatabaseDetailDto` - -| Field | Type | Required | Description | -|---|---|---|---| -| `id` | string | yes | | -| `name` | string | yes | | -| `engine` | Engine | yes | Supported query engines. | -| `host` | string | yes | | -| `port` | integer (int32) | yes | | -| `options` | ConnOptions | yes | Non-secret connection options stored in `databases.options` (jsonb). | -| `has_secret` | boolean | yes | | -| `status` | string | yes | | -| `delegated` | boolean | yes | | -| `governed` | boolean | yes | See [`Database::governed`]. Surfaced on the detail DTO so the Data API author's database picker can warn BEFORE publish that the connection it is pointed at cannot carry a governed endpoint. | -| `engine_id` | string, nullable | no | Set (as a UUID string) when this connection was materialized by an engine→workspace assignment (read-only, provisioned by egret); `None` for a self-service connection. The frontend uses it to split the workspace Engines page into... | +**Response 200 (application/json)**: `DatabaseListing`, one of `DatabaseDetailDto[]`, `DatabaseListDto[]`. | Status | Meaning | |---|---| -| 200 | Registered databases in the workspace | +| 200 | Connections granted to the workspace: the full detail for a workspace admin, id/name/engine only for a member with query:run or dataset:read | | 401 | Not authenticated | | 403 | Missing workspace permission | | 500 | Internal error | diff --git a/platform/reference/rest-api/index.md b/platform/reference/rest-api/index.md index 502f353..281686b 100644 --- a/platform/reference/rest-api/index.md +++ b/platform/reference/rest-api/index.md @@ -130,10 +130,10 @@ described by their auth line here. | Data governance | [Permission model](rest-api/perm) | 30 | | Data governance | [Policy administration](rest-api/policy-admin) | 14 | | Data governance | [Ranger-compatible policy service](rest-api/ranger) | 6 | -| Data governance | [Governance data browser](rest-api/control-plane-governance) | 7 | +| Data governance | [Governance data browser](rest-api/control-plane-governance) | 11 | | Data governance | [Classification tags](rest-api/control-plane-tags) | 8 | | Control plane | [Access control](rest-api/control-plane-access) | 26 | -| Control plane | [Engines](rest-api/control-plane-engines) | 24 | +| Control plane | [Engines](rest-api/control-plane-engines) | 26 | | Control plane | [Catalog registry](rest-api/control-plane-catalog-registry) | 14 | | Control plane | [Egret warehouses](rest-api/control-plane-catalog) | 30 | | Control plane | [Storage connections](rest-api/control-plane-storage) | 7 | diff --git a/platform/reference/rest-api/ranger.md b/platform/reference/rest-api/ranger.md index 7e40989..bf46c55 100644 --- a/platform/reference/rest-api/ranger.md +++ b/platform/reference/rest-api/ranger.md @@ -12,16 +12,16 @@ This page lists 6 endpoints (OpenAPI tag `ranger`). It is generated from `openap | Method | Path | Operation id | Notes | |---|---|---|---| -| GET | `/service/plugins/definitions` | `list_defs` | | -| GET | `/service/plugins/definitions/name/{name}` | `get_def` | | -| GET | `/service/plugins/policies/download/{service}` | `download_policies` | `GET /plugins/policies/download/\{service\}?lastKnownVersion=N` | -| GET | `/service/plugins/policies/{id}` | `get_policy` | | -| GET | `/service/plugins/services` | `list_services` | | -| GET | `/service/tags/download/{service}` | `download_tags` | `GET /tags/download/\{service\}?lastKnownVersion=N` | +| GET | `/ranger/service/plugins/definitions` | `list_defs` | | +| GET | `/ranger/service/plugins/definitions/name/{name}` | `get_def` | | +| GET | `/ranger/service/plugins/policies/download/{service}` | `download_policies` | `GET /plugins/policies/download/\{service\}?lastKnownVersion=N` | +| GET | `/ranger/service/plugins/policies/{id}` | `get_policy` | | +| GET | `/ranger/service/plugins/services` | `list_services` | | +| GET | `/ranger/service/tags/download/{service}` | `download_tags` | `GET /tags/download/\{service\}?lastKnownVersion=N` | -## GET `/service/plugins/definitions` +## GET `/ranger/service/plugins/definitions` -Operation id: `list_defs`. Auth: Bearer token equal to `ranger.auth_token` (open when the token is empty). +Operation id: `list_defs`. Auth: Session cookie. Forbidden (403): An engine credential: these routes are for the operator bearer only. **Response 200 (application/json)**: `RangerServiceDefs` @@ -32,12 +32,13 @@ Operation id: `list_defs`. Auth: Bearer token equal to `ranger.auth_token` (open | Status | Meaning | |---|---| | 200 | All stored Ranger service-definitions | -| 401 | Missing or invalid bearer token (when a ranger auth token is configured) | +| 401 | Missing or invalid credential while `ranger.auth_mode` is `required` | +| 403 | An engine credential: these routes are for the operator bearer only | | 500 | Internal error | -## GET `/service/plugins/definitions/name/{name}` +## GET `/ranger/service/plugins/definitions/name/{name}` -Operation id: `get_def`. Auth: Bearer token equal to `ranger.auth_token` (open when the token is empty). +Operation id: `get_def`. Auth: Session cookie. Forbidden (403): An engine credential: these routes are for the operator bearer only. | Parameter | In | Type | Required | Description | |---|---|---|---|---| @@ -48,13 +49,14 @@ Operation id: `get_def`. Auth: Bearer token equal to `ranger.auth_token` (open w | Status | Meaning | |---|---| | 200 | The raw Ranger service-definition | -| 401 | Missing or invalid bearer token (when a ranger auth token is configured) | +| 401 | Missing or invalid credential while `ranger.auth_mode` is `required` | +| 403 | An engine credential: these routes are for the operator bearer only | | 404 | No such service-definition | | 500 | Internal error | -## GET `/service/plugins/policies/download/{service}` +## GET `/ranger/service/plugins/policies/download/{service}` -Operation id: `download_policies`. Auth: Bearer token equal to `ranger.auth_token` (open when the token is empty). +Operation id: `download_policies`. Auth: Session cookie. Forbidden (403): An engine credential asked for a profile its engine is not bound to. `GET /plugins/policies/download/\{service\}?lastKnownVersion=N` @@ -81,13 +83,14 @@ Returns 304 when the plugin's cached version matches; 200 + `ServicePolicies` ot |---|---| | 200 | The service's enabled policies + service-def (plugin download envelope) | | 304 | Not modified — the plugin's lastKnownVersion matches the current policy version | -| 401 | Missing or invalid bearer token (when a ranger auth token is configured) | +| 401 | Missing or invalid credential (engine Basic or operator bearer) while `ranger.auth_mode` is `required` | +| 403 | An engine credential asked for a profile its engine is not bound to | | 404 | No such policy profile | | 500 | Internal error | -## GET `/service/plugins/policies/{id}` +## GET `/ranger/service/plugins/policies/{id}` -Operation id: `get_policy`. Auth: Bearer token equal to `ranger.auth_token` (open when the token is empty). +Operation id: `get_policy`. Auth: Session cookie. Forbidden (403): An engine credential: these routes are for the operator bearer only. | Parameter | In | Type | Required | Description | |---|---|---|---|---| @@ -98,13 +101,14 @@ Operation id: `get_policy`. Auth: Bearer token equal to `ranger.auth_token` (ope | Status | Meaning | |---|---| | 200 | The raw stored Ranger policy body (with id) | -| 401 | Missing or invalid bearer token (when a ranger auth token is configured) | +| 401 | Missing or invalid credential while `ranger.auth_mode` is `required` | +| 403 | An engine credential: these routes are for the operator bearer only | | 404 | No such policy | | 500 | Internal error | -## GET `/service/plugins/services` +## GET `/ranger/service/plugins/services` -Operation id: `list_services`. Auth: Bearer token equal to `ranger.auth_token` (open when the token is empty). +Operation id: `list_services`. Auth: Session cookie. Forbidden (403): An engine credential: these routes are for the operator bearer only. **Response 200 (application/json)**: array of `RangerServiceRef` @@ -120,12 +124,13 @@ Operation id: `list_services`. Auth: Bearer token equal to `ranger.auth_token` ( | Status | Meaning | |---|---| | 200 | All policy profiles (Ranger services) | -| 401 | Missing or invalid bearer token (when a ranger auth token is configured) | +| 401 | Missing or invalid credential while `ranger.auth_mode` is `required` | +| 403 | An engine credential: these routes are for the operator bearer only | | 500 | Internal error | -## GET `/service/tags/download/{service}` +## GET `/ranger/service/tags/download/{service}` -Operation id: `download_tags`. Auth: Bearer token equal to `ranger.auth_token` (open when the token is empty). +Operation id: `download_tags`. Auth: Session cookie. Forbidden (403): An engine credential asked for a profile its engine is not bound to. `GET /tags/download/\{service\}?lastKnownVersion=N` @@ -151,7 +156,8 @@ Serves the real resource→tag mapping (`catalog_tag_assignments`) for the servi |---|---| | 200 | The service's catalog tag assignments (plugin tag-download envelope) | | 304 | Not modified — the plugin's lastKnownVersion matches the current tag version | -| 401 | Missing or invalid bearer token (when a ranger auth token is configured) | +| 401 | Missing or invalid credential (engine Basic or operator bearer) while `ranger.auth_mode` is `required` | +| 403 | An engine credential asked for a profile its engine is not bound to | | 404 | No such policy profile | | 500 | Internal error | diff --git a/platform/reference/rest-api/workflow.md b/platform/reference/rest-api/workflow.md index 3f5ed19..a7b74b6 100644 --- a/platform/reference/rest-api/workflow.md +++ b/platform/reference/rest-api/workflow.md @@ -93,6 +93,7 @@ Operation id: `list_executions`. Auth: Session cookie. Forbidden (403): Missing | `total` | integer (int64) | yes | Every execution matching the filter, not just this page's. Counted by a second statement carrying the same predicates, workspace one included. | | `limit` | integer (int64) | yes | The page size actually used, after the clamp. | | `offset` | integer (int64) | yes | The offset actually used, after the clamp. | +| `worker_enabled` | boolean | yes | Whether this deployment runs the workflow worker (`workflow.worker_enabled`, `EGRET_WORKFLOW__WORKER_ENABLED`; off by default). **False: an execution is created and stays `RUNNING`, and nothing runs it until an administrator turns the... | | Status | Meaning | |---|---| @@ -145,6 +146,7 @@ Operation id: `get_execution`. Auth: Session cookie. Forbidden (403): Missing wo | `runs` | TaskRunDto[] | yes | Every task run, in `run_ix` order — the executor's creation order. | | `can_manage` | boolean | yes | Whether the caller may resume and replay this execution: they hold `workflow:manage` in this workspace. A detail page hides both actions when this is false; the routes still check for themselves. | | `effective_labels` | LabelDto[] | yes | What the run sees as `labels`: the flow's, then `labels`, the later key winning. Empty when the frozen source does not parse. | +| `worker_enabled` | boolean | yes | Whether this deployment runs the workflow worker (`workflow.worker_enabled`, `EGRET_WORKFLOW__WORKER_ENABLED`; off by default). **False: an execution is created and stays `RUNNING`, and nothing runs it until an administrator turns the... | | Status | Meaning | |---|---| @@ -306,6 +308,7 @@ Tasks retrying, paused or not yet started end `KILLED` at once; a running task i | `runs` | TaskRunDto[] | yes | Every task run, in `run_ix` order — the executor's creation order. | | `can_manage` | boolean | yes | Whether the caller may resume and replay this execution: they hold `workflow:manage` in this workspace. A detail page hides both actions when this is false; the routes still check for themselves. | | `effective_labels` | LabelDto[] | yes | What the run sees as `labels`: the flow's, then `labels`, the later key winning. Empty when the frozen source does not parse. | +| `worker_enabled` | boolean | yes | Whether this deployment runs the workflow worker (`workflow.worker_enabled`, `EGRET_WORKFLOW__WORKER_ENABLED`; off by default). **False: an execution is created and stays `RUNNING`, and nothing runs it until an administrator turns the... | | Status | Meaning | |---|---| @@ -400,6 +403,7 @@ Run a finished execution again. | `runs` | TaskRunDto[] | yes | Every task run, in `run_ix` order — the executor's creation order. | | `can_manage` | boolean | yes | Whether the caller may resume and replay this execution: they hold `workflow:manage` in this workspace. A detail page hides both actions when this is false; the routes still check for themselves. | | `effective_labels` | LabelDto[] | yes | What the run sees as `labels`: the flow's, then `labels`, the later key winning. Empty when the frozen source does not parse. | +| `worker_enabled` | boolean | yes | Whether this deployment runs the workflow worker (`workflow.worker_enabled`, `EGRET_WORKFLOW__WORKER_ENABLED`; off by default). **False: an execution is created and stays `RUNNING`, and nothing runs it until an administrator turns the... | | Status | Meaning | |---|---| @@ -457,6 +461,7 @@ W7b Task 3 fix round 1, the user's ruling of 2026-10-11, after review found the | `runs` | TaskRunDto[] | yes | Every task run, in `run_ix` order — the executor's creation order. | | `can_manage` | boolean | yes | Whether the caller may resume and replay this execution: they hold `workflow:manage` in this workspace. A detail page hides both actions when this is false; the routes still check for themselves. | | `effective_labels` | LabelDto[] | yes | What the run sees as `labels`: the flow's, then `labels`, the later key winning. Empty when the frozen source does not parse. | +| `worker_enabled` | boolean | yes | Whether this deployment runs the workflow worker (`workflow.worker_enabled`, `EGRET_WORKFLOW__WORKER_ENABLED`; off by default). **False: an execution is created and stays `RUNNING`, and nothing runs it until an administrator turns the... | | Status | Meaning | |---|---| @@ -535,6 +540,7 @@ Operation id: `create_flow`. Auth: Session cookie and `X-Egret-CSRF` header. For | `updated_at` | string (date-time) | yes | | | `validation` | ValidationReportDto | yes | A flow's validation report, over the wire. Errors block publish; warnings do not — see [`ValidationReport`]'s own doc comment. | | `can_manage` | boolean | yes | Whether the caller may save, publish, run and delete this flow: they hold `workflow:manage` in this workspace. An editor opens read-only when this is false; every mutating route still checks for itself. | +| `worker_enabled` | boolean | yes | Whether this deployment runs the workflow worker (`workflow.worker_enabled`, `EGRET_WORKFLOW__WORKER_ENABLED`; off by default). **False: an execution is created and stays `RUNNING`, and nothing runs it until an administrator turns the... | | Status | Meaning | |---|---| @@ -635,6 +641,7 @@ Operation id: `get_flow`. Auth: Session cookie. Forbidden (403): Missing workflo | `updated_at` | string (date-time) | yes | | | `validation` | ValidationReportDto | yes | A flow's validation report, over the wire. Errors block publish; warnings do not — see [`ValidationReport`]'s own doc comment. | | `can_manage` | boolean | yes | Whether the caller may save, publish, run and delete this flow: they hold `workflow:manage` in this workspace. An editor opens read-only when this is false; every mutating route still checks for itself. | +| `worker_enabled` | boolean | yes | Whether this deployment runs the workflow worker (`workflow.worker_enabled`, `EGRET_WORKFLOW__WORKER_ENABLED`; off by default). **False: an execution is created and stays `RUNNING`, and nothing runs it until an administrator turns the... | | Status | Meaning | |---|---| @@ -685,6 +692,7 @@ With `if_updated_at`, the save happens only if the draft is still the one the ed | `updated_at` | string (date-time) | yes | | | `validation` | ValidationReportDto | yes | A flow's validation report, over the wire. Errors block publish; warnings do not — see [`ValidationReport`]'s own doc comment. | | `can_manage` | boolean | yes | Whether the caller may save, publish, run and delete this flow: they hold `workflow:manage` in this workspace. An editor opens read-only when this is false; every mutating route still checks for itself. | +| `worker_enabled` | boolean | yes | Whether this deployment runs the workflow worker (`workflow.worker_enabled`, `EGRET_WORKFLOW__WORKER_ENABLED`; off by default). **False: an execution is created and stays `RUNNING`, and nothing runs it until an administrator turns the... | | Status | Meaning | |---|---| @@ -768,6 +776,7 @@ Operation id: `start_flow_execution`. Auth: Session cookie and `X-Egret-CSRF` he | `runs` | TaskRunDto[] | yes | Every task run, in `run_ix` order — the executor's creation order. | | `can_manage` | boolean | yes | Whether the caller may resume and replay this execution: they hold `workflow:manage` in this workspace. A detail page hides both actions when this is false; the routes still check for themselves. | | `effective_labels` | LabelDto[] | yes | What the run sees as `labels`: the flow's, then `labels`, the later key winning. Empty when the frozen source does not parse. | +| `worker_enabled` | boolean | yes | Whether this deployment runs the workflow worker (`workflow.worker_enabled`, `EGRET_WORKFLOW__WORKER_ENABLED`; off by default). **False: an execution is created and stays `RUNNING`, and nothing runs it until an administrator turns the... | | Status | Meaning | |---|---| diff --git a/platform/workflows/executions/detail.md b/platform/workflows/executions/detail.md index b603be4..f6fb929 100644 --- a/platform/workflows/executions/detail.md +++ b/platform/workflows/executions/detail.md @@ -39,8 +39,8 @@ The **Overview** tab is the summary. Its sections: ![A loop's execution lists one child execution per value](/img/platform/workflows/execution-children.png) -:::warning Secret inputs are visible -A `SECRET` input is stored with the execution and shown here as plain text. See [Inputs and outputs](../flows/inputs-and-outputs.md). +:::note Secret inputs +A `SECRET` input shows `******` here, in the logs and in every API answer; the real value is stored encrypted and used only by the tasks. Task outputs are not masked. See [Secret inputs](../flows/inputs-and-outputs.md#secret-inputs). ::: ## Topology diff --git a/platform/workflows/flows/inputs-and-outputs.md b/platform/workflows/flows/inputs-and-outputs.md index f964fcf..928d715 100644 --- a/platform/workflows/flows/inputs-and-outputs.md +++ b/platform/workflows/flows/inputs-and-outputs.md @@ -56,7 +56,7 @@ Read an input anywhere an expression is allowed as `{{ inputs. }}`. | `DURATION` | A text box that expects an ISO-8601 duration such as `PT1H`. | The text. | | `EMAIL` | A text box that checks for an email address. | Text. | | `URI` | A text box that expects a scheme such as `https://`. | Text. | -| `SECRET` | A masked box. | The text. See the warning below. | +| `SECRET` | A masked box. | The text. See [Secret inputs](#secret-inputs). | | `SELECT` | A drop-down of `values`. | The chosen text. | | `MULTISELECT` | A multi-choice list. | A list. | | `ARRAY` | A list of rows of `itemType`. | A list, so `{{ inputs.list \| length }}` works. | @@ -76,7 +76,15 @@ The API, a schedule, a webhook and a Flow trigger start a flow without checking ### Secret inputs -A `SECRET` input is masked in the run form. The value it receives is stored with the execution as plain text and appears in the execution's **Overview** tab and in the execution API response. Do not use it for credentials that must not be readable by people who can see executions. The `secret()` function that reads workspace secrets resolves only on Kestra plugin tasks; see [Expressions](./expressions.md). +A `SECRET` input is masked in the run form, and stays masked after the run starts: + +- Every read shows `******` in its place: the answer to **Run**, the execution's **Overview** tab, the executions lists, the topology and the API responses (`GET …/executions`, `…/executions/{id}`, `…/graph`, `…/logs`, and the replay and resume answers). +- The run logs and a failed run's message are masked too: a log line that contains the value shows `******` instead. +- The real value is stored encrypted (AES-256-GCM) with the server's `secrets.master_key`, and only the worker opens it, so `{{ inputs. }}` still renders the real value in a task. A **Replay** uses the stored value, and a loop iteration and a retry keep it; a `Subflow` child masks the inputs that its own flow declares as `SECRET`. +- Executions stored before this masking existed are masked at the first start of the upgraded server, together with the log lines they wrote. +- A server without `secrets.master_key` cannot store the value, so a start that would store one is refused with 400: *the flow declares SECRET input(s) …, and this deployment cannot store a secret value: set secrets.master_key*. See [Configuration keys](../../reference/configuration-keys.md). + +What is **not** masked, the same rule as for `secret()` values: the outputs of a task (for example a `Return` of the input), the rendered values of a task, and a value the flow passes on into a non-`SECRET` input of another flow. Do not copy a secret input into an output. The `secret()` function that reads workspace secrets resolves only on Kestra plugin tasks; see [Expressions](./expressions.md). ## Start with inputs @@ -139,7 +147,7 @@ A flow can declare `outputs:` with an `id`, a `type` and a `value`. The engine a ## Limits and known issues - Only the run form validates inputs. Server-side validation of `required`, types and `values` is not implemented. -- `SECRET` inputs are not masked after the run starts. +- The outputs and rendered values of a task that uses a `SECRET` input are not masked; only the inputs and the logs are (KI-246). - `YAML` inputs are not parsed into an object in expressions. - Flow `outputs:` are not evaluated. diff --git a/platform/workflows/runtime.md b/platform/workflows/runtime.md index d65e8bc..a0ce980 100644 --- a/platform/workflows/runtime.md +++ b/platform/workflows/runtime.md @@ -10,13 +10,15 @@ This page is for the person who runs the platform: which switches make workflows | Setting | What it does | Default | |---|---|---| -| `worker_enabled` | Starts the claim loop and the lease reaper. Without it an execution is created and then waits in *Running* for ever. | `false` | +| `worker_enabled` | Starts the claim loop and the lease reaper. Without it an execution is created and then waits in *Running* for ever; the pages say so (see below). | `false` | | `scheduler_enabled` | Starts the trigger loop that fires schedules and flow triggers. Independent of the worker. | `false` | | `webhooks_enabled` | Mounts the unauthenticated webhook routes. Off, they answer 404. | `false` | | `container_runner` | `none`, `docker` or `kubernetes`. Needed for script tasks and the plugin host. | `none` | Run the worker and the scheduler together unless you want a replica that only serves. A scheduler without a worker creates executions nobody advances, and the **Triggers** page says so. +When the worker is off, the flow page, the **Run** dialog, the **Executions** list and an execution that has not ended show the notice *The workflow worker is off on this server, so executions are created but do not run. An administrator turns it on with workflow.workerEnabled (EGRET_WORKFLOW__WORKER_ENABLED=true).* The **Triggers** page uses the same sentence. The API carries the same information as `worker_enabled` on a flow, on the executions list, on an execution and in the answer to **Run**, replay and restart. + ## What runs on the Docker demo stack The Docker demo stack (`just demo-up`) sets `EGRET_WORKFLOW__WORKER_ENABLED=true` and `EGRET_WORKFLOW__SCHEDULER_ENABLED=true` and leaves the other switches at their defaults. Each item below was tried on it. @@ -40,6 +42,7 @@ The Docker demo stack (`just demo-up`) sets `EGRET_WORKFLOW__WORKER_ENABLED=true |---|---|---| | `tick_interval_secs` | Seconds between claim ticks: the floor on how soon a ready task starts. | 1 | | `batch` | Task runs one tick may claim. | 8 | +| `max_concurrent_runs` | Claimed task runs one replica runs at the same time. A tick claims at most the free slots (and at most `batch`), so a slow task holds one slot, not the replica; a ready run beyond the bound stays unclaimed for another replica or a later tick. `0` is refused at start. Environment variable `EGRET_WORKFLOW__MAX_CONCURRENT_RUNS`. | 8 | | `trigger_interval_secs` | Seconds between trigger evaluations. | 1 | | `reaper_interval_secs` | Seconds between passes that hand back abandoned task runs. | 60 | | `working_timeout_s` | How long a task run may hold its lease before the reaper takes it back. | 3600 | @@ -86,7 +89,8 @@ Script and plugin tasks also need a `container_runner`, and the image on the pla - Flow `concurrency:`, `sla:`, `checks:` and `quotas:` are accepted and not enforced. - Flow-level `finally:` and `afterExecution:` do not run. - Deleting a flow keeps its namespace and id reserved. -- **One slow task holds up the others on its replica**. The worker runs the task runs it claimed in one tick one after another, and claims more only when they have all ended. On the demo stack a long `Sleep` kept four storage tasks of other executions in *Running* for 2 min 13 s, and no new task started, until the `Sleep` was killed. Keep long waits out of flows that share a replica with others, or kill them; a `Pause` does not hold the worker. +- A replica runs at most `max_concurrent_runs` task runs at once (8 by default). With that many long tasks running, further ready tasks wait for a free slot or another replica. +- Restarting Egret while a long task runs can leave the run in *Running* until the reaper takes it back after `working_timeout_s`, an hour by default (KI-237). - The webhook URL is served by the backend under `/workflows/v1/`, a path the frontend's web server and the Helm ingress do not forward to the backend. Route it yourself before you switch webhooks on. - Script tasks, plugin tasks and webhook calls were not run on the Docker demo stack; see the table above. diff --git a/platform/workflows/triggers/index.md b/platform/workflows/triggers/index.md index b95e769..4151120 100644 --- a/platform/workflows/triggers/index.md +++ b/platform/workflows/triggers/index.md @@ -32,7 +32,7 @@ triggers: Three things must be on, and the **Triggers** page tells you which is missing: 1. **The scheduler** (`EGRET_WORKFLOW__SCHEDULER_ENABLED=true`). Without it schedules and flow triggers do not fire, and the page says *The scheduler is off on this server, so schedules and flow triggers do not fire.* -2. **The worker** (`EGRET_WORKFLOW__WORKER_ENABLED=true`). Without it triggers create executions that never run: *The worker is off, so triggered executions are created but do not run.* +2. **The worker** (`EGRET_WORKFLOW__WORKER_ENABLED=true`). Without it triggers create executions that never run: *The workflow worker is off on this server, so executions are created but do not run. An administrator turns it on with workflow.workerEnabled (EGRET_WORKFLOW__WORKER_ENABLED=true).* 3. **Webhooks** (`EGRET_WORKFLOW__WEBHOOKS_ENABLED=true`), for webhook triggers only: *Webhooks are off on this server; their URLs answer 404.* A disabled flow fires nothing: the page says *This flow is disabled, so none of its triggers fire.* diff --git a/platform/workspace-settings/databases.md b/platform/workspace-settings/databases.md index 291e397..14798ab 100644 --- a/platform/workspace-settings/databases.md +++ b/platform/workspace-settings/databases.md @@ -10,6 +10,8 @@ A **database connection** (shown in the UI as an *engine connection*) tells the *Query engine connections this workspace can use in SQL Lab and charts.* The page needs workspace-admin rights; without them it says *Requires workspace-admin to manage engines.* A new workspace says *No engines connected yet.* +A member who is not a workspace admin but holds `query:run` or `dataset:read` still gets the list of granted connections through the API, cut down to id, name, engine and delegation, so that SQL Lab and the dataset pickers can offer them (see [SQL Lab permissions](../explore/sql-lab/permissions.md#which-databases-you-can-use)). This page shows such a member the *Requires workspace-admin to manage engines.* notice instead of the rows. + ## The connection list A summary strip counts the **connections**, how many are **ok** and how many are in **error**. **Search connections…** filters by name. If the workspace has engines assigned by the platform admins, they appear in a **Provisioned by egret** section above **Your connections**; those are read-only (*Provisioned by egret — read-only*). @@ -99,7 +101,7 @@ On the Docker demo stack, the workspace `default` connects to the demo Trino as ## Limits and known issues - Egret validates reachability on save, but a new connection shows **Unknown** until you press **Test**. -- Only workspace admins can list the workspace's connections. Members with `ws_editor`, `ws_viewer` or a custom role get *403* on the list, so the database picker in SQL Lab is empty for them, also for a provisioned engine assigned to the workspace. +- When the workspace has no connection at all, a non-admin sees the same *No engines connected yet.* empty state as an admin, and its **Add connection** button then answers 403 (KI-243). - Only Trino and StarRocks connections can be added here. External engines for Governance are registered in the Control Plane (see [Register an external engine](../control-plane/query-engines/register-external.md)). ## Further reading