Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions platform/control-plane/catalogs/catalog-detail.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,22 @@ An engine's client holds `catalog · manage`, so it can do all of these. For exa
| 400 | A rename or commit body egret cannot read | *malformed rename request: …*, *malformed multi-table commit: …*, *malformed table commit: …* |
| 413 | A rename body over 1 MiB, or a commit body over 16 MiB | *the body of a rename request is larger than 1048576 bytes* |

### Storage credentials follow the grant {#storage-credentials-follow-the-grant}

When a client loads a table, the catalog hands it short-lived S3 credentials scoped to that table's location, so it can read the table's files without static object-store keys. What those credentials allow depends on the role's grant on the table:

| The role's grant covering the table | Vended credentials |
|---|---|
| **read** (on the catalog, the namespace or the table) | Read-only: `s3:GetObject` and `s3:GetObjectVersion` on the table's location. The client can scan the table but cannot write, overwrite or delete any of its files. |
| **write** or **manage** | Read, write and delete on the table's location. |
| None | No credentials: the load itself is refused with 403. |

The same rule applies with the `X-Iceberg-Access-Delegation: vended-credentials` header, without the header, and on `GET …/tables/{t}/credentials`. Only `X-Iceberg-Access-Delegation: client-managed` asks for no credentials. A role with several grants gets the widest one that covers the table: **read** on the namespace `sales` plus **write** on `sales.orders` vends read-write-delete on `sales.orders` and read-only on every other table in `sales`. Creating or staging a table (which needs **write** on the namespace) vends read-write-delete on the new table's location. A view load vends read-only credentials on the view's metadata location.

An engine or client with only **read** grants can therefore read but not write. Its writes fail at the object store with *access denied* when it uploads the first data file, before anything reaches the catalog. PyIceberg, for example, raises `OSError: … AWS Error ACCESS_DENIED during CreateMultipartUpload operation: Access Denied` on an append. Engines provisioned by egret hold `catalog · manage` and write as before.

The credentials stay valid for about an hour, and the catalog reuses one key for every caller with the same permission on the same table. Revoking or narrowing a grant stops new credentials at once, but does not cancel the ones already handed out (KI-315).

## Catalog Clients (warehouse)

*Machine credentials (client id + secret) an engine or notebook uses to authenticate.*
Expand All @@ -120,7 +136,7 @@ A dropped table keeps its classifications in the **Recycle bin**, and **Undrop**

![The Connect tab of demo_warehouse with the REST catalog URI and the PyIceberg snippet](/img/platform/admin/cp-catalog-connect.png)

*egret exposes this warehouse as an Iceberg REST catalog. Point any Iceberg-compatible engine at it using a Catalog Client's credentials.* The tab shows the **CATALOG URI**, **WAREHOUSE** and **OAUTH TOKEN URL**, and ready-made snippets for **PyIceberg**, **Trino** and **StarRocks**. Replace the `<CLIENT_ID>:<CLIENT_SECRET>` placeholders with a client's values, and the example table `demo_ns.events` with one of yours. If the engine runs in Docker, use the egret host reachable from that container instead of the public address. On the demo stack the snippet's `s3.endpoint` is `http://rustfs:9000`, an address that resolves only inside the Docker network. The snippets ask egret to vend short-lived, table-scoped S3 credentials, so no static object-store keys are needed.
*egret exposes this warehouse as an Iceberg REST catalog. Point any Iceberg-compatible engine at it using a Catalog Client's credentials.* The tab shows the **CATALOG URI**, **WAREHOUSE** and **OAUTH TOKEN URL**, and ready-made snippets for **PyIceberg**, **Trino** and **StarRocks**. Replace the `<CLIENT_ID>:<CLIENT_SECRET>` placeholders with a client's values, and the example table `demo_ns.events` with one of yours. If the engine runs in Docker, use the egret host reachable from that container instead of the public address. On the demo stack the snippet's `s3.endpoint` is `http://rustfs:9000`, an address that resolves only inside the Docker network. The snippets ask egret to vend short-lived, table-scoped S3 credentials, so no static object-store keys are needed. A client whose role holds only **read** grants gets read-only credentials; see [Storage credentials follow the grant](#storage-credentials-follow-the-grant).

## Settings (warehouse) {#settings-warehouse}

Expand All @@ -141,7 +157,8 @@ A dropped table keeps its classifications in the **Recycle bin**, and **Undrop**

- A catalog test before attaching checks only that the host answers on its port; a TPC-H catalog has no host to check.
- Attach and detach on Trino restart the coordinator.
- A Catalog Client with only a **read** grant receives vended S3 credentials that can also write and delete the files of the table it loads, so it can damage that table without any catalog write (KI-311). Give **read** grants only to clients you would trust with write access to those tables until this is fixed.
- Vended credentials outlive a revoked or narrowed grant by up to about an hour, and callers with the same permission on a table share one key (KI-315).
- Two warehouses that share a location can let a namespace writer in one of them obtain read-write credentials on a table of the other, when their storage connections write the same endpoint differently (KI-316). Give each warehouse its own bucket or key prefix.
- Any **write** grant lets a role create and list namespaces, and a grant on one table lets it list every table name in that namespace (KI-312).

## Further reading
Expand Down
3 changes: 2 additions & 1 deletion platform/control-plane/query-engines/provision.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ A StarRocks engine has its own configuration screen with sections such as **Repl
## What happens next

1. Egret starts the cluster, creates a **policy profile** with the engine's name (for example `docs-trino`, with the tag service `docs-trino_tag`) and binds it.
2. The attached catalogs are applied to the engine (state **Applied** on the **Catalogs** tab).
2. The attached catalogs are applied to the engine (state **Applied** on the **Catalogs** tab). For an egret warehouse, egret creates a Catalog Client for the engine under the warehouse's `engine` role, which holds `catalog · manage`, so the engine reads and writes every table.
3. Assign the engine to a workspace on its **Workspaces** tab. It then appears in the workspace's SQL Lab as a connection with the engine's name, and in **Workspace settings → Connections** under **Provisioned by egret**.
4. The Governance area lists the engine under **Engine & policy**, and grants on its catalogs reach it.

Expand All @@ -97,6 +97,7 @@ The demo's Docker provisioner only starts images from `trinodb/trino`, with no p

- A Trino catalog attach or detach restarts the coordinator, so queries in flight are lost.
- The Ingress section is offered on the Docker provisioner, which refuses any ingress (KI-264).
- egret reuses an existing role named `engine` on the warehouse without checking its grants. If that role was narrowed to **read**, the engine can query but its writes fail at the object store with *access denied*, because the catalog vends read-only storage credentials (see [Storage credentials follow the grant](../catalogs/catalog-detail.md#storage-credentials-follow-the-grant)).

## Further reading

Expand Down
2 changes: 2 additions & 0 deletions platform/control-plane/query-engines/register-external.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ A Trino profile created in the console has no tag service and cannot be bound. U

*Declare which registry catalogs this engine serves, so grants on them reach its profile. egret does not create these catalogs on your engine — create them there with the same names.* Without **catalog_registry:read** the list is replaced by *Listing catalogs needs the catalog_registry:read permission, which your account does not have. You can register the engine now and declare catalogs later.*

For an egret warehouse, you create the catalog on your engine yourself with a Catalog Client from the catalog's **Connect** tab. The client's role decides what the engine may do: with only **read** grants, the storage credentials the catalog vends are read-only, so the engine can query the tables but its writes fail at the object store with *access denied*. Give the role **write** or **manage** if the engine writes. See [Storage credentials follow the grant](../catalogs/catalog-detail.md#storage-credentials-follow-the-grant).

## After you register

A dialog, *Engine registered: save its Ranger credential*, shows the engine's Ranger credential once: *Copy it now: egret shows this password once.* Its button depends on the profile:
Expand Down
9 changes: 7 additions & 2 deletions platform/deploy-operate/storage-and-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,10 @@ s3.path-style-access=true
```

With `vended-credentials-enabled`, engines get credentials scoped to the table they load and the static `s3.aws-*` keys are
not needed. A warehouse is provisioned with STS enabled and remote signing off, and egret refuses every S3 signer request
not needed. The credentials follow the client's grant: a role with only `read` on the table gets read-only credentials, and
`write` or `manage` gets read, write and delete, so an engine whose client holds only `read` grants cannot write (see
[Storage credentials follow the grant](../control-plane/catalogs/catalog-detail#storage-credentials-follow-the-grant)).
A warehouse is provisioned with STS enabled and remote signing off, and egret refuses every S3 signer request
(`…/aws/s3/sign`) with 403, whatever the client's grants.

### Setting it up in the app
Expand All @@ -107,7 +110,9 @@ not needed. A warehouse is provisioned with STS enabled and remote signing off,

:::caution One warehouse, one prefix
A warehouse without a `key_prefix` claims the whole bucket, and the catalog rejects a warehouse whose location overlaps an
existing one. Give each warehouse its own bucket or key prefix.
existing one. Give each warehouse its own bucket or key prefix. The overlap check compares the storage connections' endpoints
as written, so `http://rustfs:9000` and `http://rustfs:9000/`, or two host names for one store, are not seen as the same
store (KI-316).
:::

### Check it
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ of a real failure:
| 400 *malformed rename request*, *malformed multi-table commit* or *malformed table commit* | The request body is not the JSON the Iceberg REST API defines. |
| 413 *the body of a … is larger than … bytes* | A rename body is limited to 1 MiB, and a commit body to 16 MiB. |
| 403 on the warehouse | The token's warehouse differs from the one requested. |
| A write fails at the object store with *access denied* (PyIceberg: `ACCESS_DENIED during CreateMultipartUpload`), while reads of the same table work | The client's role has only a `read` grant covering the table, so the catalog vended read-only storage credentials. Give the role `write` on the table, its namespace or the catalog. Nothing was committed. See [Storage credentials follow the grant](../../control-plane/catalogs/catalog-detail#storage-credentials-follow-the-grant). |
| Warehouse creation refused | A warehouse without a `key_prefix` claims the whole bucket, and the catalog rejects a location that overlaps another warehouse. Give each its own bucket or prefix. |

egret's refusals use the Iceberg REST error format, `{"error": {"message", "type", "code"}}`, so the client shows the
Expand Down Expand Up @@ -107,6 +108,10 @@ The catalog hands engines short-lived, table-scoped credentials by calling STS `
without STS support cannot serve them. RustFS supports it; MinIO's current public images are no longer available. Without
vended credentials, engines need static `s3.*` keys in their catalog configuration.

Vended credentials follow the client's grant: `read` vends read-only credentials, `write` and `manage` read-write-delete.
A write by a read-only client fails at the object store, not in the catalog (see the table above). Credentials already
handed out stay valid for up to about an hour after a grant is revoked or narrowed (KI-315).

## Storage connections in the app

**Control Plane → Storage** lists registered connections. A connection whose secret cannot be read (a changed master key) or
Expand Down
21 changes: 21 additions & 0 deletions platform/deploy-operate/upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,27 @@ A role reaches no table it could not already reach by spelling the name as its g
limited to 1 MiB and 16 MiB (413). See
[What each operation needs](../control-plane/catalogs/catalog-detail#what-each-operation-needs).

### Vended credentials follow catalog grants (platform #760)

From platform #760 the S3 credentials egret's Iceberg catalog vends on a table load follow the client's grant. There is no
migration and no API change.

- A role with only a `read` grant covering the table (on the table, its namespace or the catalog) gets read-only credentials
(`s3:GetObject`, `s3:GetObjectVersion`). Before, every client that could load a table got read-write-delete credentials on
its files, even without the `X-Iceberg-Access-Delegation` header (KI-311).
- `write` and `manage` still get read-write-delete. The same rule applies on `…/tables/{t}/credentials`. View loads stay
read-only.
- **Behaviour change:** an engine or client that uses a Catalog Client with only `read` grants can still read, but it can no
longer write or delete table files. Its writes now fail at the object store with *access denied* (PyIceberg:
`ACCESS_DENIED during CreateMultipartUpload`) instead of reaching the catalog. If such an engine is meant to write, give its
role `write` or `manage`.
- Engines provisioned by egret hold `catalog · manage` and are not affected; Trino's creates, inserts, updates, deletes,
renames and drops were checked.
- Credentials vended before the upgrade, read-write-delete ones handed to readers included, stay valid until they expire,
about an hour after they were issued (KI-315).

See [Storage credentials follow the grant](../control-plane/catalogs/catalog-detail#storage-credentials-follow-the-grant).

### StarRocks 4.1.6 enforces view privileges

See [Engines and Ranger setup](./engines-and-ranger#supported-versions).
Expand Down
8 changes: 8 additions & 0 deletions platform/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,14 @@ table name is not enough. A commit needs `write` on the table its body names, an
A recursive namespace drop needs a `catalog`-level grant. Grants match names case-insensitively, as the catalog does. See
[What each operation needs](./control-plane/catalogs/catalog-detail#what-each-operation-needs).

### Why can a catalog client read a table but not write to it, with *access denied* from the object store?

The storage credentials the catalog vends follow the client's grant. A role with only `read` on the table (or on its
namespace or the catalog) gets read-only credentials, so it can scan the table but its writes fail at the object store
when it uploads the first file; PyIceberg reports `ACCESS_DENIED during CreateMultipartUpload`. Give the role `write` on the
table, its namespace or the catalog. Engines provisioned by egret hold `catalog · manage` and are not affected. See
[Storage credentials follow the grant](./control-plane/catalogs/catalog-detail#storage-credentials-follow-the-grant).

### Why did my StarRocks policies stop applying after a restart?

The activation `access_control = ranger` lives in the front end's `fe.conf`, which a recreated container regenerates, and
Expand Down
Loading
Loading