diff --git a/platform/data-api/activity.md b/platform/data-api/activity.md index 24e7a8b..1bddff3 100644 --- a/platform/data-api/activity.md +++ b/platform/data-api/activity.md @@ -6,7 +6,7 @@ sidebar_position: 7 **Activity** is the access log of the Data API: every call to one of the workspace's endpoints, with its status, duration and whether the cache answered. Use it to see who is calling what, find the call behind an error report, and spot an endpoint that has started to fail. -![The Activity page with summary tiles, a per-endpoint table and the calls list](/img/platform/data-api/activity.png) +![The Activity page for 24h: 282 calls, a 20.2% error rate shown as Failing, latency and cache-hit tiles, the per-endpoint table with No matching endpoint last, and the newest calls including 429 rate_limited](/img/platform/data-api/activity.png) Open **Explore → Data API → Activity**. diff --git a/platform/data-api/calling-endpoints.md b/platform/data-api/calling-endpoints.md index 83a2102..c01a9d2 100644 --- a/platform/data-api/calling-endpoints.md +++ b/platform/data-api/calling-endpoints.md @@ -13,14 +13,14 @@ You need: - a **published** endpoint (here, `GET /orders/by-status`); - a [service account with an OAuth2 client](./clients), and its `client_id` and `client_secret`; - an invoke [grant](./endpoints/access) for that service account on the endpoint; -- the base URL of your deployment. On the demo instance it is `http://localhost:8080`. +- the base URL of your deployment. On the Docker demo stack it is `http://egret.localtest.me:8080` (or the port set in `DEMO_PORT`). Set them as shell variables: ```bash -export BASE=http://localhost:8080 +export BASE=http://egret.localtest.me:8080 export WS=default -export CLIENT_ID=egret-dataapi-bf232700-aeb0-461f-9b13-f1ce18ba805e +export CLIENT_ID=egret-dataapi-cf9e7fe8-f970-45af-901d-66a9a24b9ab7 export CLIENT_SECRET='…the secret shown once when the client was issued…' ``` @@ -180,7 +180,7 @@ content-type: application/json retry-after: 1 ``` -On the walk-through endpoint (**Per caller (qps)** 5, **Burst** 10), forty simultaneous requests produced twelve `200` answers and twenty-eight `429`. A client should wait the `Retry-After` interval and retry, with a little jitter: +On the walk-through endpoint (**Per caller (qps)** 5, **Burst** 10), thirty simultaneous requests from one client produced 10 to 12 `200` answers and the rest `429`. A client should wait the `Retry-After` interval and retry, with a little jitter: ```bash for i in 1 2 3; do diff --git a/platform/data-api/clients.md b/platform/data-api/clients.md index 4c8b593..c2e8a6e 100644 --- a/platform/data-api/clients.md +++ b/platform/data-api/clients.md @@ -6,7 +6,7 @@ sidebar_position: 3 A consumer that calls your endpoints needs a credential. In the Data API that is an **OAuth2 client** owned by a **service account**. A service account is a machine identity: it is the "who" that endpoints grant access to and that the query engine sees. A client is one `client_id` and `client_secret` pair that lets a program prove it acts for that account. -![The Clients page with two service accounts and their clients](/img/platform/data-api/clients-list.png) +![The Clients page with three service accounts and their clients: Active, Revoked and Expired](/img/platform/data-api/clients-list.png) Open **Explore → Data API → Clients**. Each service account is a header row with its clients listed beneath it. @@ -26,7 +26,7 @@ One service account can have several clients, for example one per consuming syst |---|---| | **Client** | The service account's name and email, with the number of clients and when it was created; under it, each client's name. | | **ID** | A shortened `client_id` (`egret-da…805e`) with a copy button. | -| **Status** | **Active**, **Revoked** (with *revoked 3 minutes ago*) or **Expired** (with the expiry date). | +| **Status** | **Active**, **Revoked** (with *revoked an hour ago*) or **Expired** (with *expires 9 months ago*, which means it expired then; KI-025). | | **Last used** | When the client last minted a token, or *Never*. | **Search service accounts…** filters by name. The **⋯** menu on a service account offers **Issue client…** and **Delete**; on an active client it offers **Revoke…**. A client that is already revoked has no menu. @@ -48,7 +48,7 @@ Names are unique in a workspace. Reusing one is refused: `service account alread 2. Enter a **Name** for the credential and, optionally, **Expires**. Name it after the consumer that will use it, so revoking the right one later is obvious. Leave **Expires** blank for a client that lives until revoked. 3. Press **Create client**. -![The New client page](/img/platform/data-api/client-new.png) +![The New client page for orders-reporting-bot with the name weekly-export](/img/platform/data-api/client-new.png) The next page shows three values: @@ -56,7 +56,7 @@ The next page shows three values: |---|---| | **Client ID** | Public identifier, `egret-dataapi-` followed by a UUID. | | **Client secret** | The secret. **Shown only once.** Press **Copy** and store it in the consumer's secret manager now. | -| **Token endpoint** | Where the client exchanges its credentials, for example `http://localhost:8080/oauth/token`. | +| **Token endpoint** | Where the client exchanges its credentials, for example `http://egret.localtest.me:8080/oauth/token`. | Reloading the page or returning to it later shows *Client secret no longer available*: the secret cannot be read back, by design. If it is lost, revoke the client and issue another. @@ -86,7 +86,7 @@ The status becomes **Revoked**. A request for a token with a revoked secret is a A token is a signed, stateless credential. Revoking a client stops it minting new tokens, but a token already issued keeps working until it expires, up to 10 minutes later. In the walk-through, a token minted shortly before revocation still returned `200` afterwards. To cut a consumer off at once, remove the **grant** on the endpoint: grants are read on every request and take effect on the next call. ::: -An **Expired** client (its **Expires** date has passed) behaves like a revoked one for new tokens. +An **Expired** client (its **Expires** date has passed) behaves like a revoked one for new tokens: on the demo stack the expired client `partner-pilot` got `{"error":"invalid_client"}`. ## Delete a service account diff --git a/platform/data-api/endpoints/access.md b/platform/data-api/endpoints/access.md index 1fd2d4e..0fac986 100644 --- a/platform/data-api/endpoints/access.md +++ b/platform/data-api/endpoints/access.md @@ -6,7 +6,7 @@ sidebar_position: 7 Nobody can call an endpoint until you say so. The **Access** tab lists the workspace's service accounts and lets you grant each one permission to **invoke** this endpoint. Grants are per endpoint, so a consumer you give the sales endpoint cannot call the finance one. -![The Access tab with one granted and one ungranted service account](/img/platform/data-api/endpoint-access.png) +![The Access tab of Orders by status: finance-dashboard and orders-reporting-bot granted on this endpoint, partner-sandbox not granted](/img/platform/data-api/endpoint-access.png) ## Grant a service account access to an endpoint @@ -37,7 +37,7 @@ The status under **Invoke** is one of: Two notes appear on a row when they matter: -- **Revoking will not remove access**, shown on a granted account that also reaches invoke through the workspace: revoking removes the grant but not the access. +- **Revoking will not remove access**, meant for a granted account that also reaches invoke through the workspace: revoking removes the grant but not the access. On the demo stack this note is shown on **every** granted service account, including ones with no workspace role, and is wrong for them: revoking `finance-dashboard` on **Orders by status** made its next call answer `403 not_granted`. Trust the result of **Try it** after a revoke, not the note. - **Revoking may not remove access**, shown when Egret cannot check the inherited access, for example while the endpoint is not live. The check for inherited access runs the account's own OpenAPI document and looks for this endpoint in it, so it only works on a live endpoint. A draft or disabled endpoint says so above the table: *This endpoint is not live, so nothing is being served yet and egret cannot check workspace-inherited invoke rights against it. Grants recorded here are real and take effect when it is published.* diff --git a/platform/data-api/endpoints/config.md b/platform/data-api/endpoints/config.md index fe48123..2d74a16 100644 --- a/platform/data-api/endpoints/config.md +++ b/platform/data-api/endpoints/config.md @@ -6,7 +6,7 @@ sidebar_position: 4 The **Config** tab holds everything about an endpoint that is not its query or its response: its name and URL, whether answers are cached, how big and how slow an answer may be, and how often each caller may ask. Changes are saved with **Save** and reach consumers when you [publish](./publishing). -![The Config tab showing the Endpoint and Cache groups](/img/platform/data-api/endpoint-config.png) +![The Config tab of Orders by status: Endpoint, Cache (enabled, TTL 60, Per caller), Limits (100 rows, 30 seconds) and Rate limit (5 qps, burst 10) groups, and Validate saved draft](/img/platform/data-api/endpoint-config.png) The tab is a table with **Setting**, **Details** and **Value** columns, grouped under **Endpoint**, **Cache**, **Limits** and **Rate limit**, followed by a **Validate** check. @@ -90,7 +90,7 @@ How many requests may arrive at once, above the sustained rate, before throttlin A ceiling across all callers of this endpoint together, whatever their individual limits. `0` means no endpoint-wide limit. -With **Per caller (qps)** `5` and **Burst** `10`, forty simultaneous requests from one client produced twelve answers and twenty-eight `429` responses with `Retry-After: 1`: the ten burst slots plus the tokens that refilled while the burst ran. Sequential calls at normal speed were not throttled. +With **Per caller (qps)** `5` and **Burst** `10` on **Orders by status**, thirty simultaneous requests from one client produced 10 to 12 answers and 18 to 20 `429` responses with `Retry-After: 1`, and sixteen simultaneous requests from a second client of the same service account produced eleven answers and five `429`: the ten burst slots plus the tokens that refilled while the burst ran. Each client credential has its own bucket. Sequential calls at normal speed were not throttled. :::note With one Egret replica, rate-limit counters and the cache live in the server's memory. Running more than one replica requires a shared store (Redis or Valkey); Egret refuses to start with several replicas and no store, so limits are never silently multiplied. If the shared store cannot be reached, a call to an endpoint that has a rate limit is refused with `503 rate_limiter_unavailable`, and a cache lookup that fails is treated as a miss. diff --git a/platform/data-api/endpoints/index.md b/platform/data-api/endpoints/index.md index e731cc1..b2e811c 100644 --- a/platform/data-api/endpoints/index.md +++ b/platform/data-api/endpoints/index.md @@ -6,7 +6,7 @@ sidebar_position: 1 An endpoint is one query published at one URL. This page covers the list of endpoints, how to create an endpoint, and how the editor is organised. Each editor tab has its own page. -![The endpoints list showing six live endpoints](/img/platform/data-api/endpoints-list.png) +![The endpoints list with eight endpoints: six live, one draft and one disabled](/img/platform/data-api/endpoints-list.png) ## The endpoints list @@ -16,14 +16,14 @@ Open **Explore → Data API → Endpoints**. The list shows every endpoint in th |---|---| | **Name** | The endpoint's name, with a star to bookmark it. | | **Method & path** | The HTTP method and the path callers append to the workspace's base URL, for example `GET /orders/by-status`. | -| **State** | **Live**, **Draft**, or **Disabled** (see [lifecycle](./overview#endpoint-lifecycle)). | +| **State** | **Live**, **Draft**, or **Disabled · was live** (see [lifecycle](./overview#endpoint-lifecycle)). | | **Engine** | The engine of the endpoint's database connection, for example Trino. | | **Last edited** | When the draft last changed. | Above the table: - **Search endpoints…** filters by name. -- **State** and **Engine** narrow the list. The counters under the filters (`6 endpoints total · 6 live · 0 draft · 0 disabled`) are clickable shortcuts for the State filter. +- **State** and **Engine** narrow the list. The counters under the filters (`8 endpoints total · 6 live · 1 draft · 1 disabled`) are clickable shortcuts for the State filter. - The toolbar buttons are **Bookmarks** (show only starred endpoints), **Bulk select** (select several rows and press **Move N to Trash**), **New folder**, and **Trash**. - **Download OpenAPI** exports an OpenAPI document; see [OpenAPI export](./openapi). - **+ Endpoint** opens the create page. @@ -39,10 +39,10 @@ The **⋯** menu on a row offers: | **Open** | Opens the editor. | | **Disable** | Takes a live endpoint out of service. Calls get `404 not_found`. Its live version and grants are kept. | | **Enable** | Puts a disabled endpoint back, serving the version it had. Not available for an endpoint that was never published. | -| **Move to folder** | Moves it into a folder or a new folder. | -| **Move to Trash** | Soft-deletes the endpoint. Disabled for a **Live** endpoint: the menu says *Disable it first*. | +| **Move to new folder…** | Creates a folder and moves the endpoint into it. With folders in the workspace the menu also offers **Move to "folder"** for each one, and **Remove from folder** for an endpoint in a folder. | +| **Move to Trash** | Soft-deletes the endpoint. Greyed out for a **Live** endpoint, with the hint *Disable it first*. | -![The row menu for an endpoint](/img/platform/data-api/endpoint-actions-menu.png) +![The row menu of the live endpoint Top customers: Open, Disable, Move to new folder and a greyed-out Move to Trash with Disable it first](/img/platform/data-api/endpoint-actions-menu.png) In **Trash**, a row offers **Restore** (the endpoint returns **disabled**, never straight to serving) and **Delete permanently**, which cannot be undone. @@ -52,7 +52,7 @@ In **Trash**, a row offers **Restore** (the endpoint returns **disabled**, never 2. Fill in **Name**, **Path**, an optional **Description**, **Method**, **Database** and **SQL**. 3. Press **Create endpoint**. -![The New endpoint page filled in](/img/platform/data-api/endpoint-new.png) +![The New endpoint page filled in for Customer spend on trino-demo](/img/platform/data-api/endpoint-new.png) The create page collects only what an endpoint cannot exist without. Parameters, response shape, cache and limits are configured in the editor you land in next. Nothing is served yet: a new endpoint is a **Draft**. @@ -104,10 +104,10 @@ Opening an endpoint shows its editor. The header carries the name, a state badge | **Try it** | Make a real call as a service account. | [Try it](./endpoints/try-it) | | **Versions** | Publish, review history, roll back. | [Publishing](./endpoints/publishing) | -The header badge reads **Draft · never published**, **Live · v2**, **Live · v2 · unpublished changes** (the saved draft differs from what is served) or **Disabled · was live**. Pressing the badge opens **Versions**. +The header badge reads **Draft · never published**, **Live · v2**, **Live · v1 · unpublished changes** (the saved draft differs from what is served, as on **Top customers** in the walk-through) or **Disabled · was live**. Pressing the badge opens **Versions**. :::caution Changing the path moves the live URL -In the version of Egret used for these docs, saving a new **Path** in **Config** changes the URL that is served immediately, before you publish. A call to the old path returns `404 not_found` while the new path answers with the live version's data. Publishing then reports the change as breaking. Treat a path change as a breaking change to make deliberately. See [Publishing](./endpoints/publishing#breaking-changes). +Saving a new **Path** in **Config** changes the URL that is served immediately, before you publish. A call to the old path returns `404 not_found` while the new path answers with the live version's data. Publishing then reports the change as breaking. Treat a path change as a breaking change to make deliberately. See [Publishing](./endpoints/publishing#breaking-changes). ::: ## Limits and known issues diff --git a/platform/data-api/endpoints/parameters.md b/platform/data-api/endpoints/parameters.md index 35ad60e..d60247b 100644 --- a/platform/data-api/endpoints/parameters.md +++ b/platform/data-api/endpoints/parameters.md @@ -6,7 +6,7 @@ sidebar_position: 2 Parameters let a caller change what an endpoint returns: filter by a status, pick a date, choose a sort column. You declare each parameter on the **Query** tab and refer to it in the SQL as `{{ name }}`. Egret checks every value before the query runs and sends it to the engine as a bound value, never as pasted text. -![The Query tab with the parameters table open](/img/platform/data-api/endpoint-params.png) +![The Query tab of Orders by status with Database and parameters open: priority (enum) and min_total (number, 0 to 1000000)](/img/platform/data-api/endpoint-params.png) ## Declare a parameter diff --git a/platform/data-api/endpoints/publishing.md b/platform/data-api/endpoints/publishing.md index 2584f1e..8c32028 100644 --- a/platform/data-api/endpoints/publishing.md +++ b/platform/data-api/endpoints/publishing.md @@ -6,7 +6,7 @@ sidebar_position: 8 Consumers never see your draft. They see a **version**: a frozen snapshot of the endpoint's contract (SQL, parameters, response settings, cache, limits and rate limit) taken when you press **Publish**. You can edit the draft as much as you like without affecting anyone, and go back to an earlier version in one click. -![The Versions tab listing v2 as Live and v1](/img/platform/data-api/endpoint-versions.png) +![The Versions tab of Orders by status: v2 Live with a 1 breaking change badge over its note, and v1 with Roll back](/img/platform/data-api/endpoint-versions.png) ## Publish an endpoint @@ -41,7 +41,7 @@ Version numbers are per endpoint and never reused. A change is **breaking** when a consumer built against the live version could stop working. Before you publish, the dialog compares the draft with the live version. -![The Publish dialog listing a breaking change and requiring acknowledgement](/img/platform/data-api/publish-dialog-breaking.png) +![The Publish dialog for Top customers: column spend of statement main is gone, and Publish stays disabled until the acknowledgement is ticked](/img/platform/data-api/publish-dialog-breaking.png) Examples Egret classifies as breaking, each with its own message: @@ -71,7 +71,7 @@ Rolling back makes an earlier version the live one again. Consumers calling the 2. Press **Roll back** on the version you want back. 3. Type `rollback` to confirm and press **Roll back**. The dialog says *Every deployed client calling this endpoint's URL will immediately start receiving this version's frozen contract instead of the current live one. This does not delete anything — publish again to move forward once you're ready.* -![The rollback confirmation dialog](/img/platform/data-api/rollback-dialog.png) +![The Roll back to v1? dialog with rollback typed and the red Roll back button enabled](/img/platform/data-api/rollback-dialog.png) A message confirms *Rolled back to vN*. Nothing is deleted. The draft is untouched, so the header may read **Live · v1 · unpublished changes** when the draft is ahead of the version being served. diff --git a/platform/data-api/endpoints/response.md b/platform/data-api/endpoints/response.md index 8fa388b..c6d3059 100644 --- a/platform/data-api/endpoints/response.md +++ b/platform/data-api/endpoints/response.md @@ -6,7 +6,7 @@ sidebar_position: 5 The **Response** tab shows what a consumer actually receives when it calls the endpoint: the frozen columns of the live version, and the settings that decide how rows become JSON. Use it to rename keys, return positional arrays, drop the envelope, or build a custom document with a script. -![The Response tab with frozen columns and the Output shape settings](/img/platform/data-api/endpoint-response.png) +![The Response tab of Orders by status: frozen columns status, revenue and order_count, and the Output shape settings](/img/platform/data-api/endpoint-response.png) ## The default response @@ -72,7 +72,7 @@ Every other column is in every response. Tick **Build the response with a script** to replace the standard `data` with a document you build. This is for a shape no SQL projection can express, such as a single object that joins a total with a list. Most endpoints do not need it: **Shape** and **Naming** already cover renaming keys and objects versus arrays. -![The Response tab of an endpoint with a transform script and declared return columns](/img/platform/data-api/endpoint-response-transform.png) +![The Response tab of Yearly revenue summary: two statements' frozen columns, the transform script (227 / 8,192 bytes) and three declared returns](/img/platform/data-api/endpoint-response-transform.png) The script is Starlark, a deterministic Python subset with no input or output, no network, no clock and no randomness. It must define `transform(params, statements)`: diff --git a/platform/data-api/endpoints/statements.md b/platform/data-api/endpoints/statements.md index 14d5256..68fbfa9 100644 --- a/platform/data-api/endpoints/statements.md +++ b/platform/data-api/endpoints/statements.md @@ -6,7 +6,7 @@ sidebar_position: 3 An endpoint normally runs one query. When a response needs more than one result set, for example a page of rows and the total count for a pager, add more **statements**. When the SQL itself must change shape depending on the caller's parameters in a way `{% if %}` cannot express, build it with a short script. -![The Query tab of an endpoint with two statements, rows and total](/img/platform/data-api/endpoint-statements.png) +![The Query tab of Orders page with two statements, rows and total, and the rows statement selected](/img/platform/data-api/endpoint-statements.png) ## Add a statement diff --git a/platform/data-api/endpoints/try-it.md b/platform/data-api/endpoints/try-it.md index b5abc96..1f05117 100644 --- a/platform/data-api/endpoints/try-it.md +++ b/platform/data-api/endpoints/try-it.md @@ -20,7 +20,7 @@ A green **Preview** does not prove a consumer can call the endpoint: it checks n ## Preview a draft -![The Query tab with a Preview result](/img/platform/data-api/endpoint-query.png) +![The Query tab of Orders by status with priority 2-HIGH and a three-row Preview result](/img/platform/data-api/endpoint-query.png) 1. Open the endpoint's **Query** tab. 2. Under **Preview**, fill in a value for each parameter you want to test. Required parameters are marked with a red asterisk, and a parameter's default shows as the placeholder. @@ -40,7 +40,7 @@ What to know: ## Try it -![The Try it tab after a successful run as orders-reporting-bot](/img/platform/data-api/endpoint-try.png) +![The Try it tab after a successful run as orders-reporting-bot: Status 200 and the response body](/img/platform/data-api/endpoint-try.png) 1. Open the endpoint's **Try it** tab. 2. Under **Run as**, choose a **Service account**. diff --git a/platform/data-api/governed-connections.md b/platform/data-api/governed-connections.md index 4a66fd0..fb046a8 100644 --- a/platform/data-api/governed-connections.md +++ b/platform/data-api/governed-connections.md @@ -7,7 +7,7 @@ sidebar_position: 6 An endpoint answers with whatever the query engine returns to the identity that ran the query. If the engine filters rows or masks columns **per user** (row filters and column masks in Ranger, for example), the endpoint must run the query as the right user, or every consumer would read what the shared credential can read. This page explains who "the caller" is for the Data API, and the rule Egret enforces for governed data. :::warning Not run in the demo -The demo connection, `trino-demo`, is neither governed nor per-user, and shared settings could not be changed for this page. The rule is described from the product's messages and source, and the messages shown below appear in the endpoint editor when the condition holds. The refusal on publish and the `503 governance_stale` response were not provoked on the instance. +The demo connection, `trino-demo`, is neither governed nor per-user. Provoking the rule needs a governed connection, and the demo workspace is shared by everyone using the stack, so no connection was added or changed for this page. The rule is described from the product's messages and source, and the messages shown below appear in the endpoint editor when the condition holds. The refusal on publish and the `503 governance_stale` response were not provoked on the instance. ::: ## Who is "the caller" diff --git a/platform/data-api/openapi.md b/platform/data-api/openapi.md index f4115df..2349f20 100644 --- a/platform/data-api/openapi.md +++ b/platform/data-api/openapi.md @@ -15,7 +15,7 @@ Two places offer **Download OpenAPI**: - **Endpoints** list, top right: the document for every live endpoint the chosen account may invoke. - An endpoint's **Response** tab: the contract of that one endpoint. -![The Download OpenAPI dialog with a service account chosen](/img/platform/data-api/openapi-dialog.png) +![The Download OpenAPI dialog over the endpoints list, with orders-reporting-bot chosen](/img/platform/data-api/openapi-dialog.png) 1. Press **Download OpenAPI**. 2. Under **Service account**, choose the account the document is for. The dialog says *Choose a service account. The document lists every live endpoint that account may invoke.* (From an endpoint: *Choose the service account this document describes invoke rights for.*) @@ -37,23 +37,23 @@ The management API offers the same for administrators: `GET /api/ws/{workspace}/ ## What the document contains -From the walk-through workspace, for the service account `orders-reporting-bot`: +From the walk-through workspace on the demo stack (port 8080), for the service account `orders-reporting-bot`. `partner-sandbox`, whose only grant is on the disabled **Legacy order export**, gets an empty `paths` object: ```json { "openapi": "3.0.3", "info": { "title": "Data API", "version": "1.0.0" }, - "servers": [{ "url": "http://localhost:8080/data-api/v1/default" }], + "servers": [{ "url": "http://egret.localtest.me:8080/data-api/v1/default" }], "security": [{ "egret_oauth2": [] }], "components": { "securitySchemes": { "egret_oauth2": { "type": "oauth2", - "flows": { "clientCredentials": { "tokenUrl": "http://localhost:8080/oauth/token", "scopes": {} } } + "flows": { "clientCredentials": { "tokenUrl": "http://egret.localtest.me:8080/oauth/token", "scopes": {} } } } } }, - "paths": { "/customers/top": {}, "/orders/by-status": {}, "/orders/page": {}, "/reports/yearly": {} } + "paths": { "/customers/top": {}, "/orders/by-status": {}, "/orders/lookup": {}, "/orders/page": {}, "/orders/status-totals": {}, "/reports/yearly": {} } } ``` diff --git a/platform/data-api/overview.md b/platform/data-api/overview.md index 2f2d494..cb6d234 100644 --- a/platform/data-api/overview.md +++ b/platform/data-api/overview.md @@ -8,7 +8,7 @@ The Data API turns a SQL query into a REST endpoint. You write the query, declar Use it when a consumer that is not a person (a nightly job, a partner's backend, a mobile app, a spreadsheet macro) needs the answer to a query that you control. For a person who wants to look at data, use [Charts](../explore/charts) and [Dashboards](../explore/dashboards) instead. -![The endpoints list with six live endpoints](/img/platform/data-api/endpoints-list.png) +![The endpoints list with eight endpoints: six live, Customer spend as a draft and Legacy order export disabled](/img/platform/data-api/endpoints-list.png) ## The pieces diff --git a/platform/explore/apps/access-and-data.md b/platform/explore/apps/access-and-data.md index 5bb50e8..3201b25 100644 --- a/platform/explore/apps/access-and-data.md +++ b/platform/explore/apps/access-and-data.md @@ -4,9 +4,9 @@ sidebar_position: 4 # Access and data -Two separate questions decide what a person sees in an app: **may they open it**, and **what may the app's queries read when they do**. Egret answers the first from the app's permissions and shares, and the second from the viewer's own data permissions. +Two separate questions decide what a person sees in an app: **may they open it**, and **what may the app's queries read when they do**. Egret answers the first from the app's permissions and shares, and the second from the data permissions of the identity the query runs as. -![The Share dialog for an app](/img/platform/apps/share-dialog.png) +![The Share dialog of Revenue explorer: finance-analysts as Viewer, carla.mendes@egret.local as Editor, and marketing with No access until 2027-03-31](/img/platform/apps/share-dialog.png) ## Who can open and edit an app @@ -16,7 +16,7 @@ Access to an app comes from three places, combined: |---|---| | **Workspace role** | `app:read` lets a person open every published app; `app:write` lets them create, edit, publish and restore. By default **ws_viewer**, **ws_editor** and **ws_admin** hold `app:read`; **ws_editor** and **ws_admin** hold `app:write`. | | **A share on one app** | Gives a person or group a role **on that app only**. A share that grants write on one app makes the Apps list show write actions on that row and no other. | -| **A folder share** | Shares given to a folder apply to the apps in it. | +| **A folder share** | Shares given to a folder apply to the apps in it. The Share dialog lists them under **Inherited from folder**. | The list reflects this per row. A row you cannot write has no **Edit**, **Rename…** or **Duplicate**; a row you cannot manage has no **Share…**. @@ -27,10 +27,10 @@ To edit, you need `app:write` on the app **and** `query:run` in the workspace, b 1. In **Explore → Apps**, open the **⋯** menu of the app and choose **Share…**. 2. Choose **User** or **Group**, type the user's email or the group name. 3. Choose the role: **Viewer**, **Editor** or **Blocked**. -4. Optionally set **Access expires**. Leave it blank for no expiry. +4. Optionally set **Access expires** (*optional — leave blank for no expiry*). 5. Press **Add**. -The dialog says *Grant people and groups access to this app.* and lists everyone with direct access under **People & groups with access**, or *No direct access yet.* You can change a row's role or remove it from that list. +The dialog says *Grant people and groups access to this app.* and lists everyone with direct access under **People & groups with access**, with a **User** or **Group** badge and the expiry date when there is one. Change a row's role in its picker, or remove it with the bin icon. A **Blocked** row shows a red **No access** badge instead of a role picker. | Role | Effect | |---|---| @@ -41,12 +41,21 @@ The dialog says *Grant people and groups access to this app.* and lists everyone Sharing needs `app:manage` on the app. Shares work as on other objects; see [Shared objects](../../workspace-settings/shared-objects). :::note -Viewers are signed-in Egret users: workspace members and people shared in. There are no public links and no embedding outside Egret. +Viewers are signed-in Egret users. There are no public links and no embedding outside Egret. A user who is shared into an app but is not a member of the workspace can open it only from a direct link: the Apps list of that workspace answers `403` for them. ::: ## What the app can read -An app's code is written by its author, but it runs **as the viewer**. Every query goes through the query gateway, which executes it with the viewer's identity, so the engine decides what that person may read. Ranger row filters and column masks that apply to the viewer apply to the app. The author never holds a credential for the engine, and neither does the pod. +An app's code is written by its author, but it runs **for the viewer**: run mode starts one process per viewer and per published version, and every query goes through the query gateway with that viewer's session. Which identity the engine then sees depends on the database connection: + +| Connection | Identity the engine sees | +|---|---| +| **Per-user delegation** or **Run queries as the logged-in user** turned on | The viewer. Ranger row filters and column masks that apply to the viewer apply to the app. | +| Neither turned on | The connection's own shared user. Every viewer sees what that user can read. | + +The demo connection `trino-demo` has neither switch on, so `SELECT current_user` from an app answers `egret` for everyone. For data that needs per-person rules, turn on **Per-user delegation** or **Run queries as the logged-in user** on the connection; see [Databases](../../workspace-settings/databases). + +With a per-user connection: | If… | Then… | |---|---| @@ -54,41 +63,46 @@ An app's code is written by its author, but it runs **as the viewer**. Every que | An author with wide access builds the app | Viewers still see only what they could read themselves. | | A viewer lacks access to a table the app queries | The query fails for that viewer, with the engine's refusal, while it works for others. | -The viewer's identity reaches the engine through the connection's per-user settings, described in [Databases](../../workspace-settings/databases). A connection that does not run queries as the signed-in user shows every viewer what its shared credential can read; for data that needs per-person rules, turn on **Per-user delegation** or **Run queries as the logged-in user** on the connection. +The author never holds a credential for the engine, and neither does the app process. + +![The published Revenue explorer as a viewer sees it: the Order status picker, the result table and a bar chart of revenue by priority](/img/platform/apps/viewer.png) ### What the app cannot do - **Write data.** Queries must be read-only; see [what the gateway allows](./authoring#what-the-gateway-allows). -- **Reach the engine directly.** Pods can talk only to Egret. -- **See another viewer's session.** Run mode starts one process per viewer and per published version, never shared between people, so one viewer's data cannot leak to another through the app's memory. Idle run pods are stopped after five minutes by default. +- **Reach the engine directly.** In production, app pods can talk only to Egret. +- **See another viewer's session.** Run mode never shares a process between people, so one viewer's data cannot leak to another through the app's memory. Idle run processes are stopped after five minutes by default. - **Keep a token forever.** A viewer's session token lasts 30 minutes idle and 8 hours at most, and dies when the viewer signs out. Every gateway call is written to the audit log with the viewer, the app and the outcome, so access through apps is reviewable alongside every other access. ## When a viewer cannot open an app -Opening an app shows one of these messages when it cannot start: +Opening an app shows *Starting the app…* (*The first open after a while can take up to a minute.*) and then the app, or one of these messages when it cannot start: | Message | Meaning | |---|---| | *You can't open this app* | The viewer lacks `app:read` on it, or the workspace denies running queries. Ask a workspace admin. | | *App not found* | The app was deleted or belongs to another workspace. | -| *Nobody has published this app yet* | The author has not published a version. Ask them to publish before sharing. | +| *Nobody has published this app yet* | The author has not published a version (the detail reads *the app has no published version to run; publish it first*). Ask them to publish before sharing. | | *This app can't be opened right now* | The app is in the trash. | | *Apps are not set up on this deployment* | The apps origin is not configured. An operator needs to set `apps.origin`. | -![The viewer page when the apps origin is not configured](/img/platform/apps/view-unavailable.png) - -The last message is what the demo instance showed for **Revenue explorer**, so the running app could not be captured. +![Opening the draft-only app Shipping modes: Nobody has published this app yet](/img/platform/apps/not-published.png) ## Where apps are served -An app is served from a separate origin, a sibling subdomain of Egret's own host (for example `apps.egret.example`), inside a sandboxed iframe. Egret hands the browser a one-time ticket that lasts 30 seconds by default; the apps origin turns it into a cookie for that viewer. Apps get their own origin so that an app's JavaScript cannot read Egret's session. An operator sets this up with `apps.origin`, `apps.egret_origin` and an `[apps.pod]` launcher (`kubernetes` in production). +An app is served from a separate origin, a sibling host of Egret's own (for example `apps.egret.example`; on the demo stack `apps.localtest.me`), inside a sandboxed iframe. Egret hands the browser a one-time ticket that lasts 30 seconds by default; the apps origin turns it into a cookie for that viewer. Apps get their own origin so that an app's JavaScript cannot read Egret's session. An operator sets this up with `apps.origin`, `apps.egret_origin` and an `[apps.pod]` launcher: `kubernetes` in production, `subprocess` on the demo stack. + +:::warning Demo stack only +The `subprocess` launcher runs every app process inside the Egret container as the same user, with no isolation between apps and users, and over plain http Egret's session cookie has no `__Host-` prefix, so code on the apps host can set cookies that Egret then receives. Never put real data or untrusted authors on the demo stack. +::: ## Limits and known issues -- The running app could not be opened in the demo environment; its behaviour is from the source. - There is no public sharing and no embedding. +- A shared-in user who is not a workspace member has no list to find the app from. +- On the demo stack, viewers' apps stop opening (they stay on *Starting the app…*) about five minutes after the first viewer opened one, until Egret is restarted: the idle reaper cannot stop the app process. ## Further reading diff --git a/platform/explore/apps/authoring.md b/platform/explore/apps/authoring.md index 15d42ee..1a283d7 100644 --- a/platform/explore/apps/authoring.md +++ b/platform/explore/apps/authoring.md @@ -4,33 +4,29 @@ sidebar_position: 2 # Write an app -You write an app in marimo's own editor, opened inside Egret. This page covers the Egret side of authoring: how the draft is saved, the `egret_marimo` Python module that queries data as the viewer, and what the apps runtime allows. +You write an app in marimo's own editor, opened inside Egret. This page covers the Egret side of authoring: the editor page, how the draft is saved, the `egret_marimo` Python module that queries data, and what the apps runtime allows. Behaviour inside marimo itself is marimo's, documented at [marimo.io](https://docs.marimo.io). -:::warning Not run locally -The marimo editor needs an edit pod, which needs Kubernetes (or a local launcher). The demo instance has neither, so the editor never loaded and this page is written from the source code (`egret_marimo` in `apps-image/`, the apps crate and the editor page), plus the draft REST API, which was run. Behaviour inside marimo itself is marimo's, documented at [marimo.io](https://docs.marimo.io). -::: - -![The editor page header with Versions, Reload editor and Publish draft rev 2, and the apps-origin error shown when no pod is available](/img/platform/apps/edit-unavailable.png) +![The editor of Revenue explorer: the Egret header with Published v4, Draft revision 3, Versions, Reload editor and Publish draft rev 3, and below it the marimo notebook with the egret.sql cell, its result table and a bar chart](/img/platform/apps/editor.png) ## Open the editor -Open **Explore → Apps**, open the **⋯** menu of an app and choose **Edit**, or press **+ App** to create one and land in its editor. +Open **Explore → Apps**, open the **⋯** menu of an app and choose **Edit**, or press **+ App** to create one and land in its editor. Pressing the name of a draft-only app also opens the editor. -The page has a header and an iframe below it. +The page has an Egret header and the marimo notebook in an iframe below it. | Header element | What it shows or does | |---|---| | Back arrow | Returns to the Apps list. | | App name and badge | **Draft only** until the first publish, then **Published vN**. | -| **Draft revision N** | The draft's revision number. It increases every time the draft is saved. | +| **Draft revision N** | The draft's revision number. It increases every time the draft is saved. The header re-reads it every 5 seconds. | | *Save in the editor (Ctrl/⌘+S) before publishing.* | A reminder: publishing freezes the **saved** draft. | | **Versions** | Opens the version history. See [Publish and version an app](./publishing). | | **Reload editor** | Reloads the iframe with a fresh session. Use it if the editor stops responding. | | **Publish draft rev N** | Publishes the saved draft as a new version. | -While the editor starts you see *Starting your editor…* and *The first open after a while can take up to a minute.* The editor is marimo's own interface, not Egret's, so it does not use Egret's theme beyond light or dark. +While the editor starts you see *Starting your editor…* and *The first open after a while can take up to a minute.* The first time an author opens an edit session Egret starts one marimo process for that author in that workspace; it is shared by every app that author edits there and stopped after 30 minutes of inactivity by default. Opening the editor again within that time shows marimo's *Reconnected — You have reconnected to an existing session.* banner. -The first time an author opens an edit session Egret starts one pod for that author in that workspace. It is stopped after 30 minutes of inactivity by default, and the next open starts it again. +Opening the editor does **not** run the notebook. Cells that have not run are marked stale; press marimo's run button (bottom right) to run them all, or run one cell with its own ▶ button. The editor is marimo's own interface, not Egret's, so it does not use Egret's theme beyond light or dark. If the editor cannot open, the page shows one of these messages instead: @@ -41,9 +37,13 @@ If the editor cannot open, the page shows one of these messages instead: | *This app can't be opened right now* | The app is in the trash. | | *Apps are not set up on this deployment* | The apps origin is not configured. An operator needs to set `apps.origin`. | +:::note What the editor does not offer +Egret's proxy in front of marimo refuses some of marimo's own endpoints on purpose: the language server (`/api/lsp`), the package manager (`/api/packages`), data-source discovery (`/api/datasources`) and usage reporting (`/api/usage`) answer `403`. So there is **no autocomplete or hover help**, and marimo's package and data-source panels do not work. The browser console shows these `403`s and *Language server initialization failed*; they are expected. +::: + ## The draft -The source of an app is one Python file, stored as the **draft** in the database. Saving in marimo updates the draft and increases its revision. Two rules protect it: +The source of an app is one Python file, stored as the **draft** in the database. Saving in marimo (Ctrl+S or ⌘+S) updates the draft and increases its revision. Two rules protect it: - **One author at a time per revision.** A save names the revision it started from. If someone else saved in between, Egret answers `409` with `the draft changed since it was read` and the current revision, instead of overwriting. Reload the editor and merge by hand. There is no real-time collaboration and no locking. - **1 MiB of source at most.** A larger save is refused with `413`. @@ -52,7 +52,7 @@ The draft can also carry the grid layout marimo keeps beside the source. ## Query data with `egret_marimo` -Apps import the `egret_marimo` module, already installed in the pod image, which is a fixed set of packages. You cannot install packages at runtime; the image is curated so that egress can be closed to Egret only. +Apps import the `egret_marimo` module, already installed in the app image. The image is a fixed set of packages; you cannot install more at runtime, so that egress can be closed to Egret only. ```python import egret_marimo as egret @@ -62,49 +62,76 @@ import egret_marimo as egret |---|---| | `egret.connect(database, dialect="sql")` | Returns a DB-API 2.0 connection for the database named `database` (its name in **Workspace settings → Engines**). marimo SQL cells list it and can use it. | | `egret.sql(query, database, params=())` | Runs `query` and returns a Polars `DataFrame`. `params` is a list of values bound to `?` markers in order. | -| `egret.databases()` | Returns `(name, engine)` pairs for pickers. **Edit mode only**: a viewer's session gets a `403`, because listing a workspace's databases is an admin act. An app that needs the names when viewed carries them in its source. | +| `egret.databases()` | Returns `(name, engine)` pairs for pickers, for example `[('trino-demo', 'trino')]`. **Edit mode only**: a viewer's session gets a `403`, because listing a workspace's databases is an admin act. An app that needs the names when viewed carries them in its source. | -For example: +For example, the query cell of **Revenue explorer**, shown in the screenshot above: ```python -import marimo as mo -import egret_marimo as egret +status = mo.ui.dropdown( + {"Open": "O", "Fulfilled": "F", "Partly shipped": "P"}, + value="Open", + label="Order status", +) +status +``` -status = mo.ui.dropdown(["O", "F", "P"], value="O", label="Order status") +```python df = egret.sql( - "SELECT orderpriority, count(*) AS orders FROM tpch.tiny.orders " - "WHERE orderstatus = ? GROUP BY 1 ORDER BY 1", + "SELECT orderpriority AS priority, count(*) AS orders, " + "round(sum(totalprice), 2) AS revenue " + "FROM tpch.tiny.orders WHERE orderstatus = ? GROUP BY 1 ORDER BY 1", database="trino-demo", params=[status.value], ) -mo.vstack([mo.md("# Revenue explorer"), status, df]) ``` -This is the source of the **Revenue explorer** app in the walk-through. It was saved to the draft and published through the REST API; the rows it returns could not be viewed because the app could not be opened. +Changing the dropdown re-runs the query cell and everything that depends on it, as marimo always does. + +### Charts + +The image includes `altair` and `plotly`, but not `numpy` or `pandas`. Use Altair, which reads a Polars `DataFrame` directly, and wrap the chart in `mo.ui.altair_chart`: + +```python +import altair as alt + +chart = ( + alt.Chart(df) + .mark_bar() + .encode(x=alt.X("priority:N", title="Priority"), y=alt.Y("revenue:Q", title="Revenue")) + .properties(height=260, width=440) +) +mo.ui.altair_chart(chart) +``` + +`plotly.express` cannot be imported (*Plotly Express requires numpy to be installed*); `plotly.graph_objects` works with plain lists, for example `go.Bar(x=df["priority"].to_list(), y=df["revenue"].to_list())`. ### Errors -A failed query raises `GatewayError` with a `status` and a `detail`. `SessionExpired` is raised when the viewer's session token has expired; the viewer reloads the app. A network failure raises `GatewayError` with status `0`. +A failed query raises `GatewayError` with a `status` and a `detail`; the message reads `egret gateway error : `. `SessionExpired` is raised when the viewer's session token has expired; the viewer reloads the app. A network failure raises `GatewayError` with status `0`. + +marimo's own rule applies to every cell: a name may be defined in one cell only. Prefix scratch names with an underscore (`_r`, `_e`) to keep them local to a cell, or marimo refuses to run the cell (*This cell redefines variables from other cells*). ### What the gateway allows -Every `egret.sql` call goes to Egret's query gateway with the viewer's token. The gateway checks, in order: +Every `egret.sql` call goes to Egret's query gateway with the session's token. The gateway checks, in order: 1. The token is valid and unexpired, otherwise `401`. 2. The caller's rate limit: 20 calls per token per 2 seconds by default, otherwise `429`. -3. The viewer is still an active user, the app is not deleted and the workspace has **Apps** enabled. -4. The viewer may read the app (run mode), or may write it and run queries (edit mode). -5. The database name exists in the token's workspace. An unknown name and a name in another workspace give the same `404`. -6. The SQL is **read-only**. +3. The person is still an active user, the app is not deleted and the workspace has **Apps** enabled. +4. The person may read the app (run mode), or may write it and run queries (edit mode). +5. The database name exists in the token's workspace. An unknown name and a name in another workspace give the same `404 not found: database`. +6. The SQL is **read-only**: exactly one statement that is positively known to read. -The read-only rule is strict. The SQL must be exactly one statement that is positively known to read. These are refused, with a message that starts `data apps may only read data:`: +These were run from an app on the demo stack: -- anything other than `SELECT` or `WITH … SELECT`, such as `INSERT`, `DELETE` or DDL; -- more than one statement; -- table functions (Trino's `TABLE(catalog.system.query(...))` can pass arbitrary SQL to the underlying database); -- MySQL executable comments (`/*! … */`), `#` comments, comments with a nested `/*`, and quoting forms the engine may read differently. +| Query | Answer | +|---|---| +| `DELETE FROM memory.default.t` | `400 invalid input: data apps may only read data: this statement kind is not allowed` | +| `SELECT 1; SELECT 2` | `400 invalid input: exactly one statement is allowed per query, got 2` | +| `SELECT 1` on database `nope` | `404 not found: database` | +| `SELECT current_user AS u` on `trino-demo` | `[('egret',)]`: the connection's own user, because `trino-demo` does not run queries as the signed-in person (see [Access and data](./access-and-data#what-the-app-can-read)). | -SQL that cannot be parsed is refused too, because it cannot be shown to be read-only: *rewrite it in standard SQL*. +Also refused, with a message that starts `data apps may only read data:`: table functions (Trino's `TABLE(catalog.system.query(...))` can pass arbitrary SQL to the underlying database), MySQL executable comments (`/*! … */`), `#` comments, comments with a nested `/*`, and quoting forms the engine may read differently. SQL that cannot be parsed is refused too, because it cannot be shown to be read-only. | Gateway limit | Default | |---|---| @@ -118,9 +145,10 @@ These are operator settings under `[apps]` (`max_rows`, `timeout_secs`, `rate_ma ## Limits and known issues -- The editor and the running app could not be opened in the demo environment, so none of the marimo-side behaviour on this page was observed. -- No packages can be installed from an app; the image is fixed. +- No autocomplete: the language server is refused by design (see the note above). +- No packages can be installed from an app; the image is fixed and has no `numpy` or `pandas`, so `plotly.express` does not work. - There is no real-time co-editing; the second author to save gets a `409`. +- **Publish draft rev N** right after Ctrl+S can answer `409` because the header refreshes the revision only every 5 seconds; see [Publish and version an app](./publishing#publish-a-draft). ## Further reading diff --git a/platform/explore/apps/index.md b/platform/explore/apps/index.md index 65d31c6..233f201 100644 --- a/platform/explore/apps/index.md +++ b/platform/explore/apps/index.md @@ -4,60 +4,64 @@ sidebar_position: 1 # Apps -An app is a dashboard written in code. You build it in [marimo](https://marimo.io), a reactive Python notebook, publish it, and other people open it from Egret. Every query the app makes runs **as the person looking at it**, so row filters and column masks apply to the viewer and never to the author. +An app is a dashboard written in code. You build it in [marimo](https://marimo.io), a reactive Python notebook, publish it, and other people open it from Egret. Every query the app makes goes through Egret's query gateway under the identity of **the person looking at it**, so their own data permissions apply, not the author's. Use an app when a [dashboard](./dashboards) cannot express what you need: custom interaction, several linked queries, your own layout. For a standard chart or table, a dashboard is faster to build and needs no code. -![The Apps list with one published app](/img/platform/apps/apps-list.png) +![The Apps list with a Sales analysis folder holding two published apps, a draft-only app and another published app](/img/platform/apps/apps-list.png) -:::warning What could be run for these docs -Apps run in pods that Egret starts on demand: one hardened Kubernetes pod per author per workspace for editing, and one per viewer per published version for running. The demo instance used for these docs has no Kubernetes and no apps origin, so the **editor and the running app could not be opened**. The list, the create dialog, row actions, versions, sharing, publishing and the REST API were all run and are documented from what they did. The editor and viewer experience inside marimo is described from the source code and is marked as such. +:::note Demo stack +On the Docker demo stack the marimo processes run inside the Egret container (one per author for editing, one per viewer and version for viewing) and are served from `http://apps.localtest.me:18080`. In production they run as Kubernetes pods. Measured on the demo stack: the first editor load took 15 to 18 seconds and the first view of a published app 25 to 60 seconds, while the process started and Trino warmed up; later opens took a few seconds. ::: ## How it works 1. You create an app. Egret stores its Python source in the database as a **draft**. -2. You open the editor, a marimo notebook in an iframe. Saving in marimo writes to the draft. +2. You open the editor, a marimo notebook in an iframe. Saving in marimo (Ctrl+S or ⌘+S) writes to the draft. 3. You **publish** the draft. That freezes it as a numbered, immutable **version**. 4. Viewers press **Open**. Egret starts a marimo process for that viewer and that version. Its queries go through Egret's query gateway, which executes them under the viewer's identity. -Nothing in a pod holds credentials for a query engine. The source lives only in Postgres; a pod's disk is a disposable copy. +![The published Order trends app: a Year picker, three figures for 1995 and a line chart of orders per month](/img/platform/apps/viewer-trends.png) + +Nothing in an app process holds credentials for a query engine. The source lives only in Postgres; the process's disk is a disposable copy. ## The list -Open **Explore → Apps**. The list shows every app you can see. +Open **Explore → Apps**. The list shows every app you can see, grouped by folder, with a count such as `4 apps · 1 folder` above it. | Column | What it shows | |---|---| -| **Name** | The app's name, a star to bookmark it, and its description. | +| **Name** | The app's name, a star to bookmark it, and its description. Pressing the name opens a published app, or the editor for a draft-only app. | | **Status** | **Draft only** until the first publish, then **Published vN** for the latest version. | | **Updated** | When the app last changed. | -Above the table: **Search apps…**, and the toolbar buttons **Bookmarks**, **Bulk select**, **Trash** and **New folder**. Folders group apps. **+ App** creates one and is shown only to people who may create apps. +Above the table: **Search apps…**, and the toolbar buttons **Bookmarks**, **Bulk select**, **Trash** and **New folder**. **+ App** creates an app and is shown only to people who may create apps. A folder row has its own **⋯** menu to rename, share or delete the folder; apps not in a folder are listed under **Uncategorized**. ### Row actions -![The row menu](/img/platform/apps/row-menu.png) +![The row menu of Revenue explorer, with Open, Edit, Rename, Duplicate, the folder actions, Share and Move to Trash](/img/platform/apps/row-menu.png) | Action | What it does | |---|---| | **Open** | Runs the latest published version. Disabled for a draft-only app: *Publish a draft before it can be opened.* | | **Edit** | Opens the editor on the draft. | | **Rename…** | Changes the **Name** and **Description**. | -| **Duplicate** | Makes a copy. Needs permission to create apps in the workspace; without it you see *You can edit this app, but making a copy needs permission to create apps in this workspace.* | -| **Move to new folder…** | Puts the app in a new folder. | +| **Duplicate** | Makes a copy of the draft, owned by you. Needs permission to create apps in the workspace; without it you see *You can edit this app, but making a copy needs permission to create apps in this workspace.* | +| **Move to "folder"** | Moves the app into an existing folder. The folder it is already in is shown greyed out. | +| **Remove from folder** | Moves the app back to **Uncategorized**. Shown for an app in a folder. | +| **Move to new folder…** | Creates a folder and moves the app into it. | | **Share…** | Grants people and groups access. Shown to people who can manage the app. See [Access and data](./apps/access-and-data). | | **Move to Trash** | Soft-deletes the app. In **Trash**, **Restore** brings it back and **Delete permanently** removes the draft and every published version for good. | ## Create an app -1. Go to **Explore → Apps** and press **+ App** (or **Create your first app**). -2. Type a **Name**, for example `Revenue explorer`. +1. Go to **Explore → Apps** and press **+ App** (or **Create your first app** in an empty list). +2. Type a **Name**, for example `Shipping costs`. 3. Press **Create**. -![The Create app dialog](/img/platform/apps/create-dialog.png) +![The Create app dialog with the name Shipping costs](/img/platform/apps/create-dialog.png) -Egret creates the app with a starter notebook and takes you straight to the editor. The dialog explains it in one line: *A data app is a marimo notebook that runs SQL as whoever is looking at it. You edit a draft; publishing it makes it the version people open.* +Egret creates the app with a starter notebook and takes you straight to the editor. The dialog explains it in one line: *A data app is a marimo notebook that runs SQL as whoever is looking at it. You edit a draft; publishing it makes it the version people open.* The dialog asks only for a name; add a description later with **Rename…**. The starter source is: @@ -95,15 +99,13 @@ The next pages cover [writing the app](./apps/authoring), [publishing and versio | Create, edit, publish and restore | `app:write` (**ws_editor**, **ws_admin**), and to open the **editor** also `query:run` in the workspace | | Share an app, delete it permanently | `app:manage` on the app | -The workspace must have the **Apps** feature enabled, and an operator must have set up the apps origin and a pod launcher. Without the origin, opening an app fails with *Apps are not set up on this deployment. The apps origin is not configured. An operator needs to set apps.origin.* - -![The editor page shell when the apps origin is not configured](/img/platform/apps/edit-unavailable.png) +The workspace must have the **Apps** feature enabled, and an operator must have set up the apps origin (`apps.origin`) and a launcher for the marimo processes. Without the origin, opening an app fails with *Apps are not set up on this deployment. The apps origin is not configured. An operator needs to set apps.origin.* ## Limits and known issues - The app source is limited to 1 MiB. -- Editing and running need a configured pod launcher. This was not available in the demo environment. - An app cannot be opened until it has been published once. +- The Create app dialog sets only the name; the description and a starting database cell cannot be set there (KI-041). ## Further reading diff --git a/platform/explore/apps/publishing.md b/platform/explore/apps/publishing.md index 20b024b..45ef6b8 100644 --- a/platform/explore/apps/publishing.md +++ b/platform/explore/apps/publishing.md @@ -6,17 +6,17 @@ sidebar_position: 3 What viewers open is never your draft. It is a **version**: a frozen copy of the app's source taken when you publish. You can keep editing the draft without disturbing anyone, publish when you are ready, and go back to an earlier version when a release goes wrong. -![The Versions dialog listing v2 and v1, each with a Restore button](/img/platform/apps/versions-dialog.png) +![The Versions dialog of Revenue explorer listing v4 to v1, newest first, each with a Restore button; v4 and v3 are marked restored from an earlier version](/img/platform/apps/versions-dialog.png) ## Publish a draft 1. Open the app's editor (**⋯ → Edit**). 2. Save in the marimo editor with Ctrl+S or ⌘+S. Publishing freezes the **saved** draft, not what is on screen. -3. Press **Publish draft rev N**, where N is the draft's revision shown in the header. +3. Wait until the button shows the new revision (the header re-reads it every 5 seconds), then press **Publish draft rev N**. A message confirms *Published version N*. The header badge changes from **Draft only** to **Published vN**, and the Apps list shows the same in **Status**. -The button publishes the revision it shows. If the draft has changed since the page loaded (a second tab, or someone else with edit access), Egret refuses with *The draft changed while you were publishing. Save in the editor, then publish again.* and nothing is published. +The button publishes the revision it shows. If the draft has changed since then (a save you just made, a second tab, or someone else with edit access), Egret refuses with *The draft changed while you were publishing. Save in the editor, then publish again.* and nothing is published. On the demo stack, pressing **Publish draft rev 1** a moment after Ctrl+S answered `409` (`the draft changed since it was read`, `current_revision` 2); a few seconds later the button read **Publish draft rev 2** and publishing worked. | | | |---|---| @@ -35,10 +35,16 @@ Publishing does not check the app's queries or run it. A published app with a br Press **Versions** in the editor header. The dialog says: *Every version ever published, newest first. Restoring one publishes it again as a new version — it never moves or removes anything.* -Each row shows the version number (`v2`), when it was published and by whom, and a **Restore** button. A new app that has never been published shows *Nothing has been published yet.* +Each row shows the version number (`v4`), when it was published and by whom (as a shortened user id, for example `by 8a66d837`), *restored from an earlier version* for a restored one, and a **Restore** button. An app that has never been published shows *Nothing has been published yet.* + +:::caution The dialog is not refreshed after a publish +After you press **Publish draft rev N**, the **Versions** dialog on the same page still shows the old list (on a new app, *Nothing has been published yet.*) until you reload the page. The version was published; the Apps list and the header badge show it at once. +::: ### What a version records +The REST API returns these fields for each version: + | Field | Meaning | |---|---| | `version_no` | The version number. | @@ -52,9 +58,11 @@ Versions are append-only, so a rollback is a new publish of old source. 1. Open **Versions**. 2. Press **Restore** on the version you want back. -3. A dialog titled *Restore vN?* asks you to confirm. It says *This publishes vN's source again as a brand-new version. It does not move or remove any existing version — the app's history only ever grows.* Press **Restore this version**. +3. A dialog titled *Restore vN?* asks you to confirm: *This publishes vN's source again as a brand-new version. It does not move or remove any existing version — the app's history only ever grows.* Press **Restore this version**. + +![The Restore v2? confirmation over the Versions dialog, with Cancel and Restore this version](/img/platform/apps/restore-dialog.png) -A message confirms *Published version N*, where N is the next number. In the walk-through, restoring v1 of **Revenue explorer** created v3 (`restored_from` v1), and restoring v2 afterwards created v4. The latest version, v4, is what viewers get. +A message confirms *Published version N*, where N is the next number. The walk-through app **Revenue explorer** has four versions: v1 (a table), v2 (table and chart), v3 (restored from v1) and v4 (restored from v2). The latest, v4, is what viewers get. Restoring does not change the draft. Your draft keeps its unpublished edits. @@ -81,6 +89,7 @@ A save without `If-Match` is refused: `If-Match with the draft revision is requi - Publishing without saving first publishes the last saved draft. A brand-new app published straight away publishes the starter notebook. - There is no unpublish. To stop viewers opening an app, move it to the Trash. +- The **Versions** dialog is stale after a publish until the page is reloaded, and shows the publisher as a shortened id (KI-027). ## Further reading diff --git a/scripts/seed-apps.mjs b/scripts/seed-apps.mjs new file mode 100644 index 0000000..833fd44 --- /dev/null +++ b/scripts/seed-apps.mjs @@ -0,0 +1,153 @@ +// Seeds the Data Apps behind the apps pages and screenshots (scripts/shots/apps.mjs) on the Docker demo stack. +// Idempotent: apps are looked up by name, and an app that exists is left alone (prints "exists"). +// +// BASE=http://egret.localtest.me:18080 STATE= node scripts/seed-apps.mjs +// +// STATE is a Playwright storage state of a signed-in workspace admin. The apps query the demo database +// `trino-demo` (TPC-H `tpch.tiny`). The demo Data Apps image has altair and plotly but no numpy or pandas, +// so the charts use altair (plotly.express needs numpy). +import {request} from 'playwright'; +import {readFileSync} from 'node:fs'; + +const BASE = process.env.BASE ?? 'http://egret.localtest.me:18080'; +const WS = process.env.WS ?? 'default'; +const state = JSON.parse(readFileSync(process.env.STATE, 'utf8')); +const csrf = state.cookies.find((c) => c.name === 'egret_csrf')?.value ?? ''; +const ctx = await request.newContext({baseURL: BASE, storageState: process.env.STATE, extraHTTPHeaders: {'X-Egret-CSRF': csrf}}); +const A = `/api/ws/${WS}/apps`; +async function call(method, url, data, headers = {}) { + const r = await ctx.fetch(url, {method, headers, ...(data === undefined ? {} : {data})}); + const t = await r.text(); + if (r.status() >= 300) throw new Error(`${method} ${url}: ${r.status()} ${t.slice(0, 300)}`); + return t ? JSON.parse(t) : null; +} + +const HEADER = `import marimo + +__generated_with = "0.25.0" +app = marimo.App(width="medium") +`; +const FOOTER = ` + +if __name__ == "__main__": + app.run() +`; +const cell = (args, body, ret = '') => `\n\n@app.cell\ndef _(${args}):\n${body.replace(/^/gm, ' ')}\n return${ret ? ` ${ret}` : ''}\n`; +const app = (...cells) => HEADER + cells.join('') + FOOTER; + +const imports = cell('', 'import marimo as mo\nimport egret_marimo as egret\nimport altair as alt', '(alt, egret, mo)'); +const statusPicker = cell( + 'mo', + 'status = mo.ui.dropdown(\n {"Open": "O", "Fulfilled": "F", "Partly shipped": "P"},\n value="Open",\n label="Order status",\n)\nstatus', + '(status,)', +); +const byPriority = cell( + 'egret, status', + 'df = egret.sql(\n "SELECT orderpriority AS priority, count(*) AS orders, "\n "round(sum(totalprice), 2) AS revenue "\n "FROM tpch.tiny.orders WHERE orderstatus = ? GROUP BY 1 ORDER BY 1",\n database="trino-demo",\n params=[status.value],\n)', + '(df,)', +); +const revenueV1 = app( + imports, + cell('mo', 'mo.md(r"""\n# Revenue explorer\n""")'), + statusPicker, + byPriority, + cell('df, mo', 'mo.ui.table(df, selection=None)'), +); +const revenueV2 = app( + imports, + cell('mo', 'mo.md(r"""\n# Revenue explorer\n\nRevenue by order priority for one order status. Every query runs as the person viewing the app.\n""")'), + statusPicker, + byPriority, + cell( + 'alt, df, mo', + 'chart = (\n alt.Chart(df)\n .mark_bar()\n .encode(x=alt.X("priority:N", title="Priority"), y=alt.Y("revenue:Q", title="Revenue"), tooltip=["priority", "orders", "revenue"])\n .properties(height=260, width=440)\n)\nmo.hstack([mo.ui.table(df, selection=None), mo.ui.altair_chart(chart)], widths=[2, 3])', + ), +); + +const trends = app( + imports, + cell('mo', 'mo.md(r"""\n# Order trends\n\nOrders and revenue per month for one year.\n""")'), + cell('mo', 'year = mo.ui.dropdown([str(y) for y in range(1992, 1999)], value="1995", label="Year")\nyear', '(year,)'), + cell( + 'egret, year', + 'monthly = egret.sql(\n "SELECT month(orderdate) AS month, count(*) AS orders, "\n "round(sum(totalprice) / 1e6, 2) AS revenue_m "\n "FROM tpch.tiny.orders WHERE year(orderdate) = ? GROUP BY 1 ORDER BY 1",\n database="trino-demo",\n params=[int(year.value)],\n)', + '(monthly,)', + ), + cell( + 'mo, monthly, year', + 'mo.hstack(\n [\n mo.stat(value=f"{monthly[\'orders\'].sum():,}", label=f"Orders in {year.value}"),\n mo.stat(value=f"{monthly[\'revenue_m\'].sum():,.1f} M", label="Revenue"),\n mo.stat(value=f"{monthly[\'orders\'].max():,}", label="Most orders in one month"),\n ],\n justify="start",\n)', + ), + cell( + 'alt, mo, monthly', + 'line = (\n alt.Chart(monthly)\n .mark_line(point=True)\n .encode(x=alt.X("month:O", title="Month"), y=alt.Y("orders:Q", title="Orders"), tooltip=["month", "orders", "revenue_m"])\n .properties(height=240, width=560)\n)\nmo.ui.altair_chart(line)', + ), +); + +const lookup = app( + imports, + cell('mo', 'mo.md(r"""\n# Customer lookup\n\nEvery order of one customer, newest first.\n""")'), + cell('mo', 'customer = mo.ui.number(start=1, stop=1500, value=370, label="Customer key")\ncustomer', '(customer,)'), + cell( + 'customer, egret', + 'orders = egret.sql(\n "SELECT orderkey, orderdate, orderstatus, orderpriority, totalprice "\n "FROM tpch.tiny.orders WHERE custkey = ? ORDER BY orderdate DESC",\n database="trino-demo",\n params=[int(customer.value)],\n)', + '(orders,)', + ), + cell('mo, orders', 'mo.vstack([mo.md(f"**{orders.height} orders**"), mo.ui.table(orders, selection=None, page_size=10)])'), +); + +const draftOnly = app( + imports, + cell('mo', 'mo.md(r"""\n# Shipping modes\n\nWork in progress: not published yet.\n""")'), +); + +const existing = await call('GET', A); +async function ensure(name, description, steps) { + if (existing.find((x) => x.name === name)) { + console.log(`exists app ${name}`); + return; + } + const created = await call('POST', A, {name, description}); + for (const s of steps) await s(created.id); + console.log(`created app ${name}`); +} +const save = (source) => async (id) => { + const d = await call('GET', `${A}/${id}/draft`); + await call('PUT', `${A}/${id}/draft`, {source}, {'If-Match': String(d.revision)}); +}; +const publish = async (id) => { + const d = await call('GET', `${A}/${id}/draft`); + await call('POST', `${A}/${id}/publish`, {expected_revision: d.revision}); +}; +const restore = (n) => async (id) => { + const v = (await call('GET', `${A}/${id}/versions`)).find((x) => x.version_no === n); + await call('POST', `${A}/${id}/versions/${v.id}/restore`, {}); +}; +const share = (principal_type, principal_id, role, extra = {}) => async (id) => + call('PUT', `/api/ws/${WS}/objects/app/${id}/grants`, {principal_type, principal_id, role, ...extra}); + +// A folder for the two sales apps, so the list shows grouping. +async function folder(name) { + const list = await call('GET', `/api/ws/${WS}/folders/apps`); + return list.find((f) => f.name === name) ?? (await call('POST', `/api/ws/${WS}/folders/apps`, {name})); +} +const sales = await folder('Sales analysis'); +const intoSales = async (id) => call('PATCH', `${A}/${id}/folder`, {folder_id: sales.id}); +const bookmark = async (id) => call('POST', `${A}/${id}/bookmark`, {}); + +await ensure('Revenue explorer', 'Revenue by order priority for one order status', [ + save(revenueV1), + publish, + save(revenueV2), + publish, + restore(1), + restore(2), + share('group', 'finance-analysts', 'ws_viewer'), + share('user', 'carla.mendes@egret.local', 'ws_editor'), + share('group', 'marketing', 'ws_editor', {effect: 'deny', expires_at: '2027-03-31T00:00:00Z'}), + intoSales, + bookmark, +]); +await ensure('Order trends', 'Orders and revenue per month for a chosen year', [save(trends), publish, intoSales]); +await ensure('Customer lookup', 'Every order of one customer', [save(lookup), publish]); +await ensure('Shipping modes', 'Work in progress', [save(draftOnly)]); +console.log('seeded apps'); diff --git a/scripts/seed-data-api.mjs b/scripts/seed-data-api.mjs new file mode 100644 index 0000000..5bbbaf9 --- /dev/null +++ b/scripts/seed-data-api.mjs @@ -0,0 +1,320 @@ +// Seeds the Data API objects behind the data-api pages and screenshots (scripts/shots/data-api.mjs) on the +// Docker demo stack. Idempotent: every object is looked up by name or path first, and each step only runs +// when its result is missing, so a second run changes nothing (it prints "exists" per object). +// +// BASE=http://egret.localtest.me:18080 STATE= node scripts/seed-data-api.mjs +// ... TRAFFIC=1 node scripts/seed-data-api.mjs # also makes real calls so Activity has data +// +// STATE is a Playwright storage state of a signed-in workspace admin (the login endpoint is rate limited, so +// sign in once and reuse it). Client secrets are shown only once; the ones this script issues are kept in +// SECRETS (default: a file in the system temp dir, never the repo) so TRAFFIC can mint tokens on a later run. +// The names are the ones the pages use; nothing else on the demo stack creates Data API objects. +import {api} from './api-helper.mjs'; +import {existsSync, readFileSync, writeFileSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; + +const BASE = process.env.BASE ?? 'http://egret.localtest.me:18080'; +const WS = process.env.WS ?? 'default'; +const SECRETS = process.env.SECRETS ?? join(tmpdir(), 'egret-docs-data-api-secrets.json'); +const a = await api(process.env.STATE, BASE); +const ws = (p) => `/api/ws/${WS}/data-api/${p}`; +const must = (r, what) => { + if (r.status >= 300) throw new Error(`${what}: ${r.status} ${JSON.stringify(r.body).slice(0, 400)}`); + return r.body; +}; +const secrets = existsSync(SECRETS) ? JSON.parse(readFileSync(SECRETS, 'utf8')) : {}; + +const db = (await a.get(`/api/ws/${WS}/databases`)).body.find((d) => d.name === 'trino-demo'); +if (!db) throw new Error('database trino-demo not found (run the demo seed first)'); + +// ---- Service accounts and clients ------------------------------------------------------------------- +const accounts = {}; +for (const [name, description] of [ + ['orders-reporting-bot', 'Nightly orders report for the finance team'], + ['partner-sandbox', 'External partner, read access to nothing yet'], + ['finance-dashboard', 'Read-only reporting for the finance team'], +]) { + const list = must(await a.get(ws('service-accounts')), 'list service accounts'); + let sa = list.find((s) => s.name === name); + if (!sa) sa = must(await a.post(ws('service-accounts'), {name, description}), `create ${name}`); + else console.log(`exists service account ${name}`); + accounts[name] = sa; +} + +async function client(account, name, {revoke = false, expires_at} = {}) { + const sa = accounts[account]; + const list = must(await a.get(ws(`service-accounts/${sa.id}/clients`)), 'list clients'); + let c = list.find((x) => x.name === name); + if (!c) { + c = must(await a.post(ws(`service-accounts/${sa.id}/clients`), {name, ...(expires_at ? {expires_at} : {})}), `issue ${name}`); + const id = c.client_id ?? c.client?.client_id; + const secret = c.client_secret ?? c.secret; + if (secret) secrets[name] = {client_id: id, client_secret: secret}; + writeFileSync(SECRETS, JSON.stringify(secrets, null, 2), {mode: 0o600}); + if (revoke) must(await a.del(ws(`clients/${c.id ?? c.client?.id}`)), `revoke ${name}`); + } else console.log(`exists client ${name}`); + return c; +} +await client('orders-reporting-bot', 'nightly-orders-etl'); +await client('orders-reporting-bot', 'weekly-export'); +await client('orders-reporting-bot', 'old-laptop-test', {revoke: true}); +await client('partner-sandbox', 'partner-trial'); +await client('partner-sandbox', 'partner-pilot', {expires_at: '2026-01-31T00:00:00Z'}); // already expired +await client('finance-dashboard', 'finance-sheet'); + +// ---- Endpoints -------------------------------------------------------------------------------------- +const endpoints = async () => must(await a.get(ws('endpoints')), 'list endpoints'); +const versions = async (id) => must(await a.get(ws(`endpoints/${id}/versions`)), 'versions'); + +async function publish(id, note) { + const v = must(await a.post(ws(`endpoints/${id}/validate`), {}), `validate ${id}`); + return must(await a.post(ws(`endpoints/${id}/publish`), {note, ...(v.acknowledge_breaking ? {acknowledge_breaking: v.acknowledge_breaking} : {})}), `publish ${id}`); +} +async function grant(id, account) { + const grants = must(await a.get(ws(`endpoints/${id}/grants`)), 'grants'); + const sa = accounts[account].id; + if (JSON.stringify(grants).includes(sa)) return; + must(await a.post(ws(`endpoints/${id}/grants`), {service_account_id: sa}), `grant ${account}`); +} +/** Create the endpoint if its path is free, then run `after(ep)` once (publish steps). */ +async function endpoint(spec, after) { + const found = (await endpoints()).find((e) => e.path === `/${spec.path}` || e.name === spec.name); + if (found) { + console.log(`exists endpoint ${spec.name}`); + return found; + } + const ep = must(await a.post(ws('endpoints'), {database_id: db.id, ...spec, description: spec.description ?? ''}), `create ${spec.name}`); + if (after) await after(ep); + console.log(`created endpoint ${spec.name}`); + return ep; +} + +const PRIORITIES = ['1-URGENT', '2-HIGH', '3-MEDIUM', '4-NOT SPECIFIED', '5-LOW']; +const byStatusV1 = + 'SELECT orderstatus AS status, count(*) AS orders, round(sum(totalprice), 2) AS revenue\nFROM tpch.tiny.orders\nWHERE totalprice >= {{ min_total }}\n{% if priority %} AND orderpriority = {{ priority }}\n{% endif %}GROUP BY orderstatus\nORDER BY revenue DESC'; +const byStatusV2 = + 'SELECT orderstatus AS status,\n round(sum(totalprice), 2) AS revenue,\n count(*) AS order_count\nFROM tpch.tiny.orders\nWHERE totalprice >= {{ min_total }}\n{% if priority %} AND orderpriority = {{ priority }}\n{% endif %}GROUP BY orderstatus\nORDER BY revenue DESC'; + +await endpoint( + { + name: 'Orders by status', + path: 'orders/by-status', + description: 'Order count and revenue per order status, optionally for one priority', + statements: [{name: 'main', sql: byStatusV1}], + params: [ + {name: 'priority', type: 'enum', location: 'query', allowed: PRIORITIES, description: 'Only orders with this priority'}, + {name: 'min_total', type: 'number', location: 'query', default: '0', minimum: '0', maximum: '1000000', description: 'Smallest order total to count'}, + ], + cache: {enabled: true, ttl_seconds: 60, scope: 'per_principal'}, + limits: {max_rows: 100, timeout_seconds: 30}, + rate_limit: {per_key_qps: 5, burst: 10, per_endpoint_qps: 0}, + }, + async (ep) => { + await publish(ep.id, 'First version'); + must(await a.patch(ws(`endpoints/${ep.id}`), {statements: [{name: 'main', sql: byStatusV2}]}), 'by-status v2 draft'); + await publish(ep.id, 'Rename orders to order_count'); + await grant(ep.id, 'orders-reporting-bot'); + await grant(ep.id, 'finance-dashboard'); + }, +); + +await endpoint( + { + name: 'Orders page', + path: 'orders/page', + description: 'One page of orders since a date, keyed on orderkey, with the total for a pager', + statements: [ + {name: 'rows', sql: 'SELECT orderkey, custkey, orderstatus, totalprice, orderdate\nFROM tpch.tiny.orders\nWHERE orderdate >= {{ since }}\n AND orderkey > {{ after }}\nORDER BY orderkey\nLIMIT 25'}, + {name: 'total', sql: 'SELECT count(*) AS total\nFROM tpch.tiny.orders\nWHERE orderdate >= {{ since }}'}, + ], + params: [ + {name: 'since', type: 'date', location: 'query', required: true, description: 'First order date to include'}, + {name: 'after', type: 'number', location: 'query', default: '0', minimum: '0', description: 'Last orderkey of the previous page'}, + ], + }, + async (ep) => { + await publish(ep.id, 'Rows and total for a pager'); + await grant(ep.id, 'orders-reporting-bot'); + }, +); + +const transform = `def transform(params, statements): + total = statements["total"] + return { + "year": params["year"], + "orders": total[0]["orders"] if len(total) > 0 else 0, + "breakdown": statements["by_status"], + } +`; +await endpoint( + { + name: 'Yearly revenue summary', + path: 'reports/yearly', + description: 'Orders and revenue for one calendar year, as a single document', + statements: [ + {name: 'total', sql: 'SELECT count(*) AS orders, round(sum(totalprice), 2) AS revenue\nFROM tpch.tiny.orders\nWHERE year(orderdate) = {{ year }}'}, + {name: 'by_status', sql: 'SELECT orderstatus AS status, count(*) AS orders\nFROM tpch.tiny.orders\nWHERE year(orderdate) = {{ year }}\nGROUP BY orderstatus\nORDER BY orderstatus'}, + ], + params: [{name: 'year', type: 'number', location: 'query', required: true, minimum: '1992', maximum: '1998', description: 'Calendar year (1992-1998)'}], + output: { + shape: 'objects', + naming: 'as_is', + envelope: 'wrapped', + transform: { + script: transform, + returns: [ + {name: 'year', ty: 'bigint', required: true}, + {name: 'orders', ty: 'bigint', required: true}, + {name: 'breakdown', ty: 'array', required: true}, + ], + }, + }, + cache: {enabled: true, ttl_seconds: 300, scope: 'per_principal'}, + }, + async (ep) => { + await publish(ep.id, 'Summary document'); + await grant(ep.id, 'orders-reporting-bot'); + await grant(ep.id, 'finance-dashboard'); + }, +); + +await endpoint( + { + name: 'Top customers', + path: 'customers/top', + description: 'The twenty customers with the highest total spend', + statements: [{name: 'main', sql: 'SELECT custkey, count(*) AS orders, round(sum(totalprice), 2) AS spend\nFROM tpch.tiny.orders\nGROUP BY custkey\nORDER BY spend DESC\nLIMIT 20'}], + }, + async (ep) => { + await publish(ep.id, 'First version'); + await grant(ep.id, 'orders-reporting-bot'); + // A saved draft that drops a column: publishing it is a breaking change (Publish dialog screenshot). + must( + await a.patch(ws(`endpoints/${ep.id}`), { + statements: [{name: 'main', sql: 'SELECT custkey, count(*) AS orders\nFROM tpch.tiny.orders\nGROUP BY custkey\nORDER BY orders DESC\nLIMIT 20'}], + }), + 'top customers breaking draft', + ); + }, +); + +await endpoint( + { + name: 'Status totals', + path: 'orders/status-totals', + description: 'Orders per status; add revenue with with_revenue=true', + statements: [{name: 'main', sql: 'SELECT orderstatus AS status, count(*) AS orders\n{% if with_revenue %}, round(sum(totalprice), 2) AS revenue\n{% endif %}FROM tpch.tiny.orders\nGROUP BY orderstatus\nORDER BY orderstatus'}], + params: [{name: 'with_revenue', type: 'boolean', location: 'query', description: 'Also return revenue'}], + output: {shape: 'objects', naming: 'camel', envelope: 'wrapped'}, + }, + async (ep) => { + await publish(ep.id, 'Optional revenue column'); + await grant(ep.id, 'finance-dashboard'); + await grant(ep.id, 'orders-reporting-bot'); + }, +); + +const lookupScript = `def build_sql(params): + cols = "orderkey, orderstatus, orderdate" + if params.get("detail"): + cols = cols + ", totalprice, orderpriority, clerk" + return "SELECT " + cols + " FROM tpch.tiny.orders WHERE orderkey = {{ orderkey }}" +`; +await endpoint( + { + name: 'Order lookup', + path: 'orders/lookup', + method: 'POST', + description: 'One order by key; detail=true adds price, priority and clerk', + statements: [{name: 'main', script: lookupScript}], + params: [ + {name: 'orderkey', type: 'number', location: 'query', required: true, minimum: '1', description: 'The order key'}, + {name: 'detail', type: 'boolean', location: 'query', description: 'Include price, priority and clerk'}, + ], + rate_limit: {per_key_qps: 2, burst: 4, per_endpoint_qps: 20}, + }, + async (ep) => { + await publish(ep.id, 'Scripted lookup'); + await grant(ep.id, 'orders-reporting-bot'); + }, +); + +await endpoint({ + name: 'Customer spend', + path: 'customers/spend', + description: 'Total spend per customer, highest first (work in progress)', + statements: [{name: 'main', sql: 'SELECT custkey, round(sum(totalprice), 2) AS spend\nFROM tpch.tiny.orders\nGROUP BY custkey\nORDER BY spend DESC\nLIMIT 20'}], +}); + +await endpoint( + { + name: 'Legacy order export', + path: 'legacy/orders', + description: 'Replaced by Orders page; kept disabled for one release', + statements: [{name: 'main', sql: 'SELECT orderkey, orderstatus, totalprice FROM tpch.tiny.orders ORDER BY orderkey LIMIT 50'}], + }, + async (ep) => { + await publish(ep.id, 'Old export'); + await grant(ep.id, 'partner-sandbox'); + must(await a.post(ws(`endpoints/${ep.id}/disable`), {}), 'disable legacy'); + }, +); + +// ---- Traffic (TRAFFIC=1) ----------------------------------------------------------------------------- +if (process.env.TRAFFIC) { + const token = async (name) => { + const s = secrets[name]; + if (!s) throw new Error(`no stored secret for ${name} in ${SECRETS}`); + const r = await fetch(`${BASE}/oauth/token`, { + method: 'POST', + headers: {'content-type': 'application/x-www-form-urlencoded'}, + body: new URLSearchParams({grant_type: 'client_credentials', ...s}), + }); + const j = await r.json(); + if (!j.access_token) throw new Error(`token for ${name}: ${r.status} ${JSON.stringify(j)}`); + return j.access_token; + }; + const call = async (tok, path, {method = 'GET', headers = {}} = {}) => { + const r = await fetch(`${BASE}/data-api/v1/${WS}/${path}`, {method, headers: {authorization: `Bearer ${tok}`, ...headers}}); + await r.arrayBuffer(); + return r.status; + }; + const bot = await token('nightly-orders-etl'); + const fin = await token('finance-sheet'); + const partner = await token('partner-trial'); + const tally = {}; + const note = (s) => (tally[s] = (tally[s] ?? 0) + 1); + const pause = (ms) => new Promise((r) => setTimeout(r, ms)); + for (let i = 0; i < 6; i++) { + for (const p of ['', '?priority=2-HIGH', '?priority=1-URGENT&min_total=100000', '?min_total=250000']) { + note(await call(bot, `orders/by-status${p}`)); + note(await call(fin, `orders/by-status${p}`)); + await pause(250); + } + for (const y of [1993, 1995, 1997]) note(await call(bot, `reports/yearly?year=${y}`)); + note(await call(fin, 'reports/yearly?year=1996')); + note(await call(bot, `orders/page?since=1998-06-01&after=${i * 25000}`)); + note(await call(bot, 'customers/top')); + note(await call(fin, `orders/status-totals${i % 2 ? '?with_revenue=true' : ''}`)); + note(await call(bot, `orders/lookup?orderkey=${1 + i * 32}&detail=${i % 2 ? 'true' : 'false'}`, {method: 'POST'})); + await pause(400); + } + // Errors a consumer typically makes. + note(await call(bot, 'orders/by-status?priority=9-X')); + note(await call(bot, 'reports/yearly')); + note(await call(bot, 'reports/yearly?year=2001')); + note(await call(partner, 'orders/by-status')); + note(await call(partner, 'customers/top')); + note(await call(bot, 'orders/by-satus')); + note(await call(bot, 'legacy/orders')); + note(await call(bot, 'orders/lookup?orderkey=1')); + note(await call(bot, 'orders/by-status', {headers: {'X-Egret-Version': '1'}})); + note(await call(bot, 'orders/by-status', {headers: {'X-Egret-Version': '7'}})); + note(await call('not-a-token', 'orders/by-status')); + // A burst over the per-caller limit (5 qps, burst 10). + const burst = await Promise.all(Array.from({length: 30}, (_, i) => call(bot, `orders/by-status?min_total=${1000 + i}`))); + burst.forEach(note); + console.log('traffic', JSON.stringify(tally)); +} +console.log('seeded data-api'); diff --git a/scripts/shots/apps.mjs b/scripts/shots/apps.mjs new file mode 100644 index 0000000..4ecabb0 --- /dev/null +++ b/scripts/shots/apps.mjs @@ -0,0 +1,137 @@ +// Screenshots for the Apps pages (area: apps). Objects used (created by scripts/seed-apps.mjs): the apps +// "Revenue explorer" (v1 to v4, three shares, folder "Sales analysis"), "Order trends", "Customer lookup" and +// the draft-only "Shipping modes". The editor and the viewer are marimo, served from the apps origin +// (apps.localtest.me on the demo stack) inside an iframe; a cold start can take a minute. +const ws = '/explore/default'; +const START_MS = 180_000; + +const appHref = async (page, name) => + new URL((await page.getByRole('link', {name, exact: true}).getAttribute('href')).replace(/\/edit$/, ''), page.url()).href; +const rowActions = (name) => async (page) => { + await page.getByRole('row').filter({hasText: name}).getByRole('button', {name: 'Actions'}).click(); +}; +/** The marimo frame: the iframe whose document is on the apps origin. */ +async function appsFrame(page, selector) { + const handle = await page.locator(selector).elementHandle({timeout: 60_000}); + const frame = await handle.contentFrame(); + await frame.waitForURL((u) => u.host.startsWith('apps.'), {timeout: START_MS}); + return frame; +} +const openEditor = async (page, name) => { + await page.goto(`${await appHref(page, name)}/edit`); + return appsFrame(page, "iframe[title$=' editor']"); +}; +const openViewer = async (page, name, text) => { + await page.goto(await appHref(page, name)); + const frame = await appsFrame(page, `iframe[title='${name}']`); + await frame.getByText(text).first().waitFor({timeout: START_MS}); + return frame; +}; + +const list = [ + { + id: 'apps/apps-list', + path: `${ws}/apps`, + prepare: async (page) => { + await page.getByText('Customer lookup').waitFor(); + await page.setViewportSize({width: 1440, height: 600}); + }, + settle: 1000, + }, + { + id: 'apps/create-dialog', + path: `${ws}/apps`, + prepare: async (page) => { + await page.getByRole('button', {name: '+ App'}).click(); + await page.getByLabel('Name', {exact: true}).fill('Shipping costs'); + }, + settle: 800, + }, + {id: 'apps/row-menu', path: `${ws}/apps`, prepare: rowActions('Revenue explorer'), settle: 600}, + { + id: 'apps/editor', + path: `${ws}/apps`, + prepare: async (page) => { + await page.setViewportSize({width: 1440, height: 1100}); + const frame = await openEditor(page, 'Revenue explorer'); + await frame.getByText('import egret_marimo').first().waitFor({timeout: START_MS}); + // Run every cell (the global run button is the last one), then scroll to the query and its output. + await frame.locator('[data-testid="run-button"]').last().click(); + await frame.getByText('1-URGENT').first().waitFor({timeout: 90_000}); + // A reused session shows a "Reconnected" banner over the first cell: close it. + const reconnected = frame.getByText('Reconnected', {exact: true}); + if (await reconnected.isVisible()) await reconnected.locator('xpath=ancestor::div[.//button][1]').getByRole('button').last().click().catch(() => {}); + await frame.getByText('egret.sql(').first().evaluate((el) => el.scrollIntoView({block: 'start'})); + await page.waitForTimeout(3000); + }, + settle: 1000, + }, + { + id: 'apps/viewer', + path: `${ws}/apps`, + prepare: async (page) => { + await openViewer(page, 'Revenue explorer', '1-URGENT'); + await page.waitForTimeout(3000); + }, + settle: 1000, + }, + { + id: 'apps/viewer-trends', + path: `${ws}/apps`, + prepare: async (page) => { + await openViewer(page, 'Order trends', 'Most orders in one month'); + await page.waitForTimeout(3000); + }, + settle: 1000, + }, + { + id: 'apps/not-published', + path: `${ws}/apps`, + prepare: async (page) => { + await page.goto(await appHref(page, 'Shipping modes')); + await page.getByText('Nobody has published this app yet').waitFor(); + await page.setViewportSize({width: 1440, height: 400}); + }, + settle: 500, + }, + { + id: 'apps/versions-dialog', + path: `${ws}/apps`, + prepare: async (page) => { + await page.goto(`${await appHref(page, 'Revenue explorer')}/edit`); + await page.getByRole('button', {name: 'Versions'}).click(); + await page.getByRole('dialog').getByText('v4', {exact: true}).waitFor(); + }, + settle: 800, + }, + { + id: 'apps/restore-dialog', + path: `${ws}/apps`, + prepare: async (page) => { + await page.goto(`${await appHref(page, 'Revenue explorer')}/edit`); + await page.getByRole('button', {name: 'Versions'}).click(); + await page.getByRole('dialog').getByRole('button', {name: 'Restore'}).nth(2).click(); + }, + settle: 800, + }, + { + id: 'apps/share-dialog', + path: `${ws}/apps`, + prepare: async (page) => { + await rowActions('Revenue explorer')(page); + await page.getByRole('menuitem', {name: 'Share…'}).click(); + await page.getByText('carla.mendes@egret.local').waitFor(); + }, + settle: 800, + }, +]; + +// screenshots.mjs reuses one page, so a viewport set by one shot would leak into the next: each shot starts at +// 1440x900 unless its own prepare changes it. +export const shots = list.map((s) => ({ + ...s, + prepare: async (page) => { + await page.setViewportSize({width: 1440, height: 900}); + if (s.prepare) await s.prepare(page); + }, +})); diff --git a/scripts/shots/data-api.mjs b/scripts/shots/data-api.mjs index 6135bfd..6063c70 100644 --- a/scripts/shots/data-api.mjs +++ b/scripts/shots/data-api.mjs @@ -1,6 +1,5 @@ -// Screenshots for the Data API, Alerts & reports and Apps pages (area: data-api). -// Objects used (created by scripts/seed-data-api.sh): service accounts "orders-reporting-bot" and -// "partner-sandbox"; endpoints "Orders by status", "Orders page", "Yearly revenue summary", "Top customers". +// Screenshots for the Data API and Alerts & reports pages (Apps: scripts/shots/apps.mjs). +// Data API objects are created by scripts/seed-data-api.mjs (TRAFFIC=1 also fills Activity). const ws = '/explore/default'; const A = 'data-api'; const tall = (h) => (page) => page.setViewportSize({width: 1440, height: h}); @@ -15,7 +14,7 @@ const openEndpoint = (name, tab, height) => async (page) => { } }; -export const shots = [ +const list = [ {id: `${A}/endpoints-list`, path: `${ws}/data-api/endpoints`, settle: 1500}, { id: `${A}/endpoint-actions-menu`, @@ -55,7 +54,7 @@ export const shots = [ id: `${A}/endpoint-params`, path: `${ws}/data-api/endpoints`, prepare: async (page) => { - await openEndpoint('Orders by status', null, 1500)(page); + await openEndpoint('Orders by status', null, 1300)(page); await page.getByText('Database and parameters').click(); await page.waitForTimeout(500); }, @@ -74,7 +73,7 @@ export const shots = [ {id: `${A}/endpoint-config`, path: `${ws}/data-api/endpoints`, prepare: openEndpoint('Orders by status', 'Config', 1450), settle: 1500}, {id: `${A}/endpoint-response`, path: `${ws}/data-api/endpoints`, prepare: openEndpoint('Orders by status', 'Response', 1150), settle: 1500}, {id: `${A}/endpoint-response-transform`, path: `${ws}/data-api/endpoints`, prepare: openEndpoint('Yearly revenue summary', 'Response', 1500), settle: 2500}, - {id: `${A}/endpoint-access`, path: `${ws}/data-api/endpoints`, prepare: openEndpoint('Orders by status', 'Access'), settle: 2000}, + {id: `${A}/endpoint-access`, path: `${ws}/data-api/endpoints`, prepare: openEndpoint('Orders by status', 'Access', 640), settle: 2000}, { id: `${A}/endpoint-try`, path: `${ws}/data-api/endpoints`, @@ -100,7 +99,7 @@ export const shots = [ }, settle: 500, }, - {id: `${A}/endpoint-versions`, path: `${ws}/data-api/endpoints`, prepare: openEndpoint('Orders by status', 'Versions'), settle: 2000}, + {id: `${A}/endpoint-versions`, path: `${ws}/data-api/endpoints`, prepare: openEndpoint('Orders by status', 'Versions', 560), settle: 2000}, { id: `${A}/publish-dialog-breaking`, path: `${ws}/data-api/endpoints`, @@ -115,9 +114,9 @@ export const shots = [ id: `${A}/rollback-dialog`, path: `${ws}/data-api/endpoints`, prepare: async (page) => { - await openEndpoint('Orders by status', 'Versions')(page); + await openEndpoint('Orders by status', 'Versions', 760)(page); await page.getByRole('button', {name: 'Roll back'}).click(); - await page.waitForTimeout(800); + await page.getByPlaceholder('rollback').fill('rollback'); }, settle: 500, }, @@ -131,11 +130,12 @@ export const shots = [ }, settle: 800, }, - {id: `${A}/clients-list`, path: `${ws}/data-api/clients`, settle: 1500}, + {id: `${A}/clients-list`, path: `${ws}/data-api/clients`, prepare: tall(640), settle: 1500}, { id: `${A}/service-account-new`, path: `${ws}/data-api/clients/new`, prepare: async (page) => { + await page.setViewportSize({width: 1440, height: 560}); await page.getByLabel('Name', {exact: true}).fill('finance-dashboard'); await page.getByLabel('Description (optional)').fill('Read-only reporting for the finance team'); }, @@ -148,6 +148,7 @@ export const shots = [ await page.getByRole('button', {name: 'Actions for orders-reporting-bot'}).click(); await page.getByRole('menuitem', {name: /Issue client/}).click(); await page.waitForLoadState('networkidle'); + await page.setViewportSize({width: 1440, height: 500}); await page.getByLabel('Name', {exact: true}).fill('weekly-export'); }, settle: 800, @@ -155,63 +156,14 @@ export const shots = [ {id: `${A}/activity`, path: `${ws}/data-api/activity`, prepare: tall(1250), settle: 2500}, // Alerts & reports shots live in scripts/shots/alerts.mjs. - // ---- Apps ------------------------------------------------------------------------------------- - {id: 'apps/apps-list', path: `${ws}/apps`, settle: 2000}, - { - id: 'apps/create-dialog', - path: `${ws}/apps`, - prepare: async (page) => { - await page.getByRole('button', {name: '+ App'}).click(); - await page.getByLabel('Name', {exact: true}).fill('Revenue explorer'); - }, - settle: 800, - }, - { - id: 'apps/row-menu', - path: `${ws}/apps`, - prepare: async (page) => { - await page.getByRole('button', {name: 'Actions', exact: true}).click(); - }, - settle: 600, - }, - { - id: 'apps/edit-unavailable', - path: `${ws}/apps`, - prepare: async (page) => { - await page.getByRole('button', {name: 'Actions', exact: true}).click(); - await page.getByRole('menuitem', {name: 'Edit'}).click(); - await page.waitForTimeout(2500); - }, - settle: 500, - }, - { - id: 'apps/view-unavailable', - path: `${ws}/apps`, - prepare: async (page) => { - await page.getByText('Revenue explorer', {exact: true}).first().click(); - await page.waitForTimeout(2500); - }, - settle: 500, - }, - { - id: 'apps/versions-dialog', - path: `${ws}/apps`, - prepare: async (page) => { - await page.getByRole('button', {name: 'Actions', exact: true}).click(); - await page.getByRole('menuitem', {name: 'Edit'}).click(); - await page.getByRole('button', {name: 'Versions'}).click(); - await page.waitForTimeout(1500); - }, - settle: 500, - }, - { - id: 'apps/share-dialog', - path: `${ws}/apps`, - prepare: async (page) => { - await page.getByRole('button', {name: 'Actions', exact: true}).click(); - await page.getByRole('menuitem', {name: 'Share…'}).click(); - await page.waitForTimeout(1500); - }, - settle: 500, - }, ]; + +// screenshots.mjs reuses one page, so a viewport set by one shot would leak into the next: each Data API shot +// starts at 1440x900 unless its own prepare changes it. +export const shots = list.map((s) => !s.id.startsWith(`${A}/`) ? s : ({ + ...s, + prepare: async (page) => { + await page.setViewportSize({width: 1440, height: 900}); + if (s.prepare) await s.prepare(page); + }, +})); diff --git a/static/img/platform/apps/apps-list.png b/static/img/platform/apps/apps-list.png index b0eef2e..e801c93 100644 Binary files a/static/img/platform/apps/apps-list.png and b/static/img/platform/apps/apps-list.png differ diff --git a/static/img/platform/apps/create-dialog.png b/static/img/platform/apps/create-dialog.png index 9212a9e..cb3c08d 100644 Binary files a/static/img/platform/apps/create-dialog.png and b/static/img/platform/apps/create-dialog.png differ diff --git a/static/img/platform/apps/edit-unavailable.png b/static/img/platform/apps/edit-unavailable.png deleted file mode 100644 index 18a3936..0000000 Binary files a/static/img/platform/apps/edit-unavailable.png and /dev/null differ diff --git a/static/img/platform/apps/editor.png b/static/img/platform/apps/editor.png new file mode 100644 index 0000000..b49c957 Binary files /dev/null and b/static/img/platform/apps/editor.png differ diff --git a/static/img/platform/apps/not-published.png b/static/img/platform/apps/not-published.png new file mode 100644 index 0000000..c5b5fe5 Binary files /dev/null and b/static/img/platform/apps/not-published.png differ diff --git a/static/img/platform/apps/restore-dialog.png b/static/img/platform/apps/restore-dialog.png new file mode 100644 index 0000000..99ae8d6 Binary files /dev/null and b/static/img/platform/apps/restore-dialog.png differ diff --git a/static/img/platform/apps/row-menu.png b/static/img/platform/apps/row-menu.png index 3689439..7d17abf 100644 Binary files a/static/img/platform/apps/row-menu.png and b/static/img/platform/apps/row-menu.png differ diff --git a/static/img/platform/apps/share-dialog.png b/static/img/platform/apps/share-dialog.png index 8709c83..f877735 100644 Binary files a/static/img/platform/apps/share-dialog.png and b/static/img/platform/apps/share-dialog.png differ diff --git a/static/img/platform/apps/versions-dialog.png b/static/img/platform/apps/versions-dialog.png index e151982..b459687 100644 Binary files a/static/img/platform/apps/versions-dialog.png and b/static/img/platform/apps/versions-dialog.png differ diff --git a/static/img/platform/apps/view-unavailable.png b/static/img/platform/apps/view-unavailable.png deleted file mode 100644 index 5493ac1..0000000 Binary files a/static/img/platform/apps/view-unavailable.png and /dev/null differ diff --git a/static/img/platform/apps/viewer-trends.png b/static/img/platform/apps/viewer-trends.png new file mode 100644 index 0000000..1d07f07 Binary files /dev/null and b/static/img/platform/apps/viewer-trends.png differ diff --git a/static/img/platform/apps/viewer.png b/static/img/platform/apps/viewer.png new file mode 100644 index 0000000..c136325 Binary files /dev/null and b/static/img/platform/apps/viewer.png differ diff --git a/static/img/platform/data-api-activity.png b/static/img/platform/data-api-activity.png index fcaad69..fa24976 100644 Binary files a/static/img/platform/data-api-activity.png and b/static/img/platform/data-api-activity.png differ diff --git a/static/img/platform/data-api-client-new.png b/static/img/platform/data-api-client-new.png index fd9c5c0..f3c4371 100644 Binary files a/static/img/platform/data-api-client-new.png and b/static/img/platform/data-api-client-new.png differ diff --git a/static/img/platform/data-api-clients.png b/static/img/platform/data-api-clients.png index 55f4536..76011c3 100644 Binary files a/static/img/platform/data-api-clients.png and b/static/img/platform/data-api-clients.png differ diff --git a/static/img/platform/data-api-endpoint-new.png b/static/img/platform/data-api-endpoint-new.png index 552fc9d..62849d6 100644 Binary files a/static/img/platform/data-api-endpoint-new.png and b/static/img/platform/data-api-endpoint-new.png differ diff --git a/static/img/platform/data-api-endpoints.png b/static/img/platform/data-api-endpoints.png index 93fcee5..67a7e2d 100644 Binary files a/static/img/platform/data-api-endpoints.png and b/static/img/platform/data-api-endpoints.png differ diff --git a/static/img/platform/data-api/activity.png b/static/img/platform/data-api/activity.png index 10ab913..67c2672 100644 Binary files a/static/img/platform/data-api/activity.png and b/static/img/platform/data-api/activity.png differ diff --git a/static/img/platform/data-api/client-new.png b/static/img/platform/data-api/client-new.png index 27ee4cc..7ba7b33 100644 Binary files a/static/img/platform/data-api/client-new.png and b/static/img/platform/data-api/client-new.png differ diff --git a/static/img/platform/data-api/clients-list.png b/static/img/platform/data-api/clients-list.png index 3bac16e..b119fe6 100644 Binary files a/static/img/platform/data-api/clients-list.png and b/static/img/platform/data-api/clients-list.png differ diff --git a/static/img/platform/data-api/endpoint-access.png b/static/img/platform/data-api/endpoint-access.png index 7349314..d9551df 100644 Binary files a/static/img/platform/data-api/endpoint-access.png and b/static/img/platform/data-api/endpoint-access.png differ diff --git a/static/img/platform/data-api/endpoint-actions-menu.png b/static/img/platform/data-api/endpoint-actions-menu.png index 94c9b3c..b60cdf4 100644 Binary files a/static/img/platform/data-api/endpoint-actions-menu.png and b/static/img/platform/data-api/endpoint-actions-menu.png differ diff --git a/static/img/platform/data-api/endpoint-config.png b/static/img/platform/data-api/endpoint-config.png index f42f99c..8284064 100644 Binary files a/static/img/platform/data-api/endpoint-config.png and b/static/img/platform/data-api/endpoint-config.png differ diff --git a/static/img/platform/data-api/endpoint-new.png b/static/img/platform/data-api/endpoint-new.png index df0a128..a0ee5f6 100644 Binary files a/static/img/platform/data-api/endpoint-new.png and b/static/img/platform/data-api/endpoint-new.png differ diff --git a/static/img/platform/data-api/endpoint-params.png b/static/img/platform/data-api/endpoint-params.png index a56e02f..20dce69 100644 Binary files a/static/img/platform/data-api/endpoint-params.png and b/static/img/platform/data-api/endpoint-params.png differ diff --git a/static/img/platform/data-api/endpoint-query.png b/static/img/platform/data-api/endpoint-query.png index 22f1a02..7f2cd3f 100644 Binary files a/static/img/platform/data-api/endpoint-query.png and b/static/img/platform/data-api/endpoint-query.png differ diff --git a/static/img/platform/data-api/endpoint-response-transform.png b/static/img/platform/data-api/endpoint-response-transform.png index b155adb..e80ed8b 100644 Binary files a/static/img/platform/data-api/endpoint-response-transform.png and b/static/img/platform/data-api/endpoint-response-transform.png differ diff --git a/static/img/platform/data-api/endpoint-response.png b/static/img/platform/data-api/endpoint-response.png index b671942..4c0a0c7 100644 Binary files a/static/img/platform/data-api/endpoint-response.png and b/static/img/platform/data-api/endpoint-response.png differ diff --git a/static/img/platform/data-api/endpoint-statements.png b/static/img/platform/data-api/endpoint-statements.png index 608a4f4..5b96367 100644 Binary files a/static/img/platform/data-api/endpoint-statements.png and b/static/img/platform/data-api/endpoint-statements.png differ diff --git a/static/img/platform/data-api/endpoint-try-refused.png b/static/img/platform/data-api/endpoint-try-refused.png index 85695ee..7311ff8 100644 Binary files a/static/img/platform/data-api/endpoint-try-refused.png and b/static/img/platform/data-api/endpoint-try-refused.png differ diff --git a/static/img/platform/data-api/endpoint-try.png b/static/img/platform/data-api/endpoint-try.png index 827e0ec..3bfb904 100644 Binary files a/static/img/platform/data-api/endpoint-try.png and b/static/img/platform/data-api/endpoint-try.png differ diff --git a/static/img/platform/data-api/endpoint-versions.png b/static/img/platform/data-api/endpoint-versions.png index 515239e..9add490 100644 Binary files a/static/img/platform/data-api/endpoint-versions.png and b/static/img/platform/data-api/endpoint-versions.png differ diff --git a/static/img/platform/data-api/endpoints-list.png b/static/img/platform/data-api/endpoints-list.png index a9723d0..5db4159 100644 Binary files a/static/img/platform/data-api/endpoints-list.png and b/static/img/platform/data-api/endpoints-list.png differ diff --git a/static/img/platform/data-api/openapi-dialog.png b/static/img/platform/data-api/openapi-dialog.png index 3e91235..88d9d2c 100644 Binary files a/static/img/platform/data-api/openapi-dialog.png and b/static/img/platform/data-api/openapi-dialog.png differ diff --git a/static/img/platform/data-api/publish-dialog-breaking.png b/static/img/platform/data-api/publish-dialog-breaking.png index 27d4cc6..25ba3f4 100644 Binary files a/static/img/platform/data-api/publish-dialog-breaking.png and b/static/img/platform/data-api/publish-dialog-breaking.png differ diff --git a/static/img/platform/data-api/rollback-dialog.png b/static/img/platform/data-api/rollback-dialog.png index a3c9866..dc98e74 100644 Binary files a/static/img/platform/data-api/rollback-dialog.png and b/static/img/platform/data-api/rollback-dialog.png differ diff --git a/static/img/platform/data-api/service-account-new.png b/static/img/platform/data-api/service-account-new.png index d411807..2f381bf 100644 Binary files a/static/img/platform/data-api/service-account-new.png and b/static/img/platform/data-api/service-account-new.png differ diff --git a/static/img/platform/explore-apps.png b/static/img/platform/explore-apps.png index 5875c8e..a36c773 100644 Binary files a/static/img/platform/explore-apps.png and b/static/img/platform/explore-apps.png differ