diff --git a/platform/get-started/overview.md b/platform/get-started/overview.md index 451a437..9cb6b04 100644 --- a/platform/get-started/overview.md +++ b/platform/get-started/overview.md @@ -2,12 +2,20 @@ sidebar_position: 1 --- +import ThemedImage from '@theme/ThemedImage'; +import useBaseUrl from '@docusaurus/useBaseUrl'; + # Overview Egret is an enterprise data platform that lets many teams work on one shared data foundation. A single deployment gives the organization one source of truth, gives each team an isolated **workspace**, and governs and audits every access to data. + + It is written in Rust as a modular monolith (one server binary) and brings together tools you would otherwise assemble from separate products: @@ -20,6 +28,10 @@ from separate products: | Serving data as an API | SELECT statements with parameters published as REST endpoints | [Data API](../data-api/overview) | | Automation | A workflow engine compatible with Kestra's flow syntax, with schedules and triggers | [Workflows](../workflows/flows) | +![The E-commerce Insights dashboard of the demo stack: revenue and orders per quarter, a goal, a combined chart over time and a breakdown by product category](/img/platform/ops/get-started/demo-ecommerce-overview.png) + +*A dashboard in a workspace: the demo stack seeds this one, with 22 charts over sample data.* + ## Where to start 1. **Run it.** [Run the demo stack](./run-the-demo-stack) starts the whole platform on Docker with one command (`just demo-up`), diff --git a/platform/get-started/run-the-demo-stack.md b/platform/get-started/run-the-demo-stack.md index 87fcb76..b5f83f3 100644 --- a/platform/get-started/run-the-demo-stack.md +++ b/platform/get-started/run-the-demo-stack.md @@ -8,10 +8,10 @@ The demo stack runs the whole platform on Docker with one command: Egret, Postgr Keycloak for single sign-on, a Trino engine with sample data, Data Apps, a renderer for PNG and PDF reports and a mail catcher. It is the fastest way to try Egret: you need Docker and `just`, and no Rust toolchain, Node.js or Kubernetes. -The stack comes seeded with a workspace, a database connection, a dataset, three charts and a dashboard, so there is -something to look at as soon as you sign in. +The stack comes seeded with a workspace, a database connection, datasets, charts and two dashboards, a small **Sales overview** and +a full **E-commerce Insights** with 22 charts on three tabs, so there is something to look at as soon as you sign in. -![The seeded Sales overview dashboard on the demo stack](/img/platform/ops/get-started/demo-sales-overview.png) +![The seeded E-commerce Insights dashboard on the demo stack](/img/platform/ops/get-started/demo-ecommerce-overview.png) :::warning Demo only: never expose it to a network Every secret in the stack is a public value committed to the repository (`deploy/demo/demo.env`), the Keycloak realm accepts any @@ -142,8 +142,13 @@ In the workspace `default` (shown as **Default**): | Dataset | `orders` | TPC-H `tpch.tiny.orders`, with the metrics `revenue` (`sum(totalprice)`) and `order_count` (`count(*)`). | | Charts | **Revenue by order status**, **Revenue by month**, **Orders by priority** | A bar chart, a line chart by month and a table. | | Dashboard | **Sales overview** | The three charts. | +| Datasets | `ecom_orders`, `ecom_reviews`, `accounts`, `funnel_steps` | Virtual datasets over the TPC-H sample data, reshaped into an online store: product categories, vendors, US states, marketing sources, age groups and ratings. The values are illustrative, and the dates run from 2023 to June 2029. | +| Charts | 22, from **Revenue per quarter** to **Most recent subscription** | One of every kind: scalar and trend, progress, gauge, combo, bar, row, line, area, pie, box plot, scatter, pivot table, table, object detail, map, funnel, waterfall and Sankey. | +| Dashboard | **E-commerce Insights (egret clone)** | Three tabs, *Overview*, *Portfolio Performance* and *Website Analysis*, built after Metabase's sample dashboard of the same name. | | Mail server | Workspace **Notifications** | SMTP to Mailpit (`mailpit:1025`), sender `egret@demo.local`. | +![The Portfolio Performance tab of E-commerce Insights: a vendor donut, a box plot, best-selling products, a map of revenue by state and a scatter plot](/img/platform/ops/get-started/demo-ecommerce-portfolio.png) + And in the Control Plane, for **Add engine**: the storage connection `demo-rustfs` (the demo object store, bucket `warehouse`) and the warehouse `demo-warehouse`, whose catalog is `demo_warehouse`. diff --git a/scripts/shots/ops.mjs b/scripts/shots/ops.mjs index f53a3b8..61928b3 100644 --- a/scripts/shots/ops.mjs +++ b/scripts/shots/ops.mjs @@ -47,6 +47,16 @@ const openDashboard = async (page, name) => { await page.getByText('Showing 5 rows').first().waitFor({timeout: 60000}).catch(() => {}); }; +// "E-commerce Insights (egret clone)" has three tabs; wait until the charts of the chosen tab have drawn. +const openEcommerce = async (page, tab) => { + await page.getByText('E-commerce Insights (egret clone)', {exact: true}).first().click(); + await page.waitForLoadState('networkidle'); + await page.getByRole('tab', {name: tab}).first().click(); + // Taller than the usual 1440x900 so that three rows of charts show; the next shot starts at 900 again. + await page.setViewportSize({width: 1440, height: 1330}); + await page.waitForTimeout(7000); +}; + export const shots = [ { id: 'ops/get-started/explorer-home', @@ -152,6 +162,19 @@ export const shots = [ settle: 3000, clip: {x: 0, y: 0, width: 1440, height: 620}, }, + // The fuller dashboard of the Docker demo stack (scripts/seed_ecommerce_clone.py in the platform repo). + { + id: 'ops/get-started/demo-ecommerce-overview', + path: `${ws}/dashboards`, + prepare: async (page) => openEcommerce(page, 'Overview'), + settle: 1500, + }, + { + id: 'ops/get-started/demo-ecommerce-portfolio', + path: `${ws}/dashboards`, + prepare: async (page) => openEcommerce(page, 'Portfolio Performance'), + settle: 1500, + }, // Mailpit (outside Egret): the inbox with a report mail opened. { id: 'ops/get-started/demo-mailpit', diff --git a/static/img/architecture-dark.svg b/static/img/architecture-dark.svg new file mode 100644 index 0000000..5341d35 --- /dev/null +++ b/static/img/architecture-dark.svg @@ -0,0 +1,117 @@ + +Egret architecture + + + +One platform · many teams · one governed data foundation +TEAMS + +Analytics + +Engineering + +Finance +IDENTITY + +OIDC / SSO +Keycloak · groups from claims + + + + +Egret +one Rust binary · control plane + workspaces +WORKSPACES · ISOLATED PER TEAM + +Workspace A +datasets · charts · apps + +Workspace B +datasets · charts · apps + +Workspace C +datasets · charts · apps +WHAT EACH WORKSPACE RUNS + + + +Explore +SQL Lab · charts +dashboards · alerts + + + +Data API +SQL to REST +OAuth2 clients + + + +Workflows +flows · schedules +triggers · webhooks + + + +Data Apps +marimo apps +run as the viewer + +GOVERNANCE PLANE · EVERY REQUEST, EVERY WORKSPACE + +Cedar authorization + +Ranger policy + +Per-user delegation + +Audit log +CONTROL PLANE +Catalogs · Engines · Storage · Secrets · Users & roles +DATA PLANE + +QUERY ENGINES + +Trino + +StarRocks +Trino: Docker or Kubernetes +StarRocks: Kubernetes + +ICEBERG LAKEHOUSE +Source of truth +Embedded REST catalog (lakekeeper) +S3-compatible object storage + +EXTERNAL SOURCES + +PostgreSQL + +MySQL + +Kafka + +BigQuery + +Snowflake + ++ 30 more + + + + + + + + + +as the caller + +Rust · one binary + +Docker demo, one command + +Docker or Kubernetes + +Apache-2.0 + \ No newline at end of file diff --git a/static/img/architecture-light.svg b/static/img/architecture-light.svg new file mode 100644 index 0000000..7300292 --- /dev/null +++ b/static/img/architecture-light.svg @@ -0,0 +1,117 @@ + +Egret architecture + + + +One platform · many teams · one governed data foundation +TEAMS + +Analytics + +Engineering + +Finance +IDENTITY + +OIDC / SSO +Keycloak · groups from claims + + + + +Egret +one Rust binary · control plane + workspaces +WORKSPACES · ISOLATED PER TEAM + +Workspace A +datasets · charts · apps + +Workspace B +datasets · charts · apps + +Workspace C +datasets · charts · apps +WHAT EACH WORKSPACE RUNS + + + +Explore +SQL Lab · charts +dashboards · alerts + + + +Data API +SQL to REST +OAuth2 clients + + + +Workflows +flows · schedules +triggers · webhooks + + + +Data Apps +marimo apps +run as the viewer + +GOVERNANCE PLANE · EVERY REQUEST, EVERY WORKSPACE + +Cedar authorization + +Ranger policy + +Per-user delegation + +Audit log +CONTROL PLANE +Catalogs · Engines · Storage · Secrets · Users & roles +DATA PLANE + +QUERY ENGINES + +Trino + +StarRocks +Trino: Docker or Kubernetes +StarRocks: Kubernetes + +ICEBERG LAKEHOUSE +Source of truth +Embedded REST catalog (lakekeeper) +S3-compatible object storage + +EXTERNAL SOURCES + +PostgreSQL + +MySQL + +Kafka + +BigQuery + +Snowflake + ++ 30 more + + + + + + + + + +as the caller + +Rust · one binary + +Docker demo, one command + +Docker or Kubernetes + +Apache-2.0 + \ No newline at end of file diff --git a/static/img/platform/ops/get-started/demo-ecommerce-overview.png b/static/img/platform/ops/get-started/demo-ecommerce-overview.png new file mode 100644 index 0000000..2ee376f Binary files /dev/null and b/static/img/platform/ops/get-started/demo-ecommerce-overview.png differ diff --git a/static/img/platform/ops/get-started/demo-ecommerce-portfolio.png b/static/img/platform/ops/get-started/demo-ecommerce-portfolio.png new file mode 100644 index 0000000..90c607e Binary files /dev/null and b/static/img/platform/ops/get-started/demo-ecommerce-portfolio.png differ