Skip to content

feat(dashboards): table-backed dashboard files behind a rollout flag - #8305

Open
TheodoreSpeaks wants to merge 22 commits into
stagingfrom
codex/research-sim-dashboards
Open

TheodoreSpeaks wants to merge 22 commits into
stagingfrom
codex/research-sim-dashboards

Conversation

@TheodoreSpeaks

@TheodoreSpeaks TheodoreSpeaks commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds table-backed dashboards as a workspace file type, handled like Sim pages. A dashboard is a YAML file that selects authorized table queries and lays out metrics, ECharts charts, tables, text, and tabs; aggregation runs on the server with bounded results and timeouts. The Files viewer and the Chat resource panel render it live.

Companion: https://github.com/simstudioai/mothership/pull/531

  • Storage: creating or uploading <Name>.dashboard drops the suffix and stamps text/x-sim-dashboard, the same ingestion pattern pages use for .html. The type is metadata that content writes never reset. Dashboards live in Files and its folders, use the ordinary file tools, and appear in Chat as files. There is no separate resource type, folder tree, API, or Mothership tool.
  • Rendering: shared time-range and timezone controls, drag-to-select ranges, synchronized hover values, responsive layout, muted EMCN styling, and animated metrics. Horizontal bar charts size to their row count, keep an authored label column intact, and draw a row highlight around label and bar.
  • Queries: a session-only tables.rows.analytics operation. Conditional percentage aggregates reuse the table filter grammar, so tables need no 100/0 helper columns.
  • Rollout: the default-off dashboards flag, targetable globally or by organization and resolved from the workspace's owning organization. It gates the live viewer and the analytics operation. Public share links show a workspace-only notice and never query.
  • Write diagnostics: the v2 file create, replace and edit responses carry an optional diagnostics list for dashboard files: parse errors as path: message lines, or an empty list. Writes are never blocked, like the page lint. The OpenAPI document and CLI API are regenerated; create now documents V2CreatedFileResponse.
  • Authoring: Mothership learns the format from a sim-dashboards reference in its own research-and-deliverables skill (companion PR), the same way it learns Sim Pages. There is no Sim built-in skill; workspace built-ins stay Agent-block documents.
  • Chat previewability and tab/picker icons now come from the file's stored type, so extensionless files render and get the right icon.

Type of Change

  • New feature

Testing

  • Chart and dashboard suites (58 tests) include real-renderer checks for the horizontal bar layout: labels clear neighbouring bars at the computed height, an authored left label column stays intact, and the row highlight covers the hovered label and bar without reaching adjacent rows. Each layout regression test failed before its fix.
  • Real PostgreSQL integration tests for the analytics query compiler. A new integration test covers .dashboard ingestion (suffix dropped, type sticky across a text/plain write); it is written but was not run locally because Docker was unavailable.
  • End to end against a local stack: Mothership, given only a plain request, produced a dashboard that passes the parser, renders in the Files viewer and Chat panel, and uses only the generic file, table, and skill CLI commands.
  • Type-check (all 26 workspaces), lint:check, and all repository audits pass after merging staging.

Rollout and scope

No migrations: this branch leaves packages/db identical to staging. Enable the flag per organization in AppConfig (dashboards: { orgIds: [...] }), or set DASHBOARDS=true locally. Public live dashboards, log queries, and per-chip MIME icons in the composer are deferred. No merge or deployment is included.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

🤖 Generated with Claude Code

https://claude.ai/code/session_01S6aTRnkiu7PxYZPNPXYEMV

@vercel

vercel Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Sep 27, 2026 1:17am UTC

Request Review

@github-actions github-actions Bot added the requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep label Sep 25, 2026
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

⚠️ Cross-repo companion check

One or more companion PRs aren't merged into staging yet. Merging this without them will leave copilot and sim out of sync — merge them in lockstep.

  • ❌ simstudioai/mothership#531 — OPEN, not merged (targets staging) — feat(files): teach Sim Dashboards through a sim-dashboards reference

TheodoreSpeaks and others added 2 commits September 26, 2026 12:33
Dashboards are now ordinary workspace files, handled like Sim pages, instead
of a separate resource. Creating or uploading `<Name>.dashboard` drops the
suffix and stamps `text/x-sim-dashboard`; the type is sticky across content
writes. The file viewer renders it live behind the `dashboards` flag, and the
public share viewer shows a workspace-only notice.

- Remove the dashboard resource: sidebar page, API routes, hooks, contracts,
  application layer, Mothership dashboards/dashboard_folders tools, resource
  tags, and the per-turn dashboardsEnabled payload.
- Revert the file discovery column (0385) and drop the dashboard folder
  resource enum (0384); dashboards never shipped, so no backfill.
- Chat panel decides previewability and tab/picker icons from the file type,
  not the name, so extensionless dashboards render and get the chart icon.
- Renderer: authored left label columns are kept intact, horizontal bar
  frames grow with row count, and hovered rows get a label-and-bar highlight.
- Simplify the create-dashboard skill around one validated example.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6aTRnkiu7PxYZPNPXYEMV
Resolves conflicts in the feature-flag and skill use-case tests (moved onto
staging's central mocks) and the tab strip widths. Adapts the branch's
dashboard and analytics tests to staging's test-pattern audit, scopes the
table operation delegation invariant to operations that admit delegated
principals, and hides the rollout-gated create-dashboard built-in from the
organization composer when the dashboards flag is off.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6aTRnkiu7PxYZPNPXYEMV
@TheodoreSpeaks TheodoreSpeaks changed the title feat(dashboards): add table-backed dashboard resources feat(dashboards): table-backed dashboard files behind a rollout flag Sep 26, 2026
@TheodoreSpeaks
TheodoreSpeaks marked this pull request as ready for review September 26, 2026 19:45
@TheodoreSpeaks
TheodoreSpeaks requested a review from a team as a code owner September 26, 2026 19:45
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

@greptile-apps

greptile-apps Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[High risk] Adds table analytics API and dashboard file support behind a feature flag.

The PR is not yet safe to merge because a dashboard detail panel can fail when it inherits an aggregate-only sort.

Findings

  1. P1 Aggregate sort breaks detail panels ▶

Summary

The PR adds dashboard workspace files, a gated analytics endpoint, live Files and Chat previews, and dashboard write diagnostics. Since the previous review, it also adds a create-response revision, separates diagnostic entries, and allows panels to switch between inherited detail and aggregate query modes.

  • The new mode-switch behavior can retain an incompatible aggregate sort on a detail panel.
  • The create-response revision follows the existing content-revision mechanism.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  File[Dashboard file] --> Parse[Parse and resolve panel source]
  Parse --> Panel[Visible dashboard panel]
  Panel --> API[Session-authenticated analytics API]
  API --> Table[Authorized table query]
  Table --> Panel
Loading

Reviews (13) · Last reviewed commit: "fix(dashboards): split diagnostics, swit..."

Comment thread apps/sim/lib/workflows/skills/builtin-skills.ts Outdated
Comment thread apps/sim/components/charts/echarts-view.test.tsx Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 86 files

Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic

Comment thread apps/sim/lib/charts/option.ts
Comment thread apps/sim/lib/charts/summary.ts Outdated
Comment thread apps/sim/lib/charts/time-series.ts
Comment thread apps/sim/lib/table/analytics/query.ts
Comment thread apps/sim/components/dashboards/dashboard-controls.tsx
Comment thread apps/sim/lib/charts/bar-row-highlight.ts Outdated
- Size horizontal bar `.chart` previews by category count like dashboard panels.
- Keep the ECharts label column for percentage bar widths, resolve percentage
  grid insets for the row highlight, and keep the time axis on the queried range.
- Show small readout values with significant digits instead of rounding to 0.
- Pass the dashboard's timezone-adjusted today to the range calendar.
- Decide the Chat panel's Markdown mode from the file record.
- Replace mock-call assertions in the EChartsView tests with DOM behavior.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6aTRnkiu7PxYZPNPXYEMV
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 86 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/lib/charts/option.ts Outdated
Comment thread apps/sim/lib/charts/bar-row-highlight.ts Outdated
… resizes

Unstacked bar series sit side by side in a category row, so grouped charts keep
the ECharts label column and their rows fit every bar slot. The row highlight
redraws the active row after each render, so a resize moves it with the plot.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6aTRnkiu7PxYZPNPXYEMV
@cubic-dev-ai

cubic-dev-ai Bot commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 83 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic

TheodoreSpeaks and others added 2 commits September 26, 2026 17:26
…Pages

Drop the create-dashboard built-in skill and its rollout gating in the skill
lists. Mothership now learns the dashboard format from a sim-dashboards
reference in its own research-and-deliverables skill, the pattern Sim Pages
use; Sim workspace built-ins stay Agent-block documents. The dashboards flag
still gates the viewer and table analytics.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6aTRnkiu7PxYZPNPXYEMV
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 76 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic

Comment thread apps/sim/components/dashboards/dashboard-feature-gate.tsx Outdated
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6aTRnkiu7PxYZPNPXYEMV
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 76 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic

The v2 file create, replace and edit responses now carry `diagnostics` for a
dashboard file: its parse errors, or an empty list. Writes are never blocked,
like the page lint; table columns and queries are still checked when panels
render. Parse errors are reported as `path: message` lines, and a block with
no recognized kind names the allowed kinds and unknown keys instead of Zod's
union dump.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6aTRnkiu7PxYZPNPXYEMV
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

Comment thread apps/sim/lib/dashboards/diagnostics.ts Outdated
Comment thread apps/sim/app/api/table/[tableId]/analytics/route.test.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 86 files

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/lib/dashboards/spec.ts Outdated
Comment thread apps/docs/openapi-v2-files-audit.json
…create revision

- One diagnostics entry per parse error instead of a newline-joined string.
- A panel choosing aggregate or columns drops the other mode's inherited
  dashboard fields, so shared defaults serve table and aggregate panels.
- The v2 create response returns the revision it produced, like the replace and
  edit responses.
- The analytics route test uses the shared createMockRequest helper.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6aTRnkiu7PxYZPNPXYEMV
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 86 files

Confidence score: 3/5

  • In apps/sim/lib/dashboards/diagnostics.ts, the 128 KiB check runs only after converting a write of up to 50 MiB to a string and encoding it again, which can cause avoidable memory and CPU spikes before rejection; check the byte length first and return the size diagnostic.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/sim/lib/dashboards/diagnostics.ts">

<violation number="1" location="apps/sim/lib/dashboards/diagnostics.ts:14">
P2: A dashboard write can be 50 MiB, but this converts it to a string before `parseDashboardSpec` checks its 128 KiB cap, and that check encodes the whole string again. Check the byte length first and return the size diagnostic to avoid synchronous full-payload copies on oversized writes.</violation>
</file>

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Fix all with cubic | Re-trigger cubic

content: Buffer | string
): string[] | undefined {
if (contentType !== DASHBOARD_CONTENT_TYPE) return undefined
const { error } = parseDashboardSpec(content.toString())

@cubic-dev-ai cubic-dev-ai Bot Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A dashboard write can be 50 MiB, but this converts it to a string before parseDashboardSpec checks its 128 KiB cap, and that check encodes the whole string again. Check the byte length first and return the size diagnostic to avoid synchronous full-payload copies on oversized writes.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/sim/lib/dashboards/diagnostics.ts, line 14:

<comment>A dashboard write can be 50 MiB, but this converts it to a string before `parseDashboardSpec` checks its 128 KiB cap, and that check encodes the whole string again. Check the byte length first and return the size diagnostic to avoid synchronous full-payload copies on oversized writes.</comment>

<file context>
@@ -0,0 +1,16 @@
+  content: Buffer | string
+): string[] | undefined {
+  if (contentType !== DASHBOARD_CONTENT_TYPE) return undefined
+  const { error } = parseDashboardSpec(content.toString())
+  return error ? error.split('\n') : []
+}
</file context>
Suggested change
const { error } = parseDashboardSpec(content.toString())
if (Buffer.byteLength(content) > 128 * 1024) return ['Dashboard source exceeds 128 KB']
const { error } = parseDashboardSpec(content.toString())
Fix with cubic

Comment on lines +100 to +105
const inherited = source?.columns
? omit(defaults ?? {}, ['aggregate', 'groupBy', 'bucket'])
: source?.aggregate
? omit(defaults ?? {}, ['columns'])
: defaults
const merged = { ...inherited, ...source }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Aggregate sort breaks detail panels

When a dashboard defaults to an aggregate query sorted by an aggregate alias, a detail panel that selects columns still inherits that sort. The new mode-switch logic removes the aggregate but keeps its sort, so the detail query tries to use the alias as a table column and fails instead of showing rows. Drop or revalidate the inherited sort when switching query modes.

This branch was successfully deployed

1 active deployment
Preview — 5d0e7e48 Deployed Sep 27, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant