Skip to content

Mount Columns and Sort on the search page - #3432

Open
elizabetdev wants to merge 21 commits into
mainfrom
elizabet/search-columns-sort
Open

elizabetdev wants to merge 21 commits into
mainfrom
elizabet/search-columns-sort

Conversation

@elizabetdev

@elizabetdev elizabetdev commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Summary

Moves SELECT and ORDER BY off the query form. On a results search, Columns and Sort sit on the results table and edit those clauses. Event Deltas and Event Patterns do not show them. On a form narrower than 900px, the where clause and the time controls each take a full row, with the where clause first. Columns and Sort wrap with the table actions instead of overlapping them.

The editors landed in #3431.

Screenshots were not captured.

Screenshots or video

Screenshot 2026-11-19 at 16 39 30

How to reproduce

Starting point: yarn app:dev:local connected to the demo server, or the Vercel preview. Open a log search and run it.

On main:

  1. The query form has SELECT and ORDER BY fields.
  2. The results table has no Columns or Sort button.

On this branch:

  1. The query form has no SELECT or ORDER BY field. Group by is still there.
  2. Above the results table, Columns and Sort are next to the table controls.
  3. Click Columns. Fields lists the columns; SQL edits the SELECT clause. Reset, then Apply. A removed column is dropped from the sort when it is one of the sort fields.
  4. Click Sort. Fields lists sort fields with direction; SQL edits the ORDER BY clause. Apply runs the search.
  5. Narrow the window below 900px. The where clause takes the first row and the time controls wrap onto the next. Columns and Sort wrap with the table controls; a multi-field summary shows only the count.

How to test on Vercel preview

Preview routes: /search

Steps:

  1. Open /search, choose a log source, and run the search.
  2. Confirm the query form has no SELECT or ORDER BY field.
  3. Confirm Columns and Sort appear with the results table controls.
  4. Click Columns. Confirm Fields and SQL are shown, then click Cancel.
  5. Click Sort. Confirm Fields and SQL are shown, then click Cancel.
  6. Confirm Group by is still in the query form.

References

@github-actions github-actions Bot added the review/tier-4 Critical — deep review + domain expert sign-off label Oct 21, 2026
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Tier 3 — Standard

Introduces new logic, modifies core functionality, or touches areas with non-trivial risk.

Why this tier:

  • Diff size: 800 production lines changed (Tier 2 max: < 250)

Review process: Full human review — logic, architecture, edge cases.
SLA: First-pass feedback within 1 business day.

Stats
  • Production files changed: 9
  • Production lines changed: 800 (+ 700 in test files, excluded from tier calculation)
  • Branch: elizabet/search-columns-sort
  • Author: elizabetdev

To override this classification, remove the review/tier-3 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 107 in total (1.13%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

const knownColumns = new Set(columns.map(column => column.name));
// Field discovery is off for some teams, and `useAllFields` then returns
// [] rather than "still loading". Scalar columns are still known.
const keys =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 major — Columns and Sort pickers never offer Timestamp or other DateTime fields, nor whole Map/JSON columns

getSqlSourceKeys/getSqlSourceColumnKeys (packages/app/src/components/AdhocDashboardFilter/useAdhocFilterOptions.ts:63-88) belong to the ad hoc filter. They drop every key where isAdhocComparableDataType (common-utils/src/filters.ts:963, !/\bDateTime/) is false, and they skip non-scalar top-level columns. So Timestamp/TimestampTime, the most common sort fields, never show up under 'Add sort' or 'Search columns', and neither does a whole column such as LogAttributes. Build the clause options from all columns plus map/JSON subpaths without the DateTime filter. Add a hook test with a DateTime64 column. The popover tests pass a hardcoded fieldOptions=['Body','Timestamp'], so they cannot catch this.

Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond

isFieldMetadataDisabled || fields == null
? getSqlSourceColumnKeys(columns)
: getSqlSourceKeys(fields, columns);
return Array.from(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Keys are quoted twice; the second toQuotedClickHouseKeyExpression pass and its knownColumns set can't change anything

getSqlSourceKeys and getSqlSourceColumnKeys already run every key through toQuotedKeys → toQuotedClickHouseKeyExpression (useAdhocFilterOptions.ts:54-57), and that function leaves already-quoted input alone. Drop the extra .map(...) and the knownColumns set, and just dedupe and sort the keys.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 P0/P1 in code this PR adds or changes: fix before asking for review.
  • 🟡 P2 in code this PR changes: your call. Fix if small, otherwise reply.
  • 🔵 P3 is collapsed at the bottom: nobody needs to act on it before merge.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    priority. Reply with one sentence; if it is P0, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

1 finding(s): 🔴 0 P0/P1 · 🟡 1 P2 · 🔵 0 P3

Already reported on an earlier push — no new comments. 1 unchanged from an earlier push (already inline above).


Priority comes from a separate grading pass over the finished review. P3 is folded here instead of posted inline. No finding blocks merge automatically; a maintainer decides.

@elizabetdev
elizabetdev changed the base branch from main to elizabet/column-sort-editors October 21, 2026 15:59
@elizabetdev
elizabetdev added this pull request to stack #3433 October 21, 2026 16:00
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 474 passed • 1 skipped • 1042s

Status Count
✅ Passed 474
❌ Failed 0
⚠️ Flaky 3
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found.

This PR moves SELECT/ORDER BY editing off the query form into Columns and Sort popovers on the results table (results mode only), prunes sorts whose alias a removed column dropped, and adds a discard-confirm flow when closing a popover with unsaved edits. I traced the core clause-resolution logic (orderByAfterRemovingSelectItems, selectItemsWithRemovedAlias, orderByWithoutAliases) across its branches and it is sound: bare, double-quoted, and backtick alias forms are all matched; non-aliased columns correctly keep their sort; the empty-orderBy / default / source-default fallback chain terminates correctly. The previously-raised P1 ("removed columns leave broken sorts") is genuinely fixed, and the other prior threads (phone toolbar overlap, untouched-saved-search dirty check, Escape routing, button memoization, blank-Apply-then-Update) are addressed in the current code.

🟡 P2 — recommended

  • packages/app/src/DBSearchPage.tsx:2530 — The results-popover wiring that prunes dependent sorts when a column is removed (applySelect passing select || defaultSelect and defaultOrderBy into the pruning helpers) has no page-level test because the DBSearchPage suites stub ColumnsPopover; only the underlying utils helpers are unit-tested, so a regression in how the page feeds them would pass CI.
    • Fix: Add a DBSearchPage test that drives a real ColumnsPopover apply for a removed aliased column and asserts the resulting orderBy, or an integration test covering the applySelect → helper argument plumbing.
🔵 P3 nitpicks (1)
  • packages/app/src/DBSearchPage.tsx:559 — effectiveSelect resolves through the saved-search select and then the source defaultTableSelectExpression, but effectiveOrderBy stops at the saved-search orderBy and never resolves a source default, leaving the two "empty means default" save paths asymmetric; both are individually correct (an empty saved orderBy is re-defaulted at query time), so this is a readability concern rather than a behavior bug.
    • Fix: Add a brief comment explaining why orderBy intentionally saves empty while select is materialized, or resolve both symmetrically.

Reviewers (1): direct multi-dimensional analysis (correctness, adversarial/edge-case, frontend-race, TypeScript, testing, maintainability, prior-comment resolution, agent-native).

Testing gaps:

  • No page-level coverage of the ColumnsPopover → applySelect → sort-pruning path; the suites stub ColumnsPopover, so only the standalone utils helpers are exercised for the alias-removal logic.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

Comment thread packages/app/src/DBSearchPage.tsx Resolved
Comment thread packages/app/src/DBSearchPage.tsx Resolved
</>
);
};
const applySelect = useCallback(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — applySelect repeats toggleColumn's logic for pruning ORDER BY

applySelect repeats toggleColumn's steps (lines 1838-1872): set select, read orderBy with getValues, call orderByAfterRemovingSelectItem with defaultSearchConfig.orderBy, then setValue('orderBy'). Move this into one helper, for example pruneOrderByForRemovedItems(items), and call it from both places, so a change to the pruning rule cannot reach only one of them.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

@changeset-bot

changeset-bot Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 1a4e6ad

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@hyperdx/app Minor
@hyperdx/api Minor
@hyperdx/otel-collector Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Nov 19, 2026 1:33pm UTC
hyperdx-storybook Ready Ready Preview Nov 19, 2026 1:33pm UTC

Request Review

@greptile-apps

greptile-apps Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The changes affect the search page and its results-table workflow.

Fix sort cleanup for removed aliases before merging.

Fix All in Claude CodeFindings

  1. P1 Removed columns leave broken sorts ▶

Summary

Moves SELECT and ORDER BY into Columns and Sort popovers above search results.

  • Search results put Columns and Sort beside the table.
  • Table actions stay above the results while the table scrolls.
  • Narrow search forms put WHERE above the time controls.

The earlier toolbar-overlap thread is addressed by wrapping the controls. elizabetdev kept the where field first to preserve reading order; Greptile agreed that this matches the updated description. Both supplied threads are unnumbered.

Reviews (6) · Last reviewed commit: "fix(app): keep the clause footer above t..." · Reviewed by Greptile

Base automatically changed from elizabet/column-sort-editors to main October 23, 2026 11:59
SELECT and ORDER BY move off the query form onto the results table, and the
where row wraps under the time controls on a narrow form.

Co-authored-by: Cursor <cursoragent@cursor.com>
@jordan-simonovski
jordan-simonovski force-pushed the elizabet/search-columns-sort branch from 2096af9 to b5442e5 Compare October 23, 2026 11:59
@github-actions github-actions Bot added review/tier-3 Standard — full human review required and removed review/tier-4 Critical — deep review + domain expert sign-off labels Oct 23, 2026
Comment thread packages/app/src/DBSearchPage.tsx Resolved
);
};
const applySelect = useCallback(
(select: string, removedItems: string[]) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — applySelect repeats the ORDER BY pruning that toggleColumn already does

Lines 2307-2316 copy the getValues('orderBy') → orderByAfterRemovingSelectItem(..., defaultSearchConfig.orderBy) → setValue('orderBy') sequence from toggleColumn at lines 1851-1860. Move it into one local helper, for example pruneOrderByForRemoved(items: string[]), and call it from both, so the two column-removal paths can't drift apart.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Columns, Sort, and the table actions move onto another line instead of
overlapping when the results panel is narrow.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread packages/app/src/utils.ts Resolved
…is empty

An empty form ORDER BY means "use the saved sort". The helper treated the
empty string as already clean, so removing an aliased column after
resetting Sort left the saved alias sort in the query and it failed on an
unknown identifier. Run the empty case through the same alias removal.

Co-authored-by: Cursor <cursoragent@cursor.com>
The popover handles Escape before the input does, so closing the column
or sort suggestions also closed the popover and dropped the text that had
not been added yet. Skip that close while the suggestion list is open.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread packages/app/src/DBSearchPage.tsx Resolved
Changing an aliased column's expression or the case of AS (for example upper(Body) AS message to lower(Body) AS message) counted the old item as removed. The sort on the alias was then dropped even though the alias was still selected. Ignore a removed item whose alias is still in the new SELECT.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread packages/app/src/DBSearchPage.tsx Resolved
wrap="wrap"
// The SQL completion list overflows the editor onto these buttons.
// Fields mode uses an in-popover Autocomplete, which must stay above them.
style={

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 P2 — SQL-mode footer at max z-index covers the completion entries it overlaps

The comment says the CodeMirror completion list spills onto the footer. Raising the Reset/Cancel/Apply row to --mantine-z-index-max puts those buttons on top of the list (.cm-tooltip is z-index 100 in the same dropdown stacking context). On a short Sort editor, completion entries under the buttons can't be seen or clicked. Keep the footer below the tooltip. If Apply has to stay clickable, use tooltips({ position: 'fixed', parent }) or reserve space under the editor instead of stacking the buttons over the list.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing this. Apply has to stay reachable when the completion list is open, so the footer stays above the list. Raising the .cm-tooltip above it would also cover the Reset/Cancel/Apply row. Left as is.

if (availableWidth > 0) {
elements.floating.style.maxWidth = `${availableWidth}px`;
}
if (availableHeight > 0) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 P2 — maxHeight from the size middleware has no effect without overflow

Mantine's Popover.Dropdown sets no overflow. Content taller than availableHeight therefore spills past the capped box, and on a short viewport Apply still ends up off screen. Add overflowY: 'auto' to the dropdown, or to an inner scroll wrapper that excludes the footer, so the cap actually keeps Apply reachable.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing this. Adding overflow: auto to the dropdown clips the Fields autocomplete and the SQL completion list, which was the earlier overflow thread. The height cap keeps the popover inside the viewport; the list itself can still extend past the popover's box. Left as is.

Apply stores an empty SELECT when the draft matches the default. The alias check then read no aliases from the new SELECT and dropped a sort on an alias the default still defines. Check against the effective SELECT instead.

Co-authored-by: Cursor <cursoragent@cursor.com>
Keep the phone search-row layout, and hide live tail while the duration heatmap is open.

Co-authored-by: Cursor <cursoragent@cursor.com>
// The SQL completion list overflows the editor onto these buttons.
// Fields mode uses an in-popover Autocomplete, which must stay above them.
style={
mode === 'sql'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 P2 — In SQL mode the footer is raised above the autocomplete list, so suggestions behind it can't be seen or clicked

The footer Group spans the full popover width and gets position: relative; z-index: var(--mantine-z-index-max). The CodeMirror tooltip renders inside the editor (ColumnsPopover and SortPopover pass no parentRef), and it ends up under this footer, so any completion rows that overlap the Reset/Cancel/Apply row are hidden and can't be clicked. The e2e applyClause helper has to work around this. Fix: let the completion list render above the footer, for example by passing parentRef={document.body} to SQLInlineEditor or by giving the editor a higher stacking context, rather than raising the footer to max z-index.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing this. Raising the footer is what keeps Apply clickable while the SQL completion list is open. Passing parentRef would move the list out of the popover, and I have not verified that it stays clickable under the footer. Left as is; the e2e helper works around it.

The merge added them so the pre-commit warning check would pass. The app one puts eslint-disable above its baseline, which fails the lint job.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread packages/app/src/DBSearchPage.tsx Resolved
elements.floating.style.maxWidth = `${availableWidth}px`;
}
if (availableHeight > 0) {
elements.floating.style.maxHeight = `${availableHeight}px`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 P2 — The size middleware caps the dropdown's maxHeight, but nothing in the dropdown scrolls

Mantine's Popover.Dropdown does not set overflow by default. On a short viewport the Stack (including Apply/Cancel) therefore spills outside the bordered dropdown instead of scrolling. Put overflowY: 'auto' on an inner wrapper around the mode content, not on the footer, so the SQL completion list is not clipped. Or drop the maxHeight cap.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing this. It is the same overflow question as the earlier thread. overflowY: auto on the dropdown or a wrapper would clip the Fields and SQL completion lists, so the height cap keeps the popover on screen and leaves the lists unclipped. Left as is.

Apply stores an empty SELECT or sort when the draft matches the default. On a saved search the default is the saved clause, so Update wrote the source default SELECT and cleared the sort. Save the clause the results table runs: the saved one when the form is blank, else the source default.

Co-authored-by: Cursor <cursoragent@cursor.com>

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 1a4e6ad1 Deployed Nov 19, 2026 by vercel[bot]
Preview – hyperdx-storybook — 1a4e6ad1 Deployed Nov 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-3 Standard — full human review required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants