Skip to content

feat: Add filter variable configuration - #2836

Merged
kodiakhq[bot] merged 3 commits into
mainfrom
drew/filter-variable-modal
Aug 11, 2026
Merged

feat: Add filter variable configuration#2836
kodiakhq[bot] merged 3 commits into
mainfrom
drew/filter-variable-modal

Conversation

@pulpdrew

@pulpdrew pulpdrew commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR updates the dashboard filter configuration modal to support enabling "variable" mode for a filter, which will allow users to refer to the filter value as a variable in queries.

This feature is behind the NEXT_PUBLIC_ENABLE_DASHBOARD_VARIABLES feature toggle, since the actual variable replacement is not yet implemented.

Variable mode is off by default for existing filters and on by default for new filters, when the toggle is on. The user may specify a custom name for the variable. A default variable name is generated from the filter's name. Variable names must be unique in the dashboard, while filter names may not be.

Variable mode and broadcast mode (the current behavior) may both be enabled or disabled independently. When the feature is disabled, broadcast mode cannot be disabled and variable mode cannot be enabled - the checkboxes do not appear.

**Note: Variable substitution functionality is not implemented by this PR. ** Only configuration of variable type filters is implemented.

Screenshots or video

With the toggle enabled

Screen.Recording.2026-08-07.at.2.26.31.PM.mov

Updated tooltips and broadcast-disabled functionality

Screen.Recording.2026-08-10.at.12.33.45.PM.mov

Variable mode is not available on Preset Dashboards, and broadcast mode cannot be disabled

This works on temporary/local dashboards as well

Screen.Recording.2026-08-07.at.2.32.12.PM.mov

Import/Export preserves variable mode

Screen.Recording.2026-08-07.at.2.33.56.PM.mov

With the toggle disabled

Screen.Recording.2026-08-07.at.2.40.45.PM.mov

How to test locally

  1. Run locally so that NEXT_PUBLIC_ENABLE_DASHBOARD_VARIABLES is enabled
  2. Create a dashboard and add some filters. Observe the validation for variable name.
  3. Note that the variables cannot yet be referenced, this change covers only the configuration of variable-mode filters.

References

  • Linear Issue: Closes HDX-5043 HDX-5049
  • Related PRs:

@changeset-bot

changeset-bot Bot commented Aug 7, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 94ec6a5

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/app Patch
@hyperdx/api Patch
@hyperdx/common-utils Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Aug 7, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hyperdx-oss Ready Ready Preview Aug 11, 2026 5:47pm
hyperdx-storybook Ready Ready Preview Aug 11, 2026 5:47pm

Request Review

@greptile-apps

greptile-apps Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds dashboard filter broadcast/variable configuration, shared validation, persistence support, feature gating, and UI/E2E coverage. The HTTP APIs now reject unusable variable names, but the dashboard provisioning path still bypasses that validation.

  • Adds broadcast and variable settings to shared dashboard filter types.
  • Adds modal controls, generated variable names, uniqueness checks, and mode validation.
  • Preserves the new settings through internal and external dashboard APIs and import/export transformations.
  • Updates query filtering so broadcast-disabled filters do not apply to tiles.

Confidence Score: 4/5

The PR is not yet safe to merge because provisioned dashboards can still persist enabled variables without usable tokens.

The internal, external, and MCP dashboard write paths now enforce the variable-name contract, but ProvisionDashboardsTask still parses the unrefined shared schema and directly persists filters, leaving the previously reported provisioning failure reachable.

Files Needing Attention: packages/common-utils/src/types.ts and packages/api/src/tasks/provisionDashboards/index.ts

Important Files Changed

Filename Overview
packages/common-utils/src/types.ts Adds optional filter mode and variable-name fields, but the bare shared dashboard schema still permits tokenless enabled variables on provisioning paths.
packages/common-utils/src/dashboardValidation.ts Adds centralized checks for resolvable unique variable names and at least one enabled filter mode.
packages/api/src/routers/api/dashboards.ts Applies the new shared filter refinements to internal dashboard creation and updates.
packages/api/src/routers/external-api/v2/utils/dashboards.ts Applies equivalent filter validation to external API and MCP dashboard body schemas.
packages/app/src/DashboardFiltersModal.tsx Adds feature-gated broadcast and variable configuration, generated names, and client-side validation.
packages/app/src/hooks/useDashboardFilters.tsx Excludes broadcast-disabled filter definitions from tile query predicates.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Input[Dashboard filter configuration]
  Internal[Internal dashboard API]
  External[External API and MCP]
  Provisioner[Provisioned dashboard JSON]
  Refine[Variable-name and mode validation]
  BareSchema[Bare DashboardWithoutIdSchema]
  Mongo[(Dashboard in MongoDB)]

  Input --> Internal --> Refine --> Mongo
  Input --> External --> Refine
  Provisioner --> BareSchema --> Mongo
Loading

Fix All in Claude Code Fix All in Conductor Fix All in Cursor Fix All in Codex

Reviews (4): Last reviewed commit: "Merge branch 'main' into drew/filter-var..." | Re-trigger Greptile

Comment thread packages/app/src/DashboardFiltersModal.tsx
Comment thread packages/common-utils/src/types.ts
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

E2E Test Results

All tests passed • 278 passed • 1 skipped • 1070s

Status Count
✅ Passed 278
❌ Failed 0
⚠️ Flaky 2
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

Comment thread packages/common-utils/src/types.ts
@pulpdrew
pulpdrew marked this pull request as ready for review August 7, 2026 19:27
@github-actions github-actions Bot added the review/tier-4 Critical — deep review + domain expert sign-off label Aug 7, 2026
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

🔴 Tier 4 — Critical

Touches authentication, tenancy data models, the public API or shipped database config — or substantially changes background tasks, the OTel pipeline, image build, or release CI.

Why this tier:

  • Critical-path files (1) — tenancy, public API, or shipped database config:
    • packages/api/src/routers/external-api/v2/utils/dashboards.ts
  • Cross-layer change: touches frontend (packages/app) + backend (packages/api) + shared utils (packages/common-utils)

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats
  • Production files changed: 14
  • Production lines changed: 696 (+ 2019 in test files, excluded from tier calculation)
  • Critical-path lines changed: 7
  • Branch: drew/filter-variable-modal
  • Author: pulpdrew

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found. No P0/P1 defects: the security pass found nothing exploitable (the variable-name regex is fully anchored and linear, all display goes through JSX auto-escaping, and query substitution is explicitly not implemented in this diff), and no data-loss, auth-bypass, or happy-path crash was introduced. The backwards-compat defaulting (isBroadcastEnabled !== false, isVariableEnabled === true) was traced and correctly keeps legacy filters broadcasting. The items below are recommended and discretionary.

🟡 P2 -- recommended

  • packages/api/src/mcp/tools/dashboards/schemas.ts:997 -- mcpDashboardFilterSchema is a plain z.object that omits isBroadcastEnabled, isVariableEnabled, and variableName, so an agent calling clickstack_save_dashboard/clickstack_patch_dashboard has those keys silently stripped while UI and REST-API users can set them.
    • Fix: Add the three fields to mcpDashboardFilterSchema with descriptions so the MCP write path reaches parity with the UI and v2 REST surface.
    • agent-native
  • packages/common-utils/src/dashboardValidation.ts:180 -- validateDashboardFilterVariableNames checks existence and uniqueness of the derived name but never re-applies the length/format checks that the schema .max(64).regex() and the form's validateVariableName apply to explicit names, so the API accepts a variable-enabled filter whose long display name derives an over-length token the UI would reject.
    • Fix: Apply the DASHBOARD_VARIABLE_NAME_MAX_LENGTH/DASHBOARD_VARIABLE_NAME_REGEX checks to the resolved (derived-or-explicit) name inside validateDashboardFilterVariableNames.
    • adversarial, correctness, api-contract
🔵 P3 nitpicks (7)
  • packages/app/src/DashboardFiltersModal.tsx:529 -- Modal title "Filters and Variables" uses title case, violating the sentence-case UI-text rule in agent_docs/code_style.md.
    • Fix: Rename to "Filters and variables".
    • project-standards
  • packages/app/src/DBDashboardPage.tsx:2970 -- Tooltip label "Edit Filters and Variables" uses title case, violating the same sentence-case rule.
    • Fix: Rename to "Edit filters and variables".
    • project-standards
  • packages/app/src/DashboardFiltersModal.tsx:171 -- The auto-fill setValue('variableName', derivedVariableName) omits shouldValidate, so a variable-name error surfaced by a failed save lingers after the user fixes it by editing the display name.
    • Fix: Pass { shouldValidate: true } (or gate on an existing variableName error) so the field re-validates when its value changes.
    • julik-frontend-races
  • packages/api/src/routers/api/dashboards.ts:207 -- The /preset/:presetDashboard/filter PUT/POST endpoints extend DashboardFilterSchema but never call addFilterIssues, so a direct API caller can persist a preset filter with a duplicate variableName or with no active mode.
    • Fix: Decide whether preset filters should share the invariant and, if so, wire the validators into those handlers.
    • api-contract, kieran-typescript
  • packages/app/src/DashboardFilters.tsx:40 -- getFilterEffect reports "Available as variable ($name)" and suppresses the no-effect warning whenever isVariableEnabled is true, independent of the feature flag, so a variable-only filter is advertised as live even where the feature is off and substitution is unimplemented.
    • Fix: Gate the variable branch on the feature flag (thread showVariableOptions/useIsVariablesEnabled through) so an inert filter is not shown as active.
    • correctness, adversarial
  • packages/api/src/routers/external-api/v2/dashboards.ts:1630 -- The hand-written OpenAPI JSDoc for the external v2 filter schema was not updated for the three new fields or the new uniqueness/at-least-one-mode rules.
    • Fix: Document isBroadcastEnabled, isVariableEnabled, variableName, and the cross-field validation in the v2 spec block.
    • api-contract, correctness
  • packages/app/src/hooks/useIsVariablesEnabled.ts:10 -- The hook returns a hardcoded isLoading: false around a static env-derived constant, making !isVariablesFlagLoading && isVariablesEnabled in DBDashboardPage.tsx a permanently-false branch for a team-toggle use case that does not exist yet.
    • Fix: Return the constant directly and introduce the loading shape when an async source actually lands.
    • maintainability

Reviewers (11): correctness, security, adversarial, testing, maintainability, project-standards, api-contract, kieran-typescript, julik-frontend-races, agent-native, learnings-researcher.

Testing gaps:

  • No test exercises the modal with showVariableOptions=false (feature-flag-off / preset path); playwright.config.ts forces NEXT_PUBLIC_ENABLE_DASHBOARD_VARIABLES=true, so the flag-off default is never covered end-to-end.
  • No direct unit test for the DashboardFiltersModal form logic (auto-fill gating via hasEditedVariableName, the cross-field trigger('isVariableEnabled') re-validation, and the submit-time clearing of variableName).
  • No test asserts the API rejects a variable-enabled filter whose derived name exceeds 64 chars, nor that the client validateVariableName and server validateDashboardFilterVariableNames agree on the same inputs (duplicated uniqueness logic can drift).
  • No MCP tool-level test round-trips the new filter fields; no test covers the preset filter endpoints with these fields.
  • No test reopens the edit form on a different filter without an intervening unmount — the latent reset-vs-auto-fill ordering hazard is unreachable through today's UI but untested.

* Whether a filter definition broadcasts its selected value onto a tile
* whose source is `sourceId`.
*/
const definitionAppliesToSource = (

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes in this file prevent filters with broadcast mode disabled from being applied to any dashboard tiles.

onRemoveFilter={handleRemoveFilter}
source={source}
isLoading={isFetchingFilters || isFiltersMutationPending}
showVariableOptions={false}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The user doesn't define tiles on this dashboard, so there is no reason for variable-type filters. The UI will not show the option to enable variable mode or disable broadcast mode.

@pulpdrew
pulpdrew requested review from a team, knudtty and wrn14897 and removed request for a team and knudtty August 10, 2026 16:52
* field broadcasts, and that must not change. Read it through
* `isFilterBroadcastEnabled` rather than defaulting at each call site.
*/
isBroadcastEnabled: z.boolean().optional(),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to make these changes on the model side as well?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not, at least not outside of a separate refactor. Dashboard filters are currently stored in two tables:

  1. Dashboards - here filters is registered as a schema-less Array, like most other fields on the dashboard model. A nice refactor could be to improve the schemas across each of the fields. I can file an issue for that if you think it would be a worthwhile improvement.
  2. PresetDashboardFilters - these are filters specfically for preset dashboards (eg. Services Dashboard), and while these have a schema, these filters will not support variable mode, so we are intentionally not adding these new properties.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. I just want to make sure that, since the schema isn't persistent on the model side, those fields might be overridden. But if you're sure those corner cases are covered, I think we should be good.

@wrn14897 wrn14897 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

* Undefined / missing means DISABLED. Ignored while the dashboard-variables
* feature is off.
*/
isVariableEnabled: z.boolean().optional(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Provisioned variables can lack tokens

When provisioned dashboard JSON enables variable mode without variableName and its display name contains no token-safe characters, the provisioning task parses the unrefined DashboardWithoutIdSchema and persists the filter, causing an enabled variable with no usable $variableName reference.

Knowledge Base Used:

Fix in Claude Code Fix in Conductor Fix in Cursor Fix in Codex

@kodiakhq
kodiakhq Bot merged commit b9430a6 into main Aug 11, 2026
29 checks passed
@kodiakhq
kodiakhq Bot deleted the drew/filter-variable-modal branch August 11, 2026 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge review/tier-4 Critical — deep review + domain expert sign-off

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants