Skip to content

fix(prometheus): enforce namespace lockdown on PromQL vector selectors - #533

Merged
billzhuang merged 2 commits into
mainfrom
claude/eloquent-bell-o032i5
Jul 11, 2026
Merged

fix(prometheus): enforce namespace lockdown on PromQL vector selectors#533
billzhuang merged 2 commits into
mainfrom
claude/eloquent-bell-o032i5

Conversation

@billzhuang

@billzhuang billzhuang commented Jul 11, 2026

Copy link
Copy Markdown
Owner

Summary

The Linear Heimdall project's backlog and Todo/In Progress states are all empty, and there is no open-PR refactor queue, so per the fallback workflow I searched the codebase for a genuine, real-payoff gap rather than inventing a cosmetic refactor.

heimdall.config.yaml's namespace.locked setting is meant to be a code-enforced security boundary: kubectl, list_namespaces, helm_release, loki_query, and kubecost_query all enforce it. prometheus_query did not — PrometheusConfig had no lockedNamespace field and runPrometheusQuery never inspected the PromQL query for a namespace matcher before executing it. An operator who locks Heimdall to namespace prod for compliance reasons was silently unprotected on prometheus_query: the model could query container_memory_usage_bytes{namespace="other-team-secrets"} even with lockdown configured.

Changes

  • Extracted Loki's hardened selector/matcher parser (src/lib/loki.ts) into a new shared module src/lib/selector-lockdown.ts. PromQL and LogQL selectors use the same {label="value", label2=~"value2"} grammar, so the parser — already hardened through several prior follow-up fixes for comment/backtick/multi-selector spoofing — applies unchanged to PromQL.
  • loki.ts now re-exports validateNamespaceLockdown from the shared module (public contract unchanged; existing Loki tests pass unmodified).
  • src/lib/prometheus.ts: added lockedNamespace to PrometheusConfig and enforce it in runPrometheusQuery, mirroring Loki's fail-closed behavior (a query with no selector, a wildcard selector, or a mismatched namespace is blocked with a BLOCKED: message).
  • src/tools/prometheus.ts: makePrometheusQuery now accepts lockedNamespace, threads it into the tool config, and appends a NAMESPACE LOCKDOWN ACTIVE note to the tool description (same pattern as loki_query). prometheusPlugin.factory now passes config.namespace?.locked through.
  • Added tests covering the PromQL-shaped lockdown behavior at both the lib and tool layers (mirroring the existing Loki test coverage).

jaeger_query, newrelic_query, and datadog_query have the same gap but use different query languages (Jaeger tag filters, NRQL, Datadog query syntax) that don't share PromQL/LogQL's selector grammar, so they're out of scope for this change — each would need its own query-language-specific enforcement.

Validation

  • npm run typecheck — passes
  • npm test — full suite: 108 files / 3073 tests passing
  • npm run buildflue build --target node succeeds

Not auto-verified

  • No live Prometheus instance was queried; behavior is verified via unit tests against the pure runPrometheusQuery/validateNamespaceLockdown functions and mocked fetch, consistent with this repo's existing test strategy (no real HTTP calls in tests).

Generated by Claude Code

Summary by CodeRabbit

  • New Features

    • Added namespace lockdown for Prometheus queries.
    • Queries must target the configured namespace; unauthorized or missing namespace selectors are blocked.
    • Prometheus tool descriptions now indicate when lockdown is active and identify the locked namespace.
    • Namespace lockdown supports all query selectors and is disabled when no namespace is configured.
  • Tests

    • Added coverage for accepted, rejected, malformed, and multi-selector queries.
    • Verified lockdown behavior and configuration propagation across Prometheus tooling.

prometheus_query silently ignored heimdall.config.yaml's namespace.locked
setting: kubectl, list_namespaces, helm_release, loki_query, and
kubecost_query all code-enforce the lockdown, but prometheus_query had no
lockedNamespace field at all, letting a model query any namespace's
metrics even on an instance restricted to a single namespace.

Extracts Loki's hardened selector/matcher parser (LogQL and PromQL share
the same `{label="value"}` grammar) into selector-lockdown.ts and reuses
it for prometheus_query the same way loki_query already does — same
fail-closed behavior for missing/wildcard/mismatched namespace matchers.
@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@billzhuang, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 48 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 84209177-a7fb-4251-a57b-d79e3878a682

📥 Commits

Reviewing files that changed from the base of the PR and between a716030 and f0aac41.

📒 Files selected for processing (4)
  • src/lib/__tests__/loki.test.ts
  • src/lib/__tests__/prometheus.test.ts
  • src/lib/prometheus.ts
  • src/lib/selector-lockdown.ts
📝 Walkthrough

Walkthrough

The PR adds shared namespace-selector validation, applies it to Prometheus queries, delegates Loki validation to it, and wires locked namespaces through Prometheus tool configuration and descriptions with corresponding tests.

Changes

Namespace Lockdown

Layer / File(s) Summary
Shared selector validation and library contracts
src/lib/selector-lockdown.ts, src/lib/prometheus.ts, src/lib/loki.ts
A shared parser validates all selector groups for exact locked-namespace matchers, and both query libraries expose the shared validator.
Prometheus query enforcement
src/lib/prometheus.ts, src/lib/__tests__/prometheus.test.ts
Prometheus configuration accepts lockedNamespace; invalid queries return BLOCKED output before fetch, while valid or unlocked queries proceed.
Tool configuration and plugin wiring
src/tools/prometheus.ts, src/tools/__tests__/prometheus.test.ts
The tool forwards namespace locks from configuration and adds lockdown details to its description when enabled.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant prometheus_query
  participant makePrometheusQuery
  participant runPrometheusQuery
  participant validateNamespaceLockdown
  participant fetch
  prometheus_query->>makePrometheusQuery: submit query
  makePrometheusQuery->>runPrometheusQuery: pass lockedNamespace
  runPrometheusQuery->>validateNamespaceLockdown: validate selectors
  validateNamespaceLockdown-->>runPrometheusQuery: allow or block
  runPrometheusQuery->>fetch: execute allowed query
Loading

Possibly related PRs

  • billzhuang/heimdall#313: Introduced or refactored the shared buildLockdownNote helper used by the Prometheus tool description.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: enforcing namespace lockdown for PromQL vector selectors.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/eloquent-bell-o032i5

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request consolidates the namespace-lockdown enforcement logic for Prometheus and Loki queries into a shared utility in selector-lockdown.ts. The review feedback identifies a critical security bypass vulnerability where bare metrics or range vectors without selectors can evade the lockdown checks. Additionally, the reviewer pointed out that the parser lacks support for single-quoted string literals (which are valid in PromQL) and provided actionable suggestions to update the comment stripper, brace matcher, selector extractor, and matcher parser to handle single quotes, along with adding corresponding test coverage.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/lib/selector-lockdown.ts
Comment thread src/lib/selector-lockdown.ts Outdated
Comment thread src/lib/selector-lockdown.ts
Comment thread src/lib/selector-lockdown.ts
Comment thread src/lib/selector-lockdown.ts
Comment thread src/lib/__tests__/prometheus.test.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a716030f84

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/lib/prometheus.ts
… lockdown

Two independent review bots (Gemini, Codex) confirmed a critical bypass in
the previous commit: validateNamespaceSelectorLockdown only validates the
{...} selectors it can find, but PromQL allows a bare metric name (no
braces at all) as a valid, unscoped vector selector. A query combining one
namespace-scoped selector with a bare reference via a binary/set operator
(e.g. `up{namespace="prod"} + up`, `container_memory_usage_bytes or
kube_pod_info{namespace="prod"}`) passed validation while the bare term
queried every namespace at Prometheus.

Adds a PromQL-aware scan (lib/prometheus.ts) that walks the query and
rejects it if any metric-name-shaped identifier lacks its own selector,
recognizing PromQL's aggregation operators/functions/modifiers (including
`by (...)`/`without (...)` label-list clauses, e.g. histogram_quantile's
`by (le)`) so legitimate queries aren't broken.

Also fixes the shared selector parser (selector-lockdown.ts) to treat
single-quoted strings as opaque spans everywhere double-quoted and
backtick-quoted strings already are — PromQL and LogQL both support
single-quoted string literals, and the parser previously had no way to
recognize their boundaries, the same decoy-spoofing class of bug fixed
for backticks in prior Loki hardening.
@billzhuang
billzhuang merged commit 18fb93b into main Jul 11, 2026
7 checks passed
@billzhuang
billzhuang deleted the claude/eloquent-bell-o032i5 branch July 11, 2026 05:45

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f0aac41f7b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/lib/prometheus.ts
i = end + 1;
continue;
}
if (PROMQL_SAFE_IDENTIFIERS.has(identifier)) continue;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Require function names to be calls before exempting them

When lockdown is active, this exemption lets a bare metric whose name collides with a PromQL function/aggregator pass as soon as the expression also contains one valid namespaced selector, e.g. up{namespace="prod"} + time currently validates even though time is an unscoped vector selector if that metric exists. Prometheus documents that a lone metric name selects all series with that name and only reserves bool, on, ignoring, group_left, and group_right as metric names (https://prometheus.io/docs/prometheus/latest/querying/basics/#instant-vector-selectors), so function names should only be skipped when they are actually used as calls/operators rather than as bare identifiers.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants