Skip to content

feat: support configurable Quent telemetry exporters #1359

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
johanpel merged 1 commit into from
Jul 31, 2026
Merged

feat: support configurable Quent telemetry exporters #1359

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions docs/super-sirius/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -371,8 +371,9 @@ sirius:

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `enable_quent` | bool | true | Emit Quent telemetry using the ndjson exporter. When false, telemetry uses the noop exporter. |
| `output_directory` | string | `telemetry_data` | Directory for Quent ndjson files. |
| `enable_quent` | bool | true | Emit Quent telemetry using the configured exporter. When false, telemetry uses the noop exporter. |
| `exporter` | string | `ndjson` | Quent filesystem exporter: `ndjson`, `msgpack`, or `postcard`. |
| `output_directory` | string | `telemetry_data` | Directory for Quent telemetry files. |
| `engine_name` | string | `siriusDB` | Engine name reported in engine-level telemetry. |

Per-query labels are configured separately from YAML. They can be set with the
Expand Down Expand Up @@ -412,7 +413,8 @@ sirius:

Load that config through the normal config resolution path, usually by setting
`SIRIUS_CONFIG_FILE=/path/to/sirius.yaml`. Any Sirius query run with
`enable_quent: true` writes Quent ndjson files into `output_directory`.
`enable_quent: true` writes Quent ndjson files into `output_directory` by default. Set
`exporter: postcard` for compact benchmark or CI telemetry.

For TPC-H Parquet runs, the helper script runs queries and labels each
`(query, iteration)` pair before executing it:
Expand Down
14 changes: 8 additions & 6 deletions docs/super-sirius/quent-telemetry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ a modular instrumentation based telemetry toolkit to better understand runtime
behaviours of complex applications. When a query runs, Sirius emits structured
traces describing the engine, the plan (operators, ports, edges), executor
/task-manager threads, task queues, and per-query activity. These traces are written
as newline-delimited JSON (ndjson) files that Quent's analyzer server then ingests
as newline-delimited JSON (ndjson) files by default that Quent's analyzer server then ingests
and renders as an interactive timeline in your browser.

## 1. Enable the exporter

Telemetry is controlled entirely by the Sirius YAML config (see the
[Configuration reference](configuration.md#telemetry) for where config files are resolved). Enable
the Quent ndjson exporter and choose an output directory:
the Quent exporter and choose an output directory:

```yaml
sirius:
Expand All @@ -26,13 +26,15 @@ sirius:

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `enable_quent` | bool | `true` | Emit Quent telemetry using the ndjson exporter. When `false`, telemetry uses the no-op exporter and nothing is written. |
| `output_directory` | string | `telemetry_data` | Directory for the Quent ndjson files. |
| `enable_quent` | bool | `true` | Emit Quent telemetry using the configured exporter. When `false`, telemetry uses the no-op exporter and nothing is written. |
| `exporter` | string | `ndjson` | Quent filesystem exporter: `ndjson`, `msgpack`, or `postcard`. |
| `output_directory` | string | `telemetry_data` | Directory for Quent telemetry files. |
| `engine_name` | string | `siriusDB` | Engine name reported in engine-level telemetry. |

Load the config through the normal resolution path — usually by setting
`SIRIUS_CONFIG_FILE=/path/to/sirius.yaml` before loading the extension. Any Sirius query run with
`enable_quent: true` then writes ndjson files into `output_directory`.
`enable_quent: true` then writes ndjson files into `output_directory` by default. Set
`exporter: postcard` for compact benchmark or CI telemetry.

## 2. Label your queries (optional)

Expand All @@ -57,7 +59,7 @@ with `sirius_set_query_label`. Unlabeled queries are reported as `unnamed_query`

## 3. Generate telemetry

Run any query with the exporter enabled and ndjson files appear under `output_directory`.
Run any query with the exporter enabled and telemetry files appear under `output_directory`.

### TPC-H helper

Expand Down
174 changes: 174 additions & 0 deletions pixi.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pixi.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ rust-src = "*"
liburing = "*"
openssl = "*"
libcurl = "*"
pnpm = "*"
# Go toolchain (with cgo) to build the patched testcontainers-native bridge used
# by the C++ S3 integration tests (fetched at configure time; see
# cmake/testcontainers_native.cmake). Upstream go.mod requires >=1.25.
Expand Down
Loading
Loading