Skip to content

perf: encode a summary once per batch rather than on every observation - #81

Merged
kamilmysliwiec merged 1 commit into
nestjs:masterfrom
mwawacek:scout/summary-encode-per-batch-v3
Nov 9, 2026
Merged

kamilmysliwiec merged 1 commit into
nestjs:masterfrom
mwawacek:scout/summary-encode-per-batch-v3

Conversation

@mwawacek

@mwawacek mwawacek commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

PR Checklist

Please check if your PR fulfills the following requirements:

PR Type

What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[x] Other... Please describe: performance fix

What is the current behavior?

Every observe on a summary from TracerService re-encodes it into the outgoing batch (upsertCustomMetric), and encoding a summary sorts a copy of its sample once for each of p50, p95 and p99. The sample keeps at most 2,048 observations per label, so the cost of one observe grows with the number of observations since the last flush until the sample is full, and then stays at that level:

observations in the window one observe before after
500 147 µs < 1 µs
2,048 964 µs < 1 µs
5,000 985 µs < 1 µs
20,000 977 µs < 1 µs

That time is spent on the main thread, inside the request that calls observe. Measured on Node 22 against the built package, timing the last 200 of n observations of one summary in a single window:

const summary = new Summary("request.duration");
const buffer = new ObserveAgentSharedBuffer({ serviceId: "svc", forwardLogs: true });
for (let i = 0; i < n; i++) {
  summary.observe(Math.random() * 100);
  buffer.upsertCustomMetric(summary);
}
buffer.encodeAndWrite();

What is the new behavior?

A summary observed into the batch is remembered and encoded once, when the batch is written in encodeAndWrite (about 1-3 ms per batch in the runs above, up from about 0.1 ms). observe no longer sorts. Counters and gauges are buffered as before. The values that reach the collector are unchanged, since the summary is encoded from the same window, just once. Only the order within custom changes: summaries now come at the end of the array.

Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information

Every `observe` on a summary from `TracerService` re-encoded it into the
outgoing batch, and encoding a summary sorts a copy of its sample once
for each of p50, p95 and p99. The sample holds at most 2,048 observations
per label, so the cost of one `observe` grew with the window until the
sample was full and then stayed there: about 0.15 ms with 500
observations since the last flush, about 1 ms from 2,048 on.

A summary observed into the batch is now encoded when the batch is
written, so `observe` no longer sorts and the quantiles are worked out
once per batch.
@kamilmysliwiec
kamilmysliwiec merged commit b1b3086 into nestjs:master Nov 9, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants