Skip to content

Commit 8e89300

Browse files
committed
docs(gomodel): its perf drop is our dropped feature strip, not a regression
GoModel's 2026-07-25 numbers collapsed (rps_max_proxy 16000 -> 2576, added p99 306 -> 2552us) while the rig floor barely moved (direct_c1_p99 74 -> 81us). It is neither a GoModel regression nor box noise: - the image never changed — results/history/gomodel.jsonl records the identical enterpilot/gomodel:0.1.55 @sha256:606151f9...b562ac on both sides; - the last good run was 2026-07-24T01:20Z; commit 2951b97, later that same day, removed `-e LOGGING_ENABLED=false` from gw_launch as part of the OOTB no-feature-strips pass; - GoModel ships audit logging ON (LOGGING_ENABLED, LOGGING_LOG_BODIES and LOGGING_LOG_HEADERS all default true, .env.template:275-281 — "all requests and responses are logged to the configured storage"), so with the default sqlite backend every call now captures request + response bodies and headers on the request path; - the load sweep agrees: throughput pins at ~2500rps flat from c=32 to c=512, a saturation ceiling rather than a scheduling artefact. So the pre-07-24 numbers were the flattering ones, taken with a strip no other gateway received. Removing it was correct and it stays removed — same posture, and same disclosure, One-API already carries for its per-request quota writes. Document it in the manifest instead: a NOTE ON ITS NUMBERS block, and a warning on the env list so nobody re-adds LOGGING_ENABLED=false and silently re-inflates throughput ~6x. Also flagged, NOT root-caused: openai-responses->openai-responses measured ~100rps with p50 added ~10.09ms, ~25x worse than the other served cells and flat across concurrency — a second, distinct serialization. That one cell's perf is suspect until a targeted re-measure explains it. Run-over-run comparisons for gomodel across the 2026-07-24 boundary are invalid and must not be published as a GoModel regression.
1 parent 07232d3 commit 8e89300

1 file changed

Lines changed: 32 additions & 0 deletions

File tree

‎gateways/gomodel/gateway.sh‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,34 @@
66
# via OPENAI_BASE_URL, so /v1/chat/completions forwards there. Left unprotected (GOMODEL_MASTER_KEY
77
# unset) for a pure proxy-overhead measurement — the default posture. Image pinned in
88
# gateways/versions.env; the resolved tag is recorded in the result.
9+
#
10+
# ── NOTE ON ITS NUMBERS: GoModel AUDIT-LOGS EVERY REQUEST BY DEFAULT ─────────────────────────────
11+
# Same disclosure One-API carries, for the same reason. GoModel ships audit logging ON: LOGGING_ENABLED
12+
# defaults true, and so do LOGGING_LOG_BODIES and LOGGING_LOG_HEADERS (.env.template:275-281 — "When
13+
# enabled, all requests and responses are logged to the configured storage"; config/logging.go). With
14+
# the default STORAGE_TYPE=sqlite that means a per-request entry — full request AND response body, plus
15+
# headers — captured on the request path (internal/auditlog/middleware.go captureLoggedRequestBody /
16+
# captureLoggedResponseBody) and batch-written by the flush loop (internal/auditlog/logger.go). Its
17+
# latency/throughput therefore reflect A GATEWAY THAT AUDIT-LOGS EVERY CALL WITH BODIES, not a bare
18+
# proxy — the honest measurement of GoModel as it ships. There is no external infra involved (sqlite in
19+
# the image's own /app/data, which the upstream Dockerfile creates and does not mount as a volume), so
20+
# turning it off would be a forbidden feature strip, not a permitted run-mechanic.
21+
#
22+
# ***RUN-OVER-RUN COMPARISONS ACROSS 2026-07-24 ARE INVALID FOR GOMODEL — DO NOT PUBLISH THE DELTA AS A
23+
# GOMODEL REGRESSION.*** Every run up to and including 2026-07-24T01:20Z was measured with an
24+
# `-e LOGGING_ENABLED=false` FEATURE STRIP that no other gateway in the field received; commit 2951b97
25+
# ("bench(ootb): config-transparency mechanism") removed that strip later the same day, correctly. The
26+
# image never changed — results/history/gomodel.jsonl records the identical
27+
# enterpilot/gomodel:0.1.55 @sha256:606151f9…b562ac digest on BOTH sides of the boundary — and the
28+
# 2026-07-25 rig floor was healthy (direct_c1_p99_us 74→81us, +7..9%). So the observed
29+
# rps_max_proxy 16000→2576 and added p99 306→2552us is NOT a GoModel regression and NOT box noise: it
30+
# is the cost of the audit-log feature the strip used to hide. The pre-07-24 numbers were the
31+
# flattering ones. The load-sweep signature agrees — throughput pins at ~2500rps flat from c=32 to
32+
# c=512 (results/fanout-gomodel.log), a saturation ceiling, not a scheduling artifact.
33+
# STILL UNEXPLAINED (flagged, not fixed): the openai-responses→openai-responses cell measured ~100rps
34+
# with p50 added ≈10.09ms, ~25x worse than the other served cells and flat across concurrency. That is
35+
# a second, distinct serialization we have not root-caused; treat that ONE cell's perf as suspect until
36+
# a targeted re-measure explains it.
937
GW_KIND=docker
1038
# Self-describing manifest metadata — charts.py + the run lists read these, so a gateway
1139
# is fully defined by its own dir (add/remove a dir → it appears/disappears everywhere).
@@ -75,6 +103,10 @@ gw_launch() {
75103
# OOTB posture: default features stay ON (no LOGGING_ENABLED/budget/ratelimit/admin/mcp strips); the
76104
# only deviations are the permitted ones — provider base_urls → mock, dummy keys, OpenAI-lane
77105
# provider scope, and the STORAGE_TYPE=sqlite / MODELS_ENABLED_BY_DEFAULT run-mechanics.
106+
# LOGGING_ENABLED is DELIBERATELY ABSENT so GoModel's default audit logging (bodies + headers →
107+
# sqlite, on the request path) stays on. Re-adding LOGGING_ENABLED=false here would restore the
108+
# pre-2026-07-24 feature strip and silently re-inflate its throughput ~6x — see the "NOTE ON ITS
109+
# NUMBERS" block at the top of this file before touching this list.
78110
_gomodel_env() {
79111
local ncore=$(( ${CORES##*-} - ${CORES%%-*} + 1 ))
80112
cat <<ENV

0 commit comments

Comments
 (0)