docs: current-state pass over the timeout surface

Fix references left behind by the consolidation: renamed drain knob in
session.py docstring, argv-vs-schema 0-sentinel wording in the aggregator
snapshot/help text, config module pointers after the schema split, the
from-config flag surface in CLI_QUICK_REFERENCE, regenerate-templates
trigger lists, the config DESIGN nested-model table, and the compliance
plan's duration-floor note. schema.py re-exports HTTPClientConfig again.

Author: viraatc

AGENTS.md

@@ -128,7 +128,7 @@ The aggregator is a separate process (`python -m inference_endpoint.async_utils.
 CLI is auto-generated from `config/schema.py` Pydantic models via cyclopts. Fields annotated with `cyclopts.Parameter(alias="--flag")` get flat shorthands; all other fields get auto-generated dotted flags (kebab-case).
 
 - **CLI mode** (`offline`/`online`): cyclopts constructs `OfflineBenchmarkConfig`/`OnlineBenchmarkConfig` (subclasses in `config/schema.py`) directly from CLI args. Type locked via `Literal`. `--dataset` is repeatable with TOML-style format `[perf|acc:]<path>[,key=value...]` (e.g. `--dataset data.csv,samples=500,parser.prompt=article`). Full accuracy support via `accuracy_config.eval_method=pass_at_1` etc.
-- **YAML mode** (`from-config`): `BenchmarkConfig.from_yaml_file()` loads YAML, resolves env vars, and auto-selects the right subclass via Pydantic discriminated union. Optional `--timeout` (maps to `settings.timeouts.run_timeout_s`)/`--mode` overrides via `config.with_updates()`.
+- **YAML mode** (`from-config`): `BenchmarkConfig.from_yaml_file()` loads YAML, resolves env vars, and auto-selects the right subclass via Pydantic discriminated union. Optional `--timeout` override maps to `settings.timeouts.run_timeout_s` via `config.with_updates()`; `--mode` selects the `TestMode` passed to the runner.
 - **eval**: Not yet implemented (raises `CLIError` with a tracking issue link)
 
 ### Config Construction & Validation
@@ -315,7 +315,7 @@ All of these run automatically on commit:
 - `mypy` type checking
 - `prettier` for YAML/JSON/Markdown
 - License header enforcement
-- `regenerate-templates`: auto-regenerates YAML config templates from schema defaults when `schema.py`, `config.py`, or `regenerate_templates.py` change
+- `regenerate-templates`: auto-regenerates YAML config templates from schema defaults when any config schema module (`schema|audit|model_params|datasets|settings|timeouts`.py), `endpoint_client/config.py`, or `regenerate_templates.py` changes
 
 **IMPORTANT: Always run `pre-commit run --all-files` before every commit.** Hooks may modify files (prettier, ruff-format, license headers). If files are modified, stage the changes and commit once. Never commit without running pre-commit first.
 

docs/CLI_DESIGN.md

@@ -67,7 +67,7 @@ Both paths produce the **same subclass with the same defaults**. A YAML file wit
 
 2. **Auto-selects subclass.** `type: "offline"` → `OfflineBenchmarkConfig`, `type: "online"` → `OnlineBenchmarkConfig`, others → base `BenchmarkConfig`.
 
-3. **Optional CLI overrides.** `--timeout` and `--mode` applied via `config.with_updates(...)` which re-runs validators.
+3. **Optional CLI overrides.** `--timeout` maps into `settings.timeouts.run_timeout_s` via `with_updates(...)` (re-runs validators); `--mode` selects the `TestMode` passed to the runner and never touches the config object.
 
 ### Why subclasses?
 

docs/CLI_QUICK_REFERENCE.md

@@ -102,9 +102,8 @@ Flag names shown as `--full.dotted.path --alias`. Both forms work.
 - `--endpoint-config.api-key --api-key` - API authentication
 - `--endpoint-config.api-type --api-type` - API type: openai/sglang (default: openai)
 - `--report-dir` - Report output directory
-  Note: applies to CLI-driven `benchmark offline` / `benchmark online`; `benchmark from-config`
-  does not expose a CLI override for `report_dir`. Set it in the YAML only if you need to control
-  the output location; otherwise a default report directory is used.
+  Note: `benchmark from-config` also accepts `--report-dir` as an override of the YAML value;
+  when neither is set a default report directory is used.
 - `--timeout` - Whole-run watchdog in seconds (off by default). If it fires, the run is aborted, the report is marked INTERRUPTED, and the process exits non-zero.
 - `--enable-cpu-affinity / --no-cpu-affinity` - NUMA-aware CPU pinning (default: true)
 - `--no-early-stopping` - opt out of the MLPerf early-stopping percentile estimates in `result_summary.json` (default: on; see [early_stopping.md](early_stopping.md))
@@ -307,8 +306,8 @@ inference-endpoint init submission
 # 3. Run (YAML mode)
 inference-endpoint benchmark from-config \
   --config submission_template.yaml
-# Note: from-config only accepts --config, --timeout, and --mode via CLI.
-# Set report_dir in the YAML if you need a specific output location.
+# from-config accepts --config, --timeout, --mode, --accuracy-only, and
+# --report-dir; everything else comes from the YAML.
 ```
 
 ### Validate First

docs/DEVELOPMENT.md

@@ -198,7 +198,7 @@ docs/short-description
 
 ## YAML Config Templates
 
-Config templates in `src/inference_endpoint/config/templates/` are auto-generated from schema defaults. When you change `config/schema.py`, regenerate them:
+Config templates in `src/inference_endpoint/config/templates/` are auto-generated from schema defaults. When you change any config schema module (`config/schema.py` and its sibling domain modules), regenerate them:
 
 ```bash
 uv run python scripts/regenerate_templates.py

docs/async_utils/services/metrics_aggregator/DESIGN.md

@@ -122,7 +122,8 @@ COMPLETE event ─► trigger.fire ─► queue.enqueue(text, on_count)        [
 
 `--drain-timeout` and `--tokenizer-workers` have service-side defaults (`0`
 and `2`) so the service is launchable by hand without tuning knobs, but
-`config/schema.py` is the single source of truth: the benchmark always
+the config schema is the single source of truth (`settings.timeouts.metrics_drain_timeout_s`
+in `config/timeouts.py`, `settings.metrics_tokenizer_workers` in `config/settings.py`): the benchmark always
 forwards the schema values (`--metrics-drain-timeout`,
 `--metrics-tokenizer-workers`), overriding these defaults in normal runs.
 

docs/compliance_audit_plan.md

@@ -523,14 +523,12 @@ Two scenarios must be covered: **Offline** (`max_throughput`) and **SingleStream
 > catches a crashed run — but the examples default to equal for the clearest, least-contentious
 > comparison.
 
-> **`min_duration` is not a duration floor (current limitation).** The load-generator stop
-> check (`session.py`) halts a phase on **sample count** or **`max_duration_ms`** only;
-> `min_duration_ms` merely _derives_ a count when no explicit count is set. Because TEST04
-> drives an explicit `samples` count, each phase stops at `samples` and `min_duration_ms` is
-> **not** honored as a "run for at least 10 minutes" floor. MLCommons' 10-minute compliance
-> minimum therefore is **not** enforced today; combining a count floor with a duration floor
-> ("AND-semantics") is future work. Set `samples` large enough that each phase reaches a
-> stable throughput on its own.
+> **No duration floor (current limitation).** Runs are count-driven: the load-generator stop
+> check (`session.py`) halts a phase on **sample count** or **`runtime.max_duration_ms`**
+> only, and TEST04 drives explicit `samples` / `audit_samples` counts. MLCommons' 10-minute
+> compliance minimum therefore is **not** enforced today; combining a count floor with a
+> duration floor ("AND-semantics") is future work. Set `samples` large enough that each phase
+> reaches a stable throughput on its own.
 
 Both scenarios ship as committed configs (see also
 [`compliance/audit_test/README.md`](../src/inference_endpoint/compliance/audit_test/README.md)):

docs/config/DESIGN.md

@@ -45,13 +45,14 @@ the benchmark `type` and `endpoint_config`.
 
 Key nested models:
 
-| Model            | Purpose                                             |
-| ---------------- | --------------------------------------------------- |
-| `LoadPattern`    | Pattern type + parameters (target QPS, concurrency) |
-| `RuntimeConfig`  | Duration, sample count, RNG seeds                   |
-| `ClientSettings` | Worker count and HTTP client settings               |
-| `EndpointConfig` | Endpoint URLs, API key                              |
-| `Dataset`        | Dataset path, type (performance / accuracy)         |
+| Model              | Purpose                                                     |
+| ------------------ | ----------------------------------------------------------- |
+| `LoadPattern`      | Pattern type + parameters (target QPS, concurrency)         |
+| `RuntimeConfig`    | Sample count, perf-phase cap (`max_duration_ms`), RNG seeds |
+| `Timeouts`         | All global waits/deadlines (`settings.timeouts`)            |
+| `HTTPClientConfig` | Worker count and HTTP client settings (`settings.client`)   |
+| `EndpointConfig`   | Endpoint URLs, API key                                      |
+| `Dataset`          | Dataset path, type (performance / accuracy)                 |
 
 ### `RuntimeSettings` (frozen dataclass)
 
@@ -145,8 +146,8 @@ to the report output.
 | Consumer                        | Usage                                                        |
 | ------------------------------- | ------------------------------------------------------------ |
 | `load_generator/session.py`     | Receives `RuntimeSettings` at construction                   |
-| `load_generator/scheduler.py`   | Reads `load_pattern`, `n_samples_to_issue`, RNG seeds        |
+| `load_generator/strategy.py`    | Reads `load_pattern`, `n_samples_to_issue`, RNG seeds        |
 | `endpoint_client/config.py`     | Reads `api_type`, `num_workers`, streaming mode              |
-| `metrics/reporter.py`           | Reads `reported_metrics`, duration bounds                    |
+| `metrics/report.py`             | Reads `reported_metrics`, duration bounds                    |
 | `commands/benchmark/cli.py`     | Defines benchmark subcommands and resolves CLI vs YAML input |
 | `commands/benchmark/execute.py` | Runs the benchmark lifecycle from resolved configuration     |

src/inference_endpoint/async_utils/services/metrics_aggregator/__main__.py

@@ -145,7 +145,7 @@ async def main() -> None:
             "Wall-clock budget (seconds) to finish tokenizing buffered samples "
             "after ENDED before the aggregator emits the final snapshot with "
             "n_pending_tasks > 0 (0 = wait indefinitely, the default; the "
-            "benchmark forwards the schema default, see config/schema.py). "
+            "benchmark forwards the schema default, see config/timeouts.py). "
             "Increase for very large datasets where the end-of-run tokenize "
             "batch is big."
         ),
@@ -176,7 +176,7 @@ async def main() -> None:
             "In-process tokenizer threads for live (mid-run) ISL/OSL/TPOT "
             "(0 = no mid-run tokenization, everything defers to the "
             "end-of-run drain; the benchmark forwards the schema default, "
-            "see config/schema.py). The drain always uses the auto-sized "
+            "see config/settings.py). The drain always uses the auto-sized "
             "sharded pool — one worker process per 8-core block."
         ),
     )

src/inference_endpoint/async_utils/services/metrics_aggregator/snapshot.py

@@ -45,7 +45,7 @@ class SessionState(str, Enum):
     LIVE        → run in progress; tick task publishing live HDR-derived stats.
     DRAINING    → ``SessionEventType.ENDED`` has been received; the aggregator
                   is tokenizing the buffered samples (bounded by the
-                  ``--drain-timeout`` budget — schema default 0 = unlimited). Tick task
+                  ``--drain-timeout`` budget — argv 0 = unlimited; the schema knob ``settings.timeouts.metrics_drain_timeout_s`` uses None, converted at the argv boundary). Tick task
                   continues at this stage, still HDR-derived; no new events
                   will arrive.
     COMPLETE    → terminal clean state. The ``publish_final()`` snapshot

src/inference_endpoint/config/schema.py

@@ -51,6 +51,7 @@
 )
 
 from ..core.types import APIType
+from ..endpoint_client.config import HTTPClientConfig
 from ..exceptions import CLIError
 from ..utils import WithUpdatesMixin
 from .audit import AuditConfig, AuditTestId, OutputCachingTestConfig
@@ -97,6 +98,7 @@
     "EarlyStoppingConfig",
     "EndpointConfig",
     "EvalMethod",
+    "HTTPClientConfig",
     "LoadPattern",
     "LoadPatternType",
     "ModelParams",