Merge branch 'main' into arekay/consolidate_llama_examples

Author: arekay-nv

.pre-commit-config.yaml

@@ -51,7 +51,11 @@ repos:
 
       - id: regenerate-templates
         name: Regenerate YAML templates from schema defaults
-        entry: python scripts/regenerate_templates.py
+        # uv pins the interpreter to this project's environment: a bare
+        # `python` resolves whatever is first on PATH, which can import
+        # inference_endpoint from a sibling checkout and regenerate the
+        # templates against the wrong schema.
+        entry: uv run --no-sync python scripts/regenerate_templates.py
         language: system
         pass_filenames: false
         files: ^(src/inference_endpoint/config/(schema\.py|templates/.*)|src/inference_endpoint/endpoint_client/config\.py|scripts/regenerate_templates\.py)$

AGENTS.md

@@ -25,4 +25,13 @@ This section lists third-party assets and their attributions.
   (instance IDs such as `django__django-16899`). Cite: Jimenez et al.,
   "SWE-bench: Can Language Models Resolve Real-world Github Issues?", ICLR 2024.
 
+### mini-swe-agent
+
+- Source: https://github.com/SWE-agent/mini-swe-agent/tree/v2.3.0
+- License: MIT
+- Copyright: Copyright (c) 2025 Kilian A. Lieret and Carlos E. Jimenez
+- Description: The SWE-bench service templates and optional Qwen tool schemas,
+  parser, and model extension are derived from mini-swe-agent v2.3.0. The full
+  upstream notice is retained in the swebench_service package.
+
 For a complete list of dependencies and their licenses, see the requirements files in the `requirements/` directory.

ATTRIBUTION

@@ -76,13 +76,13 @@ Dataset Manager ──> Load Generator ──> Endpoint Client ──> External
                     Metrics Collector (EventRecorder + MetricsReporter)
 ```
 
-| Component           | Purpose                                                                              |
-| ------------------- | ------------------------------------------------------------------------------------ |
-| **Load Generator**  | Central orchestrator: `BenchmarkSession` owns lifecycle, `Scheduler` controls timing |
-| **Endpoint Client** | Multi-process HTTP workers communicating via ZMQ IPC                                 |
-| **Dataset Manager** | Loads JSONL, HuggingFace, CSV, JSON, Parquet datasets                                |
-| **Metrics**         | SQLite-backed event recording, aggregation (QPS, latency, TTFT, TPOT)                |
-| **Config**          | Pydantic-based YAML schema, CLI auto-generated via cyclopts                          |
+| Component           | Purpose                                                                                                                           |
+| ------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
+| **Load Generator**  | Central orchestrator: `BenchmarkSession` owns lifecycle, `Scheduler` controls timing                                              |
+| **Endpoint Client** | Multi-process HTTP workers communicating via ZMQ IPC                                                                              |
+| **Dataset Manager** | Loads JSONL, HuggingFace, CSV, JSON, Parquet datasets                                                                             |
+| **Metrics**         | SQLite-backed event recording, aggregation (QPS, latency, TTFT, TPOT), MLPerf early-stopping percentile estimates (on by default) |
+| **Config**          | Pydantic-based YAML schema, CLI auto-generated via cyclopts                                                                       |
 
 ### Benchmark Modes
 

README.md

@@ -32,12 +32,12 @@ enable_cpu_affinity: true # Auto-compute NUMA-aware plan (default)
 # enable_cpu_affinity: false  # Disabled
 ```
 
-**Auto mode allocation** (default 6 physical cores for loadgen):
+**Auto mode allocation** (default 5 physical cores for loadgen, `DEFAULT_LOADGEN_CORES`):
 
 - 1 core: Session thread (scheduler, busy-wait timing)
 - 1 core: Event loop thread (uvloop, response handling)
-- 4 cores: ZMQ I/O threads
-- Remaining physical cores: Workers (one per core with all SMT siblings)
+- Remaining cores: ZMQ I/O threads (up to 4, sharing the leftover loadgen cores)
+- All other physical cores: Workers (one per core with all SMT siblings)
 
 ## Platform Notes
 

docs/CLIENT_PERFORMANCE_TUNING.md

@@ -108,6 +108,7 @@ Flag names shown as `--full.dotted.path --alias`. Both forms work.
   the output location; otherwise a default report directory is used.
 - `--timeout` - Global timeout in seconds
 - `--enable-cpu-affinity / --no-cpu-affinity` - NUMA-aware CPU pinning (default: true)
+- `--no-early-stopping` - opt out of the MLPerf early-stopping percentile estimates in `result_summary.json` (default: on; see [early_stopping.md](early_stopping.md))
 
 **Online-specific:**
 
@@ -142,6 +143,7 @@ Format is auto-detected from file extension. Override with `format=<ext>` in the
 
 - Max throughput testing
 - Metrics: QPS, latency, TTFT, TPOT
+- Ordinary configured scoring remains available, but external scorers are skipped
 - Fastest - no response collection overhead
 
 **acc** - Accuracy only (collect all responses)
@@ -172,6 +174,10 @@ inference-endpoint benchmark offline \
 
 > **Note:** Submission runs (`type: submission`) are YAML-only — they require `submission_ref` and `benchmark_mode` fields not exposed in CLI.
 
+Report directories contain a sanitized `config.yaml`: credentials and other
+secret values are replaced with `<redacted>`. Restore those values before
+reusing that file as benchmark input.
+
 ## Load Patterns
 
 **max_throughput** - Offline mode