fix(test-harness): implement num_prompt_special_tokens on FakeTokenizer (cherry-pick to release/0.13.0)

[ajcasagrande]

Summary

• Cherry-pick of 9b60a3d479 from main into release/0.13.0
• Original PR: fix(test-harness): implement num_prompt_special_tokens on FakeTokenizer #1322 — fix(test-harness): implement num_prompt_special_tokens on FakeTokenizer

Conflicts

None — clean cherry-pick onto prerequisite base.

Stack

This is PR 4 of 6. Merge after #1377, #1378, and #1371.

1. fix(mmap): make get_conversation thread-safe, prefault pages, drop executor hop (#1245) #1377 — fix(mmap): make get_conversation thread-safe, prefault pages, drop executor hop #1245 (mmap thread-safety)
2. fix(dataset): restore random_pool batch-size support with --input-file (#1274) #1378 — fix(dataset): restore random_pool batch-size support with --input-file #1274 (random_pool batch-size)
3. fix(config): stop silently dropping CLI flags passed with --config (cherry-pick to release/0.13.0) #1371 — fix(config): stop silently dropping CLI flags passed with --config #1280 (config flag routing)
4. This PR — fix(test-harness): implement num_prompt_special_tokens on FakeTokenizer #1322 (FakeTokenizer)
5. feat: Support stateful responses API with previous_response_id chaining (cherry-pick to release/0.13.0) #1375 — feat: Support stateful responses API with previous_response_id chaining #1347 (stateful responses / previous_response_id)
6. feat(kubernetes): add native Kubernetes execution + prerequisite stack (cherry-pick #1245+#1274+#1280+#1322+#1325+#1326+#1347 to release/0.13.0) #1379 — combined feat: control-plane groundwork for k8s execution — dead-component detection, result finalization, ZMQ reconnect #1325+ feat(k8s): AIPerf goes k8s-native — operator, CRDs, JobSet execution, and the aiperf kube CLI #1326 (k8s-native)

[github-actions]

Try out this PR

Quick install:

pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@ec7709d6f6c300e6417bfc3d33c1ae82059a9458

Recommended with virtual environment (using uv):

uv venv --python 3.12 && source .venv/bin/activate
uv pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@ec7709d6f6c300e6417bfc3d33c1ae82059a9458

---

Last updated for commit: ec7709d • Browse code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!