Skip to content

Harden benchmark worker and offline transform sandbox#11

Open
anupsv wants to merge 19 commits into
mainfrom
swift-only-migration-update
Open

Harden benchmark worker and offline transform sandbox#11
anupsv wants to merge 19 commits into
mainfrom
swift-only-migration-update

Conversation

@anupsv

@anupsv anupsv commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • keep hidden benchmark oracle tokens out of the runtime worker by switching benchmark decode to begin/step requests
  • run correctness and timed benchmark in separate worker processes
  • tighten runtime/offline sandboxes: no network, no child process execution, no worker writes, transform writes only to weights
  • validate overlaid editable paths after copy and remove stale make-golden docs

Validation

  • bash -n benchmark.sh .github/scripts/run-offline.sh .github/scripts/overlay-editable-paths.sh setup.sh
  • swift test --filter BenchmarkScriptTests
  • swift test --filter 'BenchmarkScriptTests|BenchmarkSupportTests|GoldenTests'
  • CLANG_MODULE_CACHE_PATH=$PWD/.build/clang-module-cache swift build -c release

Note: a full local swift test still requires mlx.metallib for existing unguarded MLX runtime tests; the focused non-runtime hardening tests pass.


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.

@anupsv anupsv had a problem deploying to benchmark-private-prompts June 23, 2026 19:37 — with GitHub Actions Error
@anupsv anupsv had a problem deploying to benchmark-private-prompts June 23, 2026 19:39 — with GitHub Actions Failure
@anupsv anupsv had a problem deploying to benchmark-private-prompts June 23, 2026 19:52 — with GitHub Actions Failure
@anupsv anupsv had a problem deploying to benchmark-private-prompts June 23, 2026 20:04 — with GitHub Actions Failure
@anupsv anupsv had a problem deploying to benchmark-private-prompts June 23, 2026 20:14 — with GitHub Actions Failure
@anupsv anupsv deployed to benchmark-private-prompts June 23, 2026 20:25 — with GitHub Actions Active
@anupsv anupsv had a problem deploying to benchmark-private-prompts June 24, 2026 17:20 — with GitHub Actions Failure
@anupsv anupsv had a problem deploying to benchmark-private-prompts June 24, 2026 17:32 — with GitHub Actions Failure
@anupsv anupsv had a problem deploying to benchmark-private-prompts June 24, 2026 17:43 — with GitHub Actions Failure
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant