Skip to content

Sandbox/judge vendor spike: Testpress labs vs Judge0 vs Piston (+ grade QUIZ CODE_OUTPUT) #62

Description

@teetangh

2-week timebox. CODE_LAB embed currently only works against Fermion labs (Testpress-dependent); DSA judge functions are dead exports; QUIZ CODE_OUTPUT is a dead enum (excluded from authoring validation/course.ts:163, always-wrong in grading learning.ts:188-198).

  • If Testpress outreach lands: wire real school key → existing lab embed path E2E
  • Judge0 self-host docker-compose spike (py/js/java/c++; per-user rate caps)
  • Piston public API as zero-infra fallback for quick runs
  • Feature flag judge.provider; wire ONE surface end-to-end: CODE_OUTPUT questions finally gradeable via code execution + stdout compare
  • Meter all runs into SandboxSession; abuse cap/day server-side
  • Decision doc: p95 latency, cost @10k runs/mo, ops burden

Spec: temp prompt library 06-sandbox-judge.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    v1V1 (6-12 months)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions