feat: add DELETE endpoints for evaluations/submissions + fix schema-lock contention - #242
Merged
Conversation
ensure_schema() ran ALTER TABLE ... ADD COLUMN IF NOT EXISTS unconditionally on every process start (API and worker), even though the columns are normally already present. ALTER TABLE takes an exclusive lock, and Postgres queues later requests -- including plain SELECTs -- behind a pending exclusive lock request, so a long-running job transaction could block not just new process startups but every already-running process's unrelated queries too. Add _add_column_if_missing/_drop_not_null_if_needed helpers that check information_schema first (a non-blocking catalog SELECT) and only take the exclusive lock when a migration is actually needed. A 5s lock_timeout remains as a safety net for the rare case a migration is genuinely needed while something else is busy, so that case fails fast with a clear error instead of hanging indefinitely. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds DELETE /api/admin/evaluations/{id} (restricted to standalone
provider-route evals, i.e. submission_id is null) and
DELETE /api/admin/submissions/{id}, for the admin UI's new delete
buttons on the provider-tests and standings tables.
Both endpoints clear referencing Artifact/JobQueue rows first since
those FKs have no ON DELETE cascade in Postgres, then delete the
target row (submission deletion cascades to its TrainRun/EvaluationRun
rows via the existing ORM-level cascade="all, delete-orphan").
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
github-actions
Bot
added
eval
Merged
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
validator/src/eval_backend/db.py:ensure_schema()no longer takes an exclusiveALTER TABLElock on every process boot when the schema is already up to date — checksinformation_schemafirst. Keeps a 5slock_timeoutas a safety net. Fixes a real issue hit this session where a busy job's transaction blocked every new API/worker startup, and by lock-queue fairness, blocked an already-running API server's unrelated queries too.validator/src/eval_backend/api/routes.py: newDELETE /api/admin/evaluations/{id}(standalone provider-route evals only) andDELETE /api/admin/submissions/{id}, backing the new delete buttons in the admin UI (companion PR inmini-router.github.io). Both clear referencingArtifact/JobQueuerows first (noON DELETEcascade on those FKs in Postgres).Test plan
pytestpasses (19 passed, 18 skipped)DELETE /evaluations/{id}returns 404 for unknown id, 204 + row gone for a real standalone eval;DELETE /submissions/{id}returns 204 + row gone from leaderboard, 404 on repeat delete