Skip to content

feat(ml-api): split liveness/readiness probes and drain on shutdown - #1020

Merged
Userunknown84 merged 1 commit into
Userunknown84:mainfrom
pavsoss:feat/1009-health-shutdown
Jul 29, 2026
Merged

feat(ml-api): split liveness/readiness probes and drain on shutdown#1020
Userunknown84 merged 1 commit into
Userunknown84:mainfrom
pavsoss:feat/1009-health-shutdown

Conversation

@pavsoss

@pavsoss pavsoss commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

/health was a static {"status":"ok"}, so an orchestrator could route traffic to an instance that would fail every prediction, and SIGTERM dropped in-flight requests.

  • GET /health/live — always 200 (process up).
  • GET /health/ready — 200 only when the serving state is loaded and the spam-words DB and rate-limit store respond; otherwise 503 in the standard error envelope.
  • GET /health retained as a backward-compatible alias.
  • In-flight requests are tracked; on SIGTERM the app enters draining (readiness starts returning 503 so load balancers stop routing) and waits for in-flight requests to finish before exit.

Test plan

/health/live is 200 independent of dependencies; /health/ready is 503 when the model isn't loaded / a dependency is down and 200 when healthy; entering drain flips readiness to 503; /health behaves as before.

Closes #1009

@vercel

vercel Bot commented Jul 28, 2026

Copy link
Copy Markdown

@pavsoss is attempting to deploy a commit to the Aditya Sharma's projects Team on Vercel.

A member of the Team first needs to authorize it.

@Userunknown84
Userunknown84 merged commit d8e34b1 into Userunknown84:main Jul 29, 2026
4 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Readiness/liveness health probes and graceful shutdown with in-flight request draining

2 participants