[Feature]: Readiness/liveness health probes and graceful shutdown with in-flight request draining

[pavsoss]

Problem

/health returns a static {"status": "ok"} regardless of whether the model is loaded or dependencies (spam-words SQLite, rate-limit store) are reachable, so an orchestrator can route traffic to an instance that will 5xx every prediction. There is also no graceful shutdown: a SIGTERM drops in-flight requests immediately.

Proposed solution

Split health into liveness and readiness, make readiness reflect real dependencies, and drain in-flight requests on shutdown.

Scope

• GET /health/live — process is up (always 200).
• GET /health/ready — 200 only when the serving state is populated and the spam-words DB + rate-limit store respond; otherwise 503 with the standard error envelope ([Feature]: Standardize Flask ML API errors behind a machine-readable error envelope (backward compatible) #986).
• Keep GET /health as a backward-compatible alias.
• Track in-flight requests in before_request/after_request.
• A SIGTERM handler that flips readiness to draining (so /health/ready returns 503 and load balancers stop routing), waits for in-flight requests to finish up to a bounded timeout, then exits.
• README + OpenAPI paths; regression coverage for ready/live states and the drain flag.

Acceptance criteria

• /health/ready is 503 before the model loads / when a dependency is down, and 200 otherwise; /health/live is independent of dependencies.
• /health still responds as today (no breaking change).
• After SIGTERM, readiness reports draining and in-flight requests complete before exit (covered by a test with a fake slow request).

Out of scope

Kubernetes manifests/probe configuration; multi-process (gunicorn) worker coordination.

[github-actions]

👋 Welcome back! Thanks for your continued contribution. Your issue/PR is received and waiting for maintainer review.

[github-actions]

The issue highlights problems with the health endpoint of a service, which always returns {"status": "ok"} regardless of readiness or dependency state, leading to orchestration errors. Additionally, there is no graceful shutdown mechanism, resulting in dropped in-flight requests during termination. The proposed solution involves splitting health checks into liveness (/health/live) and readiness (/health/ready), with readiness reflecting actual dependency states and transitioning to a draining mode during shutdown. This ensures no breaking changes, supports backward compatibility, and allows graceful termination by waiting for in-flight requests to complete before exiting. Necessary tests, documentation, and compatibility are specified, but Kubernetes-specific configurations are out of scope.

[pavsoss]

@Userunknown84 Please assign this to me