[Feature]: Structured JSON logging with request-ID correlation and secret/PII redaction for the Flask ML API

[pavsoss]

Problem

The Flask ML API logs ad-hoc: scattered app.logger.warning(f"...") calls in the zero-trust decorators, no consistent format, and no correlation to a request. A request_id is captured in capture_request_id() (from X-Request-ID) but never appears in any log line, so a failing prediction can't be traced across the request lifecycle. Some lines also log request.remote_addr and raw query values, leaking client data into logs with no redaction.

This is complementary to (not a duplicate of) the Prometheus /metrics work in #984 — metrics are aggregate counters; this is per-request, correlated, structured log records.

Proposed solution

Introduce a single logging layer that emits machine-parseable JSON, stamps every record with the current request_id, logs request start/finish with latency and status, and scrubs secrets/PII.

Scope

• A logging module (backend/logging_config.py) providing configure_logging() with a JSON formatter and a filter that injects g.request_id; generate a UUID request_id when the X-Request-ID header is absent.
• Structured access logs in before_request/after_request (method, path, status, latency_ms, request_id, response size).
• Route the error handlers (ApiError, 400/403/404/500) through structured logging with the ErrorCode and request_id; replace the ad-hoc f-string warnings.
• A README "Logging" section.

Suggested PR split

• Part 1 — logging infrastructure: the logging module, the UUID request-id fallback, and request-lifecycle access logs.
• Part 2 — redaction + error/audit coverage (builds on Part 1): a redaction filter that scrubs X-Internal-Secret, OAuth tokens, API keys, and email addresses/message bodies from records; structured logging applied to the error handlers and zero-trust warnings; regression coverage asserting the JSON shape, request-id propagation, and that secrets never appear.

Acceptance criteria

• Every log line is valid JSON containing request_id, level, msg, and a timestamp.
• Exactly one access log per request with latency_ms and status.
• No secret/PII substring appears in emitted records (covered by a test).
• Existing endpoints and response bodies are unchanged.

Out of scope

Log shipping/aggregation backends; frontend logging.

[github-actions]

👋 Welcome back! Thanks for your continued contribution. Your issue/PR is received and waiting for maintainer review.

[github-actions]

The issue highlights the need for improved logging in a Flask-based ML API. Current logging practices are inconsistent, lack request-level correlation, and may expose sensitive information. To address this, it proposes introducing structured JSON logging with request_id correlation, per-request access logs, and automated redaction of secrets/PII. The solution involves creating a centralized logging module, implementing structured error handling, and establishing tests to ensure compliance. The suggested implementation is split into two stages: setting up logging infrastructure and adding redaction/audit coverage. The scope excludes log aggregation and frontend logging features.

[pavsoss]

@Userunknown84 Please assign this to me