Replace run_report_data table with S3 CSV files for plate reader data

[wasimxyz]

Summary

• The SpectraMax plate reader Lambda now writes well data to a CSV in the S3 processed bucket instead of inlining the entire DataFrame as JSON in the API request body, avoiding Vercel's function payload limit for large kinetic files.
• The run_report_data Postgres table is dropped; the plate reader UI fetches and parses the CSV from S3 at render time.
• The get_run_report_data MCP tool is removed; clients use list_run_files + get_file_download_url to access processed data.

Changes

Lambda

• spectramax_plate_reader/process_file.py — Writes well data to a local CSV, uploads it to the S3 processed bucket, and registers it as a category="processed" file record. Removes report_data from the update_file call.
• api_client.py — Removes the report_data parameter from update_file().

Web app API

• PATCH /files/:fileId — Removes the report_data insert block and the runReportData delete on reprocess transition.
• GET /runs/:runId — Removes the runReportData query and report_data from the JSON response.
• file-reprocessing.ts — Removes the tx.delete(runReportData) transaction; now a simple status update.

Web app UI

• instrument-runs.ts — Adds getProcessedCsvData() helper that fetches processed CSV files from S3 and parses them into RawWellRow[]. Removes getRunReportData() and RunReportEntry type.
• run-detail.ts — RunDetailProps.reportData replaced with wellData: RawWellRow[].
• Run detail page — Calls getProcessedCsvData(runFiles) instead of getRunReportData(run.id).
• plate-reader-run-detail.tsx — Refactored to consume wellData rows directly instead of RunReportEntry wrappers.
• default-run-detail.tsx — Removes reportData splitting logic.
• delete-run-dialog.tsx — hasReportData prop replaced with hasProcessedFiles, computed from processed file count.
• run-analysis-section.tsx — Simplified to a standalone "coming soon" section (no more analysisData prop).
• run-report-section.tsx — Simplified to show only processed images; removes RunReportEntry dependency.

MCP

• tools.ts — Removes the get_run_report_data tool registration.
• prompts.ts — Updates run_analysis and compare_runs prompts to use list_run_files + get_file_download_url.

Schema

• schema.ts — Deletes the runReportData table definition.
• drizzle/0004_keen_gorilla_man.sql — DROP TABLE "run_report_data" CASCADE.

Tests

• Lambda integration tests — SpectraMax tests verify processed CSV file records and S3 uploads instead of report_data; reprocess test verifies CSV file idempotency.
• Lambda/watcher conftest — Removes run_report_data from truncation lists.
• Web app integration tests — Removes report_data assertions from files, runs, and MCP tests.

Docs

• lambda.md, mcp.md, architecture.md, guides/adding-an-instrument.md — Updated to reflect the S3 CSV pattern.

Breaking changes

• The GET /api/v1/instruments/:instrumentId/runs/:runId response no longer includes the report_data field. Any external consumers reading report_data from this endpoint must switch to list_run_files to find processed CSVs and download them via presigned URLs.
• The get_run_report_data MCP tool is removed. MCP clients referencing it will get a tool-not-found error and should use list_run_files + get_file_download_url instead.
• The report_data field in PATCH /api/v1/files/:fileId request bodies is now ignored.
• Migration 0004 drops the run_report_data table — this is irreversible and must be run after all Lambda deployments are using the new CSV path.

Driveby changes

• docs/guides/adding-an-instrument.md — Renumbered sub-headings under Step 4 from ### 1 through ### 5 to ### 4.1 through ### 4.5 for correct nesting.
• docs/architecture.md — Added L -->|processed data| S3 arrow to the system overview diagram.

Testing

• Lambda integration tests pass (pytest -m integration in lambda/)
• Web app integration tests pass (npm test in web-app/)
• MCP protocol tests pass (tool list no longer includes get_run_report_data)
• make check-all passes (formatting, lint, type checks)
• Deploy to staging and verify a SpectraMax plate reader file processes end-to-end: raw .xls completes, processed CSV appears in S3 and as a file record, plate map grid renders on the run detail page
• Verify reprocessing a plate reader file replaces (not duplicates) the processed CSV
• Verify the delete dialog still requires confirmation for runs with processed files
• Run migration 0004 against staging DB and confirm the run_report_data table is dropped cleanly

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
data-hub	Ready	Preview, Comment	Apr 17, 2026 9:20pm