feat(api): USCIS H-1B Employer Data Hub import, h1b_sponsors table, and sponsor_likelihood matching (#262)

[Taleef7]

Closes #262 (Epic #245 — Feed Engine).

Summary of Changes

1. Database Migration (db/migrations/015_h1b_sponsors.sql):

   • Creates global reference table h1b_sponsors with columns id, employer_name, employer_name_normalized, fiscal_year, approvals, denials, and created_at.
   • Adds unique index on (employer_name_normalized, fiscal_year) and lookup index on (employer_name_normalized).
   • Alters jobs.sponsor_likelihood to text to accommodate JSON-serialized { status: 'known_sponsor', approvals, denials } payloads.

2. Employer Normalization & Sponsorship Lookup (apps/api/src/lib/sponsorship.ts):

   • normalizeEmployerName(name): strips punctuation, maps & to and, normalizes whitespace/case, and repeatedly strips stacked trailing corporate suffixes (llc, inc, corp, holdings, group, etc.) while protecting suffix-only names (e.g. 'CO Inc' -> 'co').
   • lookupSponsorLikelihood(company): exact normalized lookup over the last 3 fiscal years present in h1b_sponsors ((select coalesce(max(fiscal_year), 0) from h1b_sponsors) - 2), returning { status: 'known_sponsor', approvals, denials } when positive filings exist. Fails closed (returns null) on missing pool, empty string, zero filings, or query error.

3. USCIS H-1B CSV Import Script (apps/api/scripts/import-h1b.ts):

   • Local CSV parsing with csv-parse/sync (BOM and relax column count handling).
   • Case-insensitive header matching for fiscal year, employer name, and approval/denial counts.
   • Aggregates multi-state rows per (employerNameNormalized, fiscalYear).
   • Batch upsert into h1b_sponsors (500 records per batch) with ON CONFLICT DO UPDATE.
   • Backfills existing jobs with sponsor_likelihood JSON payloads for matching companies.

4. Types & Job Stores Integration:

   • Extended SponsorLikelihood in apps/api/src/types.ts and apps/web/src/types/job.ts with KnownSponsorLikelihood.
   • Updated job-store.postgres.ts and in-memory store to serialize sponsorLikelihood on createJob and safely parse it on read.
   • In apps/api/src/lib/discovery.ts, insertIfNew enriches jobs with deps.lookupSponsor(job.company) and preserves pre-existing signals.
   • Wired lookupSponsor: lookupSponsorLikelihood into routes/discovery.ts.

5. Testing & Verification:

   • Unit tests in sponsorship.test.ts (normalization edge cases + mock pool queries) and import-h1b.test.ts (header resolution + row aggregation).
   • Integration test in discovery.test.ts verifying lookupSponsor enrichment.
   • 336 API unit tests pass. Full npm run check (monorepo lint + typecheck + Next.js build + API build) clean.

[Taleef7]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 681e9af736

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".