Skip to content

BACKLOG.md

Latest commit

 

History

History
82 lines (64 loc) · 7.76 KB

BACKLOG.md

File metadata and controls

82 lines (64 loc) · 7.76 KB
Raw

mySommelier — Backlog

AI-powered LCBO Vintages recommender. Builds your taste profile, ranks the bi-weekly Vintages drops by what you'd actually love, and updates itself as you rate the picks. Public site with weekly Saturday email.

Live: my-sommelier-rinalabs.vercel.app · Code: github.com/rinaLabs/mySommelier

Stack

  • Next.js 16 (App Router) + TypeScript, Tailwind v4 + shadcn/ui, deployed to Vercel
  • Supabase — Postgres + pgvector. Drizzle ORM owns the schema.
  • Gemini via Vercel AI SDK (@ai-sdk/google)
    • gemini-embedding-001 — wine + profile embeddings (3072-dim)
    • gemini-2.5-flash — onboarding extraction, "why this pick" rationales, the chat agent
  • Resend — weekly Saturday email
  • Data source: lcbo.com/graphql (LCBO's Adobe Commerce / Magento GraphQL endpoint). Per-release category UIDs surface ~100 fresh Vintages wines per drop. No scraping.

Public-repo posture (locked)

This repo is public. Implications baked into every commit:

  • No secrets in source. .env.local gitignored, .env.example committed with placeholders.
  • No personal data in code. Profiles live only in production DB. Forks get a clean slate via the onboarding form.
  • /p/{slug} is a generic public read route — no special-cased users.
  • LICENSE: MIT. README explains what it is, how to run it, architecture.

Shipped

The full retrieval + agent + feedback loop is live. Highlights:

  • Onboarding — 4-question form ("name, wines you've loved, wines you didn't, price ceiling"). Gemini extracts varietals / regions / specific wines from the prose; named wines are searched against the full LCBO catalogue and seeded as "loved" so the profile vector is anchored to real bottles.
  • Ranking — pgvector cosine over the embedded catalogue, MMR for varietal/colour diversity, sugar ceiling derived from the user's loved-wine sugar levels, region nudge when the user named regions in onboarding.
  • Profile page (/p/{slug}) — taste fingerprint card + 5 picks with one-line rationale + rate buttons + "your wines" gallery + email opt-in + chat.
  • Refinement chat — Gemini-2.5-flash agent with tools: rerank (structured filters), searchByVibe (free-text descriptors), findSimilar (anchor wine + modifier), explainPick, logWine (logs any LCBO wine, not just current picks).
  • Learning loop — loved/disliked ratings re-blend the profile vector; tried-wines roll into a gallery and feed recomputeProfileVector.
  • Weekly email — Resend + Vercel cron Saturday 8am ET. Idempotent multi-shot cron (commit ba4b9d0) survives a late LCBO drop. One-click unsubscribe + "tune your profile" deep link.
  • LCBO ingestion — bi-weekly sync against lcbo.com/graphql, embedding pipeline is idempotent (skip if unchanged).
  • Eval suite v1 — three personas with rule-based ground-truth rate() functions, NDCG@5/Recall@5, three coherence checks, varietal/colour-diversity sanity. Results serialize to evals/results/latest.json for trend tracking.
  • Eval suite v2 (in progress) — region-aware ranking, trace logging (TRACE_LOG=1 writes JSONL to evals/traces/), data-quality eval (bun run evals:data) covering field coverage, sanity bounds, embedding presence, nearest-neighbour sanity.

Known issues (from bun run evals:data)

  • varietal coverage at 84% of vintages wines (target: 95%). LCBO's GraphQL feed leaves the field null on ~16% of new releases. Downstream effect: the ranker can't reason about grape for those wines.
  • style coverage at 88% (target: 90%). Same root cause.
  • Fix: ingestion fallback that extracts varietal/style from name + shortDescription via Gemini when the structured field is null. See Eval roadmap → ingestion fallback below.

Eval roadmap

The eval suite is the case-study artifact. Hamel-style levels: L1 = deterministic assertions, L2 = trace review (human + LLM-as-judge), L3 = production A/B (out of scope pre-launch).

  • Category 1 — Onboarding faithfulness (L1). Assertion: every entry in statedTerms.varietals / regions / wines appears (substring or fuzzy) in the input prose. Catches Gemini hallucination during extraction. Material lives in evals/traces/onboarding.jsonl.
  • Category 2 — Recommendation accuracy (mixed).
    • L1 hard constraints: no pick exceeds price ceiling, no pick exceeds max(loved-sugar) + 2, no pick is a duplicate of a tried wine, no pick is is_buyable=false or stale.
    • L1 rationale faithfulness: every claim in rationaleFor output is grounded in the wine row.
    • L2 vibe match: pairwise LLM-as-judge ("set A vs set B for this profile, which is better?"). Validate the judge against ~25 hand-labeled pairs before trusting at scale.
  • Category 3 — Chat behavior (L1). Assert tool calls from evals/traces/chat.jsonl: "remove the rosé" → rerank with excludeVarietalIncludes: ['rosé']. "I tried Tawse Chardonnay, loved it" → logWine with the right name + rating. Deterministic, no LLM judge needed.
  • Category 4 — Feedback loop (L1). After a loved rating, assert cosine(post_profileVector, lovedWine.embedding) > cosine(pre_profileVector, lovedWine.embedding). Catches a quietly-broken recomputeProfileVector.
  • Ingestion fallback — when LCBO GraphQL omits varietal/style, fill via Gemini from name + shortDescription. Drives data-quality coverage above thresholds.
  • L2 trace review workflow — labelling UI or notebook over the evals/traces/*.jsonl files. Curate ~30 walkthroughs per category, hand-label, compute judge agreement.

Product roadmap

  • Photo upload (M3-5) — bottle label → Gemini 2.5 vision → SKU lookup → log + rank. Text fallback when OCR misses.
  • Preferred-store filter — onboarding adds local LCBO store(s). Ranking joins with inventory so we stop recommending wines users can't actually buy nearby.
  • Auth — open question. Slug-based public access has been fine for the demo. If we add real signups beyond the demo crowd, Supabase Auth (email OTP) + RLS on tried_wines is the path.
  • Compliance for public launch — privacy policy, ToS, 19+ age gate (Ontario), one-click unsubscribe (already shipped).
  • Multi-profile Venn — "pick a bottle for me + friends". Add friends by email; ranking computes a centroid of their profile vectors with a min-similarity-per-person floor so nobody gets stuck with something they'd hate. Genuinely interesting product surface — but it ships only once there are real friends with real profiles in the system. Faking it with seeded profiles would be theatre.

Permanently OUT (do not add)

  • Cellar management / drink-window tracking
  • Vintage variation reasoning
  • Restaurant wine-list OCR
  • Multi-region (no SAQ, BCL, US)
  • Native mobile apps
  • Affiliate / purchase facilitation (regulatory mess)

Locked decisions

  • Data source: lcbo.com/graphql (LCBO's own Adobe Commerce / Magento GraphQL). No scraper. Switched from api.lcbo.dev after that endpoint froze 2025-12-18.
  • Backend: Supabase (Postgres + pgvector + Storage). Drizzle owns the schema.
  • LLM provider: Gemini for both embeddings and generative. ~10× cheaper than OpenAI, free tier usable for forks, vision strength for future bottle-photo work. Vercel AI SDK abstracts the provider — eval suite can re-run against OpenAI to falsify the choice quantitatively.
  • Profile representation: 3072-dim embedding from Gemini, blended from onboarding text + loved/disliked feedback. No fine-tuning.
  • Vector index: brute-force scan over ~100–1,200 wines (sub-50ms). pgvector HNSW caps at 2,000 dims; if we outgrow this we'll switch to halfvec(3072) or Matryoshka-truncate to 1,536.
  • Email v1: Resend for transactional. No auth-emails today (no auth).