Skip to content

engineering.md

Latest commit

 

History

History
203 lines (159 loc) · 12.4 KB

engineering.md

File metadata and controls

203 lines (159 loc) · 12.4 KB
Raw

GlowUp — Engineering Design Doc

Author: Antigravity & User Status: Draft v0.1 Last updated: 2026-05-30 Reviewers: TBD

1. Summary

GlowUp is a lightweight, high-performance single-screen web application for virtual hairstyle try-ons. The application is built using a Python FastAPI backend and a vanilla HTML/CSS/JS frontend served directly by FastAPI, mimicking the design patterns from the @reference/moodjar-example.

The system takes a user-uploaded selfie and a hairstyle preset, and performs two tasks: it calls Google Vertex AI's Imagen 3 model (Image Editing / Inpainting) to perform a photorealistic hair swap, and calls Google Gemini 3.5 Flash with structured JSON output to generate a sassy stylist review and vibe rating. The frontend renders this with a buttery-smooth 60fps glassmorphism "before/after" comparison slider.

2. Assumptions

The technical architecture is built on the following constraints:

  • Target scale: <10k DAU. Designed to run on a single lightweight VM or serverless container (e.g., Cloud Run).
  • Latency budget: p95 <10 seconds. The core image-to-image API call is heavy; we must manage user expectations through a highly engaging loading state.
  • Platform: Responsive mobile-first web app, served from a single FastAPI URL.
  • Cost ceiling: $0.05 per try-on (Vertex AI Imagen 3 edit + Gemini 3.5 Flash call). We assume a budget cap or local testing keys in v1.
  • Out of scope: Multi-user synchronization, database storage for images (all uploads are short-lived in-memory streams or ephemeral local disk paths), image history, and native sharing APIs.

3. Goals & non-goals

Goals (v1):

  • Realistic Hair Edit: Perform seamless image-to-image hair editing using the selected preset prompt on the user's face.
  • Low-Friction UI: Zero login or setup required. Drop photo → click button → see result.
  • Interactive Sliding Comparison: A high-framerate vertical slider that overlays the "before" and "after" images, aligning them pixel-for-pixel with zero layout shifting.
  • Sassy Stylist Note: Dynamic structured text feedback generated via Gemini detailing a "vibe rating" and stylist commentary.
  • High-Performance Static Serving: FastAPI serving vanilla JS/CSS assets in <200ms.

Non-goals (v1):

  • User Accounts / Databases: We do not persist files long-term. Images are processed in-memory or saved in ephemeral temp directories and returned directly in the response payload.
  • Infinite Scalability: We will not set up distributed task queues (like Celery/Redis) in v1. The FastAPI request handlers will directly orchestrate the API calls.
  • Custom prompts: No text input from the user is allowed for the generation prompt to avoid prompt injection and poor-quality AI outputs.

4. Architecture

flowchart TD
  Client[Vanilla HTML/CSS/JS Client] -->|1. POST /api/try-on: file + preset| API[FastAPI Server]
  API -->|2. Inpaint/Edit Request: selfie + preset prompt| Imagen[Google Imagen 3 API]
  API -->|3. Structured Comment Request: preset| Gemini[Google Gemini 3.5 Flash]
  Imagen -->|Edited Image Bytes| API
  Gemini -->|Sassy JSON Comment| API
  API -->|4. Returns Edited Image base64 + Stylist JSON| Client
Loading

What's here:

  • FastAPI Core (backend/main.py): The single backend application that serves frontend files under / and routes /api/try-on.
  • AI Orchestrator (backend/ai_service.py): Handles authentication with Vertex AI and Google GenAI APIs, executes Imagen editing pipelines, and runs structured prompts against Gemini.
  • Vanilla JS Client (frontend/app.js & frontend/slider.js): Interactive frontend controller handling image uploads, slider clipping math (clip-path), and animating Sassoon's stylist card.

What's deliberately NOT here:

  • No Cloud Database (PostgreSQL/Firestore): No user data is stored.
  • No Cloud Storage Buckets (GCS/S3): Uploads are processed purely in memory as byte-streams, and edited images are returned back to the client as base64-encoded strings, avoiding storage overhead.
  • No Queueing System (Celery/RabbitMQ): Requests are handled synchronously. At <10k DAU, FastAPI's async event loop can handle concurrent waiting IO bounds easily.

5. Key components

FastAPI Application (backend/main.py)

  • Responsibility: API routing and static file hosting.
  • Tech choice: Python 3.11, FastAPI, Uvicorn.
  • Why this choice: Standard, highly efficient, async-native micro-framework.
  • Interface:
    • POST /api/try-on (Form data: image: File, preset: String) → returns {"image": "base64_string", "stylist_review": {...}}
    • GET /api/health → returns {"status": "glowing"}

AI Service (backend/ai_service.py)

  • Responsibility: Authenticating and invoking Google GenAI APIs.
  • Tech choice: google-genai Python SDK.
  • Why this choice: Official, modern SDK supporting structured JSON generation and Google Vertex AI Imagen models.
  • Interface:
    • async def edit_hairstyle(image_bytes: bytes, preset_id: str) -> bytes
    • async def generate_sassy_comment(preset_id: str) -> dict

Slider Component (frontend/slider.js)

  • Responsibility: buttery-smooth vertical dividing slider.
  • Tech choice: Vanilla JavaScript, requestAnimationFrame, CSS clipping masks.
  • Why this choice: Prevents layout shifting and handles high-frequency mouse/touch moves at native 60fps.
  • Interface:
    • initializeSlider(containerId: string): Binds dragging events to mousemove and touchmove, updating the CSS variable --clip-percent.

6. Data model

All session state lives in client-side memory or local memory. The backend has no persistent database. We define strict schemas for the API contract:

// Client-side representation of the complete API Response
type TryOnResponse = {
  editedImageBase64: string; // The transformed image as a base64 string
  stylistReview: {
    vibeRating: number;      // Sassy score from 0.0 to 10.0
    stylistNote: string;     // The custom dynamic critique text
    caption: string;         // A funny brief title (e.g., "Bangs or Bust")
  }
};

// Preset mapping configuration
type HairstylePreset = {
  id: "bangs" | "pixie" | "spikes" | "waves";
  label: string;
  imagenPrompt: string;      // The precise description sent to Imagen 3 Inpainting
  baseVibeScore: number;
};

Notes:

  • No PII Persistence: Selfie data is discarded immediately after the request finishes. No image files are saved to the server's filesystem, preventing data privacy leaks.

7. API surface

POST /api/try-on

  • Input: Multi-part Form Data
    • selfie: Binary file (PNG or JPEG, <5MB)
    • preset: String (must be one of: "bangs", "pixie", "spikes", "waves")
  • Output: JSON 
    {
  "image": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA...",
  "stylist_review": {
    "vibeRating": 9.5,
    "stylistNote": "Darling, these bangs were made for your cheekbones. Work.",
    "caption": "Bangs or Bust"
  }
}
  • Errors:
    • 400 Bad Request: Missing file, unsupported format, or invalid preset string. Client displays an elegant inline popup asking to upload a valid photo.
    • 422 Unprocessable Entity: AI model was unable to process the image (e.g., face not detected by Imagen). Client displays Sassoon's warning: "Sassoon couldn't find a clear face. Try another photo!"
    • 503 Service Unavailable: API timeout from Vertex AI / Google GenAI SDK. Client retries once or fails gracefully.
  • Latency budget: <10 seconds total. Budget allocation: 6.5s Imagen edit, 1.5s Gemini comment, 1s network transmission and frontend rendering.

8. Key trade-offs (with rejected alternatives)

Decision: Direct synchronous Base64 return vs Asynchronous URL generation with storage

  • Chose: Direct Base64 return in a synchronous request.
  • Considered: Uploading generated images to Google Cloud Storage (GCS) and returning a temporary signed URL.
  • Why we picked this: Storing photos introduces significant database, security, and cleanup overhead. By returning base64 directly and discarding uploaded bytes instantly, we make the app highly secure, private, and extremely cheap to run since we store absolutely nothing.

Decision: Direct Client-to-API Vertex calls vs Backend proxy server

  • Chose: Backend proxy server (FastAPI).
  • Considered: Direct calls from JS client to Vertex AI / Google GenAI API.
  • Why we picked this: Direct client calls would expose our Google API keys and GCP service accounts in the browser, allowing anyone to steal them and accumulate massive costs. The FastAPI backend acts as a secure, authenticated shield.

9. Risks & unknowns

  • Risk: Face editing artifacts — Likelihood: Medium. Inpainting sometimes deforms minor parts of the face or creates weird skin boundaries.
    • Mitigation: We design precise, structured positive and negative prompts for Imagen (e.g., inpaint only the hair bounding region, preserve original facial structure, no distortions).
  • Risk: Heavy Latency (>8s) — Likelihood: High. Generative image models are computationally expensive.
    • Mitigation: We make the wait entertaining! Sassoon's animated micro-interactions and witty stylist commentary keep the user engaged, masking the wait time.
  • Risk: Rate limits on Vertex AI — Likelihood: Low to Medium.
    • Mitigation: If rate limits are hit, backend returns a friendly HTTP 429 and the client asks the user to wait a moment.

10. Testing strategy

We use pytest for backend unit and integration testing. We deliberately mock external API calls to avoid flakes and costs.

Unit tests:

  • test_validate_image_format(): Verifies that only image/jpeg and image/png files are accepted, and files >5MB are rejected with an HTTP 400.
  • test_get_style_prompt(): Asserts that passing bangs returns the correct text description prompt, and passing an invalid preset ID raises a ValueError.
  • test_sassy_stylist_parsing(): Asserts that the JSON parser handles and successfully falls back if Gemini returns malformed JSON structure for the stylist commentary.

Integration tests (one per happy path):

  • test_try_on_success_flow(): Spins up a mock FastAPI client. Sends a valid mock image and preset to POST /api/try-on. Mocks the google-genai SDK response for both Imagen and Gemini. Asserts that the response is HTTP 200 and matches the TryOnResponse format exactly.

Deliberately not tested (and why):

  • Actual AI model output content: We do not unit-test whether the AI-generated comment is "sassy enough" or if the hair looks good, as this is purely qualitative.
  • Frontend mouse/touch dragging: Verifying the 60fps clip-path alignment is performed manually via visual inspection, avoiding complex and flaky Selenium/Playwright browser orchestration in v1.

11. Rollout & monitoring

  • Rollout: Serve the page using FastAPI's static directory router. Deploy on Google Cloud Run for automatic serverless scaling.
  • Monitoring: We monitor:
    • Latency of the Vertex/Imagen API calls (using Prometheus metrics or Cloud Logging).
    • Rate of HTTP 5xx errors (indicates Vertex API failure or backend crash).
    • Hit count of the try-on button to track user volume.
  • Rollback: Simply redeploy the previous stable Docker container on Cloud Run in <1 minute.

12. Cost & capacity

  • Per-user cost:
    • Imagen 3 Inpainting/Edit: ~$0.03 per image.
    • Gemini 3.5 Flash comment: ~$0.0001 per call (negligible).
    • Total: ~$0.0301 per hairstyle try-on.
  • Monthly budget at v1 scale (1,000 DAU, 3 try-ons per user per day):
    • 3,000 try-ons/day × 30 days = 90,000 generations/month.
    • Cost: 90,000 × $0.0301 = ~$2,709/month.
  • What breaks at 10× scale: Vertex AI quota limits. We would need to request a quota increase from Google Cloud support.

13. Open questions

  • Does Imagen 3 support precise automated masking of the hair region, or must we pass a calculated bounding box or pre-segmented mask? (Owner: Eng Team)
  • How does Imagen 3 handle highly diverse hairstyles (e.g., transitioning from extremely short buzzcut to voluminous long waves)? (Owner: Design & Eng Team)

14. Out of scope (will not do)

  • No image-to-image tuning sliders: Users cannot adjust strength, opacity, or mask boundaries.
  • No background replacement: Only the hair area is updated; the background of the original selfie is strictly preserved.
  • No native mobile app wrappers (Capacitor/React Native): Strictly a responsive web app.