Vakira Content Pipeline

Layer 1 of Vakira's 3-layer content architecture. Generates and maintains the question bank that powers Lingua — completely independent from the main app.

---

Architecture Overview

Layer 1: Content Pipeline (this folder)
   ↓ writes questions
Layer 2: Supabase (content_pool table)
   ↓ serves questions instantly
Layer 3: Vakira App (backend/)

Key principle: Layer 1 and Layer 3 never talk directly. Only communication is through the database. This means:

• Pipeline can go down without affecting users
• App can restart without affecting pipeline
• Either can be scaled or replaced independently

---

What It Does

• Generates 300 questions per topic (100 per difficulty tier)
• Covers all 4 skills × 5 levels × 5 topics = 100 topic combinations
• Total: ~30,000 questions for $3.60 one-time cost
• Runs nightly via GitHub Actions to refill low-stock combinations
• Validates every question before storing (no bad questions reach users)
• Tags each question with content traits (not exam names) for personalization

---

Folder Structure

content-pipeline/
├── README.md                 ← you are here
├── pipeline.py               ← main runner (entry point)
├── config.py                 ← settings (reads shared .env)
├── requirements.txt          ← minimal deps (openai + supabase only)
└── generators/
    ├── __init__.py
    ├── lingua_gen.py         ← question generation logic
    ├── prompts.py            ← LLM prompts + trait tagging
    └── validators.py         ← quality checks before DB insert

---

Setup

1. Install dependencies

cd content-pipeline
pip install -r requirements.txt

2. Environment variables

Uses the same .env as the main app (in repo root):

SUPABASE_URL=https://xxx.supabase.co
SUPABASE_SERVICE_KEY=eyJ...
OPENAI_API_KEY=sk-...

# Optional: override default model (default: gpt-4o-mini)
PIPELINE_MODEL=gpt-4o-mini

3. Run Supabase migration

Run supabase/migrations/content_pool.sql in your Supabase SQL Editor before running the pipeline.

This creates:

• content_pool — the question bank
• user_exercise_log — per-user answer tracking + spaced repetition
• user_topic_mastery — computed progress per topic
• user_content_traits — personalization traits
• pipeline_run_log — audit trail of all pipeline runs

---

Usage

Seed (first time — generates everything)

# Generate all skills (~20-30 minutes, ~$3.60)
python pipeline.py seed

# Generate one skill only (faster for testing)
python pipeline.py seed --skill grammar
python pipeline.py seed --skill vocabulary
python pipeline.py seed --skill pronunciation
python pipeline.py seed --skill storytelling

Check stock levels

python pipeline.py stats

Output:

[10:30:00] 📊 Current stock levels:
[10:30:00]   grammar:       7,500 questions, 0 low-stock combinations
[10:30:00]   vocabulary:    7,500 questions, 0 low-stock combinations
[10:30:00]   pronunciation: 7,500 questions, 0 low-stock combinations
[10:30:00]   storytelling:  7,500 questions, 0 low-stock combinations
[10:30:00]
[10:30:00]   TOTAL: 30,000 questions across 400 combinations

Refill low-stock combinations

# Runs automatically nightly via GitHub Actions
# Or manually:
python pipeline.py refill

Refill triggers when any combination drops below REFILL_THRESHOLD (default: 20 questions).

Quality check

python pipeline.py quality

Shows questions auto-flagged by the system:

• too_easy — >98% users get it right (after 20+ attempts)
• too_hard — <5% users get it right (after 20+ attempts)

---

GitHub Actions (Automated Nightly Refill)

File location: .github/workflows/content-pipeline.yml

Runs at 2:00 AM IST every night in refill mode.

Setup secrets

Go to: GitHub Repo → Settings → Secrets and variables → Actions

Add these secrets:

Secret	Value
SUPABASE_URL	Your Supabase project URL
SUPABASE_SERVICE_KEY	Service role key (not anon key)
OPENAI_API_KEY	OpenAI API key

Manual trigger

GitHub → Actions tab → "Vakira Content Pipeline" → Run workflow

Choose mode:

• refill — top up low-stock (default, safe to run anytime)
• seed — generate everything from scratch
• stats — just show stock levels
• quality — show flagged questions

---

Content Design

No hardcoded exam names

Questions use trait tags instead of exam names:

["error_spotting", "competitive", "formal", "subject_verb"]
["fill_blank", "conversational", "articles", "grammar"]
["reading_comp", "competitive", "inference", "formal"]
["idioms", "competitive", "vocabulary"]

This means:

• Same question can serve SSC, Bank PO, UPSC aspirants
• Works for international users too
• No code change needed when new exams are added
• User personalization is trait-based (not exam-based)

Difficulty tiers

Tier	Description	Target accuracy
1	Easy — basic rules, common vocabulary	75-90%
2	Medium — requires rule understanding	50-75%
3	Hard — competitive exam level, tricky options	30-55%

Auto-reclassification: after 20+ user attempts, questions outside their target range are flagged for review.

Indian context

All questions use:

• Indian names (Rahul, Priya, Amit, Sneha)
• Indian locations (Mumbai office, Bangalore tech park, Delhi market)
• Indian professional contexts (IT company, government office, bank)
• Common Indian English errors as wrong options
• "Indian learners often..." in explanations

---

Cost

Item	Cost
Seed (30,000 questions, one-time)	~$3.60
Nightly refill (typical)	~$0.05
Monthly maintenance	~$0.50
Per user served (amortized at 1000 users)	~$0.004

Model: gpt-4o-mini (cheapest OpenAI model, good quality for exercises)

---

Spaced Repetition (SM-2 Algorithm)

The app tracks each user's answer and schedules reviews using SM-2:

Performance	Next review
Wrong	1 day
Right (1st time)	1 day
Right (2nd time)	6 days
Right (3rd time)	~13 days
Right (4th time)	~25 days
Mastered (5+ correct)	Retired from rotation

This is the same algorithm used by Anki — proven to maximize retention.

---

Adaptive Difficulty

Tier selection is automatic based on recent performance:

Accuracy > 80% → Tier 3 (hard)
Accuracy 55-80% → Tier 2 (medium)
Accuracy < 55% → Tier 1 (easy)

Recalculated every session. User is always in their optimal challenge zone.

---

Adding New Content

Add a new topic

1. Add topic to CURRICULUM in lingua_gen.py
2. Also add to backend/lingua.py (must stay in sync)
3. Run: python pipeline.py seed --skill grammar
4. New questions available immediately

Add a new skill

1. Add curriculum to CURRICULUM dict in lingua_gen.py
2. Add generator function if needed
3. Run seed for that skill
4. Update frontend to show new skill

Change generation model

# In .env
PIPELINE_MODEL=gpt-4o  # use stronger model for quality

---

Monitoring

Every pipeline run is logged to pipeline_run_log table:

SELECT run_type, skill, questions_generated, questions_failed,
       cost_usd, duration_sec, status, started_at
FROM pipeline_run_log
ORDER BY started_at DESC
LIMIT 20;

Stock levels:

SELECT * FROM get_stock_levels()
WHERE count < 20
ORDER BY count ASC;

Flagged questions:

SELECT skill, topic_id, quality_flag_reason, global_accuracy, total_attempts
FROM content_pool
WHERE quality_flagged = TRUE
ORDER BY total_attempts DESC;

---

FAQ

Q: What if the pipeline fails mid-run? A: Already-inserted questions are safe (committed to DB). Re-run the same command — it checks existing stock and only generates what's missing.

Q: What if OpenAI is down? A: App falls back to LLM on-demand (current behavior). Users are unaffected. Pipeline retries next night.

Q: Can I run this on my laptop? A: Yes. python pipeline.py seed --skill grammar works from any machine with the .env file.

Q: What if a question is wrong/misleading? A: Flag it manually in Supabase: UPDATE content_pool SET quality_flagged=TRUE WHERE id='...'. It will never be shown to users again.

Q: How do I regenerate flagged questions? A: The refill mode automatically generates replacements for flagged questions that drop stock below threshold.