Skill Spiderweb RAG & Hackathon Candidate Discovery Ranker

This repository contains:

1. AI Recruiter RAG & Relation spiderweb dashboard (Next.js & FastAPI backend).
2. Redrob Hackathon Candidate Discovery Ranker (CLI tool reproducing candidate matching).

---

🚀 Hackathon Submission Replication Command

Organizers can run our two-stage ranking pipeline end-to-end with the following command:

python run_challenge.py --candidates ./data/candidates.jsonl --out ./data/challenge_submission.csv

Full verification (rank → validate → honeypot-in-top-100 audit → optional determinism check):

python verify_submission_full.py --candidates ./data/candidates.jsonl --check-determinism

Or via shell script:

bash scripts/verify_submission.sh --check-determinism

For Organizers

Check	How to verify
One-command replication	python run_challenge.py --candidates ./data/candidates.jsonl --out ./data/challenge_submission.csv
Format validation	python validate_submission.py ./data/challenge_submission.csv
Honeypot defense	python verify_submission_full.py --skip-run (audits top-100)
Determinism	python verify_submission_full.py --check-determinism
Docker sandbox	docker build -t team-ranker . then mount /data volume

Architecture: Stage 1 streams 100k JSONL and keeps top 2,000 by keyword heap; Stage 2 applies honeypot/consulting filters, technical fit × behavioral multiplier, deterministic candidate_id tie-break, and profile-grounded reasoning.

Constraints Satisfied:

• Time Limit: Completes in ~5–6 seconds wall-clock on the 100,000-candidate pool (organizer sandbox allows 5 min).
• Memory: Uses < 20 MB RAM due to streaming.
• Offline / CPU-only: No network requests or GPU requirements during execution.
• Determinism: Equal scores are tie-broken alphabetically using candidate_id ascending.
• Trap Defenses: 0% honeypot rates in the top-100 output.

---

🛠️ Setup Instructions

Prerequisites

• Python 3.10+
• Node.js 18+

Install Dependencies

Create a virtual environment and install the required modules:

python -m venv venv
venv\Scripts\activate   # On Windows
pip install -r requirements.txt

Link Challenge Data Folder

Create a directory junction pointing to the official challenge downloads directory:

# On Windows cmd
mklink /j challenge_assets "C:\Path\To\India_runs_data_and_ai_challenge"

---

🐳 Docker Sandbox Replication (Optional)

You can build and run the pipeline in a CPU-locked, offline container to verify reproducibility:

1. Build Container:

   docker build -t team-ranker .

2. Execute Ranking: Mount your local data directory and run the container:

   docker run --rm -v "C:\Users\rentk\Downloads\semantic-job-tracker\data:/data" team-ranker --candidates /data/candidates.jsonl --out /data/challenge_submission.csv

---

🖥️ Running the Application Dashboard (Local Sandbox)

Our visual sandbox dashboard allows you to execute the ranker, view logs live, inspect honeypot counts, and browse the validated ranked candidate list in a premium dark-slate UI.

1. Start FastAPI Backend:

   venv\Scripts\python.exe -m src.main

   (Runs on http://127.0.0.1:8000)

2. Start Next.js Frontend:

   npm run dev -- -p 3001

   (Runs on http://127.0.0.1:3001)

3. Open http://localhost:3001 in your browser and switch to the "🏆 Hackathon Arena" tab.

---

🔬 Technical Overview (How it Works)

We utilize a Two-Stage Ingestion Pipeline to score 100,000 candidate profiles under 5 seconds:

Stage 1: High-Speed Scanning (~3 seconds)

Streams the JSON-Lines candidate file line-by-line (avoiding reading 500MB into memory at once). A fast keyword calculator matches core tech skills (like vector, embeddings, retrieval) to index and extract the top 2,000 candidates in a min-heap.

Stage 2: Deep Screening & Scoring (~0.5 seconds)

1. Honeypot Excluder: Rejects profiles displaying logical contradictions:
   • Stated experience years $<$ 3 but title contains "Principal", "Director", "Lead", or "Architect".
   • Expected salary range is negative or invalid.
   • Stated notice period is anomalous or $>$ 180 days.
   • Impossible skill duration (skill months exceeding total experience months).
   • Keyword stuffing (advanced/expert proficiency with 0 months used).
   • Worked at a company before its founded year.
2. IT Services Blacklist: Rejects candidates who have strictly worked at IT services firms (TCS, Infosys, Wipro, etc.) without product company experience.
3. Technical Fit Score (70%): Evaluates experience years (5–9 target) and specific vector search (Pinecone, Milvus, Qdrant, FAISS) and eval framework competencies.
4. Behavioral Signal Score (30%): Applies multipliers based on notice period, login freshness, recruiter response rate, and location (Noida/Pune or Tier-1 relocation willing).
5. Deterministic Tie-breaking: Sorts descending by final score, breaking ties alphabetically by candidate_id ascending.
6. Descriptive Reasoning: Formulates 1-2 sentence profiles using real fields to avoid template patterns.

---

Quick Start — Run Backend & Frontend

Terminal 1 — Backend

venv\Scripts\activate
venv\Scripts\python.exe -m src.main

Backend runs at http://127.0.0.1:8000

Terminal 2 — Frontend

npm install
npm run dev -- -p 3001

Frontend runs at http://localhost:3001

Open http://localhost:3001 in your browser.

First time only — install Python deps:

python -m venv venv
venv\Scripts\activate
pip install -r requirements.txt

Load the 100k candidate pool (optional, for the spiderweb graph):

venv\Scripts\python.exe ingest_candidates_pool.py

Then go to Skill Spiderweb Graph tab, type a search query, and click Filter Graph.

---

Future Work (Post-Hackathon)

Applicant Resume Ranker tab (deferred): paste a job description, upload 1–3 PDF resumes, receive a 0–100 fit score and version comparison. This targets job seekers (YouTube masterclass flow), not the recruiter/hackathon ranker built here.