Skip to content
BranchesTags

Repository files navigation

AstraQ Logo

AstraQ Backend

Hybrid RAG + Knowledge Graph API for Satellite Data Intelligence

CI Python 3.11 FastAPI Neo4j Firebase License: MIT

The intelligent backend powering AstraQ — an AI assistant for ISRO's MOSDAC satellite data portal. Answers natural-language questions about Indian satellites and their data products by combining semantic document retrieval (RAG) with a Neo4j knowledge graph, synthesized by Google Gemini.

Architecture • Tech Stack • Getting Started • API Reference • Deployment • Contributing

Architecture

Query Processing Pipeline

flowchart TD
    Q[User Question] --> R{Intent Router<br/>Regex-based, no LLM cost}
    R -->|greeting / memory| CHAT[Deterministic Reply<br/>Zero API cost]
    R -->|glossary term| GLOSS[Glossary Lookup<br/>Cached]
    R -->|listing question| KG[Knowledge Graph<br/>Cypher Templates]
    R -->|explanation| RAG[FAISS Retrieval<br/>fastembed ONNX]
    R -->|entity + explanation| BOTH[KG + RAG Fusion]
    KG -->|no results| RAG
    RAG --> GEM{Gemini Synthesis<br/>Model Fallback Chain}
    BOTH --> GEM
    GEM -->|quota exhausted| FB[Deterministic Fallback<br/>Retrieved Snippets]
    GEM --> A[Answer + Sources + Citations]
    KG --> A
    CHAT --> A
    GLOSS --> A
Loading

System Design

flowchart LR
    subgraph Client
        FE[React SPA]
    end
    subgraph Backend["FastAPI Service"]
        MW[Middleware<br/>CORS + Security Headers + Request-ID]
        ROUTES[API Routers]
        SVC[Chat Service<br/>Intent Routing + Fusion]
        RL[Rate Limiter]
        AC[Answer Cache]
    end
    subgraph Data
        FS[(Firestore)]
        NEO[(Neo4j AuraDB)]
        FAISS[(FAISS Index)]
    end
    GEM[Gemini API]
    FBA[Firebase Auth]

    FE -->|Bearer Token| MW --> ROUTES --> SVC
    ROUTES --> RL
    SVC --> AC
    ROUTES -->|Verify Token| FBA
    SVC --> FS
    SVC --> NEO
    SVC --> FAISS
    SVC --> GEM
Loading

Design Principles

Principle Implementation
Cheap first Greetings, memory, and glossary terms never touch an LLM. Answers are cached so repeated questions cost nothing.
Graceful degradation If Neo4j is paused, KG questions fall back to RAG. If Gemini quota is exhausted, retrieved document snippets are returned directly.
Free-tier friendly Embeddings run via fastembed (ONNX runtime, no PyTorch), fitting the entire service in a 512 MB instance.
Lazy loading FAISS index and Gemini clients initialize on first use, not at import time. Boot succeeds with missing credentials.
Offline testable All 61 tests run without network access, credentials, or external services.

Tech Stack

Concern Technology
API Framework FastAPI + Uvicorn (ASGI)
Authentication Firebase Auth (Bearer ID tokens, server-side verification with revocation check)
Session Storage Google Cloud Firestore (users/{uid}/threads/{id}/messages)
Knowledge Graph Neo4j AuraDB (Satellite→Product→Parameter, Region, Payload, Algorithm)
Vector Search FAISS + all-MiniLM-L6-v2 via fastembed (ONNX, no PyTorch)
LLM Synthesis Google Gemini (configurable model fallback chain)
Rate Limiting In-memory sliding window (single-instance design)
Caching In-memory TTL cache for repeated queries
Containerization Docker (multi-stage build, ~180 MB image)
CI/CD GitHub Actions (lint, test, Docker build, secret scanning)

Getting Started

Prerequisites

  • Python 3.11+
  • Firebase project with Authentication + Firestore enabled
  • Neo4j AuraDB Free instance (optional — app starts degraded without it)
  • Google Gemini API key (optional — deterministic fallbacks work without it)

Installation

git clone https://github.com/a6hinandh/AstraQ_Backend.git
cd AstraQ_Backend

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

pip install -r requirements.txt -r requirements-dev.txt

Environment Configuration

cp .env.example .env
Variable Required Description
FIREBASE_SERVICE_ACCOUNT_B64 Yes (prod) Base64-encoded Firebase service account JSON
GEMINI_API_KEY Yes Google AI Studio API key
GEMINI_MODELS No Comma-separated model fallback chain (default: gemini-2.5-flash-lite,gemini-2.5-flash)
NEO4J_URI No neo4j+s://<id>.databases.neo4j.io
NEO4J_USERNAME / NEO4J_PASSWORD No AuraDB credentials
ENABLE_KG No true to enable knowledge graph features
AUTH_REQUIRED No false for local dev (skips token verification)
FRONTEND_ORIGINS Yes (prod) Comma-separated allowed CORS origins
CHAT_RATE_LIMIT_PER_MINUTE No Per-user rate limit (default: 20)
LOG_FORMAT No json (production) or text (local dev)

Running Locally

uvicorn backend.main:app --reload
# API: http://127.0.0.1:8000
# OpenAPI docs: http://127.0.0.1:8000/docs
# Health check: http://127.0.0.1:8000/health

The app boots without any credentials in degraded mode. Component status is reported at /health.

Running Tests

pytest                    # 61 tests, fully offline
ruff check backend rag_pipeline kg_pipeline tests

Tests use a FakeFirestore fixture and monkeypatched RAG/KG entry points — no network calls, no credentials required.

Pipeline Operations

# Rebuild the FAISS vector index (after corpus changes)
pip install -r requirements-pipeline.txt
python -m rag_pipeline.build_vector_index

# Populate the Neo4j knowledge graph
python kg_pipeline/populate_kg.py

The FAISS index ships with the repo (rag_pipeline/faiss_store/, ~13 MB) so RAG works immediately after clone.

API Reference

Base path: /api. Full interactive documentation at /docs (OpenAPI/Swagger UI).

Authentication

All endpoints require Authorization: Bearer <Firebase ID token> unless noted. With AUTH_REQUIRED=false (dev only), unauthenticated requests use a shared anonymous user.

Core Endpoints

Method Endpoint Auth Description
POST /chat Yes Main Q&A — returns answer, sources, mode, thread_id
POST /suggestions Yes AI-generated follow-up question chips
GET /threads Yes List all user threads
GET /thread/{id} Yes Full message history with citations
DELETE /threads/{id} Yes Delete thread and messages
POST /threads/{id}/favorite Yes Toggle favorite status
GET /threads/favorites Yes List favorited threads
POST /threads/{id}/share Yes Generate public share token
DELETE /threads/{id}/share Yes Revoke share link
GET /share/{token} No Public read-only conversation view
GET /kg/graph Yes Full knowledge graph (nodes + edges, capped at 200)
GET /kg/satellites Yes Satellite catalog with product lists
GET /kg/satellites/{name} Yes Satellite detail (products, payloads, metadata)
GET /search?q= Yes Personal message history search
GET /search/docs?q= Yes Semantic document corpus search
POST /upload Yes Text extraction from PDF/DOCX/TXT (max 10 MB)
GET/PATCH /user/me Yes User preferences
POST /feedback Yes Submit rating and feedback
GET /health No Component status (firebase, neo4j, faiss)
GET /health/ready No Readiness probe for orchestrators

Rate Limiting

Endpoints marked with rate limiting enforce a per-user sliding window. Responses include:

  • X-RateLimit-Limit — maximum requests per window
  • X-RateLimit-Remaining — remaining requests
  • 429 Too Many Requests with Retry-After header when exceeded

Error Responses

Standard FastAPI error shape: { "detail": "message" } with appropriate HTTP status codes (401, 404, 422, 429, 503).

Full API documentation: docs/API.md

Repository Structure

backend/
├── main.py                  # App factory, middleware, CORS, health probes
├── api/
│   ├── routes/              # One router per concern (chat, threads, kg, search, share, upload)
│   ├── schemas.py           # Pydantic request/response models
│   ├── router_logic.py      # Intent classification (regex-based)
│   └── deps.py              # Dependency injection (auth, rate limiting)
├── services/                # Chat orchestration, follow-up generation
├── auth/                    # Firebase token verification
├── session/                 # Firestore + Neo4j data access layer
├── rate_limit.py            # In-memory sliding window rate limiter
├── answer_cache.py          # In-memory TTL answer cache
├── domain_terms.py          # Satellite/product keyword vocabularies
└── logging_config.py        # Structured JSON logging with request-id

rag_pipeline/
├── build_vector_index.py    # Corpus → FAISS index builder
├── faiss_store/             # Pre-built FAISS index (committed, ~13 MB)
└── retrieve.py              # Query expansion → retrieval → Gemini synthesis

kg_pipeline/
├── populate_kg.py           # JSON → Neo4j graph population
├── kg_nl_demo.py            # NL → Cypher translation + execution
└── queries.py               # Cypher template library

static_pipeline/             # MOSDAC scraping & parsing (offline, not deployed)
├── crawlers/                # Web crawlers for MOSDAC portal
├── parsers/                 # PDF/DOCX/HTML parsing
└── output/                  # Parsed corpus (source for FAISS index)

tests/                       # 61 offline tests (FakeFirestore, monkeypatched externals)
docs/                        # Architecture, API reference, deployment guide

Deployment

Docker (Render Web Service)

The recommended deployment target is Render's Docker web service:

  1. Connect repo → New Web Service (Docker runtime)
  2. Set environment variables (see table above)
  3. Health check path: /health/ready
  4. Instance: Free tier (512 MB) is sufficient

Full step-by-step guide: docs/DEPLOYMENT.md

Operational Characteristics

Aspect Details
Cold start ~50s after 15 min idle (free tier). Frontend shows warm-up banner.
Memory Fits in 512 MB — fastembed ONNX, no PyTorch/spaCy
Scaling Single-instance by design (in-memory cache/limiter). Swap for Redis to scale horizontally.
Neo4j pauses AuraDB Free pauses after ~3 idle days. KG falls back to RAG; resume from console.
Observability Structured JSON logs with request-id propagation via X-Request-ID header

Architecture Deep-Dive

See docs/ARCHITECTURE.md for:

  • Complete request lifecycle (POST /api/chat)
  • Firestore data model
  • Neo4j schema
  • FAISS index specifications
  • Operational design decisions and rationale

Security

  • Authentication: Firebase ID tokens verified server-side with revocation checking on every request
  • Authorization: All user data (threads, messages, preferences) is scoped by Firebase Auth UID
  • Secrets management: Service account keys are never in the Docker image; injected via FIREBASE_SERVICE_ACCOUNT_B64 at runtime
  • Rate limiting: Per-user sliding window prevents abuse (configurable via environment)
  • CORS: Restricted to explicitly configured frontend origins
  • Input validation: Pydantic models enforce type safety; message length capped at 4000 chars; file uploads capped at 10 MB
  • Prompt injection guard: Retrieved RAG context is sanitized before LLM synthesis
  • No PII in logs: Request-id tracing without user content in structured logs

Contributing

See CONTRIBUTING.md for development setup, coding standards, and PR requirements.

License

This project is licensed under the MIT License — see the LICENSE file for details.

Acknowledgments

  • ISRO / MOSDAC — Source satellite data documentation, product specifications, and ATBDs
  • AstraMind — Student-led team of data engineers, AI researchers, and space enthusiasts
  • Built with FastAPI, Firebase, Neo4j, and the open-source Python ecosystem

About

Hybrid RAG + Knowledge Graph API for intelligent access to ISRO's MOSDAC satellite data. Combines FAISS vector search, Neo4j knowledge graph, and Google Gemini synthesis behind a FastAPI service with Firebase auth.

astraq-amb.onrender.com

Topics

ai-assistantsdockerfaissfastapifirebasegeminiisroknowledge-graphmosdacneo4jnlppythonragsatellite-datavector-search

Resources

Readme
License

Contributing

Contributing

Security policy

Security policy
Activity

Stars

0 stars

Watchers

0 watching

Forks

1 fork
Report repository

Releases

Packages

Contributors

Languages