Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
e11a209
add kgrag layer 4
Feb 2, 2026
7dfd58d
fix some formatting issue
Feb 2, 2026
ec5e2d6
Layer 3 of knowledge graph
Mar 5, 2026
bd107db
layer 2 of the project
Mar 5, 2026
bebc66b
layer 1 of kgrag
Mar 5, 2026
4e30946
Fill in the most important functions of KGRag
Mar 8, 2026
36457f3
Fill in the core missing functions
Mar 8, 2026
411ba59
Finished the less important functions
Mar 8, 2026
1ebf39c
Adding testing files
Mar 8, 2026
44825be
Fix issues found in the testing
Mar 8, 2026
42964ec
Implement Phase 2: 8 runnable scripts for KG-RAG pipeline
Mar 8, 2026
6e119ba
Extract common functions from the scripts
Mar 8, 2026
35919aa
Add test function for the utility files
Mar 8, 2026
aa4fa5d
Add the final run.sh script so that one can call it directly.
Mar 9, 2026
7be0ce1
Add two read me files for the library and the example
Mar 9, 2026
4e994c9
fix the issue so that run.sh is runable. However further update are e…
Mar 10, 2026
3cf11b4
Rewrite preprocessing and embedding scripts for mellea-style pipeline
Mar 11, 2026
b130db0
Enhance run_kg_update.py to match mellea's architecture and functiona…
Mar 11, 2026
eb017a3
Add comprehensive KG-RAG pipeline architecture documentation
Mar 11, 2026
60073db
Add run_kg_update.py as Step 3 to run.sh pipeline
Mar 11, 2026
d8a603d
Enable run_kg_update.py to load LLM/Neo4j config from .env file
Mar 11, 2026
e40da15
Fix run_kg_update.py to properly use RITS model from environment
Mar 11, 2026
272de0d
Update KG-RAG example README with three-stage pipeline documentation
Mar 11, 2026
2b36a1e
Remove large data files from git tracking (keep locally)
Mar 11, 2026
fca3a15
Add dataset directory documentation for large data files
Mar 11, 2026
523b244
update the run.sh for step 0
Mar 12, 2026
6e08755
Fix the run_kg_update and run_qa
Mar 16, 2026
6c70efe
fix the run_eval.py and readme
Mar 16, 2026
fe0f99e
remove some intermediate files
Mar 16, 2026
e025a6e
remove the data from git
Mar 16, 2026
2ad1b85
Refactor the scripts to better utilize the library
Mar 18, 2026
8d75eab
Address some of the issue from comments
Mar 18, 2026
427f644
Additional changes found during review
Mar 18, 2026
f3af8ba
Address additional issues of the comments from the PR
Mar 18, 2026
7a15435
Fix the embeder after refactoring
Mar 18, 2026
6125e56
Update embedder
Mar 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -443,3 +443,11 @@ pyrightconfig.json
.ionide

# End of https://www.toptal.com/developers/gitignore/api/python,direnv,visualstudiocode,pycharm,macos,jetbrains

# KG-RAG Example Data (Large Files - Keep Locally)
docs/examples/kgrag/.env
docs/examples/kgrag/dataset/*.jsonl.bz2
docs/examples/kgrag/dataset/movie/*.json
docs/examples/kgrag/dataset/movie/*.bz2
docs/examples/kgrag/output/
docs/examples/kgrag/data/
137 changes: 137 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

**mellea-contribs** is an incubation repository for contributions to the Mellea ecosystem. It provides a library for incubating generative programming tools and utilities that integrate with the Mellea LLM agent framework.

- **Tech Stack**: Python 3.10+, PDM build system, Mellea framework with LiteLLM
- **Key Dependencies**: rapidfuzz (fuzzy matching), eyecite (legal citations), playwright (web scraping), markdown
- **License**: Apache License 2.0

## Common Development Commands

### Setup and Installation

```bash
# Install with development dependencies
pdm install --group dev

# Or using uv (faster):
uv pip install -e . --group dev
```

### Code Quality

```bash
# Format code (ruff is used for both formatting and linting)
ruff format .
ruff check . --fix

# Type checking
mypy mellea_contribs/

# Run all linters individually
isort mellea_contribs/ test/
pylint mellea_contribs/ test/
ruff check mellea_contribs/ test/

# Run pre-commit hooks
pre-commit run --all-files
```

### Testing

```bash
# Run all tests
pytest

# Run a specific test file
pytest test/test_citation_exists.py

# Run a specific test function
pytest test/test_citation_exists.py::test_function_name

# Run tests with verbose output
pytest -v

# Run only tests marked as qualitative (LLM-dependent)
pytest -m qualitative

# Run tests excluding qualitative tests (useful in CI)
pytest -m "not qualitative"

# Run tests with asyncio verbose output (asyncio_mode is auto-configured)
pytest -v --asyncio-mode=auto
```

### Documentation

```bash
# Build Sphinx documentation
sphinx-build -b html docs/ docs/_build/
```

## Architecture Overview

### Module Organization

**`mellea_contribs/tools/`** - Reusable LLM-based selection and ranking algorithms:
- `top_k.py`: Generic Top-K selection engine using LLM judgment. Selects top K from N items with rejection sampling and caching.
- `double_round_robin.py`: Pairwise comparison scoring engine. Performs all-pair comparisons and returns items ranked by accumulated scores.

**`mellea_contribs/reqlib/`** - Domain-specific validators extending Mellea's validation framework:
- `citation_exists.py`: Validates legal case citations via case.law metadata + fuzzy matching
- `is_appellate_case.py`: Classifies cases as appellate by court abbreviation patterns
- `import_repair.py`: Fixes Python import errors in LLM-generated code via AST analysis
- `import_resolution.py`: Parses and resolves module not found / import errors with confidence-scored suggestions
- `grounding_context_formatter.py`: Structures multi-field context for LLM prompts (auto-skips empty fields)
- `common_aliases.py`: Module name mappings and relocations for import resolution
- `statute_data.py`: Legal statute data handling

**`mellea_contribs/kg/`** - Knowledge Graph database abstraction:
- `base.py`: Core data structures (GraphNode, GraphEdge, GraphPath)
- `graph_dbs/base.py`: GraphBackend abstract interface
- `graph_dbs/neo4j.py`: Production Neo4j implementation (requires [kg] dependencies)
- `graph_dbs/mock.py`: In-memory mock backend for testing without infrastructure
- `components/`: Query, result, traversal components (minimal Layer 4 implementations)

**Installation**: `pip install mellea-contribs[kg]` for Neo4j support

### Design Patterns

**Mellea Integration Pattern**: All validators are Requirement classes with a `validation_fn(output) → ValidationResult`. This enables iterative LLM refinement via the Instruct-Validate-Repair loop.

**Caching Strategy**: Tools use decorator-based caching keyed on item hash + context + prompts to avoid redundant LLM calls.

**Model Interaction**: Uses `mellea.instruct()` with grounding context, system prompts for output formatting (JSON arrays, single tokens), and rejection sampling (loop_budget=2) for reliability.

**Data Validation Layers**:
- Legal citations: Direct lookup → fuzzy match → LLM judgment
- Python imports: Static AST analysis + dynamic error parsing
- Court classification: Pattern matching + database lookup

### Test Infrastructure

- Large test databases: `test/data/citation_exists_database.json` (~2.8MB)
- Qualitative test markers for LLM-dependent tests (auto-xfail in CI when `MELLEA_SKIP_QUALITATIVE` env var is set)
- Neo4j integration tests: Marked with `@pytest.mark.neo4j`, skipped unless NEO4J_URI is set
- Separate database for CI efficiency
- Shared fixtures via `test/conftest.py`
- Async-first testing: `asyncio_mode = "auto"` (no explicit marking needed)

## Code Quality Standards

- **Docstrings**: Google-style convention (enforced via ruff rule D)
- **Complexity**: Maximum cyclomatic complexity of 20 (ruff C901)
- **Type Hints**: Full type annotation with mypy checking enabled
- **Imports**: isort enforced for consistent ordering with `combine-as-imports`
- **Pre-commit hooks**: Automatic formatting and validation on commit

## Release and Versioning

- **Semantic Versioning**: Angular commit parser (feat → minor, fix → patch)
- **Automated Releases**: python-semantic-release on main branch
- **CI/CD**: GitHub Actions workflows (ci.yml, cd.yml, pypi.yml)
- **PyPI Publishing**: Automated on semantic version tags
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
<img src="https://github.com/generative-computing/mellea-contribs/raw/main/mellea-contribs.jpg" height=100>


# Mellea Contribs

The `mellea-contribs` repository is an incubation point for contributions to
Expand Down
52 changes: 52 additions & 0 deletions docs/examples/kgrag/.env_template
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .env_template should be cleaned. Do not try to make it RITS specific. Make it LLM server ergodic. Especially, make it work for openai, ollama, etc.

Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Graph Database Configuration (Neo4j default; adjust for other graph DBs)
NEO4J_URI=bolt://localhost:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=password

# Data Directory
KG_BASE_DIRECTORY=./dataset
DATA_PATH=./data

# ---------------------------------------------------------------------------
# Primary LLM — any OpenAI-compatible endpoint
# ---------------------------------------------------------------------------
# Option A: OpenAI
# API_KEY=sk-...
# MODEL_NAME=gpt-4o-mini

# Option B: Local Ollama
# API_BASE=http://localhost:11434/v1
# API_KEY=ollama
# MODEL_NAME=llama3.2

# Option C: vLLM / self-hosted
# API_BASE=http://localhost:8000/v1
# API_KEY=dummy
# MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct

# Option D: Azure OpenAI
# API_BASE=https://<your-resource>.openai.azure.com/openai/deployments/<deployment>/
# API_KEY=<azure-key>
# MODEL_NAME=gpt-4o-mini

# ---------------------------------------------------------------------------
# Optional: Separate evaluation model (defaults to primary LLM if unset)
# ---------------------------------------------------------------------------
# EVAL_API_BASE=...
# EVAL_API_KEY=...
# EVAL_MODEL_NAME=...

# ---------------------------------------------------------------------------
# Optional: Embedding model for vector entity alignment
# ---------------------------------------------------------------------------
# EMB_API_BASE=http://localhost:11434/v1
# EMB_API_KEY=ollama
# EMB_MODEL_NAME=nomic-embed-text
# VECTOR_DIMENSIONS=768

# Request Configuration
MAX_RETRIES=3
TIME_OUT=1800

# OpenTelemetry — disable if you don't have a collector running
OTEL_SDK_DISABLED=true
Binary file added docs/examples/kgrag/GraphRag.drawio.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading