diff --git a/README.md b/README.md index 317804712..c9a6e22c8 100644 --- a/README.md +++ b/README.md @@ -5,15 +5,22 @@
+

+ Package version + + License - MIT + +

[Quickstart](#quickstart) • [Main Features](#main-features) • [MCP Server](#mcp-server) • +[How it works](#how-it-works) • [Benchmarks](#benchmarks)
-Semble is a fast code search library for local and remote repositories. It combines static [Model2Vec](https://github.com/MinishLab/model2vec) embeddings using [potion-code-16M](https://huggingface.co/minishlab/potion-code-16M) with [BM25](https://github.com/xhluca/bm25s) and a specialized hybrid reranking stack to deliver near-transformer accuracy at a fraction of the cost. As an [MCP server](#mcp-server), it gives agents (Claude Code, Cursor, Codex, OpenCode, etc.) instant access to any codebase: repos are cloned and indexed on demand. +Semble is a code search library built for agents. It returns the exact code snippets they need instantly, cutting both token usage and waiting time on every step. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Run it as an [MCP server](#mcp-server) and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo, cloned and indexed on demand. ## Quickstart @@ -31,26 +38,27 @@ index = SembleIndex.from_path("./my-project") # Index a remote git repository index = SembleIndex.from_git("https://github.com/MinishLab/model2vec") +# Search the index with a natural-language or code query results = index.search("save model to disk", top_k=3) +# Find code similar to a specific result +related = index.find_related(results[0], top_k=3) + # Each result exposes the matched chunk result = results[0] result.chunk.file_path # "model2vec/model.py" result.chunk.start_line # 127 result.chunk.end_line # 150 result.chunk.content # "def save_pretrained(self, path: PathLike, ..." - -# Find code similar to a specific result -related = index.find_related(results[0], top_k=3) ``` ## Main Features - **Fast**: indexes a repo in ~250 ms and answers queries in ~1.5 ms, all on CPU. -- **Accurate**: NDCG@10 of 0.854 on our benchmarks, on par with code-specialized transformer models, at a fraction of the size and cost. +- **Accurate**: NDCG@10 of 0.854 on our [benchmarks](#benchmarks), on par with code-specialized transformer models, at a fraction of the size and cost. - **Local and remote**: pass a local path or a git URL. - **MCP server**: drop-in tool for Claude Code, Cursor, Codex, OpenCode, and any other MCP-compatible agent. -- **Lightweight**: CPU-only, minimal dependencies. +- **Zero setup**: runs on CPU with no API keys, GPU, or external services required. ## MCP Server @@ -104,21 +112,41 @@ Add to `~/.cursor/mcp.json` (or `.cursor/mcp.json` in your project): | `search` | Search a codebase with a natural-language or code query. Pass `repo` as a git URL or local path. | | `find_related` | Given a file path and line number, return chunks semantically similar to the code at that location. | +## How it works + +Semble splits each file into code-aware chunks using [Chonkie](https://github.com/chonkie-inc/chonkie), then scores every query against the chunks with two complementary retrievers: static [Model2Vec](https://github.com/MinishLab/model2vec) embeddings using the code-specialized [potion-code-16M](https://huggingface.co/minishlab/potion-code-16M) model for semantic similarity, and [BM25](https://github.com/xhluca/bm25s) for lexical matches on identifiers and API names. The two score lists are fused with Reciprocal Rank Fusion (RRF). + +After fusing, results are reranked with a set of code-aware signals: + +
+Ranking signals + +- **Adaptive weighting.** Symbol-like queries (`Foo::bar`, `_private`, `getUserById`) get more lexical weight, while natural-language queries stay balanced between semantic and lexical retrievers. +- **Definition boosts.** A chunk that defines the queried symbol (a `class`, `def`, `func`, etc.) is ranked above chunks that merely reference it. +- **Identifier stems.** Query tokens matching identifier stems in a chunk receive an additional weight. +- **File coherence.** When multiple chunks from the same file match the query, the file is boosted so the top result reflects broad file-level relevance rather than a single out-of-context chunk. +- **Noise penalties.** Test files, `compat/`/`legacy/` shims, example code, and `.d.ts` declaration stubs are down-ranked so canonical implementations surface first. + +
+ +Because the embedding model is static with no transformer forward pass at query time, all of this runs in milliseconds on CPU. + ## Benchmarks -Quality and speed across all methods on ~1,250 queries over 63 repositories in 19 languages. X-axis is total latency (index + first query); y-axis is NDCG@10. Marker size reflects model parameter count. +We benchmark quality and speed across all methods on ~1,250 queries over 63 repositories in 19 languages. The x-axis is total latency (index + first query); the y-axis is NDCG@10. Marker size reflects model parameter count. ![Speed vs quality](assets/images/speed_vs_ndcg_cold.png) | Method | NDCG@10 | Index time | Query p50 | |--------|--------:|-----------:|----------:| -| ripgrep | 0.126 | — | 12 ms | -| ColGREP | 0.693 | 5.8 s | 124 ms | +| CodeRankEmbed Hybrid | 0.862 | 57 s | 16 ms | +| **semble** | **0.854** | **263 ms** | **1.5 ms** | | CodeRankEmbed | 0.765 | 57 s | 16 ms | -| semble | 0.854 | **263 ms** | **1.5 ms** | -| CodeRankEmbed Hybrid | **0.862** | 57 s | 16 ms | +| ColGREP | 0.693 | 5.8 s | 124 ms | +| BM25 | 0.673 | 263 ms | 0.02 ms | +| ripgrep | 0.126 | — | 12 ms | -The 137M-parameter CodeRankEmbed Hybrid leads NDCG@10 by 0.008. Semble indexes 218x faster and answers queries 11x faster. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and methodology. +Semble achieves 99% of the performance of the 137M-parameter [CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed) Hybrid, while indexing 218x faster and answering queries 11x faster. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and methodology. ## License diff --git a/benchmarks/README.md b/benchmarks/README.md index 97516075a..eec807b63 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -15,11 +15,12 @@ Quality and speed across all methods. | Method | NDCG@10 | Index | Query p50 | |---|---:|---:|---:| -| ripgrep | 0.126 | — | 12 ms | -| ColGREP | 0.693 | 5.8 s | 124 ms | +| CodeRankEmbed Hybrid | 0.862 | 57 s | 16 ms | +| **semble** | **0.854** | **263 ms** | **1.5 ms** | | CodeRankEmbed | 0.765 | 57 s | 16 ms | -| semble | 0.854 | **263 ms** | **1.5 ms** | -| CodeRankEmbed Hybrid | **0.862** | 57 s | 16 ms | +| ColGREP | 0.693 | 5.8 s | 124 ms | +| BM25 | 0.673 | 263 ms | 0.02 ms | +| ripgrep | 0.126 | — | 12 ms | | ![Speed vs quality (cold)](results/speed_vs_ndcg_cold.png) | ![Speed vs quality (warm)](results/speed_vs_ndcg_warm.png) | |:--:|:--:|