From fceca6e2affc92a910f9954268a996c249c16691 Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 11 May 2026 15:03:50 +0200 Subject: [PATCH 01/14] Make readme more compact --- README.md | 146 ++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 99 insertions(+), 47 deletions(-) diff --git a/README.md b/README.md index cd522fac0..2e2d259f3 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@

semble logo
- Fast and Accurate Code Search for Agents
- Uses ~98% fewer tokens than grep+read + Fast, Token-Efficient Code Search for AI Agents
+ ~98% fewer tokens than grep+read  •  250 ms indexing  •  1.5 ms queries  •  no API keys

@@ -16,22 +16,34 @@ +[Why Semble?](#why-semble) • [Quickstart](#quickstart) • [MCP Server](#mcp-server) • [Bash / AGENTS.md](#bash-integration) • [CLI](#cli) • -[Python API](#python-api) • [Benchmarks](#benchmarks)
-Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~98% fewer tokens than grep+read and cutting latency on every step. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Run it as an [MCP server](#mcp-server) or call it from the shell via [AGENTS.md](#bash-integration) and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. +Semble is a code search tool built for agents. Instead of grepping for keywords and reading full files, agents query in natural language and get back only the relevant snippets — using ~98% fewer tokens, with indexing in ~250 ms and queries in ~1.5 ms on CPU. No API keys, GPU, or external services. Works via MCP or shell instructions in AGENTS.md / CLAUDE.md, with any agent (Claude Code, Cursor, Codex, OpenCode, etc.). + +## Why Semble? + +Agents are bad at code search when they rely on `grep`, `rg`, and full-file reads. They often miss semantic matches, over-read irrelevant files, and burn context on code that does not answer the query. + +Semble gives agents a dedicated retrieval layer for code: + +- **Token-efficient:** returns focused snippets instead of full files, using ~98% fewer tokens than grep+read. +- **Semantic + lexical:** combines Model2Vec embeddings, BM25, reciprocal rank fusion, and code-aware reranking. +- **Fast on CPU:** indexes an average repository in ~250 ms and answers queries in ~1.5 ms. +- **Local-first:** no API keys, GPUs, hosted vector database, or external service required. +- **Agent-native:** works through MCP or shell instructions in AGENTS.md / CLAUDE.md. ## Quickstart -Your agent will automatically use Semble whenever it needs to find code. Instead of grepping with a keyword and reading full files, it queries in natural language (e.g. `"How is authentication handled?"`) and gets back only the relevant context. Semble can be set up as an MCP server or as a bash tool: +Your agent will automatically use Semble whenever it needs to find code. Instead of grepping with a keyword and reading full files, it queries in natural language (e.g. `"How is authentication handled?"`) and gets back only the relevant context. -### MCP +### MCP (Claude Code) Add Semble to Claude Code (requires [uv](https://docs.astral.sh/uv/getting-started/installation/)): @@ -39,31 +51,54 @@ Add Semble to Claude Code (requires [uv](https://docs.astral.sh/uv/getting-start claude mcp add semble -s user -- uvx --from "semble[mcp]" semble ``` -Using another agent harness? See [MCP Server](#mcp-server) for setup instructions for Codex, OpenCode, Cursor, and other MCP clients. +Using Codex, OpenCode, or Cursor? See [MCP Server](#mcp-server) for setup instructions. ### Bash / AGENTS.md -Install Semble first, then add the [code search snippet](#bash-integration) to your `AGENTS.md` or `CLAUDE.md`: +Install Semble, then add the snippet below to your `AGENTS.md` or `CLAUDE.md`: ```bash pip install semble # Install with pip uv tool install semble # Or install with uv ``` -> Note: for Claude Code or Codex CLI sub-agents, use the [bash integration](#bash-integration) instead of, or alongside, MCP. +
+AGENTS.md / CLAUDE.md snippet + +```markdown +## Code Search + +Use `semble search` to find code by describing what it does or naming a symbol/identifier, instead of grep: + +​```bash +semble search "authentication flow" ./my-project +semble search "save_pretrained" ./my-project +semble search "save model to disk" ./my-project --top-k 10 +​``` + +Use `semble find-related` to discover code similar to a known location (pass `file_path` and `line` from a prior search result): + +​```bash +semble find-related src/auth.py 42 ./my-project +​``` + +`path` defaults to the current directory when omitted; git URLs are accepted. -To update Semble, see [Updating](#updating). +If `semble` is not on `$PATH`, use `uvx --from "semble[mcp]" semble` in its place. -Curious how many tokens Semble has saved you? Run `semble savings` to see. See [Savings](#savings) for details. +### Workflow -## Main Features +1. Start with `semble search` to find relevant chunks. +2. Inspect full files only when the returned chunk is not enough context. +3. Optionally use `semble find-related` with a promising result's `file_path` and `line` to discover related implementations. +4. Use grep only when you need exhaustive literal matches or quick confirmation of an exact string. +``` -- **Fast**: indexes an average repo in ~250 ms and answers queries in ~1.5 ms, all on CPU. -- **Accurate**: NDCG@10 of 0.854 on our [benchmarks](#benchmarks), on par with code-specialized transformer models, at a fraction of the size and cost. -- **Token-efficient**: returns only the relevant chunks, using [~98% fewer tokens than grep+read](#token-efficiency). -- **Zero setup**: runs on CPU with no API keys, GPU, or external services required. -- **MCP server**: drop-in tool for Claude Code, Cursor, Codex, OpenCode, and any other MCP-compatible agent. -- **Local and remote**: pass a local path or a git URL. +
+ +> Note: for Claude Code or Codex CLI sub-agents, use the [bash integration](#bash-integration) instead of, or alongside, MCP. + +Curious how many tokens Semble has saved you? Run `semble savings` to see. ## MCP Server @@ -173,21 +208,30 @@ Semble also ships as a standalone CLI for use outside of MCP. This is useful in # Search a local repo semble search "authentication flow" ./my-project +# Find code similar to a known location +semble find-related src/auth.py 42 ./my-project +``` + +`path` defaults to the current directory when omitted; git URLs are accepted. If `semble` is not on `$PATH`, use `uvx --from "semble[mcp]" semble` in its place. + +
+More CLI examples + +```bash # Search for a symbol or identifier semble search "save_pretrained" ./my-project # Search a remote repo (cloned on demand) semble search "save model to disk" https://github.com/MinishLab/model2vec -# Find code similar to a known location (file_path and line from a prior search result) -semble find-related src/auth.py 42 ./my-project +# Limit results +semble search "save model to disk" ./my-project --top-k 10 ``` -`path` defaults to the current directory when omitted; git URLs are accepted. - -If `semble` is not on `$PATH`, use `uvx --from "semble[mcp]" semble` in its place. +
-### Savings +
+Savings `semble savings` shows how many tokens semble has saved across all your searches: @@ -210,17 +254,10 @@ semble savings --verbose # also show breakdown by call type Stats are stored in `~/.semble/savings.jsonl`. -### Updating - -To update/upgrade Semble to the latest version: - -```bash -pip install --upgrade semble # with pip -uv tool upgrade semble # with uv -uv cache clean semble # for MCP users (restart your MCP client after) -``` +
-## Python API +
+Library usage Semble can also be used as a Python library for programmatic access, useful when building custom tooling or integrating search directly into your own code. @@ -247,24 +284,20 @@ result.chunk.end_line # 150 result.chunk.content # "def save_pretrained(self, path: PathLike, ..." ``` -## How it works - -Semble splits each file into code-aware chunks using [Chonkie](https://github.com/chonkie-inc/chonkie), then scores every query against the chunks with two complementary retrievers: static [Model2Vec](https://github.com/MinishLab/model2vec) embeddings using the code-specialized [potion-code-16M](https://huggingface.co/minishlab/potion-code-16M) model for semantic similarity, and [BM25](https://github.com/xhluca/bm25s) for lexical matches on identifiers and API names. The two score lists are fused with Reciprocal Rank Fusion (RRF). - -After fusing, results are reranked with a set of code-aware signals: +
-Ranking signals +Updating -- **Adaptive weighting.** Symbol-like queries (`Foo::bar`, `_private`, `getUserById`) get more lexical weight, while natural-language queries stay balanced between semantic and lexical retrievers. -- **Definition boosts.** A chunk that defines the queried symbol (a `class`, `def`, `func`, etc.) is ranked above chunks that merely reference it. -- **Identifier stems.** Query tokens are stemmed and matched against identifier stems in a chunk, giving an additional weight to chunks that contain them. For example, querying `parse config` boosts chunks containing `parseConfig`, `ConfigParser`, or `config_parser`. -- **File coherence.** When multiple chunks from the same file match the query, the file is boosted so the top result reflects broad file-level relevance rather than a single out-of-context chunk. -- **Noise penalties.** Test files, `compat/`/`legacy/` shims, example code, and `.d.ts` declaration stubs are down-ranked so canonical implementations surface first. +To update/upgrade Semble to the latest version: -
+```bash +pip install --upgrade semble # with pip +uv tool upgrade semble # with uv +uv cache clean semble # for MCP users (restart your MCP client after) +``` -Because the embedding model is static with no transformer forward pass at query time, all of this runs in milliseconds on CPU. + ## Benchmarks @@ -293,6 +326,25 @@ Agents using grep+read spend most of their context budget on irrelevant code. Se Semble uses **98% fewer tokens** on average, and reaches 94% recall at a budget of only 2k tokens, while grep+read needs a full 100k context window to reach 85%. See [benchmarks](benchmarks/README.md#token-efficiency) for details. +## How it works + +Semble splits each file into code-aware chunks using [Chonkie](https://github.com/chonkie-inc/chonkie), then scores every query against the chunks with two complementary retrievers: static [Model2Vec](https://github.com/MinishLab/model2vec) embeddings using the code-specialized [potion-code-16M](https://huggingface.co/minishlab/potion-code-16M) model for semantic similarity, and [BM25](https://github.com/xhluca/bm25s) for lexical matches on identifiers and API names. The two score lists are fused with Reciprocal Rank Fusion (RRF). + +After fusing, results are reranked with a set of code-aware signals: + +
+Ranking signals + +- **Adaptive weighting.** Symbol-like queries (`Foo::bar`, `_private`, `getUserById`) get more lexical weight, while natural-language queries stay balanced between semantic and lexical retrievers. +- **Definition boosts.** A chunk that defines the queried symbol (a `class`, `def`, `func`, etc.) is ranked above chunks that merely reference it. +- **Identifier stems.** Query tokens are stemmed and matched against identifier stems in a chunk, giving an additional weight to chunks that contain them. For example, querying `parse config` boosts chunks containing `parseConfig`, `ConfigParser`, or `config_parser`. +- **File coherence.** When multiple chunks from the same file match the query, the file is boosted so the top result reflects broad file-level relevance rather than a single out-of-context chunk. +- **Noise penalties.** Test files, `compat/`/`legacy/` shims, example code, and `.d.ts` declaration stubs are down-ranked so canonical implementations surface first. + +
+ +Because the embedding model is static with no transformer forward pass at query time, all of this runs in milliseconds on CPU. + ## License MIT From f13d68980bd646394dd2c649ac60be49c9ac3113 Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 11 May 2026 15:30:28 +0200 Subject: [PATCH 02/14] Make readme more compact --- README.md | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-) diff --git a/README.md b/README.md index 2e2d259f3..c0e37a2a9 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,6 @@ -[Why Semble?](#why-semble) • [Quickstart](#quickstart) • [MCP Server](#mcp-server) • [Bash / AGENTS.md](#bash-integration) • @@ -25,19 +24,7 @@ -Semble is a code search tool built for agents. Instead of grepping for keywords and reading full files, agents query in natural language and get back only the relevant snippets — using ~98% fewer tokens, with indexing in ~250 ms and queries in ~1.5 ms on CPU. No API keys, GPU, or external services. Works via MCP or shell instructions in AGENTS.md / CLAUDE.md, with any agent (Claude Code, Cursor, Codex, OpenCode, etc.). - -## Why Semble? - -Agents are bad at code search when they rely on `grep`, `rg`, and full-file reads. They often miss semantic matches, over-read irrelevant files, and burn context on code that does not answer the query. - -Semble gives agents a dedicated retrieval layer for code: - -- **Token-efficient:** returns focused snippets instead of full files, using ~98% fewer tokens than grep+read. -- **Semantic + lexical:** combines Model2Vec embeddings, BM25, reciprocal rank fusion, and code-aware reranking. -- **Fast on CPU:** indexes an average repository in ~250 ms and answers queries in ~1.5 ms. -- **Local-first:** no API keys, GPUs, hosted vector database, or external service required. -- **Agent-native:** works through MCP or shell instructions in AGENTS.md / CLAUDE.md. +Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~98% fewer tokens than grep+read and cutting latency on every step. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Run it as an [MCP server](#mcp-server) or call it from the shell via [AGENTS.md](#bash-integration) and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. ## Quickstart From 23a842a30913f9129c34ed164784742f15d0fe92 Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 09:55:24 +0200 Subject: [PATCH 03/14] Update docs --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c0e37a2a9..036a312dc 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@

semble logo
- Fast, Token-Efficient Code Search for AI Agents
- ~98% fewer tokens than grep+read  •  250 ms indexing  •  1.5 ms queries  •  no API keys + Fast and Accurate Code Search for Agents
+ Uses ~98% fewer tokens than grep+read

From 30a14d54b8af442c5fc56ba726721615a3fe594f Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 10:24:00 +0200 Subject: [PATCH 04/14] Add docs dir --- README.md | 39 ++++++++++----------------------------- docs/python-api.md | 26 ++++++++++++++++++++++++++ 2 files changed, 36 insertions(+), 29 deletions(-) create mode 100644 docs/python-api.md diff --git a/README.md b/README.md index 036a312dc..1ec7f172d 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,15 @@ Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~98% fewer tokens than grep+read and cutting latency on every step. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Run it as an [MCP server](#mcp-server) or call it from the shell via [AGENTS.md](#bash-integration) and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. +## Main Features + +- **Fast**: indexes an average repo in ~250 ms and answers queries in ~1.5 ms, all on CPU. +- **Accurate**: NDCG@10 of 0.854 on our [benchmarks](#benchmarks), on par with code-specialized transformer models, at a fraction of the size and cost. +- **Token-efficient**: returns only the relevant chunks, using [~98% fewer tokens than grep+read](#token-efficiency). +- **Zero setup**: runs on CPU with no API keys, GPU, or external services required. +- **MCP server**: drop-in tool for Claude Code, Cursor, Codex, OpenCode, and any other MCP-compatible agent. +- **Local and remote**: pass a local path or a git URL. + ## Quickstart Your agent will automatically use Semble whenever it needs to find code. Instead of grepping with a keyword and reading full files, it queries in natural language (e.g. `"How is authentication handled?"`) and gets back only the relevant context. @@ -243,35 +252,7 @@ Stats are stored in `~/.semble/savings.jsonl`. -
-Library usage - -Semble can also be used as a Python library for programmatic access, useful when building custom tooling or integrating search directly into your own code. - -```python -from semble import SembleIndex - -# Index a local directory -index = SembleIndex.from_path("./my-project") - -# Index a remote git repository -index = SembleIndex.from_git("https://github.com/MinishLab/model2vec") - -# Search the index with a natural-language or code query -results = index.search("save model to disk", top_k=3) - -# Find code similar to a specific result -related = index.find_related(results[0], top_k=3) - -# Each result exposes the matched chunk -result = results[0] -result.chunk.file_path # "model2vec/model.py" -result.chunk.start_line # 127 -result.chunk.end_line # 150 -result.chunk.content # "def save_pretrained(self, path: PathLike, ..." -``` - -
+Need programmatic access? See [Python API](docs/python-api.md).
Updating diff --git a/docs/python-api.md b/docs/python-api.md new file mode 100644 index 000000000..7f72f65a0 --- /dev/null +++ b/docs/python-api.md @@ -0,0 +1,26 @@ +# Python API + +Semble can be used as a Python library for programmatic access, useful when building custom tooling or integrating search directly into your own code. + +```python +from semble import SembleIndex + +# Index a local directory +index = SembleIndex.from_path("./my-project") + +# Index a remote git repository +index = SembleIndex.from_git("https://github.com/MinishLab/model2vec") + +# Search the index with a natural-language or code query +results = index.search("save model to disk", top_k=3) + +# Find code similar to a specific result +related = index.find_related(results[0], top_k=3) + +# Each result exposes the matched chunk +result = results[0] +result.chunk.file_path # "model2vec/model.py" +result.chunk.start_line # 127 +result.chunk.end_line # 150 +result.chunk.content # "def save_pretrained(self, path: PathLike, ..." +``` From 8eb380226048717202b28019e11069045f12910f Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 10:31:30 +0200 Subject: [PATCH 05/14] Update docs --- README.md | 78 ++++++++++++++++++++++++++++++++----------------------- 1 file changed, 46 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index 1ec7f172d..485f6987a 100644 --- a/README.md +++ b/README.md @@ -26,15 +26,6 @@ Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~98% fewer tokens than grep+read and cutting latency on every step. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Run it as an [MCP server](#mcp-server) or call it from the shell via [AGENTS.md](#bash-integration) and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. -## Main Features - -- **Fast**: indexes an average repo in ~250 ms and answers queries in ~1.5 ms, all on CPU. -- **Accurate**: NDCG@10 of 0.854 on our [benchmarks](#benchmarks), on par with code-specialized transformer models, at a fraction of the size and cost. -- **Token-efficient**: returns only the relevant chunks, using [~98% fewer tokens than grep+read](#token-efficiency). -- **Zero setup**: runs on CPU with no API keys, GPU, or external services required. -- **MCP server**: drop-in tool for Claude Code, Cursor, Codex, OpenCode, and any other MCP-compatible agent. -- **Local and remote**: pass a local path or a git URL. - ## Quickstart Your agent will automatically use Semble whenever it needs to find code. Instead of grepping with a keyword and reading full files, it queries in natural language (e.g. `"How is authentication handled?"`) and gets back only the relevant context. @@ -96,6 +87,15 @@ If `semble` is not on `$PATH`, use `uvx --from "semble[mcp]" semble` in its plac Curious how many tokens Semble has saved you? Run `semble savings` to see. +## Main Features + +- **Fast**: indexes an average repo in ~250 ms and answers queries in ~1.5 ms, all on CPU. +- **Accurate**: NDCG@10 of 0.854 on our [benchmarks](#benchmarks), on par with code-specialized transformer models, at a fraction of the size and cost. +- **Token-efficient**: returns only the relevant chunks, using [~98% fewer tokens than grep+read](#token-efficiency). +- **Zero setup**: runs on CPU with no API keys, GPU, or external services required. +- **MCP server**: drop-in tool for Claude Code, Cursor, Codex, OpenCode, and any other MCP-compatible agent. +- **Local and remote**: pass a local path or a git URL. + ## MCP Server Semble can run as an MCP server so agents can search any codebase directly. Repos are cloned and indexed on demand, and indexes are cached for the lifetime of the session. Local paths are watched for file changes and re-indexed automatically. @@ -252,7 +252,35 @@ Stats are stored in `~/.semble/savings.jsonl`.
-Need programmatic access? See [Python API](docs/python-api.md). +
+Library usage + +Semble can also be used as a Python library for programmatic access, useful when building custom tooling or integrating search directly into your own code. + +```python +from semble import SembleIndex + +# Index a local directory +index = SembleIndex.from_path("./my-project") + +# Index a remote git repository +index = SembleIndex.from_git("https://github.com/MinishLab/model2vec") + +# Search the index with a natural-language or code query +results = index.search("save model to disk", top_k=3) + +# Find code similar to a specific result +related = index.find_related(results[0], top_k=3) + +# Each result exposes the matched chunk +result = results[0] +result.chunk.file_path # "model2vec/model.py" +result.chunk.start_line # 127 +result.chunk.end_line # 150 +result.chunk.content # "def save_pretrained(self, path: PathLike, ..." +``` + +
Updating @@ -269,30 +297,16 @@ uv cache clean semble # for MCP users (restart your MCP client after) ## Benchmarks -We benchmark quality and speed across all methods on ~1,250 queries over 63 repositories in 19 languages. The x-axis is total latency (index + first query); the y-axis is NDCG@10. Marker size reflects model parameter count. - -![Speed vs quality](https://raw.githubusercontent.com/MinishLab/semble/main/assets/images/speed_vs_ndcg_cold.png) - -| Method | NDCG@10 | Index time | Query p50 | -|--------|--------:|-----------:|----------:| -| CodeRankEmbed Hybrid | 0.862 | 57 s | 16 ms | -| **semble** | **0.854** | **263 ms** | **1.5 ms** | -| CodeRankEmbed | 0.765 | 57 s | 16 ms | -| ColGREP | 0.693 | 5.8 s | 124 ms | -| BM25 | 0.673 | 263 ms | 0.02 ms | -| grepai | 0.561 | 35 s | 48 ms | -| probe | 0.387 | — | 207 ms | -| ripgrep | 0.126 | — | 12 ms | - -Semble achieves 99% of the performance of the 137M-parameter [CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed) Hybrid, while indexing 218x faster and answering queries 11x faster. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and methodology. - -### Token efficiency - -Agents using grep+read spend most of their context budget on irrelevant code. Semble returns only the chunks that match, keeping token usage low even at high recall. +We benchmark quality and speed across ~1,250 queries over 63 repositories in 19 languages (left), and token efficiency against grep+read at equivalent recall levels (right). -![Token efficiency: recall vs. retrieved tokens](https://raw.githubusercontent.com/MinishLab/semble/main/assets/images/token_efficiency.png) + + + + + +
Speed vs qualityToken efficiency: recall vs. retrieved tokens
-Semble uses **98% fewer tokens** on average, and reaches 94% recall at a budget of only 2k tokens, while grep+read needs a full 100k context window to reach 85%. See [benchmarks](benchmarks/README.md#token-efficiency) for details. +The quality benchmark (left) scores retrieval quality (NDCG@10) against total latency across methods on ~1,250 natural-language and code queries; semble achieves 99% of the quality of the 137M-parameter [CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed) Hybrid while indexing 218x faster. The token efficiency benchmark (right) measures how many tokens each method needs to reach a given recall level; semble uses 98% fewer tokens on average and hits 94% recall at only 2k tokens, while grep+read needs a full 100k context window to reach 85%. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and full methodology. ## How it works From 72710570fecbe174b5936b0cdde0bf8dbb33ddbd Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 10:43:46 +0200 Subject: [PATCH 06/14] Update docs --- README.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 485f6987a..b4dfeecfb 100644 --- a/README.md +++ b/README.md @@ -28,8 +28,6 @@ Semble is a code search library built for agents. It returns the exact code snip ## Quickstart -Your agent will automatically use Semble whenever it needs to find code. Instead of grepping with a keyword and reading full files, it queries in natural language (e.g. `"How is authentication handled?"`) and gets back only the relevant context. - ### MCP (Claude Code) Add Semble to Claude Code (requires [uv](https://docs.astral.sh/uv/getting-started/installation/)): @@ -49,6 +47,8 @@ pip install semble # Install with pip uv tool install semble # Or install with uv ``` +Note that for sub-agent support in Claude Code or Codex, you need the bash integration. Once installed, run `semble savings` to see how many tokens Semble has saved you. See [Bash / AGENTS.md](#bash-integration) for more info. +
AGENTS.md / CLAUDE.md snippet @@ -83,9 +83,6 @@ If `semble` is not on `$PATH`, use `uvx --from "semble[mcp]" semble` in its plac
-> Note: for Claude Code or Codex CLI sub-agents, use the [bash integration](#bash-integration) instead of, or alongside, MCP. - -Curious how many tokens Semble has saved you? Run `semble savings` to see. ## Main Features @@ -153,7 +150,7 @@ Add to `~/.cursor/mcp.json` (or `.cursor/mcp.json` in your project): ## Bash integration -An alternative to MCP is to invoke Semble via Bash. For Claude Code and Codex CLI, this is the only option for sub-agents, which cannot call MCP tools directly (both lazy-load MCP schemas at the top-level agent only). +An alternative to MCP is to invoke Semble via Bash. For Claude Code and Codex CLI, this is the only option for sub-agents, which cannot call MCP tools directly, though it can also be used alongside MCP for the top-level agent. To add Bash support, append the following to your `AGENTS.md` or `CLAUDE.md`: From 3792c95f60ced5bc41a01560d3d9bc2f02778c74 Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 10:45:45 +0200 Subject: [PATCH 07/14] Update docs --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b4dfeecfb..0cfc84a6f 100644 --- a/README.md +++ b/README.md @@ -307,7 +307,7 @@ The quality benchmark (left) scores retrieval quality (NDCG@10) against total la ## How it works -Semble splits each file into code-aware chunks using [Chonkie](https://github.com/chonkie-inc/chonkie), then scores every query against the chunks with two complementary retrievers: static [Model2Vec](https://github.com/MinishLab/model2vec) embeddings using the code-specialized [potion-code-16M](https://huggingface.co/minishlab/potion-code-16M) model for semantic similarity, and [BM25](https://github.com/xhluca/bm25s) for lexical matches on identifiers and API names. The two score lists are fused with Reciprocal Rank Fusion (RRF). +Semble splits each file into code-aware chunks using [tree-sitter](https://github.com/tree-sitter/py-tree-sitter), then scores every query against the chunks with two complementary retrievers: static [Model2Vec](https://github.com/MinishLab/model2vec) embeddings using the code-specialized [potion-code-16M](https://huggingface.co/minishlab/potion-code-16M) model for semantic similarity, and [BM25](https://github.com/xhluca/bm25s) for lexical matches on identifiers and API names. The two score lists are fused with Reciprocal Rank Fusion (RRF). After fusing, results are reranked with a set of code-aware signals: From 13c1f63dffa9ab3b788873df2d5248a0d62ae6ff Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 10:50:58 +0200 Subject: [PATCH 08/14] Update docs --- README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 0cfc84a6f..fd805d158 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@
-Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~98% fewer tokens than grep+read and cutting latency on every step. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Run it as an [MCP server](#mcp-server) or call it from the shell via [AGENTS.md](#bash-integration) and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. +Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~98% fewer tokens than grep+read. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Run it as an [MCP server](#mcp-server) or call it from the shell via [AGENTS.md](#bash-integration) and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. ## Quickstart @@ -47,8 +47,6 @@ pip install semble # Install with pip uv tool install semble # Or install with uv ``` -Note that for sub-agent support in Claude Code or Codex, you need the bash integration. Once installed, run `semble savings` to see how many tokens Semble has saved you. See [Bash / AGENTS.md](#bash-integration) for more info. -
AGENTS.md / CLAUDE.md snippet @@ -83,14 +81,16 @@ If `semble` is not on `$PATH`, use `uvx --from "semble[mcp]" semble` in its plac
+Once installed, run `semble savings` to see how many tokens Semble has saved you. Note that for sub-agent support in Claude Code or Codex, you need the full [Bash / AGENTS.md](#bash-integration) setup below. + ## Main Features - **Fast**: indexes an average repo in ~250 ms and answers queries in ~1.5 ms, all on CPU. - **Accurate**: NDCG@10 of 0.854 on our [benchmarks](#benchmarks), on par with code-specialized transformer models, at a fraction of the size and cost. -- **Token-efficient**: returns only the relevant chunks, using [~98% fewer tokens than grep+read](#token-efficiency). +- **Token-efficient**: returns only the relevant chunks, using [~98% fewer tokens than grep+read](#benchmarks). - **Zero setup**: runs on CPU with no API keys, GPU, or external services required. -- **MCP server**: drop-in tool for Claude Code, Cursor, Codex, OpenCode, and any other MCP-compatible agent. +- **MCP server**: works with Claude Code, Cursor, Codex, OpenCode, and any other MCP-compatible agent. - **Local and remote**: pass a local path or a git URL. ## MCP Server @@ -243,7 +243,7 @@ semble savings --verbose # also show breakdown by call type All time 1.4k [██████████████░░] ~1.2M tokens (89%) ``` -**How savings are calculated:** for each call, semble records the total character count of the unique files containing returned chunks and the character count of the snippets returned. Estimated tokens saved is `(file chars − snippet chars) / 4` (4 chars per token). This is a conservative estimate: the baseline is reading matched files in full, which is how coding agents often explore unfamiliar code. +Savings are calculated as follows: for each call, semble records the total character count of the unique files containing returned chunks and the character count of the snippets returned. Estimated tokens saved is `(file chars − snippet chars) / 4` (4 chars per token). This is a conservative estimate: the baseline is reading matched files in full, which is how coding agents often explore unfamiliar code. Stats are stored in `~/.semble/savings.jsonl`. @@ -303,7 +303,7 @@ We benchmark quality and speed across ~1,250 queries over 63 repositories in 19 -The quality benchmark (left) scores retrieval quality (NDCG@10) against total latency across methods on ~1,250 natural-language and code queries; semble achieves 99% of the quality of the 137M-parameter [CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed) Hybrid while indexing 218x faster. The token efficiency benchmark (right) measures how many tokens each method needs to reach a given recall level; semble uses 98% fewer tokens on average and hits 94% recall at only 2k tokens, while grep+read needs a full 100k context window to reach 85%. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and full methodology. +The quality benchmark (left) scores retrieval quality (NDCG@10) against total latency; semble achieves 99% of the quality of the 137M-parameter [CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed) Hybrid while indexing 218x faster. The token efficiency benchmark (right) measures how many tokens each method needs to reach a given recall level; semble uses 98% fewer tokens on average and hits 94% recall at only 2k tokens, while grep+read needs a full 100k context window to reach 85%. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and full methodology. ## How it works From 4dec27272541009bb9c4b8231d36b30b60725277 Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 10:53:14 +0200 Subject: [PATCH 09/14] Drop docs dir --- docs/python-api.md | 26 -------------------------- 1 file changed, 26 deletions(-) delete mode 100644 docs/python-api.md diff --git a/docs/python-api.md b/docs/python-api.md deleted file mode 100644 index 7f72f65a0..000000000 --- a/docs/python-api.md +++ /dev/null @@ -1,26 +0,0 @@ -# Python API - -Semble can be used as a Python library for programmatic access, useful when building custom tooling or integrating search directly into your own code. - -```python -from semble import SembleIndex - -# Index a local directory -index = SembleIndex.from_path("./my-project") - -# Index a remote git repository -index = SembleIndex.from_git("https://github.com/MinishLab/model2vec") - -# Search the index with a natural-language or code query -results = index.search("save model to disk", top_k=3) - -# Find code similar to a specific result -related = index.find_related(results[0], top_k=3) - -# Each result exposes the matched chunk -result = results[0] -result.chunk.file_path # "model2vec/model.py" -result.chunk.start_line # 127 -result.chunk.end_line # 150 -result.chunk.content # "def save_pretrained(self, path: PathLike, ..." -``` From 96479dcc6c1a1a896a5393358e8d98e4a1e3766e Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 10:59:10 +0200 Subject: [PATCH 10/14] Update docs --- README.md | 38 ++++++++++++++------------------------ 1 file changed, 14 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index fd805d158..f75e8dd32 100644 --- a/README.md +++ b/README.md @@ -83,6 +83,16 @@ If `semble` is not on `$PATH`, use `uvx --from "semble[mcp]" semble` in its plac Once installed, run `semble savings` to see how many tokens Semble has saved you. Note that for sub-agent support in Claude Code or Codex, you need the full [Bash / AGENTS.md](#bash-integration) setup below. +
+Updating Semble + +```bash +pip install --upgrade semble # with pip +uv tool upgrade semble # with uv +uv cache clean semble # for MCP users (restart your MCP client after) +``` + +
## Main Features @@ -201,16 +211,6 @@ Semble also ships as a standalone CLI for use outside of MCP. This is useful in # Search a local repo semble search "authentication flow" ./my-project -# Find code similar to a known location -semble find-related src/auth.py 42 ./my-project -``` - -`path` defaults to the current directory when omitted; git URLs are accepted. If `semble` is not on `$PATH`, use `uvx --from "semble[mcp]" semble` in its place. - -
-More CLI examples - -```bash # Search for a symbol or identifier semble search "save_pretrained" ./my-project @@ -219,9 +219,12 @@ semble search "save model to disk" https://github.com/MinishLab/model2vec # Limit results semble search "save model to disk" ./my-project --top-k 10 + +# Find code similar to a known location +semble find-related src/auth.py 42 ./my-project ``` -
+`path` defaults to the current directory when omitted; git URLs are accepted. If `semble` is not on `$PATH`, use `uvx --from "semble[mcp]" semble` in its place.
Savings @@ -279,19 +282,6 @@ result.chunk.content # "def save_pretrained(self, path: PathLike, ..."
-
-Updating - -To update/upgrade Semble to the latest version: - -```bash -pip install --upgrade semble # with pip -uv tool upgrade semble # with uv -uv cache clean semble # for MCP users (restart your MCP client after) -``` - -
- ## Benchmarks We benchmark quality and speed across ~1,250 queries over 63 repositories in 19 languages (left), and token efficiency against grep+read at equivalent recall levels (right). From d781c0d35d6f7da6ef6854299b52132ffc9bdf5d Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 11:03:57 +0200 Subject: [PATCH 11/14] Update docs --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index f75e8dd32..1f38e06af 100644 --- a/README.md +++ b/README.md @@ -18,13 +18,13 @@ [Quickstart](#quickstart) • [MCP Server](#mcp-server) • -[Bash / AGENTS.md](#bash-integration) • +[Bash / AGENTS.md](#bash-agentsmd-1) • [CLI](#cli) • [Benchmarks](#benchmarks) -Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~98% fewer tokens than grep+read. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Run it as an [MCP server](#mcp-server) or call it from the shell via [AGENTS.md](#bash-integration) and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. +Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~98% fewer tokens than grep+read. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Run it as an [MCP server](#mcp-server) or call it from the shell via [AGENTS.md](#bash-agentsmd-1) and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. ## Quickstart @@ -81,7 +81,7 @@ If `semble` is not on `$PATH`, use `uvx --from "semble[mcp]" semble` in its plac -Once installed, run `semble savings` to see how many tokens Semble has saved you. Note that for sub-agent support in Claude Code or Codex, you need the full [Bash / AGENTS.md](#bash-integration) setup below. +Once installed, run `semble savings` to see how many tokens Semble has saved you. Note that for sub-agent support in Claude Code or Codex, you need the full [Bash / AGENTS.md](#bash-agentsmd-1) setup below.
Updating Semble @@ -158,7 +158,7 @@ Add to `~/.cursor/mcp.json` (or `.cursor/mcp.json` in your project): | `find_related` | Given a file path and line number, return chunks semantically similar to the code at that location. | -## Bash integration +## Bash / AGENTS.md An alternative to MCP is to invoke Semble via Bash. For Claude Code and Codex CLI, this is the only option for sub-agents, which cannot call MCP tools directly, though it can also be used alongside MCP for the top-level agent. From 282cda85f3f8dbc0dad9798817fab5b268d98db2 Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 11:05:23 +0200 Subject: [PATCH 12/14] Update docs --- README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 1f38e06af..41fd48e9a 100644 --- a/README.md +++ b/README.md @@ -18,13 +18,13 @@ [Quickstart](#quickstart) • [MCP Server](#mcp-server) • -[Bash / AGENTS.md](#bash-agentsmd-1) • +[Bash / AGENTS.md](#bash-agentsmd) • [CLI](#cli) • [Benchmarks](#benchmarks) -Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~98% fewer tokens than grep+read. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Run it as an [MCP server](#mcp-server) or call it from the shell via [AGENTS.md](#bash-agentsmd-1) and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. +Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~98% fewer tokens than grep+read. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Run it as an [MCP server](#mcp-server) or call it from the shell via [AGENTS.md](#bash-agentsmd) and any agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo. ## Quickstart @@ -81,7 +81,7 @@ If `semble` is not on `$PATH`, use `uvx --from "semble[mcp]" semble` in its plac
-Once installed, run `semble savings` to see how many tokens Semble has saved you. Note that for sub-agent support in Claude Code or Codex, you need the full [Bash / AGENTS.md](#bash-agentsmd-1) setup below. +Once installed, run `semble savings` to see how many tokens Semble has saved you. Note that for sub-agent support in Claude Code or Codex, you need the full [Bash / AGENTS.md](#bash-agentsmd) setup below.
Updating Semble @@ -158,6 +158,8 @@ Add to `~/.cursor/mcp.json` (or `.cursor/mcp.json` in your project): | `find_related` | Given a file path and line number, return chunks semantically similar to the code at that location. | + + ## Bash / AGENTS.md An alternative to MCP is to invoke Semble via Bash. For Claude Code and Codex CLI, this is the only option for sub-agents, which cannot call MCP tools directly, though it can also be used alongside MCP for the top-level agent. From 8190fce500e0fb46b7824ed61dffbb677ef209b9 Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 11:08:45 +0200 Subject: [PATCH 13/14] Update docs --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 41fd48e9a..eb8e5ff5b 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,8 @@ Semble is a code search library built for agents. It returns the exact code snip ## Quickstart +Your agent queries Semble in natural language (e.g. `"How is authentication handled?"`) and gets back only the relevant code snippets, without grepping or reading full files. Set it up as an MCP server or via AGENTS.md: + ### MCP (Claude Code) Add Semble to Claude Code (requires [uv](https://docs.astral.sh/uv/getting-started/installation/)): From 9c5b2103f647b71ff3643bbd11113bce26e80b2c Mon Sep 17 00:00:00 2001 From: Pringled Date: Mon, 18 May 2026 11:09:52 +0200 Subject: [PATCH 14/14] Update docs --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index eb8e5ff5b..588124530 100644 --- a/README.md +++ b/README.md @@ -209,7 +209,7 @@ This writes [`.claude/agents/semble-search.md`](src/semble/agents/semble-search. ## CLI -Semble also ships as a standalone CLI for use outside of MCP. This is useful in scripts or anywhere you want search results without an MCP session. +Semble also ships as a standalone CLI. This is useful in scripts or anywhere you want search results without an MCP session. ```bash # Search a local repo