Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ jobs:
node-version: "22"

- name: Setup pnpm
uses: pnpm/action-setup@v2
uses: pnpm/action-setup@v4
with:
version: 9
version: 11.3.0
run_install: false

- name: Get pnpm store directory
Expand Down
2 changes: 1 addition & 1 deletion web/docs/docs/Agents/_category_.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"label": "Marmot for Agents",
"position": 5,
"position": 2,
"link": {
"type": "doc",
"id": "Agents/index"
Expand Down
44 changes: 43 additions & 1 deletion web/docs/docs/Agents/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,52 @@ sidebar_position: 1

# Marmot for Agents

**Marmot for Agents** plugs your LLM agents into the catalog. They read it for context and write back the lineage they generate.
An agent acting without context is guessing. It doesn't know which table holds customer orders, who owns the payments service, what a column means, or what breaks downstream if it changes something. **Marmot for Agents** ends that context starvation: it plugs your LLM agents into the catalog so they read it for context and write back the lineage they generate.

import { CalloutCard, DocCard, DocCardGrid, FeatureCard, FeatureGrid } from '@site/src/components/DocCard';

## What your agents can do

<FeatureGrid>
<FeatureCard
title="Discover assets"
description="Find any service, API, queue, topic, table or pipeline by name, type or metadata."
icon="mdi:magnify"
/>
<FeatureCard
title="Resolve ownership"
description="Answer who owns an asset and who to contact before acting on it."
icon="mdi:account-search-outline"
/>
<FeatureCard
title="Understand meaning"
description="Pull business definitions and glossary terms so answers use your language."
icon="mdi:book-open-page-variant-outline"
/>
<FeatureCard
title="Trace lineage"
description="See upstream and downstream dependencies to reason about impact."
icon="mdi:source-branch"
/>
</FeatureGrid>

## Two ways to connect

<DocCardGrid>
<DocCard
title="MCP Server"
description="The fastest path: point any client that speaks MCP (Claude, Cursor, ChatGPT, Cline) at Marmot and start asking."
docId="MCP/index"
icon="mdi:protocol"
/>
<DocCard
title="SDK"
description="Build agents directly against the catalog with the typed Python, Go or TypeScript client."
docId="SDK/index"
icon="mdi:code-braces"
/>
</DocCardGrid>

## Supported frameworks

<DocCardGrid>
Expand Down
2 changes: 1 addition & 1 deletion web/docs/docs/MCP/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 6
sidebar_position: 3
---

# Model Context Protocol (MCP)
Expand Down
150 changes: 84 additions & 66 deletions web/docs/docs/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ sidebar_position: 1

# Introduction

Marmot is an open-source data catalog designed for teams who want powerful data discovery without enterprise complexity. Built with a focus on simplicity and speed, Marmot helps you catalog assets across your entire data stack - from databases and APIs to message queues and data pipelines.
Marmot is the open source **context layer** for your whole stack: a single catalog for every asset your systems and teams depend on, from services, APIs, queues, topics and brokers to databases, tables and pipelines. It exists to solve **context starvation**, the moment an engineer or an AI agent has to act without knowing what exists, who owns it, what it means, or what it connects to.

Marmot is built so both **humans and agents** can ask that question and get a real answer. [Catalog your assets](Populating/index.md) once, enrich them with ownership and business context, and expose them through the UI, a [REST API](api-reference.md), and a built-in [MCP server](MCP/index.md) that lets AI agents read your catalog, then write back the [lineage](open-lineage.md) they generate.

<div style={{maxWidth: '480px'}}>
<iframe width="100%" style={{aspectRatio: '16 / 9', border: 'none', borderRadius: '12px'}} src="https://www.youtube.com/embed/_JBcQGj_bFU" title="Marmot Demo" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowFullScreen></iframe>
Expand All @@ -20,32 +22,30 @@ import { CalloutCard, DocCard, DocCardGrid, FeatureCard, FeatureGrid } from '@si
icon="mdi:rocket-launch"
/>

## Why Marmot?
## Built for agents

Unlike traditional catalogs that require extensive infrastructure and configuration, Marmot ships as a **single binary** with an intuitive UI, making it easy to deploy and start cataloging in minutes.
AI agents are only as good as the context they can reach. Through a built-in MCP server and our SDKs, Marmot gives your agents a live, governed map of every asset in your stack: what exists, who owns it, what it means, and how it all connects.

<FeatureGrid>
<FeatureCard
title="Deploy in Minutes"
description="Single binary, Docker, or Kubernetes - no complex setup required"
icon="mdi:lightning-bolt"
/>
<FeatureCard
title="Powerful Search"
description="Query language with full-text, metadata, and boolean operators"
icon="mdi:magnify"
/>
<FeatureCard
title="Track Lineage"
description="Interactive dependency graphs to understand data flows and impact"
icon="mdi:source-branch"
<DocCardGrid>
<DocCard
title="Marmot for Agents"
description="Plug your LLM agents into the catalog: they read it for context and write back the lineage they generate."
docId="Agents/index"
icon="mdi:robot-outline"
/>
<FeatureCard
title="Flexible Integrations"
description="CLI, REST API, Terraform, and Pulumi - catalog assets your way"
icon="mdi:puzzle"
<DocCard
title="MCP Server"
description="Let Claude, Cursor, ChatGPT and any MCP client answer questions backed by your real catalog."
docId="MCP/index"
icon="mdi:protocol"
/>
</FeatureGrid>
</DocCardGrid>

## Why Marmot?

Most catalogs were built to help a data team document tables. Marmot is built to feed context to whoever needs it, human or agent, across every kind of asset: services, APIs, queues, topics, brokers, databases, tables and pipelines.

That means agents are first class, not an afterthought. A native MCP server and our SDKs are part of the core, so your agents read the same governed context your team does. And Marmot stays light enough to actually adopt: a single binary backed only by PostgreSQL, with no platform team required.

## Architecture

Expand All @@ -55,62 +55,80 @@ Marmot is built entirely in Go with PostgreSQL being the only external dependenc
<img src="/img/marmot-diagram.png" alt="Marmot architecture diagram" style={{maxWidth: '100%', borderRadius: '8px'}} />
</div>

## Features

### Search Everything

Find any data asset across your entire organisation in seconds. Combine full-text search with structured queries using metadata filters, boolean logic, and comparison operators.

### Interactive Lineage Visualisation
## What Marmot stores

Trace data flows from source to destination with interactive dependency graphs. Understand upstream and downstream dependencies, identify bottlenecks, and analyse impact before making changes.
Marmot is a context layer, so it stores **metadata about your assets**, not the data inside them. That means schemas, field names and types, ownership, descriptions, tags, lineage and statistics. The rows in your tables, the messages on your topics and the payloads behind your APIs never enter Marmot.

### Metadata-First Architecture
Plugins read a source's structure and metadata into PostgreSQL; the data itself never moves. The easiest path is to run it on our platform, isolated per customer and under strict access controls. Need everything to stay within your control? Run Marmot yourself, free or with an enterprise license, in your own cloud on AWS, Google Cloud, Azure, OVHcloud or anywhere else.

Store rich metadata for any asset type. From tables and topics to APIs and dashboards - if it matters to your data stack, you can catalog it in Marmot.

### Team Collaboration

Assign ownership, document business context, and create glossaries. Keep your entire team aligned with centralised knowledge about your data assets.

### Data Products

Group related assets into logical collections. Use manual assignment for curated bundles or dynamic rules that use the query language to automatically include matching assets as your catalog grows.
<CalloutCard
title="Building for a regulated environment?"
description="Run it yourself so nothing leaves your VPC, and read the source to verify exactly what is collected."
href="/pricing#contact"
buttonText="Talk to us"
variant="secondary"
icon="mdi:shield-check-outline"
/>

## Getting Started
## Features

Ready to dive in? Here's where to go next:
Everything you need to turn scattered assets into a context layer that humans and agents can both query.

<DocCardGrid>
<DocCard
title="Populating Your Catalog"
description="Learn how to add assets using plugins, CLI, or API"
docId="Populating/index"
icon="mdi:database-plus"
<FeatureGrid>
<FeatureCard
title="Discovery for humans and agents"
description="Find any asset across your whole stack in seconds, from the UI or straight through MCP."
icon="mdi:magnify"
docId="queries"
/>
<DocCard
title="Data Products"
description="Group assets into logical collections with manual or dynamic rules"
docId="data-products"
icon="mdi:package-variant-closed"
<FeatureCard
title="Lineage and impact"
description="Trace how data flows and what depends on what, so people and agents can reason about change before they make it."
icon="mdi:source-branch"
docId="open-lineage"
/>
<DocCard
title="Glossary"
description="Define business terms and create a shared vocabulary"
<FeatureCard
title="Context that gives meaning"
description="Ownership, business definitions, tags and custom fields turn raw assets into answers, not guesses."
icon="mdi:tag-text-outline"
docId="glossary"
icon="mdi:book-alphabet"
/>
<FeatureCard
title="Every asset, one catalog"
description="Services, APIs, queues, topics, brokers, databases, tables and pipelines, all in a single context layer."
icon="mdi:shape-outline"
docId="Populating/index"
/>
<FeatureCard
title="Built for agents"
description="A native MCP server and our SDKs let your agents read context and write back the lineage they generate."
icon="mdi:robot-outline"
docId="Agents/index"
/>
<FeatureCard
title="Data products"
description="Group related assets into curated bundles or dynamic rules that grow with your catalog."
icon="mdi:package-variant-closed"
docId="data-products"
/>
</FeatureGrid>

## Get started

Pick a starting point. The [Quick Start](quick-start.md) walks you from an empty deployment to a populated catalog, step by step.

<DocCardGrid>
<DocCard
title="Query Language"
description="Use Marmot's powerful search capabilities"
docId="queries"
icon="mdi:code-tags"
title="Quick Start"
description="Spin up Marmot with Docker Compose in a couple of minutes, then start cataloging."
docId="quick-start"
icon="mdi:rocket-launch-outline"
/>
<DocCard
title="Deployment Options"
description="Deploy to production with Docker, Helm, or the CLI"
docId="Deploy/index"
icon="mdi:cloud-upload"
title="Marmot for agents"
description="Connect your agents through the built-in MCP server and our SDKs."
docId="Agents/index"
icon="mdi:robot-outline"
/>
</DocCardGrid>

Expand Down
13 changes: 10 additions & 3 deletions web/docs/docs/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ title: Quick Start
Get Marmot running in seconds with Docker Compose.

import { CalloutCard, DocCard, DocCardGrid } from '@site/src/components/DocCard';
import { Steps, Step } from '@site/src/components/Steps';

<CalloutCard
title="Just Want to Explore?"
Expand All @@ -17,11 +18,13 @@ import { CalloutCard, DocCard, DocCardGrid } from '@site/src/components/DocCard'
icon="mdi:rocket-launch"
/>

## Prerequisites
<Steps>
<Step title="Install the prerequisites">

[Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) installed.

## Create Your Docker Compose File
</Step>
<Step title="Create your Docker Compose file">

Create a `docker-compose.yaml`:

Expand Down Expand Up @@ -61,14 +64,18 @@ volumes:
marmot_data:
```

## Start Marmot
</Step>
<Step title="Start Marmot">

```bash
docker compose up -d
```

Open [http://localhost:8080](http://localhost:8080) and log in with `admin` / `admin`.

</Step>
</Steps>

## Next Steps

<DocCardGrid>
Expand Down
2 changes: 1 addition & 1 deletion web/docs/docusaurus.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ try {

const config: Config = {
title: "Marmot",
tagline: "Modern Data Discovery for Modern Teams",
tagline: "The open source context layer for agents and humans",
favicon: "img/favicon.ico",

// Set the production url of your site here
Expand Down
36 changes: 2 additions & 34 deletions web/docs/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,39 +42,6 @@
"typescript": "~5.8.3",
"unplugin-icons": "~22.1.0"
},
"pnpm": {
"overrides": {
"node-forge": ">=1.4.0",
"image-size": ">=1.4.0",
"webpack-dev-server": ">=4.16.0",
"estree-util-value-to-estree": ">=3.4.1",
"mdast-util-to-hast": ">=13.2.1",
"@babel/runtime-corejs3": ">=7.27.1",
"@babel/runtime": ">=7.27.1",
"@babel/helpers": ">=7.27.1",
"glob": ">=10.5.0",
"brace-expansion": ">=5.0.6",
"@isaacs/brace-expansion": ">=5.0.5",
"picomatch": ">=4.0.4",
"yaml": ">=2.8.3",
"minimatch": ">=10.2.3",
"on-headers": ">=1.1.0",
"compression": ">=1.8.1",
"mermaid": ">=11.10.0",
"cookie": ">=0.7.0",
"serialize-javascript": ">=7.0.5",
"fast-xml-parser": ">=5.5.7",
"svgo": ">=3.3.3",
"dompurify": ">=3.3.2",
"lodash": ">=4.18.1",
"qs": ">=6.14.2",
"ajv": ">=8.18.0",
"path-to-regexp": "~0.1.13",
"webpack": "~5.104.1",
"uuid": ">=9.0.1",
"ws": ">=8.20.1"
}
},
"browserslist": {
"production": [
">0.5%",
Expand All @@ -89,5 +56,6 @@
},
"engines": {
"node": ">=20.0"
}
},
"packageManager": "pnpm@11.3.0"
}
Loading
Loading