Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Repository Overview

This is a collection of Jupyter notebooks demonstrating how to use the Vectara RAG (Retrieval Augmented Generation) platform with various integrations including LangChain, LlamaIndex, and DSPy. The notebooks are designed to run in Google Colab.

## Key Technologies

- **Vectara**: RAG-as-a-service platform providing text extraction, ML-based chunking, Boomerang embeddings, hybrid search, and LLM summarization (Mockingbird)
- **LangChain**: Use `langchain-vectara` package for integration
- **LlamaIndex**: Use `llama-index-indices-managed-vectara` package (v0.4.0+ uses API v2)
- **vectara-agentic**: Vectara's agentic RAG package built on LlamaIndex

## Environment Variables

Notebooks typically require these environment variables:
- `VECTARA_API_KEY`: Vectara API key
- `VECTARA_CORPUS_KEY`: Vectara corpus identifier
- `OPENAI_API_KEY`: Required for some notebooks that use OpenAI models

## Running Notebooks

Notebooks are designed to run in Google Colab. Each notebook includes a Colab badge link at the top. They can also be run locally with Jupyter:

```bash
pip install jupyter
jupyter notebook notebooks/
```

## Data Files

Sample data for notebooks is in `data/`:
- PDF files (transformer papers, policy docs, etc.)
- Text files (state_of_the_union.txt, paul_graham_essay.txt)

## Notebook Patterns

1. **File upload to Vectara**: Use `add_files()` or `insert_file()` - Vectara handles chunking and embedding
2. **Querying**: Use `as_query_engine()` for retrieval, `as_chat_engine()` for conversational interfaces
3. **Streaming**: Set `streaming=True` in query engine for streamed responses
4. **Reranking options**: MMR (diversity), Slingshot (multilingual), UDF (custom functions), chain reranker
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,16 @@
This repository contains example code for Vectara.
* Notebooks used in our blog posts
* Examples for how to use Vectara with LlamaIndex, LangChain and DSPy.

## API Examples Tutorial Series

A step-by-step tutorial series in `notebooks/api-examples/` covering the Vectara API v2:

1. [Corpus Creation](notebooks/api-examples/1-corpus-creation.ipynb) — Create and configure corpora
2. [Data Ingestion](notebooks/api-examples/2-data-ingestion.ipynb) — Upload and index documents
3. [Query API](notebooks/api-examples/3-query-api.ipynb) — Search, retrieval, and generation
4. [Agent API](notebooks/api-examples/4-agent-api.ipynb) — Build RAG agents
5. [Sub-Agents](notebooks/api-examples/5-sub-agents.ipynb) — Multi-agent orchestration
6. [Artifacts](notebooks/api-examples/6-artifacts.ipynb) — Working with artifacts
7. [Lambda Tools for Data Analysis](notebooks/api-examples/7-lambda-tools-data-analysis.ipynb) — NumPy/Pandas lambda tools for agent data analysis
8. [Reranker Instructions](notebooks/api-examples/8-reranker-instructions.ipynb) — Using reranker instructions with qwen3-reranker for role-based intent steering and jargon resolution
17 changes: 2 additions & 15 deletions notebooks/api-examples/1-corpus-creation.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,7 @@
"cell_type": "markdown",
"id": "db0855d0",
"metadata": {},
"source": [
"## About Vectara\n",
"\n",
"[Vectara](https://vectara.com/) is the Agent Operating System for trusted enterprise AI: a unified Agentic RAG platform with built-in multi-modal retrieval, orchestration, and always-on governance. Deploy it on-prem (air-gapped), in your VPC, or as SaaS.\n",
"\n",
"Vectara provides a complete API-first platform for building production RAG and agentic applications:\n",
"\n",
"- **Simple Integration**: RESTful APIs and SDKs for Python, TypeScript, and Java make integration straightforward\n",
"- **Flexible Deployment**: Choose SaaS, VPC, or on-premises deployment based on your security and compliance requirements\n",
"- **Multi-Modal Support**: Index and search across text, tables, and images from various document formats\n",
"- **Advanced Retrieval**: Hybrid search combining semantic and keyword matching with multiple reranking options\n",
"- **Grounded Generation**: LLM responses with citations and factual consistency scores to reduce hallucinations\n",
"- **Enterprise-Ready**: Built-in access controls, audit logging, and compliance certifications (SOC2, HIPAA)"
]
"source": "## About Vectara\n\n[Vectara](https://vectara.com/) is the Agent Platform for trusted enterprise AI: a unified Agentic RAG platform with built-in multi-modal retrieval, orchestration, and always-on governance. Deploy it on-prem (air-gapped), in your VPC, or as SaaS.\n\nVectara provides a complete API-first platform for building production RAG and agentic applications:\n\n- **Simple Integration**: RESTful APIs and SDKs for Python, TypeScript, and Java make integration straightforward\n- **Flexible Deployment**: Choose SaaS, VPC, or on-premises deployment based on your security and compliance requirements\n- **Multi-Modal Support**: Index and search across text, tables, and images from various document formats\n- **Advanced Retrieval**: Hybrid search combining semantic and keyword matching with multiple reranking options\n- **Grounded Generation**: LLM responses with citations and factual consistency scores to reduce hallucinations\n- **Enterprise-Ready**: Built-in access controls, audit logging, and compliance certifications (SOC2, HIPAA)"
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -350,4 +337,4 @@
},
"nbformat": 4,
"nbformat_minor": 5
}
}
17 changes: 2 additions & 15 deletions notebooks/api-examples/2-data-ingestion.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,7 @@
"cell_type": "markdown",
"id": "cell-2",
"metadata": {},
"source": [
"## About Vectara\n",
"\n",
"[Vectara](https://vectara.com/) is the Agent Operating System for trusted enterprise AI: a unified Agentic RAG platform with built-in multi-modal retrieval, orchestration, and always-on governance. Deploy it on-prem (air-gapped), in your VPC, or as SaaS.\n",
"\n",
"Vectara provides a complete API-first platform for building production RAG and agentic applications:\n",
"\n",
"- **Simple Integration**: RESTful APIs and SDKs for Python, TypeScript, and Java make integration straightforward\n",
"- **Flexible Deployment**: Choose SaaS, VPC, or on-premises deployment based on your security and compliance requirements\n",
"- **Multi-Modal Support**: Index and search across text, tables, and images from various document formats\n",
"- **Advanced Retrieval**: Hybrid search combining semantic and keyword matching with multiple reranking options\n",
"- **Grounded Generation**: LLM responses with citations and factual consistency scores to reduce hallucinations\n",
"- **Enterprise-Ready**: Built-in access controls, audit logging, and compliance certifications (SOC2, HIPAA)"
]
"source": "## About Vectara\n\n[Vectara](https://vectara.com/) is the Agent Platform for trusted enterprise AI: a unified Agentic RAG platform with built-in multi-modal retrieval, orchestration, and always-on governance. Deploy it on-prem (air-gapped), in your VPC, or as SaaS.\n\nVectara provides a complete API-first platform for building production RAG and agentic applications:\n\n- **Simple Integration**: RESTful APIs and SDKs for Python, TypeScript, and Java make integration straightforward\n- **Flexible Deployment**: Choose SaaS, VPC, or on-premises deployment based on your security and compliance requirements\n- **Multi-Modal Support**: Index and search across text, tables, and images from various document formats\n- **Advanced Retrieval**: Hybrid search combining semantic and keyword matching with multiple reranking options\n- **Grounded Generation**: LLM responses with citations and factual consistency scores to reduce hallucinations\n- **Enterprise-Ready**: Built-in access controls, audit logging, and compliance certifications (SOC2, HIPAA)"
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -1075,4 +1062,4 @@
},
"nbformat": 4,
"nbformat_minor": 5
}
}
17 changes: 2 additions & 15 deletions notebooks/api-examples/3-query-api.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -27,20 +27,7 @@
"cell_type": "markdown",
"id": "db0855d0",
"metadata": {},
"source": [
"## About Vectara\n",
"\n",
"[Vectara](https://vectara.com/) is the Agent Operating System for trusted enterprise AI: a unified Agentic RAG platform with built-in multi-modal retrieval, orchestration, and always-on governance. Deploy it on-prem (air-gapped), in your VPC, or as SaaS. Vectara agents deliver grounded answers and safe actions with source citations, step-level audit trails, fine-grained access controls, and real-time policy and factual-consistency enforcement, so teams ship faster with lower risk, and with trusted, production-grade AI agents at scale.\n",
"\n",
"Vectara provides a complete API-first platform for building production RAG and agentic applications:\n",
"\n",
"- **Simple Integration**: RESTful APIs and SDKs (Python, JavaScript) for quick integration into any stack\n",
"- **Flexible Deployment**: Choose SaaS, VPC, or on-premises deployment based on your requirements\n",
"- **Multi-Modal Support**: Index and search across text, tables, and images from PDFs, documents, and structured data\n",
"- **Advanced Retrieval**: Hybrid search combining semantic and keyword matching with state-of-the-art reranking\n",
"- **Grounded Generation**: LLM responses with citations and factual consistency scores to reduce hallucinations\n",
"- **Enterprise-Ready**: Built-in access controls, audit logging, and compliance (SOC2, HIPAA) from day one"
]
"source": "## About Vectara\n\n[Vectara](https://vectara.com/) is the Agent Platform for trusted enterprise AI: a unified Agentic RAG platform with built-in multi-modal retrieval, orchestration, and always-on governance. Deploy it on-prem (air-gapped), in your VPC, or as SaaS. Vectara agents deliver grounded answers and safe actions with source citations, step-level audit trails, fine-grained access controls, and real-time policy and factual-consistency enforcement, so teams ship faster with lower risk, and with trusted, production-grade AI agents at scale.\n\nVectara provides a complete API-first platform for building production RAG and agentic applications:\n\n- **Simple Integration**: RESTful APIs and SDKs (Python, JavaScript) for quick integration into any stack\n- **Flexible Deployment**: Choose SaaS, VPC, or on-premises deployment based on your requirements\n- **Multi-Modal Support**: Index and search across text, tables, and images from PDFs, documents, and structured data\n- **Advanced Retrieval**: Hybrid search combining semantic and keyword matching with state-of-the-art reranking\n- **Grounded Generation**: LLM responses with citations and factual consistency scores to reduce hallucinations\n- **Enterprise-Ready**: Built-in access controls, audit logging, and compliance (SOC2, HIPAA) from day one"
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -863,4 +850,4 @@
},
"nbformat": 4,
"nbformat_minor": 5
}
}
17 changes: 2 additions & 15 deletions notebooks/api-examples/4-agent-api.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -27,20 +27,7 @@
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## About Vectara\n",
"\n",
"[Vectara](https://vectara.com/) is the Agent Operating System for trusted enterprise AI: a unified Agentic RAG platform with built-in multi-modal retrieval, orchestration, and always-on governance. Deploy it on-prem (air-gapped), in your VPC, or as SaaS. Vectara agents deliver grounded answers and safe actions with source citations, step-level audit trails, fine-grained access controls, and real-time policy and factual-consistency enforcement, so teams ship faster with lower risk, and with trusted, production-grade AI agents at scale.\n",
"\n",
"Vectara provides a complete API-first platform for building production RAG and agentic applications:\n",
"\n",
"- **Simple Integration**: RESTful APIs and SDKs (Python, JavaScript) for quick integration into any stack\n",
"- **Flexible Deployment**: Choose SaaS, VPC, or on-premises deployment based on your security requirements\n",
"- **Multi-Modal Support**: Index and search across text, tables, and images from PDFs, documents, and structured data\n",
"- **Advanced Retrieval**: Hybrid search combining semantic and keyword matching with state-of-the-art reranking\n",
"- **Grounded Generation**: LLM responses with citations and factual consistency scores to reduce hallucinations\n",
"- **Enterprise-Ready**: Built-in access controls, audit logging, and compliance (SOC2, HIPAA) from day one"
]
"source": "## About Vectara\n\n[Vectara](https://vectara.com/) is the Agent Platform for trusted enterprise AI: a unified Agentic RAG platform with built-in multi-modal retrieval, orchestration, and always-on governance. Deploy it on-prem (air-gapped), in your VPC, or as SaaS. Vectara agents deliver grounded answers and safe actions with source citations, step-level audit trails, fine-grained access controls, and real-time policy and factual-consistency enforcement, so teams ship faster with lower risk, and with trusted, production-grade AI agents at scale.\n\nVectara provides a complete API-first platform for building production RAG and agentic applications:\n\n- **Simple Integration**: RESTful APIs and SDKs (Python, JavaScript) for quick integration into any stack\n- **Flexible Deployment**: Choose SaaS, VPC, or on-premises deployment based on your security requirements\n- **Multi-Modal Support**: Index and search across text, tables, and images from PDFs, documents, and structured data\n- **Advanced Retrieval**: Hybrid search combining semantic and keyword matching with state-of-the-art reranking\n- **Grounded Generation**: LLM responses with citations and factual consistency scores to reduce hallucinations\n- **Enterprise-Ready**: Built-in access controls, audit logging, and compliance (SOC2, HIPAA) from day one"
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -527,4 +514,4 @@
},
"nbformat": 4,
"nbformat_minor": 4
}
}
Loading
Loading