diff --git a/src/oss/python/integrations/providers/moorcheh.mdx b/src/oss/python/integrations/providers/moorcheh.mdx new file mode 100644 index 000000000..7d0130b30 --- /dev/null +++ b/src/oss/python/integrations/providers/moorcheh.mdx @@ -0,0 +1,71 @@ +--- +keywords: [moorcheh, vectorstore, semantic-search, embeddings] +--- + +# Moorcheh + +>[Moorcheh](https://www.moorcheh.ai/) is a lightning-fast semantic search engine and vector store. Instead of using simple distance metrics like L2 or Cosine, Moorcheh uses Maximally Informative Binarization (MIB) and Information-Theoretic Score (ITS) to retrieve accurate document chunks. + +This page covers how to use Moorcheh within LangChain for vector storage, semantic search, and generative AI responses. + +## Installation and Setup + +Install the Python integration package: + +```bash +pip install langchain-moorcheh +``` + +Get a Moorcheh API key from the [Moorcheh Console](https://console.moorcheh.ai/) and set it as an environment variable: + +```bash +export MOORCHEH_API_KEY="your-api-key-here" +``` + +## Vector Store + +Moorcheh provides a vector store wrapper that allows you to store, search, and retrieve document embeddings efficiently. + +See a [detailed usage example](/docs/integrations/vectorstores/moorcheh). + +```python +from langchain_moorcheh import MoorchehVectorStore + +# Initialize the vector store +store = MoorchehVectorStore( + api_key="your-api-key", + namespace="your_namespace", + namespace_type="text" # or "vector" +) + +# Add documents +from langchain_core.documents import Document +documents = [ + Document(page_content="Your document content here", metadata={"source": "example"}) +] +store.add_documents(documents=documents) +``` + +## Generative AI + +Moorcheh supports generative AI responses using various LLM models including Claude 3, allowing you to get AI-generated answers based on your stored documents. + +```python +# Get an AI-generated answer based on your documents +query = "What are the main topics covered in the documents?" +answer = store.generative_answer( + query, + ai_model="anthropic.claude-3-7-sonnet-20250219-v1:0" +) +print(answer) +``` + +## Core Features + +- **Document Management**: Add and delete documents with unique IDs +- **Semantic Search**: Find relevant documents using natural language queries +- **Generative AI**: Get AI-generated answers using various LLM models +- **Namespace Organization**: Organize your data into separate namespaces +- **Metadata Support**: Store and retrieve documents with custom metadata + +For more detailed examples and advanced usage, see the [Moorcheh vectorstore integration](/docs/integrations/vectorstores/moorcheh). diff --git a/src/oss/python/integrations/vectorstores/index.mdx b/src/oss/python/integrations/vectorstores/index.mdx index 55ef09ea8..c74b761bf 100644 --- a/src/oss/python/integrations/vectorstores/index.mdx +++ b/src/oss/python/integrations/vectorstores/index.mdx @@ -598,6 +598,7 @@ vector_store = QdrantVectorStore( | [FAISS](/oss/integrations/vectorstores/faiss) | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | | [InMemoryVectorStore](https://python.langchain.com/api_reference/core/vectorstores/langchain_core.vectorstores.in_memory.InMemoryVectorStore.html) | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | | [Milvus](/oss/integrations/vectorstores/milvus) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| [Moorcheh](/oss/integrations/vectorstores/moorcheh) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | [MongoDBAtlasVectorSearch](/oss/integrations/vectorstores/mongodb_atlas) | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | | [openGauss](/oss/integrations/vectorstores/opengauss) | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ✅ | | [PGVector](/oss/integrations/vectorstores/pgvector) | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | @@ -673,6 +674,7 @@ vector_store = QdrantVectorStore( + @@ -790,6 +792,7 @@ vector_store = QdrantVectorStore( + diff --git a/src/oss/python/integrations/vectorstores/moorcheh.mdx b/src/oss/python/integrations/vectorstores/moorcheh.mdx new file mode 100644 index 000000000..1d82931ed --- /dev/null +++ b/src/oss/python/integrations/vectorstores/moorcheh.mdx @@ -0,0 +1,169 @@ +--- +title: "Moorcheh" +description: "Lightning-fast semantic search engine and vector store using Maximally Informative Binarization (MIB) and Information-Theoretic Score (ITS)" +--- + +# Moorcheh + +[Moorcheh](https://www.moorcheh.ai/) is a lightning-fast semantic search engine and vector store. Instead of using simple distance metrics like L2 or Cosine, Moorcheh uses Maximally Informative Binarization (MIB) and Information-Theoretic Score (ITS) to retrieve accurate document chunks. + +The following tutorial will allow you to use Moorcheh and LangChain to upload and store text documents and vector embeddings as well as retrieve relevant chunks for all of your queries. + +## Setup + +First, install the necessary package: + +```bash +pip install langchain-moorcheh +``` + +## Initialization + +Get started with Moorcheh + +1. Sign up or log in at the [Moorcheh Console](https://console.moorcheh.ai/). +2. Go to the "API Keys" tab and generate an API key. +3. Save the key as an environment variable named `MOORCHEH_API_KEY` (you'll use it below). +4. To create a namespace for storing data: + - In the Console, open the "Namespaces" tab and click "Create namespace"; or + - Initialize it programmatically using the vector store code in the next section. +5. Use your API key to create namespaces, upload documents, and retrieve answers. + +For more information about the Moorcheh SDK functions, see the [GitHub repository](https://github.com/moorcheh-ai/moorcheh-python-sdk). + +## Importing Packages + +Import the below packages: + +```python +from langchain_moorcheh import MoorchehVectorStore +from moorcheh_sdk import MoorchehClient + +import logging +import os +from uuid import uuid4 +import asyncio +from typing import Any, List, Optional, Literal, Tuple, Type, TypeVar, Sequence +from langchain_core.documents import Document +from langchain_core.embeddings import Embeddings +from langchain_core.vectorstores import VectorStore +from google.colab import userdata +``` + +## Code Setup + +Set your Moorcheh API Key in your environment variables: + +```python +MOORCHEH_API_KEY = os.environ['MOORCHEH_API_KEY'] +``` + +Set up your namespace name, type, and create the vector store: + +```python +namespace = "your_namespace_name" +namespace_type = "text" # or vector +store = MoorchehVectorStore( + api_key=MOORCHEH_API_KEY, + namespace=namespace, + namespace_type=namespace_type + ) +``` + +## Adding Documents + +```python +document_1 = Document( + page_content="Brewed a fresh cup of Ethiopian coffee and paired it with a warm croissant.", + metadata={"source": "blog"}, +) + +document_2 = Document( + page_content="Tomorrow's weather will be sunny with light winds, reaching a high of 78°F.", + metadata={"source": "news"}, +) + +document_3 = Document( + page_content="Experimenting with LangChain for an AI-powered note-taking assistant!", + metadata={"source": "tweet"}, +) + +document_4 = Document( + page_content="Local bakery donates 500 loaves of bread to the community food bank.", + metadata={"source": "news"}, +) + +document_5 = Document( + page_content="That concert last night was absolutely unforgettable—what a performance!", + metadata={"source": "tweet"}, +) + +document_6 = Document( + page_content="Check out our latest article: 5 ways to boost productivity while working from home.", + metadata={"source": "website"}, +) + +document_7 = Document( + page_content="The ultimate guide to mastering homemade pizza dough.", + metadata={"source": "website"}, +) + +document_8 = Document( + page_content="LangGraph just made multi-agent workflows way easier—seriously impressive!", + metadata={"source": "tweet"}, +) + +document_9 = Document( + page_content="Oil prices rose 3% today after unexpected supply cuts from major exporters.", + metadata={"source": "news"}, +) + +document_10 = Document( + page_content="I really hope this post doesn't vanish into the digital void…", + metadata={"source": "tweet"}, +) + +documents = [ + document_1, + document_2, + document_3, + document_4, + document_5, + document_6, + document_7, + document_8, + document_9, + document_10, +] + +uuids = [str(uuid4()) for _ in range(len(documents))] + +store.add_documents(documents=documents, ids=uuids) +``` + +## Delete Documents + +```python +store.delete(ids=["chunk_id_here"]) +``` + +## Query Engine + +Once your namespace has been created and you have uploaded documents into it, you can ask queries about the documents directly through the vector store. Set the query and LLM you would like to answer your query. For more information on supported LLMs, please visit our [Github page](https://github.com/moorcheh-ai/moorcheh-python-sdk). + +```python +query = "Give me a brief summary of the provided documents" +answer = store.generative_answer(query, ai_model = "anthropic.claude-3-7-sonnet-20250219-v1:0") +print(answer) +``` + +## Further Resources + +For more information about Moorcheh, feel free to visit the resources below: + +* [Github page](https://github.com/moorcheh-ai/moorcheh-python-sdk) +* [Examples Github page](https://github.com/moorcheh-ai/moorcheh-examples) +* [Website](https://www.moorcheh.ai/) +* [Documentation](https://console.moorcheh.ai/docs) +* [Youtube](https://www.youtube.com/@moorchehai) +* [X](https://x.com/moorcheh_ai)