LLM Chatbot with Ollama and MCP Tools

A sophisticated LLM chatbot that uses Ollama for language model inference and MCP (Model Context Protocol) tools for enhanced functionality. All communication between components uses JSON format.

Features

• Ollama Integration: Seamless integration with Ollama API for LLM inference
• MCP Tools: Extensible tool system with JSON-based definitions
• JSON Communication: All components communicate using JSON format
• Multiple Tools: Web search, calculator, file operations, and weather tools
• Conversation History: Maintains conversation context across interactions
• Health Monitoring: Built-in health checks for all components
• Error Handling: Robust error handling with JSON error responses

Architecture

User Input → Chat Interface → Message Router → Ollama Client → LLM Response
                                    ↓
                              MCP Server ← Tool Definitions (JSON)
                                    ↓
                              Tool Execution → JSON Response

Prerequisites

• Python 3.8+
• Ollama installed and running
• API keys for external services (optional)

Installation

1. Clone the repository:

   git clone <repository-url>
   cd llm-chatbot

2. Install dependencies:

   pip install -r requirements.txt

3. Install and start Ollama:

   # Install Ollama (follow instructions at https://ollama.ai)
   # Start Ollama service
   ollama serve

4. Pull a model (optional, but recommended):

   # You can use any available model. Some options:
   ollama pull gemma3n:e2b
   ollama pull llama2
   ollama pull qwen2.5:7b

5. Set up environment variables (optional):

   cp .env.example .env
   # Edit .env with your API keys

Configuration

Environment Variables

Create a .env file in the project root:

# Ollama Configuration
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=gemma3n:e2b

# MCP Server Configuration
MCP_SERVER_PORT=8000

# API Keys (optional)
WEB_SEARCH_API_KEY=your_search_api_key
WEATHER_API_KEY=your_weather_api_key

Tool Definitions

Tools are defined in config/tool_definitions.json. Each tool has:

• name: Unique identifier
• description: Human-readable description
• inputSchema: JSON Schema for parameters
• annotations: Metadata about tool behavior

Usage

Running the Full Application

python main.py

Running Only the MCP Server

python main.py server

Interactive Commands

Once the application is running, you can use these commands:

• quit or exit: Exit the application
• clear: Clear conversation history
• health: Check system health
• history: Show conversation history

Available Tools

1. Web Search

• Name: web_search
• Description: Search the web for current information
• Parameters: query (required), max_results (optional)

2. Calculator

• Name: calculate
• Description: Perform mathematical calculations
• Parameters: expression (required)

3. File Operations

• Name: file_operations
• Description: Read, write, or list files
• Parameters: operation, path (required), content (for write)

4. Weather

• Name: weather
• Description: Get current weather information
• Parameters: location (required), units (optional)

JSON Communication Examples

Ollama Request

{
  "model": "llama2",
  "messages": [
    {
      "role": "user",
      "content": "What's the weather in New York?"
    }
  ],
  "tools": [
    {
      "name": "weather",
      "description": "Get current weather information",
      "inputSchema": {
        "type": "object",
        "properties": {
          "location": {"type": "string"},
          "units": {"type": "string", "enum": ["metric", "imperial"]}
        },
        "required": ["location"]
      }
    }
  ]
}

MCP Tool Call

{
  "name": "weather",
  "arguments": {
    "location": "New York",
    "units": "metric"
  }
}

MCP Response

{
  "content": [
    {
      "type": "text",
      "text": "Weather for New York: 22°C, Partly Cloudy"
    }
  ],
  "isError": false
}

Development

Project Structure

llm-chatbot/
├── src/
│   ├── client/
│   │   ├── ollama_client.py      # Ollama API client
│   │   └── chat_interface.py     # User interface
│   ├── server/
│   │   ├── mcp_server.py         # MCP server implementation
│   │   └── tool_handlers.py      # Tool implementations
│   ├── utils/
│   │   └── message_router.py     # Message routing logic
│   └── config/
│       └── settings.py           # Configuration management
├── config/
│   └── tool_definitions.json     # Tool definitions
├── requirements.txt
├── main.py
└── README.md

Adding New Tools

1. Define the tool in config/tool_definitions.json:

   {
     "name": "my_tool",
     "description": "Description of my tool",
     "inputSchema": {
       "type": "object",
       "properties": {
         "param1": {"type": "string"}
       },
       "required": ["param1"]
     },
     "annotations": {
       "title": "My Tool",
       "readOnlyHint": true
     }
   }

2. Implement the tool handler in src/server/tool_handlers.py:

   class MyTool(BaseTool):
       async def execute(self, arguments: Dict[str, Any]) -> Dict[str, Any]:
           # Your tool implementation
           return {
               "content": [{"type": "text", "text": "Result"}],
               "isError": False
           }

3. Register the tool in MCPServer.initialize_tool_handlers():

   return {
       # ... existing tools
       "my_tool": MyTool()
   }

Testing

Run tests with pytest:

pytest tests/

Troubleshooting

Ollama Connection Issues

1. Check if Ollama is running:

   curl http://localhost:11434/api/tags

2. Start Ollama:

   ollama serve

3. Verify model availability:

   ollama list

MCP Server Issues

1. Check server health:

   curl http://localhost:8000/health

2. View available tools:

   curl http://localhost:8000/tools/list

API Key Issues

If you see warnings about missing API keys:

1. Set the required environment variables in .env
2. Restart the application
3. Some tools will work without API keys (with mock responses)

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• Ollama for the local LLM inference
• MCP Specification for the tool protocol
• FastAPI for the web framework