MongoDB Connection Pool Visualizer

A real-time visualization tool for MongoDB connection pool events that supports multiple clients simultaneously.

Overview

This application uses a producer-consumer pattern to visualize MongoDB connection pool states and metrics:

• Consumer (MongoDb.ConnectionPoolVisualizer): Blazor web application that visualizes connection pool data
• Producer (MongoDb.ConnectionPoolMonitor): Standalone application that monitors MongoDB connection pools and sends events to the visualizer

The application allows you to monitor the following in real-time:

• Connection pool states (Ready, Clearing, Cleared, Closed)
• Connection metrics (Added, Checking Out, Checked Out, Checking In, Checked In, Removed)
• Multiple MongoDB clients with clear visual separation
• Connection pool events across different MongoDB driver implementations

Application Architecture

This solution consists of two separate applications:

1. ConnectionPoolVisualizer (Consumer)

A Blazor web application that:

• Provides a real-time visualization of MongoDB connection pool states and metrics
• Receives events via a REST API endpoint from any producer
• Uses SignalR to update the UI in real-time
• No longer directly connects to MongoDB (relies on external producer applications)

2. ConnectionPoolMonitor (Producer)

A console application that:

• Creates a single shared MongoClient instance for monitoring
• Monitors MongoDB connection pool events through event subscriptions
• Forwards events to the Visualizer application via HTTP POST
• Includes a test command to generate database activity
• Uses the same MongoClient instance for both monitoring and testing

Getting Started

Running the Applications

Each application has its own separate solution file and can be run independently:

1. Start the ConnectionPoolVisualizer (Consumer):

   cd MongoDb.ConnectionPoolVisualizer
   dotnet run --launch-profile https
   # Or to build and run through Visual Studio:
   # dotnet build MongoDb.ConnectionPoolVisualizer.sln

2. Start the ConnectionPoolMonitor (Lightweight Producer):

   cd MongoDb.ConnectionPoolMonitor
   dotnet run
   # Or to build and run through Visual Studio:
   # dotnet build MongoDb.ConnectionPoolMonitor.sln

   The Monitor now uses a lightweight architecture that simply forwards events without tracking metrics locally.

3. In the ConnectionPoolMonitor console:

   • Press S to start monitoring
   • Press T to generate test activity that will be visualized
   • Press X to stop monitoring
   • Press Q to quit the application

Connection Configuration

Both applications use appsettings.json for configuration:

• ConnectionPoolMonitor settings:

  • ConnectionStrings:MongoDB: MongoDB connection string
  • VisualizerApi:BaseUrl: URL for the Visualizer API endpoint
  • MongoDb:DatabaseName: Database to use for test operations
  • MongoDb:CollectionName: Collection to use for test operations
  • MongoDb:ClientId: Identifier for this client (used by visualizer for grouping)

• ConnectionPoolVisualizer settings:

  • No MongoDB connection settings required (the visualizer is a pure consumer that receives events via API)

Using the Applications

Client Identification

The ConnectionPoolVisualizer supports distinguishing between different MongoDB clients:

1. Each Monitor instance has a Client ID:

   • Default value is "Primary-MongoClient" with a random suffix
   • Can be configured in appsettings.json by adding a ClientId property under the MongoDb section

2. The Visualizer groups connection pools by Client ID in the interface for clear visual separation

3. If using multiple monitor processes, configure each with a different Client ID in its appsettings.json file

Using Only the Visualizer with a Custom Producer

The ConnectionPoolVisualizer is designed to be used independently of the provided C# ConnectionPoolMonitor. You can:

1. Run only the ConnectionPoolVisualizer application
2. Create your own custom producer in any programming language
3. Send connection pool events to the Visualizer API

This is particularly useful if:

• You're using MongoDB with a different driver (Python, Node.js, Java, etc.)
• You want to integrate with an existing application
• You need custom monitoring logic specific to your use case

Creating Custom Producers

You can create producers in any programming language by sending POST requests to the Visualizer API:

1. Minimal Event Design:

   • The API now uses a minimal design with only essential fields
   • No optional fields - producers should send only what's required
   • Status is inferred from event type - no need to send it
   • Metrics are calculated by the Visualizer - no need to send them

2. Event Types and Required Fields:

   • All events: eventType, clusterId, serverId, timestamp
   • Connection events: additionally include connectionId and connectionState
   • Example formats are provided in the API Specification section below

Important Implementation Notes

Shared MongoDB Client

The MongoDb.ConnectionPoolMonitor application uses a shared MongoClient instance for both monitoring and test operations. This is crucial because:

1. Each MongoClient instance creates its own connection pool
2. To monitor a connection pool, we must use the same MongoClient instance that creates it
3. Using separate MongoClient instances would result in monitoring a different pool than the one we are testing

Technical Stack

• C# (.NET 9.0)
• Blazor Server
• MongoDB C# Driver
• SignalR for real-time updates
• RESTful API for language-agnostic integration

Prerequisites

• .NET 9.0 SDK
• MongoDB instance (local or remote)

API Specification

The ConnectionPoolVisualizer provides a RESTful API endpoint that accepts MongoDB connection pool events:

Event Submission Endpoint

POST https://localhost:5001/api/ConnectionPoolEvents

Required Fields for All Events

• eventType (string): Type of the connection pool event (e.g., "PoolOpened", "ConnectionCheckedOut")
• clusterId (string): Identifier for the MongoDB cluster
• serverId (string): Identifier for the MongoDB server (typically the host:port)
• timestamp (ISO 8601 datetime): When the event occurred

Required Fields for Connection Events Only

• connectionId (string): The unique identifier of the connection
• connectionState (string): The state of the connection (e.g., "Created", "CheckedOut")

Note: The Visualizer infers pool status from the eventType, so no status field is required.

Health Check Endpoint

GET https://localhost:5001/api/ConnectionPoolEvents/health

Returns a simple health status response to verify the API is running.

Sample Events

Pool Opened Event:

{
  "eventType": "PoolOpened",
  "clusterId": "cluster1",
  "serverId": "localhost:27017",
  "timestamp": "2025-05-14T14:30:00Z"
}

Connection Checkout Event:

{
  "eventType": "ConnectionCheckedOut",
  "clientId": "MyMongoClient",
  "clusterId": "cluster1",
  "serverId": "localhost:27017",
  "connectionId": "6f8a023b-4b2c-43a0-8c82-1a25890cf47e",
  "connectionState": "CheckedOut", 
  "timestamp": "2025-05-14T14:30:05Z"
}