Gmail AI Agent

A production-grade AI-powered Gmail agent built with Agno, Composio, and Claude. Handles email reading, drafting, searching, and management through natural language — with real-time status streaming, HMAC webhook security, replay attack prevention, per-user rate limiting, and Helicone cost tracking.

Built as a standalone microservice that receives webhook requests, processes them through an AI agent with Gmail tool access, and streams status updates back via callbacks.

Architecture

Webhook Request (signed)
    │
    ▼
┌─────────────────────────┐
│   Validation Pipeline   │
│  HMAC → Parse → Replay  │
│  → Rate Limit → Accept  │
└──────────┬──────────────┘
           │ 202 Accepted
           ▼
┌─────────────────────────┐
│   Background Agent Run  │
│                         │
│  Helicone-proxied       │
│  Claude + Composio      │
│  Gmail tools            │
│                         │
│  Per-user OAuth scoping │
│  Tool output sanitized  │
│  HTML/CSS stripped      │
└──────────┬──────────────┘
           │
           ▼
┌─────────────────────────┐
│   Callback Protocol     │
│  thinking → tool_call   │
│  → tool_result → final  │
│                         │
│  Real-time status       │
│  streaming to client    │
└─────────────────────────┘

Key Features

Security

• HMAC-SHA256 webhook verification with constant-time comparison
• Replay attack prevention — 5-minute dedup window (in-memory, Redis-ready)
• Per-user rate limiting — 30 req/min request throttle + 10 req/day via Helicone
• User ID cross-validation — header must match payload sender
• Error masking — raw API errors never exposed to end users

Agent

• Stateless per-request architecture — new agent instance per webhook, zero server-side state
• Agno + Composio — agent framework with OAuth-scoped Gmail tools per user
• Claude (Anthropic) via Helicone proxy — cost tracking, token attribution, rate limiting
• Tool output sanitization — smart JSON parsing, HTML/CSS stripping, recursive truncation (50K char limit)
• Conversation context — last 10 messages included for continuity

Observability

• Helicone integration — per-user cost tracking, token usage, rate limit policies
• Structured logging — ISO timestamps, level filtering
• Sentry SDK (optional) — error tracking in production
• Health check endpoint — GET /health for uptime monitoring

Deployment

• Docker — non-root user, single worker, production-ready
• Railway — Hobby tier ($5/mo), always-on, auto-restart on failure
• Any PaaS — stateless design works anywhere (Render, Fly.io, etc.)

Tech Stack

Layer	Technology
Framework	FastAPI + Uvicorn
Agent	Agno 2.5.4
LLM	Anthropic Claude (claude-sonnet-4-20250514)
Tools	Composio (Gmail OAuth — 13 actions)
Observability	Helicone (free tier) + optional Sentry
Security	HMAC-SHA256, replay guard, rate limiter
HTTP Client	httpx (async, connection pooling)
Deployment	Docker, Railway

Gmail Actions Available

Action	Description
GMAIL_FETCH_EMAILS	Fetch emails (max 5 per request)
GMAIL_GET_EMAIL	Get specific email by ID
GMAIL_SEND_EMAIL	Send new email
GMAIL_REPLY_TO_THREAD	Reply to email thread
GMAIL_CREATE_EMAIL_DRAFT	Create draft
GMAIL_GET_PEOPLE_CONTACTS	List contacts
GMAIL_LIST_LABELS	List Gmail labels
GMAIL_ADD_LABEL_TO_EMAIL	Add label
GMAIL_REMOVE_LABEL_FROM_EMAIL	Remove label
GMAIL_FETCH_MESSAGE_DETAIL	Get full message detail
GMAIL_MODIFY_THREAD_LABELS	Modify thread labels
GMAIL_LIST_THREADS	List threads
GMAIL_GET_THREAD	Get thread detail

Quick Start

Prerequisites

• Python 3.12+
• Anthropic API key
• Composio account + Gmail OAuth setup
• Helicone API key (free tier)

Local Setup

git clone https://github.com/maaz-codes/gmail-ai-agent.git
cd gmail-ai-agent
pip install -r requirements.txt
cp .env.example .env
# Fill in your API keys in .env
uvicorn app.main:app --reload --port 8000

Docker

docker build -t gmail-ai-agent .
docker run -p 8000:8000 --env-file .env gmail-ai-agent

Railway

1. Connect repo to Railway
2. Set environment variables from .env.example
3. Deploy — uses Dockerfile automatically

Webhook API

POST /webhook

{
  "message_id": "unique-message-id",
  "sender_id": "user-uuid",
  "content": "Show me my latest emails",
  "conversation_history": [
    {"role": "user", "content": "..."},
    {"role": "assistant", "content": "..."}
  ]
}

Required Headers:

• x-agent-signature: HMAC-SHA256 signature of the request body
• x-user-id: Must match sender_id in payload

Response: 202 Accepted (processing happens in background)

Callback Protocol

The agent streams status updates to your callback URL:

┌─────────────┬──────────────────────────────┐
│    Type     │             When             │
├─────────────┼──────────────────────────────┤
│ thought     │ Agent starts processing      │
├─────────────┼──────────────────────────────┤
│ tool_call   │ Agent invokes a Gmail action │
├─────────────┼──────────────────────────────┤
│ tool_result │ Gmail action returns data    │
├─────────────┼──────────────────────────────┤
│ text        │ Final response or error      │
└─────────────┴──────────────────────────────┘

Testing

# Unit tests
pytest tests/ -v

# Manual webhook trigger
python trigger_webhook.py

Security Considerations

• Replay guard and rate limiter are in-memory — works for single-instance deployment. For multi-instance scaling, replace with Redis.
• HMAC secret must be shared between your webhook sender and this agent.
• Composio OAuth tokens are managed by Composio — this agent never touches user credentials directly.
• Helicone rate limits are enforced server-side — even if in-memory rate limiter is bypassed, Helicone provides a second layer.

Project Structure

app/
├── main.py              # FastAPI app + lifespan
├── config.py            # Pydantic settings (env vars)
├── agent/
│   ├── gmail_agent.py   # Stateless agent executor
│   └── prompts.py       # System prompt
├── tools/
│   └── composio_tools.py # Tool factory + output sanitizer
├── routes/
│   ├── webhook.py       # POST /webhook
│   └── health.py        # GET /health
├── security/
│   ├── hmac.py          # HMAC-SHA256 verification
│   └── replay_guard.py  # Dedup cache
├── middleware/
│   └── rate_limit.py    # Per-user throttle
├── callbacks/
│   └── callback_client.py # Async webhook responder
├── gateway/
│   └── helicone.py      # Helicone-proxied Anthropic client
└── models/
    └── schemas.py       # Pydantic models

Built By

Maaz Khan — AI Agents Engineer specializing in agentic systems, RAG pipelines, and production AI infrastructure.