Skip to content

sree-sanak/llm-accounting

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

llm-accounting

Per-customer, per-feature AI cost attribution with signed usage receipts.

Most teams shipping AI features can't answer "how much did this customer cost me last month?" or "which feature is burning tokens?" Bills arrive as a single opaque number per provider. This project is a small gateway that sits in front of OpenAI, Anthropic, Google, and OpenRouter, meters each call, and emits an HMAC-signed receipt tagged with customer, job, task, workflow, and model.

It's a demo — JSON file storage, mock adapters when no keys are set, HMAC-SHA256 receipts rather than provider-signed x402 — not a production system. The structure (gateway → receipt → analytics) is what's worth reading.

How it works

  client ─┐
          │ POST /v1/ai { customer_id, workflow, task, provider, model, messages }
          v
   ┌─────────────┐        ┌────────────────────┐
   │  Gateway    │──────► │ Vendor adapter     │  real API if key set,
   │  (FastAPI)  │        │ (OpenAI/Anthropic/ │  otherwise mock
   │             │ ◄──────│  Google/OpenRouter)│
   │             │        └────────────────────┘
   │             │
   │             │──► Pricing (hardcoded per-1k token table)
   │             │──► Receipt (HMAC-SHA256 over canonical JSON)
   │             │──► Storage (JSON files: jobs, tasks, receipts)
   └─────────────┘
          │
          v
   ┌─────────────┐
   │  Next.js    │  /job-runner    run a workflow, watch tasks + cost
   │  frontend   │  /receipts      filter, inspect, verify signature
   │             │  /dashboard     spend by provider / workflow / customer
   └─────────────┘

The gateway generates a request_id, calls the vendor adapter, computes cost from token counts and the pricing table in backend/app/config.py, and writes a receipt. Each receipt is a canonical-JSON HMAC, so tampering with stored fields breaks verification. Job totals roll up as tasks complete.

What it tracks

Every receipt carries: customer_id, job_id, task_id, request_id, provider, model, input_tokens, output_tokens, unit_price, cost_usd, latency_ms, error, metadata, signature. That's enough to answer the questions the system exists for — per-customer spend, per-feature spend, fallback cost, model mix — without relying on the vendors' dashboards.

Tech stack

Layer Technology
Backend FastAPI, Python 3, httpx for real vendor calls
Adapters OpenAI, Anthropic, Google, OpenRouter (real + mock modes, toggled by presence of API key)
Storage JSON files under backend/data/ — easy to inspect, not meant for concurrency
Signatures HMAC-SHA256 over canonical JSON; secret in HMAC_SECRET env var
Token counting len(text) // 4 heuristic for mock mode; real responses use provider-reported token counts
Frontend Next.js 14 App Router, TypeScript, Tailwind, Recharts

Getting started

# Backend
cd backend
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000
# Docs at http://localhost:8000/docs

# Frontend (new terminal)
cd frontend
npm install
npm run dev
# http://localhost:3000

By default the backend uses mock adapters. To hit real APIs, set any of OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY, OPENROUTER_API_KEY in the environment and restart. See SETUP_REAL_APIS.md.

For production use you'd want: a real database, tiktoken (or provider-reported counts only) instead of the character heuristic, a rotating HMAC secret — or swap HMAC for actual x402 provider-signed receipts — and per-customer rate limiting.

API

Endpoint Purpose
POST /v1/ai Execute an AI call; returns output, usage, receipt_id, cost
GET /v1/receipts List receipts; filter by customer_id, job_id, provider, model, from, to
GET /v1/receipts/{id} Fetch a single receipt
POST /v1/receipts/{id}/verify Recompute HMAC, return valid: bool
GET /v1/analytics/summary Aggregate spend; group_by = customer/workflow/task/model/provider

Demo workflows

job-runner ships with two scripted workflows worth clicking through:

  • Support Ticket Autopilot — classify (OpenAI mini) → retrieve policy (tool) → draft (Claude Opus) → safety check (Gemini). Three providers, one job, cost rolls up per task.
  • Invoice Extraction — first task fails deliberately, retry falls over to a more expensive model. Shows how fallback inflates per-job cost.

Repo layout

backend/app/
  routes/        ai_gateway, receipts, analytics
  services/      gateway, pricing, receipt, analytics
  adapters/      openai, anthropic, google, openrouter (real + mock)
  utils/         signature (HMAC), token_estimator, id_generator
  storage.py     JSON file I/O with a thread lock
  config.py      pricing table, env, CORS

frontend/app/
  job-runner/    run a workflow, live task timeline
  receipts/      table + detail view + verify button
  dashboard/     KPI cards, provider pie, workflow bars

License

MIT.

About

AI cost attribution. Per-customer, per-feature, per-workflow tracking across OpenAI, Anthropic, Google, and OpenRouter with verifiable receipts.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages