EvalUI – LLM Observability Dashboard

EvalUI is a portfolio‑grade LLM observability dashboard that visualises the full lifecycle of a prompt through an LLM pipeline. It showcases dual‑backend tracing, LLM‑as‑a‑judge evaluation, and a race‑track UI that compares traces side‑by‑side from Langfuse and Arize Phoenix.

---

Table of Contents

• Overview
• Key Features
• Tech Stack
• Project Structure
• Quick Start
• Running the Demo Scripts
• Deployment (Live Demo)
• Contributing
• Security & Public Release
• License

---

Overview

EvalUI was built to demonstrate modern observability patterns for LLM applications:

• Instrumentation as the single source of truth – OpenInference emits a single span stream that is exported to two OpenTelemetry exporters (Langfuse and Arize Phoenix).
• Dual‑backend comparison – The Next.js frontend renders the same trace twice, letting developers see latency, cost and evaluation differences side‑by‑side.
• LLM‑as‑a‑Judge – DeepEval runs four metrics (Faithfulness, Contextual Precision, Answer Relevancy, Hallucination) using Anthropic Claude Haiku 4.5 and writes the scores back to both backends.
• HTML‑in‑Canvas – Experimental Chrome canvas APIs (drawElementImage) provide high‑performance rendering with a graceful Tailwind CSS fallback.

---

Key Features

• Dual‑Backend Tracing – OpenInference → Langfuse & Arize Phoenix.
• Metric Evaluation – DeepEval + Claude Haiku 4.5.
• Race‑Track UI – Side‑by‑side view of Input, Retrieval, Generation, Evaluation stages.
• HTML‑in‑Canvas Rendering – Chrome origin trial (drawElementImage) with fallback.
• Zero‑Trust Access – Cloudflare OTP during development; disabled for public release.
• Secrets Management – Infisical integration; no secrets live in the repo.

---

Tech Stack

Layer	Technology
Frontend	Next.js 16 (App Router), Tailwind CSS, Framer Motion, TypeScript
Hosting	Vercel (Free/Hobby)
Tracing & Observability	OpenInference (Python), Langfuse Cloud, Arize Phoenix Cloud
Evaluation	DeepEval (Python) – Claude Haiku 4.5
LLM Generation	Google Gemini 2.5 Flash
Secret Management	Infisical (cloud sync)

---

Project Structure

README.md                ← this file
evalui/                  ← Next.js app (frontend + API routes)
├─ app/                  ← pages & components
├─ lib/                  ← helpers, types, OTel clients
├─ public/               ← static assets
scripts/                 ← Python demo & evaluation scripts
│  ├─ demo_app.py        ← generates three demo traces
│  └─ evaluate.py        ← runs DeepEval against a trace
scoping.md               ← architectural contract
RUNBOOK.md               ← step‑by‑step build guide
HOW_TO_DEMO.md           ← quick demo guide
package.json, tsconfig.* ← Node tooling

---

Quick Start

Prerequisites – Node ≥ 20, pnpm (or npm), Python ≥ 3.11, Infisical CLI installed and logged in.

# Clone the repo (public URL will be added after release)
git clone https://github.com/your‑org/evalui.git
cd evalui/evalui

# Python environment
python -m venv .venv
source .venv/bin/activate
pip install -r scripts/requirements.txt

# Node dependencies
pnpm install   # or `npm ci`

# Start the dev server
pnpm dev       # http://localhost:3000

The dashboard will be available at http://localhost:3000.

---

Running the Demo Scripts

1. Inject secrets via Infisical (once per terminal):

   INFI="infisical run --env=prod \
          --path=/langfuse_cloud \
          --path=/phoenix_arise \
          --path=/llms"

2. Generate three demo traces:

   $INFI python scripts/demo_app.py --scenario=direct
   $INFI python scripts/demo_app.py --scenario=rag
   $INFI python scripts/demo_app.py --scenario=hallucinate

   Each run prints a TRACE_ID=….

3. Score a trace (replace <TRACE_ID>):

   $INFI python scripts/evaluate.py --trace-id=<TRACE_ID>

4. Refresh the dashboard cache (optional, for live demo):

   curl https://evalui.davidcockson.com/api/revalidate

For full instructions, see HOW_TO_DEMO.md.

---

Deployment (Live Demo)

The production instance lives at https://evalui.davidcockson.com and is deployed on Vercel.

• Zero‑Trust Access is disabled for the public site – the dashboard is fully open.
• Secrets (Infisical, API keys) remain server‑side only and are never exposed to the browser.
• ISR (revalidate = 60) keeps the API cheap on free tiers.

---

Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository.
2. Create a feature branch (git checkout -b feat/your‑feature).
3. Open a Pull Request with a clear description and screenshots (if UI changes).
4. Ensure all linting passes (pnpm lint) and tests (if any) succeed.
5. CI will run automatically on the PR.

When adding new environment variables, update the Infisical configuration and add the variable to .env.example (with placeholder values).

---

Security & Public Release

• Do not commit secrets – the .gitignore already excludes .env*, .infisical.json, .deepeval/, and the Python virtual‑env folder.
• Verify the repo is clean before making it public:

  git status                # should show a clean working tree
  git ls-files | grep -E '\.env|\.infisical|\.deepeval'   # should return nothing

• If you decide to initialise a git repository at the project root (/home/dave/Documents/GitHub/evalui), add an additional .gitignore there to exclude the outer .venv/ and any IDE settings.
• The MIT license grants permissive reuse; feel free to adapt the UI or data contract for your own projects.

---

License

MIT © 2026 David Cockson