CeloJudge

AI reviewer agent for the Celo Agentic Payments & DeFAI Hackathon

CeloJudge autonomously inspects digital artifacts — websites, API endpoints, MCP servers, and skill files — and produces a structured, LLM-powered review with scores, strengths, weaknesses, and actionable recommendations.

Features

🔍 Four fetchers: website, API endpoint, MCP server, skill file (SKILL.md)
🧠 LLM scoring engine: Gemini or Claude, configurable via env var
📊 Five criteria: clarity, functionality, usefulness, technical quality, documentation
🤖 Askbots integration: registers as an OpenClaw molt bot, polls projects, auto-submits reviews, solves anti-human math challenge
🏛️ Aigora integration: stub ready to wire up once API docs are confirmed
⛓️ Celo attribution: toDataSuffix(['celojudge', '<attributionTag>']) on every onchain transaction
✅ Validated output: Zod schema ensures review JSON always matches the contract

Output Schema

Every review produces a JSON object matching this schema:

{
  "artifact_type": "website" | "api" | "mcp_server" | "skill_file",
  "artifact_url": "https://...",
  "scores": {
    "clarity": 8,
    "functionality": 7,
    "usefulness": 9,
    "technical_quality": 6,
    "documentation": 7
  },
  "overall_score": 74,
  "strengths": "2-3 sentences about what works well.",
  "weaknesses": "2-3 sentences about what needs improvement.",
  "recommendations": "2-3 concrete, actionable suggestions."
}

Score Weights

Criterion	Weight
functionality	25%
usefulness	25%
clarity	20%
technical_quality	20%
documentation	10%

Overall score = weighted average × 10 (scales 1–10 to 10–100).

Quick Start

1. Install

cd celojudge
npm install
npm run build

2. Configure

cp .env.example .env
# Edit .env and fill in your API keys

Required for LLM scoring (choose one):

LLM_PROVIDER=gemini        # or "claude"
GEMINI_API_KEY=AIza...     # if using Gemini
ANTHROPIC_API_KEY=sk-ant-... # if using Claude

3. Run a review

# Website
npx ts-node src/cli.ts review https://example.com --type website --pretty

# API endpoint
npx ts-node src/cli.ts review https://jsonplaceholder.typicode.com/posts/1 --type api

# Skill file (GitHub blob URL auto-converted to raw)
npx ts-node src/cli.ts review https://github.com/owner/repo/blob/main/SKILL.md --type skill_file

# After building:
node dist/cli.js review https://example.com --type website --pretty

CLI Reference

celojudge review <url> [options]
  -t, --type <type>        Artifact type: website | api | mcp_server | skill_file  [required]
  --submit-askbots         Auto-submit to matching Askbots projects after review
  --submit-aigora          Submit to Aigora (stub)
  --record-onchain         Record a hash of the review on the Celo blockchain
  --project-id <id>        Target a specific Askbots project
  --pretty                 Pretty-print JSON output

celojudge askbots:register [options]
  --name <name>            Bot name
  --description <desc>     Bot description
  --celo-address <addr>    Celo wallet address for payouts

celojudge askbots:status   Check Askbots registration status
celojudge askbots:projects List available Askbots projects

celojudge aigora:register [options]
  --name <name>            Agent name (stub)
  --description <desc>     Agent description (stub)

Askbots Integration

Askbots (https://main--askbots.netlify.app) is a feedback marketplace where bots earn USDT by reviewing websites, APIs, MCP servers, and skill files.

Setup

Step 1: Register (one-time)

node dist/cli.js askbots:register \
  --name "CeloJudge" \
  --description "AI reviewer agent for Celo hackathon" \
  --celo-address 0xYourCeloWallet

Copy the printed apiKey to your .env:

ASKBOTS_API_KEY=askbots_...
ASKBOTS_AGENT_ID=...

Step 2: Check status

node dist/cli.js askbots:status

Step 3: List available projects

node dist/cli.js askbots:projects

Step 4: Review + auto-submit

node dist/cli.js review https://example.com --type website --submit-askbots --pretty

How it works

GET /api/projects — find projects matching this bot's skills
Filter by propertyType and propertyUrl matching the reviewed artifact
POST /api/projects/:id/respond — submit answers mapped from the CeloJudge review
Solve the anti-human math challenge within 2 seconds
POST /api/projects/:id/verify-challenge — earn $0.10 USDT to your Celo wallet

Anti-human challenge

After submitting a response, Askbots returns a math challenge (e.g. "What is 847293 * 193847 + 582910384?"). CeloJudge uses BigInt arithmetic to evaluate it precisely and respond within the 2-second timeout.

Aigora Integration

⚠️ STUB: The Aigora API is not yet publicly documented. This module is ready to wire up.

The Aigora hackathon track (Track 4: Best Feedback for Aigora) appears to require:

aigoraProfileUrl: your public Aigora profile (https://aigora.org/services/<id>)
aigoraFeedbackIssueUrl: a GitHub issue created via the aigora-feedback skill in trionlabs/aigora-skills

To activate the Aigora module

Get API credentials from aigora.org

Add to .env:

AIGORA_API_KEY=your_key_here
AIGORA_API_BASE_URL=https://aigora.org/api  # or confirm real URL

Open src/integrations/aigora/client.ts and replace the stub endpoint paths with the confirmed routes
Remove the // TODO: confirm against real Aigora API docs once available comments

Celo Attribution Tag

Every onchain transaction this agent makes must include the ERC-8021 attribution tag:

import { toDataSuffix } from '@celo/attribution-tags';

// Single tag
await wallet.sendTransaction({ to, value, data: toDataSuffix('<attributionTag>') });

// Multiple codes (keep your existing tag alongside the registered one)
const data = toDataSuffix(['celojudge', '<attributionTag>']);
await wallet.sendTransaction({ to, value, data });

The attributionTag (format: celo_xxxxxxxxxxxx) is returned when you register your project on celobuilders.xyz. Set it as:

CELO_ATTRIBUTION_TAG=celo_xxxxxxxxxxxx

Tests

# Unit tests only (no network)
npm test -- tests/scorer.test.ts tests/schema.test.ts

# All tests including integration (requires network)
npm test

# Watch mode
npm run test:watch

Test coverage:

tests/scorer.test.ts — scoring engine weight/range logic (no network)
tests/schema.test.ts — Zod schema validation (no network)
tests/fetchers.test.ts — live fetcher tests against real public URLs

Project Structure

celojudge/
├── src/
│   ├── config/
│   │   └── index.ts          # All env vars, never hardcoded
│   ├── core/
│   │   ├── types.ts           # Zod schemas + TypeScript types
│   │   └── scorer.ts          # LLM prompt, Gemini/Claude calls, score computation
│   ├── fetchers/
│   │   ├── index.ts           # Route by artifact type
│   │   ├── website.ts         # HTML → clean text
│   │   ├── api.ts             # HTTP endpoint inspector
│   │   ├── mcp.ts             # MCP server discovery
│   │   └── skill-file.ts      # Markdown SKILL.md reader
│   ├── integrations/
│   │   ├── askbots/
│   │   │   ├── client.ts      # Full Askbots API implementation
│   │   │   └── index.ts       # High-level orchestration
│   │   └── aigora/
│   │       ├── client.ts      # Stub (TODO: real API docs)
│   │       └── index.ts       # Graceful fallback orchestration
│   ├── index.ts               # Main runReview() pipeline
│   └── cli.ts                 # Commander.js CLI
├── tests/
│   ├── scorer.test.ts         # Scoring engine unit tests
│   ├── schema.test.ts         # Zod schema unit tests
│   └── fetchers.test.ts       # Fetcher integration tests
├── .env.example               # Template for all required env vars
├── package.json
├── tsconfig.json
└── README.md

Environment Variables

Variable	Required	Description
`LLM_PROVIDER`	Yes	`gemini`, `claude`, or `groq`
`GEMINI_API_KEY`	If provider=gemini	Google AI Studio key
`ANTHROPIC_API_KEY`	If provider=claude	Anthropic API key
`GROQ_API_KEY`	If provider=groq	Groq API key
`GROQ_MODEL`	Optional	Groq model to use (default: `llama-3.3-70b-versatile`)
`ASKBOTS_API_KEY`	For Askbots	From `askbots:register`
`ASKBOTS_AGENT_ID`	For Askbots	From `askbots:register`
`AIGORA_API_KEY`	For Aigora stub	From aigora.org (TBD)
`AIGORA_API_BASE_URL`	For Aigora stub	Default: `https://aigora.org/api`
`CELO_WALLET_ADDRESS`	For payouts	Your Celo wallet
`CELO_PRIVATE_KEY`	For onchain recording	Your Celo wallet private key
`CELO_RPC_URL`	Optional	Default: `https://forno.celo.org`
`CELO_ATTRIBUTION_TAG`	For onchain txns	From celobuilders.xyz registration

License

MIT

Name		Name	Last commit message	Last commit date
.devin		.devin
celojudge		celojudge
frontend		frontend
src		src
tests		tests
.gitignore		.gitignore
INSTALLATION.md		INSTALLATION.md
LICENSE		LICENSE
RAILWAY_DEPLOYMENT.md		RAILWAY_DEPLOYMENT.md
README.md		README.md
START.bat		START.bat
VERCEL_DEPLOYMENT.md		VERCEL_DEPLOYMENT.md
jest.config.json		jest.config.json
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json

Navigation Menu

Repository files navigation

CeloJudge

Features

Output Schema

Score Weights

Quick Start

1. Install

2. Configure

3. Run a review

CLI Reference

Askbots Integration

Setup

How it works

Anti-human challenge

Aigora Integration

To activate the Aigora module

Celo Attribution Tag

Tests

Project Structure

Environment Variables

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

CeloJudge

Features

Output Schema

Score Weights

Quick Start

1. Install

2. Configure

3. Run a review

CLI Reference

Askbots Integration

Setup

How it works

Anti-human challenge

Aigora Integration

To activate the Aigora module

Celo Attribution Tag

Tests

Project Structure

Environment Variables

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages