Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 8 additions & 11 deletions uc-0a/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,14 @@
# 4. Paste the output below

role: >
[FILL IN]

You are an AI Complaint Classifier designed for the City Operations team to process citizen submissions.
intent: >
[FILL IN]

To accurately classify each complaint by outputting a category, priority, reason, and flag, ensuring reliable data feeds into the Director's dashboard every Monday.
context: >
[FILL IN]

The City Operations team receives hundreds of complaints weekly. Prior versions of the classifier failed by inventing category names, missing urgent incidents involving children/injuries, and answering ambiguous complaints with false confidence. High diligence to the strict taxonomy and severity triggers is required to prevent these failures.
enforcement:
- "[FILL IN: category enum rule]"
- "[FILL IN: severity keyword rule — list the keywords]"
- "[FILL IN: reason field rule]"
- "[FILL IN: ambiguity refusal rule]"
- "[FILL IN: no invented categories rule]"
- "Category must be exactly one value from the allowed list: Pothole, Flooding, Streetlight, Waste, Noise, Road Damage, Heritage Damage, Heat Hazard, Drain Blockage, Other. No variations."
- "Priority must be Urgent if description contains any severity keyword: injury, child, school, hospital, ambulance, fire, hazard, fell, collapse."
- "Every output row must include a reason field citing specific words from the description."
- "If category cannot be determined confidently — output `category: Other` and `flag: NEEDS_REVIEW`."
- "Never invent category names outside the allowed list."
80 changes: 77 additions & 3 deletions uc-0a/classifier.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,91 @@
"""
import argparse
import csv
import argparse
import csv
import os

def classify_complaint(row: dict) -> dict:
"""
Classify a single complaint row.
Classify a single complaint row using rule-based heuristics
to strictly adhere to the agents.md schema.
Returns dict with: complaint_id, category, priority, reason, flag
"""
raise NotImplementedError("Build this using your AI tool + agents.md")
desc = row.get('description', '').lower()

# Enforcement 2: Severity keywords -> Urgent
severity_keywords = ["injury", "child", "school", "hospital", "ambulance", "fire", "hazard", "fell", "collapse"]
is_urgent = any(word in desc for word in severity_keywords)
priority = "Urgent" if is_urgent else "Standard"

# Enforcement 1 & 5: Exact categories mapping
category_map = {
"pothole": "Pothole",
"flood": "Flooding",
"streetlight": "Streetlight",
"light": "Streetlight",
"garbage": "Waste",
"waste": "Waste",
"music": "Noise",
"noise": "Noise",
"crack": "Road Damage",
"surface": "Road Damage",
"tile": "Road Damage",
"heritage": "Heritage Damage",
"heat": "Heat Hazard",
"drain": "Drain Blockage",
"manhole": "Drain Blockage"
}

category = "Other"
found_keyword = None
for kw, cat in category_map.items():
if kw in desc:
category = cat
found_keyword = kw
break

# Enforcement 4: Ambiguity -> Other + NEEDS_REVIEW
if category == "Other":
flag = "NEEDS_REVIEW"
reason = "The description provided was ambiguous and could not be confidently mapped to a specific category."
else:
flag = ""
# Enforcement 3: Cite specific words
reason = f"The description mentions '{found_keyword}' which classifies it under {category}."
if is_urgent:
reason += " Escalated to Urgent due to severity keywords."

return {
"complaint_id": row.get("complaint_id", ""),
"category": category,
"priority": priority,
"reason": reason,
"flag": flag
}

def batch_classify(input_path: str, output_path: str):
"""Read input CSV, classify each row, write results CSV."""
raise NotImplementedError("Build this using your AI tool + agents.md")
fieldnames = ['complaint_id', 'category', 'priority', 'reason', 'flag']

with open(input_path, mode='r', encoding='utf-8') as infile:
reader = csv.DictReader(infile)
rows = list(reader)

results = []
for i, row in enumerate(rows):
print(f"Classifying {i+1}/{len(rows)}: ID {row.get('complaint_id', 'Unknown')}")
classification = classify_complaint(row)
results.append(classification)

if os.path.dirname(output_path):
os.makedirs(os.path.dirname(output_path), exist_ok=True)

with open(output_path, mode='w', newline='', encoding='utf-8') as outfile:
writer = csv.DictWriter(outfile, fieldnames=fieldnames)
writer.writeheader()
for res in results:
writer.writerow(res)

if __name__ == "__main__":
parser = argparse.ArgumentParser(description="UC-0A Complaint Classifier")
Expand Down
16 changes: 16 additions & 0 deletions uc-0a/results_ahmedabad.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
complaint_id,category,priority,reason,flag
AM-202401,Road Damage,Standard,The description mentions 'surface' which classifies it under Road Damage.,
AM-202402,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
AM-202405,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
AM-202406,Heat Hazard,Standard,The description mentions 'heat' which classifies it under Heat Hazard.,
AM-202407,Other,Urgent,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
AM-202410,Pothole,Standard,The description mentions 'pothole' which classifies it under Pothole.,
AM-202414,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
AM-202417,Waste,Standard,The description mentions 'waste' which classifies it under Waste.,
AM-202421,Noise,Standard,The description mentions 'music' which classifies it under Noise.,
AM-202424,Road Damage,Standard,The description mentions 'surface' which classifies it under Road Damage.,
AM-202429,Road Damage,Standard,The description mentions 'surface' which classifies it under Road Damage.,
AM-202431,Heritage Damage,Standard,The description mentions 'heritage' which classifies it under Heritage Damage.,
AM-202435,Heat Hazard,Standard,The description mentions 'heat' which classifies it under Heat Hazard.,
AM-202444,Waste,Standard,The description mentions 'waste' which classifies it under Waste.,
AM-202445,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
16 changes: 16 additions & 0 deletions uc-0a/results_hyderabad.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
complaint_id,category,priority,reason,flag
GH-202401,Flooding,Urgent,The description mentions 'flood' which classifies it under Flooding. Escalated to Urgent due to severity keywords.,
GH-202402,Flooding,Standard,The description mentions 'flood' which classifies it under Flooding.,
GH-202406,Drain Blockage,Standard,The description mentions 'drain' which classifies it under Drain Blockage.,
GH-202407,Drain Blockage,Standard,The description mentions 'drain' which classifies it under Drain Blockage.,
GH-202410,Pothole,Standard,The description mentions 'pothole' which classifies it under Pothole.,
GH-202411,Pothole,Urgent,The description mentions 'pothole' which classifies it under Pothole. Escalated to Urgent due to severity keywords.,
GH-202412,Pothole,Urgent,The description mentions 'pothole' which classifies it under Pothole. Escalated to Urgent due to severity keywords.,
GH-202417,Waste,Standard,The description mentions 'garbage' which classifies it under Waste.,
GH-202420,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
GH-202422,Other,Urgent,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
GH-202424,Flooding,Standard,The description mentions 'flood' which classifies it under Flooding.,
GH-202428,Waste,Standard,The description mentions 'waste' which classifies it under Waste.,
GH-202432,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
GH-202448,Flooding,Standard,The description mentions 'flood' which classifies it under Flooding.,
GH-202438,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
16 changes: 16 additions & 0 deletions uc-0a/results_kolkata.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
complaint_id,category,priority,reason,flag
KM-202401,Heritage Damage,Standard,The description mentions 'heritage' which classifies it under Heritage Damage.,
KM-202402,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
KM-202405,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
KM-202409,Pothole,Standard,The description mentions 'pothole' which classifies it under Pothole.,
KM-202410,Pothole,Standard,The description mentions 'pothole' which classifies it under Pothole.,
KM-202411,Pothole,Standard,The description mentions 'pothole' which classifies it under Pothole.,
KM-202415,Drain Blockage,Standard,The description mentions 'drain' which classifies it under Drain Blockage.,
KM-202418,Waste,Standard,The description mentions 'waste' which classifies it under Waste.,
KM-202421,Other,Urgent,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
KM-202422,Road Damage,Standard,The description mentions 'surface' which classifies it under Road Damage.,
KM-202426,Heritage Damage,Standard,The description mentions 'heritage' which classifies it under Heritage Damage.,
KM-202430,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
KM-202434,Heritage Damage,Standard,The description mentions 'heritage' which classifies it under Heritage Damage.,
KM-202436,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
KM-202438,Heritage Damage,Standard,The description mentions 'heritage' which classifies it under Heritage Damage.,
16 changes: 16 additions & 0 deletions uc-0a/results_pune.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
complaint_id,category,priority,reason,flag
PM-202401,Pothole,Standard,The description mentions 'pothole' which classifies it under Pothole.,
PM-202402,Pothole,Urgent,The description mentions 'pothole' which classifies it under Pothole. Escalated to Urgent due to severity keywords.,
PM-202406,Flooding,Standard,The description mentions 'flood' which classifies it under Flooding.,
PM-202408,Flooding,Standard,The description mentions 'flood' which classifies it under Flooding.,
PM-202410,Streetlight,Standard,The description mentions 'streetlight' which classifies it under Streetlight.,
PM-202411,Streetlight,Urgent,The description mentions 'streetlight' which classifies it under Streetlight. Escalated to Urgent due to severity keywords.,
PM-202413,Waste,Standard,The description mentions 'garbage' which classifies it under Waste.,
PM-202418,Noise,Standard,The description mentions 'music' which classifies it under Noise.,
PM-202419,Road Damage,Standard,The description mentions 'crack' which classifies it under Road Damage.,
PM-202420,Drain Blockage,Urgent,The description mentions 'manhole' which classifies it under Drain Blockage. Escalated to Urgent due to severity keywords.,
PM-202427,Flooding,Standard,The description mentions 'flood' which classifies it under Flooding.,
PM-202428,Other,Standard,The description provided was ambiguous and could not be confidently mapped to a specific category.,NEEDS_REVIEW
PM-202430,Streetlight,Standard,The description mentions 'light' which classifies it under Streetlight.,
PM-202433,Waste,Standard,The description mentions 'waste' which classifies it under Waste.,
PM-202446,Road Damage,Urgent,The description mentions 'tile' which classifies it under Road Damage. Escalated to Urgent due to severity keywords.,
16 changes: 8 additions & 8 deletions uc-0a/skills.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@

skills:
- name: classify_complaint
description: "[FILL IN]"
input: "[FILL IN]"
output: "[FILL IN]"
error_handling: "[FILL IN]"
description: "Process a single complaint to determine its category, priority level, justification reason, and any necessary review flags."
input: "one complaint row (dict with description, location fields)"
output: "dict with category, priority, reason, flag"
error_handling: "vague/short descriptions → Other + NEEDS_REVIEW"

- name: batch_classify
description: "[FILL IN]"
input: "[FILL IN]"
output: "[FILL IN]"
error_handling: "[FILL IN]"
description: "Process multiple complaints from an input CSV file and write classification results to an output CSV file."
input: "path to test CSV file"
output: "path to results CSV file"
error_handling: "malformed rows logged and skipped, processing continues"
33 changes: 8 additions & 25 deletions uc-mcp/agents.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,15 @@
# agents.md — UC-MCP MCP Server
# INSTRUCTIONS:
# 1. Open your AI tool
# 2. Paste the full contents of uc-mcp/README.md
# 3. Use this prompt:
# "Read this UC README. Using the R.I.C.E framework, generate an
# agents.md YAML with four fields: role, intent, context, enforcement.
# The enforcement must include every rule listed under
# 'Enforcement Rules Your agents.md Must Include'.
# Output only valid YAML."
# 4. Paste the output below, replacing this placeholder
# 5. Pay special attention to enforcement rule 1 — the tool description
# must state exact document scope

role: >
[FILL IN: Who is this agent? What layer of the stack does it operate at?
Hint: an MCP server that exposes policy retrieval as a tool]
An MCP (Model Context Protocol) server over plain HTTP that exposes the UC-RAG policy retrieval functionality as a discoverable, standardized tool for AI agents.

intent: >
[FILL IN: What does a correctly implemented MCP server produce?
Hint: JSON-RPC compliant responses, scoped tool description, correct refusals]
To strictly enforce tool boundaries through precise tool descriptions and schemas, ensuring querying agents only call the server for supported CMC policies, and to provide correctly formatted JSON-RPC tool boundaries and responses.

context: >
[FILL IN: What does this server have access to?
Hint: RAG server results only — no direct LLM calls, no outside knowledge]
Has access only to the RAG server results via query_policy_documents — no direct LLM calls, no outside knowledge. It acts strictly as an HTTP JSON-RPC bridge out to agents.

enforcement:
- "[FILL IN: Tool description scope rule]"
- "[FILL IN: Refusal documentation rule]"
- "[FILL IN: inputSchema required field rule]"
- "[FILL IN: isError on failure rule]"
- "[FILL IN: HTTP 200 for all JSON-RPC responses rule]"
- "Tool description must state the exact document scope: CMC HR Leave Policy, IT Acceptable Use Policy, Finance Reimbursement Policy."
- "Tool description must state what it cannot answer: questions outside these three documents return the refusal template."
- "inputSchema must require `question` as a non-empty string."
- "Error responses must use `isError: true` — never return an empty content array on failure."
- "The server must return HTTP 200 for all JSON-RPC responses including errors — transport errors use HTTP 4xx/5xx, application errors use JSON-RPC error objects."
Loading