Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,21 @@ The canonical backend test command is run from the repository root, so the
app/env/bin/python -m pytest -q
```

## Task cost report

Every pull request and delegated coding task must include a short cost report.
Record measurements before finishing the work, and use `N/A` when a tool does
not expose them; never invent token counts. The canonical template and the
definitions of each field are in
[`docs/development/TASK_COST_REPORT.md`](docs/development/TASK_COST_REPORT.md).

The report belongs in the PR description (or in its final handoff comment) and
must distinguish simulated tests from live provider calls. Unit tests and
simulated E2E tests normally cost **0 external LLM tokens**. Live Wizard/LLM
calls must include the provider-reported prompt, completion and total tokens
when available. Media generation count and elapsed time should be recorded as
well.

### The clean-repo guard

`scripts/verify_clean_repo.py` enforces that certain **locally-generated or
Expand Down
48 changes: 48 additions & 0 deletions docs/development/TASK_COST_REPORT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Task cost report

This lightweight report makes the cost of delegated work and live application
checks visible without adding telemetry or a billing system to HocusPocus.

## When to write it

Add the report to every pull request and to the final human handoff for work
that does not create a PR. Start collecting the values before running live
provider calls or long-running checks so that retries are included.

## Template

Copy this block into the PR description or final handoff comment:

```md
## Coste de la tarea

- Tests simulados: 0 tokens externos
- Tests reales: N/A
- Llamadas LLM externas: 0
- Tokens de prompt: N/A
- Tokens de respuesta: N/A
- Tokens totales: N/A
- Generaciones de imágenes/audio/vídeo: 0
- Tiempo transcurrido: N/A
- Proveedores/modelos: N/A
```

Replace `N/A` with provider-reported values when a tool exposes them. Keep
separate counts for independent providers, and include retries in the totals.
For HocusPocus tasks, Activity/task metadata may already contain
`prompt_tokens`, `completion_tokens`, `total_tokens`, `calls`, provider/model,
`task_id`, `root_task_id` and duration; copy those values instead of estimating.

## Counting rules

- Unit tests, static checks, builds and simulated E2E tests use **0 external
LLM tokens**, even if they exercise an LLM-shaped adapter.
- A live Wizard or LLM integration test counts the provider's reported usage.
- Image, audio and video generation is counted separately from text tokens.
- Codex, Luna, Cursor and Grok usage is reported only when their own tool
exposes a reliable number. Otherwise write `N/A`.
- Never include API keys, prompts containing secrets, or private user content in
the report.

This is intentionally a reporting convention, not telemetry: it has no network
calls, does not change runtime behavior, and does not block local-first use.