Skip to content

Commit 52cc32e

Browse files
committed
docs: rewrite README for community appeal
Authored with a pi-taskflow flow (discover → 4 parallel section writers → critic review → assemble), then fact-checked against the source. - Lead with the shorthand on-ramp ("you already know the API") and a punchy declarative tagline; add npm/license/Pi badges. - New "Watch it run" hero: the real DAG progress render (topological layers, ┌├└ concurrency rail, ↳ layer-skipping edge, gate BLOCK) with a guide to reading it. - Expand the comparison table (quality gates, progress visibility, ergonomics). - Quickstart now goes shorthand (single/parallel/chain) → full DSL (map + reduce). - Honest "What it doesn't do (yet)": no detached background run, no output:file, map needs a JSON array, cycles rejected. - Fix accuracy: default concurrency 8, --append-system-prompt invocation, output is text|json only, agentScope table; corrected/validated JSON examples.
1 parent 6c573ce commit 52cc32e

1 file changed

Lines changed: 237 additions & 53 deletions

File tree

‎README.md‎

Lines changed: 237 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -1,94 +1,238 @@
11
# pi-taskflow
22

3+
[![npm](https://img.shields.io/npm/v/pi-taskflow.svg)](https://www.npmjs.com/package/pi-taskflow)
4+
[![license](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
5+
[![Pi coding agent](https://img.shields.io/badge/for-Pi%20coding%20agent-7c3aed.svg)](https://pi.dev)
6+
37
> Lightweight workflow orchestration for the [Pi coding agent](https://pi.dev).
48
5-
Describe a multi-phase workflow once; let a deterministic runtime orchestrate
6-
isolated subagents to execute it. Intermediate results stay **out of your main
7-
context** — only the final answer comes back. Inspired by Claude Code's Dynamic
8-
Workflows, rebuilt as a lightweight, declarative pi extension.
9+
**Orchestrate your Pi subagents. Not by prompting — by declaring.**
10+
11+
If you've used the built-in subagent tool's `task` / `tasks` / `chain`, you
12+
already know the shorthand — your runs just get tracked, resumable, and
13+
saveable as a one-word `/tf:<name>` command.
914

1015
```bash
1116
pi install npm:pi-taskflow
1217
```
1318

19+
Fan out one subagent per item, gate the results with an adversarial review, and
20+
get back only the final report — none of the intermediate transcripts ever touch
21+
your conversation.
22+
1423
## Why
1524

16-
`subagent` is great for a single delegated task. But when a job needs **many
17-
coordinated steps**, **fan-out over dozens of items**, **cross-checked review**,
18-
or a **repeatable** pipeline, you want orchestration — without bloating your
19-
conversation with every step's transcript.
25+
The built-in subagent tool is great for a single delegated task. But when a job
26+
needs many coordinated steps, fan-out over dozens of items, cross-checked review,
27+
or a repeatable pipeline, you want orchestration — without the intermediate
28+
transcripts eating your context window.
2029

2130
`pi-taskflow` moves the plan into a small declarative definition. The runtime
2231
holds the DAG, the loops, and the intermediate results; your context receives
2332
only the final phase's output.
2433

25-
| | `subagent` | `pi-taskflow` |
34+
| | `subagent` tool | `pi-taskflow` |
2635
|---|---|---|
2736
| Who drives | the model, turn by turn | the runtime, from a definition |
2837
| Intermediate results | in your context window | in the runtime (not your context) |
2938
| Reusable | re-described each time | saved as `/tf:<name>` |
3039
| Scale | a few tasks | dynamic `map` fan-out |
31-
| Resumable | no | yes (cross-session) |
40+
| Resumable | no | yes (cross-session, cached phases skip) |
41+
| Quality gates | no | `gate` phases with `VERDICT: BLOCK / PASS` |
42+
| Progress visibility | opaque while running | live DAG render with timing + cost |
43+
| Ergonomics | inline JSON each time | shorthand (`task`/`tasks`/`chain`) or DSL |
44+
45+
## Show me
46+
47+
Describe a pipeline once, then run it from a pi session by name:
48+
49+
> `/tf:summarize-files dir=src`
50+
51+
The runtime fans out one subagent per file, merges the summaries in a `reduce`
52+
phase, and returns only the final overview. Every intermediate transcript stays
53+
in the runtime — never in your context window. (Full definition in
54+
[Quickstart](#then-go-declarative) below.)
55+
56+
## Quickstart
57+
58+
### Shorthand: same effort as `subagent`, but tracked & resumable
59+
60+
**Single task** — one agent, one job:
61+
62+
```jsonc
63+
{ "task": "Summarize the architecture of src/", "agent": "explorer" }
64+
```
65+
66+
**Parallel tasks** — fire several at once, outputs merge:
67+
68+
```jsonc
69+
{ "tasks": [
70+
{ "task": "Audit auth in src/api", "agent": "analyst" },
71+
{ "task": "Audit input validation in src/api", "agent": "analyst" }
72+
] }
73+
```
3274

33-
## Concepts
75+
**Chain** — sequential, each step sees the previous one's output:
76+
77+
```jsonc
78+
{ "chain": [
79+
{ "task": "List the public API of src/lib", "agent": "scout" },
80+
{ "task": "Write docs for:\n{previous.output}", "agent": "writer" }
81+
] }
82+
```
3483

35-
A **taskflow** is a set of **phases** forming a DAG via `dependsOn`. Each phase
36-
delegates to a subagent (an isolated `pi` process). Phases in the same DAG layer
37-
run concurrently (bounded by `concurrency`).
84+
`agent` is optional (defaults to the first available agent). Add `name` to label
85+
the run and enable saving it as a reusable command.
3886

39-
### Phase types
87+
Try it inline — tell the model something like:
4088

41-
| type | meaning |
42-
|------|---------|
43-
| `agent` | one subagent runs `task` |
44-
| `parallel` | run `branches[]` concurrently |
45-
| `map` | fan out over an array — one subagent per item, `{item}` bound |
46-
| `gate` | quality/adversarial-review step |
47-
| `reduce` | aggregate several phases' outputs into one |
89+
> Run a chain: first explore the auth flow, then summarize findings.
4890
49-
### Interpolation
91+
The model calls the `taskflow` tool; you get live progress, per-step timing,
92+
token cost, and a run record. Ask to `save` it and you get `/tf:<name>`.
5093

51-
- `{args.X}` — invocation argument
52-
- `{steps.ID.output}` — a prior phase's text output
53-
- `{steps.ID.json}` / `{steps.ID.json.field}` — prior output parsed as JSON
54-
- `{item}` / `{item.field}` — current item inside a `map` phase
55-
- `{previous.output}` — the immediately-upstream phase output
94+
### Then go declarative
5695

57-
## Example
96+
When your pipeline outgrows the shorthand — when you need dynamic fan-out,
97+
intermediate JSON routing, or quality gates — graduate to the full DSL:
5898

5999
```jsonc
60100
{
61101
"name": "summarize-files",
102+
"description": "Discover files, summarize each, produce a report",
62103
"args": { "dir": { "default": "." } },
63-
"concurrency": 4,
104+
"concurrency": 8,
64105
"phases": [
65106
{ "id": "discover", "type": "agent", "agent": "scout",
66-
"task": "List source files under {args.dir}. Output ONLY a JSON array [{\"file\":\"\"}].",
107+
"task": "List source files under {args.dir} (non-recursive).\nOutput ONLY a JSON array [{\"file\":\"\"}]. No prose.",
67108
"output": "json" },
68-
{ "id": "summarize", "type": "map", "over": "{steps.discover.json}", "as": "item",
69-
"agent": "scout", "task": "Read {item.file} and summarize it in one sentence.",
109+
{ "id": "summarize", "type": "map",
110+
"over": "{steps.discover.json}", "as": "item",
111+
"agent": "scout",
112+
"task": "Read {item.file} and give a one-sentence summary.",
70113
"dependsOn": ["discover"] },
71-
{ "id": "report", "type": "reduce", "from": ["summarize"], "agent": "writer",
114+
{ "id": "report", "type": "reduce", "from": ["summarize"],
115+
"agent": "writer",
72116
"task": "Combine into a short overview:\n{steps.summarize.output}",
73117
"dependsOn": ["summarize"], "final": true }
74118
]
75119
}
76120
```
77121

78-
## Usage
122+
What this does:
123+
124+
1. **`discover`** — an agent lists every file in the directory and outputs a JSON array.
125+
2. **`summarize`** — a `map` fans out, spawning one subagent per file in parallel
126+
(throttled to 8 concurrent). Each gets `{item.file}` bound to its file path.
127+
3. **`report`** — a `reduce` merges all summaries into one clean overview.
128+
129+
Intermediate outputs never enter your context. The runtime owns them. You get
130+
only the final report back.
131+
132+
Save it once → `/tf:summarize-files` forever.
79133

80-
The model calls the `taskflow` tool; you can also drive it directly:
134+
## Watch it run
135+
136+
This is the live progress render for a real run — the `self-improve` flow that
137+
writes and verifies its own test suites, caught here mid-block by a quality gate:
81138

82139
```
83-
/tf list # saved flows
84-
/tf run <name> [args] # run a saved flow
85-
/tf show <name> # print a definition
86-
/tf runs # recent run history
87-
/tf resume <runId> # continue a paused/failed run (cached phases skipped)
88-
/tf:<name> [args] # shortcut per saved flow
140+
⊗ taskflow self-improve 6/7 · blocked · $0.095
141+
✓ discover agent deepseek-v4-flash 10t ↑38k ↓6.7k $0.011
142+
┌ ✓ write-runner-tests agent claude-sonnet-4-6 10t ↑13 ↓6.6k $0.020
143+
├ ✓ write-store-tests agent claude-sonnet-4-6 10t ↑11 ↓10k $0.018
144+
├ ✓ write-agents-tests agent claude-sonnet-4-6 10t ↑28 ↓13k $0.030
145+
└ ✓ fix-stability agent claude-sonnet-4-6 10t ↑13 ↓3.9k $0.012
146+
✓ verify gate BLOCK 3 type errors in test files deepseek-v4-flash
147+
⊘ report reduce skipped · Gate blocked ↳ fix-stability
89148
```
90149

91-
Tool actions: `run` (inline `define` or saved `name`), `save`, `resume`, `list`.
150+
**How to read it — the layout *is* the DAG:**
151+
152+
- **Header** — `⊗` means the flow is blocked (a gate halted it); `6/7` phases
153+
processed, aggregate cost `$0.095`.
154+
- **Status icons** — `✓` done, `◐` running, `✗` failed, `⊘` skipped, `○` pending.
155+
- **Rail `┌ ├ └`** — phases in the same DAG layer, running concurrently. The four
156+
`write-*`/`fix-stability` tasks all fan out from `discover`. A blank gutter is
157+
a single-phase layer.
158+
- **`↳`** — a long (layer-skipping) dependency. `report` depends on `verify` (the
159+
adjacent layer, implied by position) *and* `fix-stability` two layers back, so
160+
only that skip edge is annotated.
161+
- **Gate** — `verify` emitted `VERDICT: BLOCK`, so the runtime skipped `report`
162+
and ended the run as `blocked`, surfacing the reason.
163+
- **Detail** — per phase: model, token counts (`↑`in `↓`out), cost, and timing.
164+
Fan-out phases also show sub-task progress.
165+
166+
## Phase types
167+
168+
| type | meaning | required fields |
169+
|------|---------|-----------------|
170+
| `agent` | one subagent runs a single task | `task` |
171+
| `parallel` | run `branches[]` concurrently | `branches` (array of `{task, agent?}`) |
172+
| `map` | fan out over an array — one subagent per item, `{item}` bound | `over`, `task` |
173+
| `gate` | quality/review step that can **halt the flow** | `task` |
174+
| `reduce` | aggregate `from[]` phase outputs into one | `from`, `task` |
175+
176+
Every phase needs `id`. Optional fields: `agent`, `dependsOn`, `output`,
177+
`model`, `thinking`, `tools`, `cwd`, `concurrency`, `final`, `optional`.
178+
179+
### `output` format
180+
181+
- `output: "text"` (default) — the raw subagent output.
182+
- `output: "json"` — the subagent output is parsed as JSON and exposed via
183+
`{steps.ID.json}` / `{steps.ID.json.field}`. Set this on phases whose output
184+
a downstream `map` or `reduce` needs to consume as structured data.
185+
186+
There is no `output: "file"`. For file-based output, have the agent write to
187+
disk with a `write` tool call.
188+
189+
### Gate phases (quality control)
190+
191+
A `gate` runs an agent to review upstream output and can **block the rest
192+
of the workflow**. End the gate task's instructions by asking the agent to
193+
emit a verdict the runtime can read:
194+
195+
- a final line `VERDICT: PASS` or `VERDICT: BLOCK` (also accepts `OK`, `FAIL`,
196+
`STOP`, `REJECT`, `HALT` — last occurrence wins), or
197+
- JSON like `{"continue": false, "reason": "missing auth checks"}` /
198+
`{"verdict": "block", "reason": "..."}`.
199+
200+
On **BLOCK**, downstream phases are skipped and the run ends as `blocked` with
201+
the reason surfaced. **Ambiguous output fails open** (treated as PASS) — a gate
202+
never halts the flow by accident.
203+
204+
```
205+
Review the audit results below. If any endpoint is missing auth, end with
206+
"VERDICT: BLOCK" and a one-line reason; otherwise end with "VERDICT: PASS".
207+
208+
{steps.audit.output}
209+
```
210+
211+
## Interpolation
212+
213+
| placeholder | resolves to |
214+
|---|---|
215+
| `{args.X}` | invocation argument |
216+
| `{steps.ID.output}` | a prior phase's text output |
217+
| `{steps.ID.json}` | prior output parsed as JSON (or `{steps.ID.json.field}`) |
218+
| `{item}` / `{item.field}` | current item inside a `map` phase |
219+
| `{previous.output}` | the immediately-upstream phase output |
220+
221+
## Commands
222+
223+
Saved flows become CLI shortcuts. All commands work in the pi session:
224+
225+
| Command | What it does |
226+
|---|---|
227+
| `/tf list` | List all saved flows |
228+
| `/tf run <name> [args]` | Run a saved flow (e.g. `/tf run summarize-files dir=src`) |
229+
| `/tf show <name>` | Print a flow's definition |
230+
| `/tf runs` | Browse recent run history (interactive TUI) |
231+
| `/tf resume <runId>` | Continue a paused/failed run — cached phases skip automatically |
232+
| `/tf:<name> [args]` | Shortcut — runs the flow in one tap |
233+
234+
Tool actions (used by the model): `run` (inline `define` or saved `name`),
235+
`save`, `resume`, `list`.
92236

93237
## Storage
94238

@@ -98,30 +242,70 @@ Tool actions: `run` (inline `define` or saved `name`), `save`, `resume`, `list`.
98242
.pi/taskflows/runs/<runId>.json # run state (resume); gitignore this
99243
```
100244

245+
Agent discovery scope (set via `agentScope` in the flow definition):
246+
247+
| value | discovers agents from |
248+
|---|---|
249+
| `"user"` (default) | `~/.pi/agent/agents/*.md` |
250+
| `"project"` | `.pi/agents/*.md` (walks up the tree) |
251+
| `"both"` | user + project; project wins on name collision |
252+
101253
## Agents
102254

103-
Taskflow reuses your existing pi agents (`~/.pi/agent/agents/*.md`,
104-
`.pi/agents/*.md`) and honors `subagents.agentOverrides` in settings. Reference
105-
agents by `name`.
255+
Taskflow reuses your existing pi agent files (`~/.pi/agent/agents/*.md`,
256+
`.pi/agents/*.md`). Reference agents by `name` in a phase or shorthand.
257+
258+
When running a phase, the runtime extracts the agent's `systemPrompt` from its
259+
`.md` frontmatter and passes it via `--append-system-prompt` (written to a temp
260+
file). Phase-level overrides for `model`, `thinking`, and `tools` are passed as
261+
`--model` / `--thinking` / `--tools` flags to the subagent invocation.
262+
263+
Settings from `~/.pi/agent/settings.json` (the `subagents.agentOverrides` map)
264+
are honored, letting you tweak model, thinking, or tools per agent across all flows.
265+
266+
## Status & limits
267+
268+
- **v0.0.4** — DSL + DAG runtime (`agent`/`parallel`/`map`/`gate`/`reduce`),
269+
inline + saved flows, cross-session resume, live progress, isolated context.
270+
Default `concurrency` is 8 (set on the flow; per-phase `concurrency` overrides
271+
for that phase).
272+
- A run executes as one streaming tool call (live progress while it runs).
273+
- `map` requires the upstream phase to emit a JSON array (`output: "json"`).
274+
- Gate verdicts are **fail-open**: if the agent output contains no recognizable
275+
verdict marker (`VERDICT: BLOCK/PASS/OK/FAIL/STOP/REJECT/HALT` or
276+
`{continue: false}` / `{verdict: "block"}`), the gate passes. This prevents
277+
an accidental missing verdict from blocking your workflow.
278+
279+
### What it doesn't do (yet)
280+
281+
- **No detached background execution.** A run needs the pi session to stay open.
282+
True background execution is on the roadmap.
283+
- **No `output: "file"`.** Outputs are text/JSON only. Write files via agent
284+
tool calls if needed.
285+
- **`map` requires a JSON array.** The `over` field must resolve to
286+
`{steps.ID.json}` where the upstream phase emitted `output: "json"`. If the
287+
source is a plain text list, wrap it in a single-agent phase that outputs JSON.
288+
- **Cycles are rejected at validation.** The DAG must be acyclic.
106289

107290
## Development
108291

109292
```bash
110293
npm install
111294
npm run typecheck
112-
node --experimental-strip-types --test test/interpolate.test.ts test/schema.test.ts test/runtime.test.ts
295+
node --experimental-strip-types --test test/interpolate.test.ts \
296+
test/schema.test.ts test/runtime.test.ts test/runner.test.ts \
297+
test/store.test.ts test/agents.test.ts test/render.test.ts \
298+
test/desugar.test.ts
113299

114300
# real end-to-end (spawns live subagents; needs model access)
115301
PI_TASKFLOW_PI_BIN=pi node --experimental-strip-types test/e2e.mts
116302
```
117303

118-
## Status & limits
304+
## Contributing
119305

120-
- **v0.1** — DSL + DAG runtime (`agent`/`parallel`/`map`/`gate`/`reduce`),
121-
inline + saved flows, cross-session resume, live progress, isolated context.
122-
- A run executes as one streaming tool call (live progress while it runs). True
123-
detached background execution is on the roadmap.
124-
- `map` requires the upstream phase to emit a JSON array (`output: "json"`).
306+
Contributions welcome! This is a young project — open an issue or PR on
307+
[GitHub](https://github.com/heggria/pi-taskflow). Tests live in `test/`, the
308+
runtime in `extensions/`.
125309

126310
## License
127311

0 commit comments

Comments
 (0)