feat: DSTsx v2 — complete all planned features + MCP integration

[Copilot]

Implements the full v2 roadmap for DSTsx (TypeScript port of DSPy), adding all planned features from scratch. 218 tests pass across 46 test files (up from 160/29 in v1).

New Modules

• TypedPredictor<T> / TypedChainOfThought<T> — JSON output with optional Zod schema validation, markdown fence stripping, configurable retry; returns TypedPrediction<T> with .typed
• Parallel — Promise.all fan-out; run() → Prediction[], forward() → first result for Module compat
• Refine — self-critique loop: inner module → built-in critic predictor → re-run with feedback
• NativeReAct — OpenAI tools API / Anthropic tool_use based ReAct; parses tool_calls from raw response

New Primitives & Helpers

• Image — fromURL / fromBase64 / fromFile; toOpenAIContentPart() / toAnthropicContentBlock() serializers for multi-modal LM calls
• majority(field) — voting reducer across Prediction[]; drop-in reduceFunc for BestOfN / Ensemble
• StreamChunk type + LM.stream() fallback + real streaming in OpenAI and Anthropic adapters + Predict.stream(inputs)

New Optimizers

• BootstrapFewShotWithOptuna — pure-TS TPE (Tree-structured Parzen Estimator) Bayesian demo subset search
• BootstrapFinetune — collects traces, exports JSONL in "openai" or "generic" format for fine-tuning
• GRPO — Group Relative Policy Optimization; each step generates groupSize instruction variants, ranks by relative advantage (score − mean) / std
• SIMBA — stochastic mini-batch ascent with Fisher-Yates sampling; accepts candidates that improve on random batches
• AvatarOptimizer — per-predictor persona search; proposes numAvatars role descriptions via LM, keeps best-scoring

Experiment Tracking

Abstract Tracker + TrackerEvent; concrete ConsoleTracker and JsonFileTracker (async JSONL append). Custom trackers extend Tracker:

class MLflowTracker extends Tracker {
  log(event: TrackerEvent) { /* ship to MLflow/W&B */ }
  async flush() {}
}

Infrastructure

• DiskCache — SHA-256 keyed JSON files, TTL + LRU eviction; LM accepts cacheDir for two-level caching
• ProgramOfThought sandbox — new sandbox: "worker" | "function" | "none" option; "worker" uses node:worker_threads with true cancellation on timeout
• MCP — MCPToolAdapter (MCP server tools → DSTsx Tool for ReAct) + DSTsxMCPServer (DSTsx modules → MCP tool definitions)
• GitHub Actions — ci.yml (typecheck + test + build on push/PR) + publish.yml (npm publish on release)
• typedoc.json — enables npm run docs

Usage snapshot

import { GRPO, SIMBA, AvatarOptimizer, NativeReAct, Image, StreamChunk } from "dstsx";

// Streaming
for await (const chunk of predict.stream({ question: "..." })) {
  process.stdout.write(chunk.delta);
}

// Native tool calling
const agent = new NativeReAct("question -> answer", tools);

// Multi-modal
const result = await captioner.forward({ image: Image.fromURL(url), question: "Describe." });

// Optimizers
const optimized = await new GRPO({ numSteps: 20, groupSize: 8 }).compile(program, trainset, metric);

Original prompt

Build this library as per the following specs upto v1.0, document usage for all apis of the library. Once complete also create a v2 roadmap extension based on what features from dspy are still missing in ours that need implementations.

The user has attached the following file paths as relevant context:

• REQUIREMENTS.md

Created from VS Code.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.