From 8139e3cdbf368c394356b9c9c73284835ccf2aed Mon Sep 17 00:00:00 2001 From: Tee Date: Sun, 31 May 2026 01:12:19 +0100 Subject: [PATCH] docs: add VS Code attach snippet, fix duplicate table, expand extension points, add batch execution design doc --- ARCHITECTURE.md | 7 +- Readme.md | 12 +- docs/batch-execution.md | 2 + docs/design/batch-execution-design.md | 195 ++++++++++++++++++++++++++ docs/remote-debugging.md | 38 +++++ 5 files changed, 246 insertions(+), 8 deletions(-) create mode 100644 docs/design/batch-execution-design.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index db758a28..de167558 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -94,9 +94,12 @@ graph TD ## Extension Points -Future contributors can extend the debugger in several ways: +Contributors can extend the debugger in the following ways: - **New Commands**: Add new subcommands to the `cli` module to support different debugging workflows. - **Custom Inspectors**: Create new inspectors in the `inspector` module to track specific resources or state types (e.g., event logs). - **UI Enhancements**: Improve the TUI in the `ui` module to provide better visualization of complex data structures or execution paths. -- **Backend Adapters**: (Future) Extend `ContractExecutor` to support connecting to remote RPC nodes for live debugging. +- **Backend Adapters**: Extend `ContractExecutor` to support connecting to remote RPC nodes for live debugging. +- **Plugin System**: Plugins are loaded from TOML manifests (`plugin.toml`) and can register new CLI subcommands, intercept execution hooks, and publish custom output. See [`docs/plugin-api.md`](docs/plugin-api.md) and [`docs/architecture-plugins.md`](docs/architecture-plugins.md) for the full plugin lifecycle and API surface. +- **Remote Debug Server**: The `server` / `remote` command pair exposes the debugger over a line-delimited JSON wire protocol (TCP), supporting token authentication and native TLS. New transports or protocol extensions can be added in `src/server/`. See [`docs/architecture-remote.md`](docs/architecture-remote.md) for the design. +- **Batch Executor**: The `BatchExecutor` in `src/batch.rs` runs multiple argument sets in parallel using Rayon. It can be extended to support new result aggregation strategies, custom assertion formats, or streaming output. See [`docs/design/batch-execution-design.md`](docs/design/batch-execution-design.md) for the canonical design reference. diff --git a/Readme.md b/Readme.md index ee7d3abb..6d203bb0 100644 --- a/Readme.md +++ b/Readme.md @@ -136,6 +136,12 @@ For precise control, use `{"type": "...", "value": ...}`: Filter large storage outputs by key pattern using `--storage-filter`: +| Pattern | Type | Matches | +|------------------|--------|----------------------------------------| +| `balance:*` | Prefix | Keys starting with `balance:` | +| `re:^user_\d+$` | Regex | Keys matching the regex | +| `total_supply` | Exact | Only the key `total_supply` | + ```bash # Prefix match: keys starting with "balance:" soroban-debug run --contract token.wasm --function mint \ @@ -213,12 +219,6 @@ An exported trace includes versioning, metadata, and full execution state: } ``` -| Pattern | Type | Matches | -|------------------|--------|----------------------------------------| -| `balance:*` | Prefix | Keys starting with `balance:` | -| `re:^user_\d+$` | Regex | Keys matching the regex | -| `total_supply` | Exact | Only the key `total_supply` | - ### Interactive Command Start an interactive debugging session: diff --git a/docs/batch-execution.md b/docs/batch-execution.md index 84223ab5..ce2f57e6 100644 --- a/docs/batch-execution.md +++ b/docs/batch-execution.md @@ -4,6 +4,8 @@ The batch execution feature allows you to run the same contract function with mu See the [Scenario Cookbook](scenario-cookbook.md) for examples of complex test patterns that can be used in batch execution. +For the architectural rationale, parallelism model, and extension points, see the [Batch Execution Design](design/batch-execution-design.md) document. + ## Usage ```bash diff --git a/docs/design/batch-execution-design.md b/docs/design/batch-execution-design.md new file mode 100644 index 00000000..7945fe49 --- /dev/null +++ b/docs/design/batch-execution-design.md @@ -0,0 +1,195 @@ +# Batch Execution Design + +> **Canonical design reference.** For user-facing usage instructions see +> [`docs/batch-execution.md`](../batch-execution.md). For the implementation +> summary produced during the initial feature work see +> [`BATCH_EXECUTION_SUMMARY.md`](../../BATCH_EXECUTION_SUMMARY.md). + +## Goals + +1. Allow users to run the same contract function against many argument sets in a single invocation. +2. Execute those argument sets in parallel so wall-clock time scales with CPU cores, not input count. +3. Aggregate individual results into a deterministic, human- and machine-readable summary. +4. Keep batch mode a composable layer on top of the existing single-run execution path rather than a separate code path. + +## Non-Goals + +- Interactive / step-through debugging of individual batch items (use single-run mode for that). +- Ordered / sequential execution (ordering is not guaranteed between items; each item is independent). +- Distributed execution across multiple hosts. + +--- + +## Architecture + +### Components + +``` +CLI (--batch-args ) + │ + ▼ + BatchExecutor (src/batch.rs) + │ + ├─ load_batch_file() – deserialise JSON array of BatchItems + │ + ├─ execute_batch() – par_iter() over items → Vec + │ │ + │ └─ execute_single() – delegates to ContractExecutor (single-run path) + │ + ├─ summarize() – fold Vec → BatchSummary + │ + └─ display_results() – render to stdout (text or JSON) +``` + +### Key Types + +| Type | Location | Purpose | +|------|----------|---------| +| `BatchItem` | `src/batch.rs` | Deserialised entry from the JSON file: `args`, optional `expected`, optional `label` | +| `BatchResult` | `src/batch.rs` | Outcome of a single execution: result value, pass/fail status, duration, error | +| `BatchSummary` | `src/batch.rs` | Aggregate counts (total, passed, failed, errors) and total duration | +| `BatchExecutor` | `src/batch.rs` | Orchestrates the above; receives a shared reference to `ContractExecutor` | + +--- + +## Parallelism Model + +Batch execution uses **Rayon's work-stealing thread pool** (`par_iter()`). + +``` +items: Vec + │ + ▼ rayon::par_iter() +┌──────────────────────────────────────┐ +│ Thread 0 │ Thread 1 │ Thread N │ (Rayon pool, N = logical CPU cores) +│ item[0] │ item[1] │ item[k] │ +│ ↓ │ ↓ │ ↓ │ +│ exec_single│ exec_single│ exec_single│ +└──────────────────────────────────────┘ + │ + ▼ collect() +results: Vec (order matches input order after collect) +``` + +**Thread safety constraints:** + +- `ContractExecutor` is instantiated fresh per item inside `execute_single`. Each invocation gets its own `soroban-env-host` `Host` instance, so there is no shared mutable state between parallel executions. +- The JSON input file is loaded once on the main thread and shared as an immutable slice. +- Result collection via `collect()` on a `ParallelIterator` is deterministic in output order (Rayon preserves index order for `collect`). + +--- + +## Result Aggregation + +After all executions complete, `summarize()` produces a `BatchSummary`: + +```rust +BatchSummary { + total: usize, // number of BatchItems run + passed: usize, // items where actual result == expected (or no expected set) + failed: usize, // items where actual result != expected + errors: usize, // items that panicked or returned an execution error + duration: Duration,// wall-clock time from start of par_iter to end of collect +} +``` + +### Pass/Fail Logic + +| Condition | Classification | +|-----------|----------------| +| No `expected` field set and execution succeeded | **Passed** | +| `expected` field set and `actual == expected` (string comparison) | **Passed** | +| `expected` field set and `actual != expected` | **Failed** | +| Execution returned an error or panicked | **Error** | + +--- + +## Input File Format + +```json +[ + { + "args": "[\"Alice\", \"Bob\", 100]", + "expected": "void", + "label": "transfer 100 units" + }, + { + "args": "[\"Charlie\", \"Dave\", 0]", + "label": "zero-value transfer (no assertion)" + } +] +``` + +- The top-level value **must** be a JSON array. +- `args` is a **JSON string** (not a nested array) containing the argument list exactly as it would be passed to `--args`. +- `expected` is an optional string compared against the serialised return value. +- `label` is optional and used only for human-readable output. + +--- + +## CLI Integration + +``` +soroban-debug run \ + --contract path/to/contract.wasm \ + --function transfer \ + --batch-args path/to/batch.json +``` + +When `--batch-args` is provided: + +1. The `run` command delegates to `BatchExecutor::load_batch_file()`. +2. `execute_batch()` is called instead of the normal single-run path. +3. Results are printed to stdout; exit code is `0` if all items pass, non-zero otherwise. + +`--batch-args` is mutually exclusive with `--args` in the CLI argument parser. + +--- + +## VS Code Extension + +The extension exposes batch mode via the `batchArgs` field in `launch.json`: + +```json +{ + "name": "Soroban: Batch Test", + "type": "soroban", + "request": "launch", + "contractPath": "${workspaceFolder}/target/wasm32-unknown-unknown/release/contract.wasm", + "entrypoint": "transfer", + "batchArgs": "${workspaceFolder}/tests/batch_inputs.json" +} +``` + +When `batchArgs` is set the adapter passes `--batch-args` to the spawned `soroban-debug server` process. Breakpoints and stepping are skipped in batch mode; use single-run mode to debug individual failing cases. + +--- + +## Performance Characteristics + +| Batch size | Approximate speed-up vs sequential | +|-----------|--------------------------------------| +| 10 items | ~10× | +| 100 items | ~50× (plateau depends on core count) | + +Speed-up is roughly linear up to the Rayon thread-pool size (default: logical CPU count) then plateaus. Contract execution time dominates; scheduling overhead is negligible for all practical batch sizes. + +--- + +## Extension Points + +| What to extend | Where | +|---------------|-------| +| New result aggregation strategies (e.g., percentiles, histogram) | `BatchExecutor::summarize()` in `src/batch.rs` | +| Custom assertion formats (e.g., regex match, numeric tolerance) | `BatchItem` struct + `execute_single()` comparison logic | +| Streaming / incremental output | `BatchExecutor::display_results()` — replace `collect` + render with a channel-based sink | +| Different parallelism back-ends (e.g., async, remote dispatch) | Replace `par_iter()` in `execute_batch()` with the desired scheduler | + +--- + +## Related Documents + +- [`docs/batch-execution.md`](../batch-execution.md) — user guide with CLI examples and output format +- [`docs/batch-result-buckets.md`](../batch-result-buckets.md) — result classification details +- [`BATCH_EXECUTION_SUMMARY.md`](../../BATCH_EXECUTION_SUMMARY.md) — implementation summary from the original feature PR +- [`ARCHITECTURE.md`](../../ARCHITECTURE.md) — system-level overview including the Batch Executor extension point diff --git a/docs/remote-debugging.md b/docs/remote-debugging.md index c1db1e24..38864cfd 100644 --- a/docs/remote-debugging.md +++ b/docs/remote-debugging.md @@ -53,6 +53,44 @@ soroban-debug remote \ --args '["user1", 100]' ``` +### Connect from VS Code (Attach) + +You can also use the VS Code extension to attach to the remote debugger server. Add a configuration of type `"soroban"` and `"request": "attach"` to your `.vscode/launch.json`. + +Below is a complete snippet matching the TLS client setup above: + +```json +{ + "version": "0.2.0", + "configurations": [ + { + "name": "Soroban: Attach to Remote Debugger", + "type": "soroban", + "request": "attach", + "host": "localhost", + "port": 9229, + "contractPath": "${workspaceFolder}/contract.wasm", + "entrypoint": "increment", + "args": ["user1", 100], + "token": "your-token-here", + "tlsCert": "${workspaceFolder}/path/to/client-cert.pem", + "tlsKey": "${workspaceFolder}/path/to/client-key.pem" + } + ] +} +``` + +#### Configuration Properties + +- **host**: The hostname or IP address of the remote debug server (defaults to `127.0.0.1`). +- **port**: The TCP port the server is listening on. +- **contractPath**: Path to the contract WASM compiled locally. +- **entrypoint**: The function name to execute. +- **args**: JSON array of arguments for the entrypoint. +- **token**: The authentication token required by the server. +- **tlsCert**: Optional path to your client TLS certificate file. +- **tlsKey**: Optional path to your client TLS private key file. + ### Timeouts and Retries (network instability) Remote sessions often run across CI, containers, or flaky links. The remote client supports deterministic timeouts and controlled retries for **idempotent** operations.