Skip to content

[codex] add metaswarm review adapter integration#49

Merged
0xzr merged 7 commits into
mainfrom
codex/metaswarm-review-adapter
Jun 17, 2026
Merged

[codex] add metaswarm review adapter integration#49
0xzr merged 7 commits into
mainfrom
codex/metaswarm-review-adapter

Conversation

@0xzr

@0xzr 0xzr commented Jun 17, 2026

Copy link
Copy Markdown
Owner

Summary

Adds an experimental metaswarm external-tools integration for freellmpool as a review/second-opinion adapter.

What changed

  • Added integrations/metaswarm/freellmpool-review-adapter.sh, a self-contained review-only adapter with health, review, and unsupported implement commands.
  • Added integrations/metaswarm/README.md with install instructions, .metaswarm/external-tools.yaml config, environment variables, and fail-closed no-key behavior.
  • Updated README, docs/INTEGRATIONS.md, docs/AGENTS.md, docs/llms.txt, docs/index.html, and CHANGELOG.md so metaswarm support is discoverable.
  • Added pytest coverage for shell syntax, no-key auth_missing behavior, and docs linkage.

Why

Metaswarm can use external review tools, and freellmpool is a good fit for independent review panels because it can fan out to multiple configured free-tier providers while keeping the implementation role with a coding agent.

Behavior and safety

The adapter is deliberately review-only. If no strong review provider is configured, review returns a structured JSON envelope with error_type: "auth_missing" before making provider calls. The raw log lists required environment variable names, not secret values.

Validation

  • bash -n integrations/metaswarm/freellmpool-review-adapter.sh
  • python3 -m pytest tests/test_metaswarm_integration.py tests/test_release_metadata.py tests/test_contributor_surface.py
  • python3 -m pytest tests/test_cli.py tests/test_tokenmax.py
  • ruff check .
  • git diff --check
  • high-confidence secret scan over changed/untracked PR files
  • full suite: python3 -m pytest → 427 passed

Notes

Local untracked docs/POLISH_PLAN.md was pre-existing and is intentionally excluded from this PR.

Summary by Sourcery

Add an experimental metaswarm external-tools adapter to use freellmpool as a review-only, second-opinion tool and document its setup and behavior.

New Features:

  • Introduce a metaswarm freellmpool review adapter script providing health and review commands for use as an external-tools reviewer/second opinion.

Enhancements:

  • Document metaswarm integration and configuration across integrations, agents, README, and site index so the adapter is discoverable.
  • Extend changelog with an unreleased entry describing the new metaswarm review adapter integration.

Tests:

  • Add tests for the metaswarm adapter shell syntax, fail-closed no-key auth_missing review behavior, and documentation/linkage coverage.

@sourcery-ai

sourcery-ai Bot commented Jun 17, 2026

Copy link
Copy Markdown

Reviewer's Guide

Adds an experimental metaswarm external-tools review-only adapter script for freellmpool, documents how to use it as a reviewer/second-opinion in metaswarm, and adds tests to ensure shell syntax correctness, fail-closed auth_missing behavior when no strong provider keys are configured, and that docs are properly linked.

Sequence diagram for metaswarm freellmpool review-only adapter

sequenceDiagram
    actor Metaswarm
    participant Adapter as freellmpool_review_adapter.sh
    participant FreellmpoolCLI as freellmpool_CLI
    participant Provider as Strong_provider_panel

    Metaswarm->>Adapter: review --worktree --rubric-file --spec-file
    Adapter->>Adapter: parse_args
    Adapter->>Adapter: count_configured_strong_providers

    alt [no strong providers configured]
        Adapter->>Adapter: emit_error command=review error_type=auth_missing
        Adapter-->>Metaswarm: JSON envelope (error_type=auth_missing)
    else [one or more strong providers]
        Adapter->>Adapter: build_keyed_env
        loop for each model in STRONG_MODELS (up to MAX_MODELS)
            Adapter->>FreellmpoolCLI: ask --model <model> --json
            FreellmpoolCLI->>Provider: upstream LLM call
            Provider-->>FreellmpoolCLI: review response
            FreellmpoolCLI-->>Adapter: JSON review
        end
        Adapter->>FreellmpoolCLI: ask --model <first_success_model> --json (synthesis)
        FreellmpoolCLI->>Provider: upstream LLM call (synthesis)
        Provider-->>FreellmpoolCLI: synthesized verdict
        FreellmpoolCLI-->>Adapter: JSON synthesis
        Adapter->>Adapter: emit_json command=review exit_code=0
        Adapter-->>Metaswarm: JSON envelope (verdict, findings, summary)
    end
Loading

File-Level Changes

Change Details Files
Introduce a metaswarm external-tools review-only adapter shell script that wraps freellmpool for health checks and adversarial code review, including strong-provider routing, timeout handling, error classification, and JSON envelope output.
  • Add a Bash adapter implementing health, review, and unsupported implement commands with strict error handling and secure temp directory usage.
  • Implement argument parsing for metaswarm worktree, rubric, spec, attempt, and timeout parameters.
  • Provide safe_invoke utility with timeout support and a fallback when the timeout command is unavailable.
  • Implement classify_error to map exit codes and stderr patterns into metaswarm-style error_type values such as timeout, tool_not_installed, rate_limited, auth_expired, context_too_large, and tool_crash.
  • Emit metaswarm-style JSON envelopes via a Python helper, including schema_version, tool, command, model, attempt, exit_code, git metadata, duration, cost, raw_log, and error_type fields.
  • Implement health command to report freellmpool availability, version, auth_valid state, and configured strong providers/models using freellmpool.config.configured_providers when available.
  • Implement implement command as a hard-failing, review-only guard that always returns unsupported_role and logs a clear message.
  • Implement review command that validates inputs, checks for freellmpool CLI presence, enforces presence of at least one configured strong provider before any provider calls, builds a diff/spec/rubric-based prompt, runs a strong-model panel or tokenmax/ask depending on FREELLMPOOL_REVIEW_MODE, synthesizes panel results when possible, writes a raw log file, and returns a JSON envelope summarizing the run.
  • Support environment-based configuration for tool name/cmd, default model, routing strategy, review mode, max models/tokens, strong providers/models, and per-model/synthesis timeouts, along with a log directory for raw logs.
integrations/metaswarm/freellmpool-review-adapter.sh
Document the metaswarm freellmpool review adapter and make it discoverable across integration, agents, README, and website docs.
  • Add a dedicated README describing installation, metaswarm external-tools.yaml configuration, supported commands, JSON envelope behavior, environment variables, and fail-closed auth_missing behavior with no provider keys.
  • Update integrations documentation to introduce metaswarm as an experimental review-only adapter, with copy-paste setup instructions and explicit mention of fail-closed behavior and required provider keys.
  • Update agents documentation with a section on metaswarm external-tools review, describing how the adapter runs adversarial review on git diffs against spec/rubric and returns metaswarm-style JSON, including auth_missing behavior when no strong provider key is configured.
  • Update the main README to mention the metaswarm review adapter in the integrations/agents section so users can discover it from the project landing page.
  • Update the HTML docs index to advertise using the metaswarm adapter to add model-pool review to metaswarm via a link to the integrations/metaswarm directory.
  • Ensure llms.txt mentions metaswarm so the integration is indexed for docs/tests.
integrations/metaswarm/README.md
docs/INTEGRATIONS.md
docs/AGENTS.md
README.md
docs/index.html
docs/llms.txt
Add tests to validate the metaswarm adapter’s shell syntax, fail-closed no-key auth_missing behavior, and documentation linkage.
  • Add a pytest module that verifies the Bash adapter passes bash -n syntax checks.
  • Provide a helper that constructs an isolated environment with empty freellmpool config/keys, no provider API key env vars, and a custom METASWARM_LOG_DIR for tests.
  • Add an integration-style test that creates a temporary git repo, spec, and rubric files, invokes the adapter review command without provider keys, and asserts the JSON payload reports exit_code 2, error_type auth_missing, contains the expected key names in raw_log, and does not leak secrets.
  • Add a docs linkage test that asserts metaswarm is mentioned in integrations, agents, README, llms.txt, the adapter script contains auth_missing, and the integration README advertises review-only behavior and external-tools.yaml configuration.
tests/test_metaswarm_integration.py
Update changelog to record the new metaswarm review adapter integration and its fail-closed behavior.
  • Add an Unreleased section describing the experimental metaswarm review adapter integration, its setup docs, and the no-key auth_missing smoke coverage.
CHANGELOG.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@0xzr

0xzr commented Jun 17, 2026

Copy link
Copy Markdown
Owner Author

Final consensus pass on head be5ba66:

  • GitHub CI: quickstart, test (3.11), test (3.12), and test (3.13) all pass.
  • Local focused validation: bash -n integrations/metaswarm/freellmpool-review-adapter.sh; python3 -m pytest tests/test_metaswarm_integration.py tests/test_release_metadata.py tests/test_contributor_surface.py -q (17 passed); ruff check tests/test_metaswarm_integration.py; git diff --check.
  • Secret scan over origin/main..HEAD: clean for high-confidence API key/private-key patterns.
  • freellmpool review (nvidia/moonshotai/kimi-k2.6): PASS; warnings only, no blockers.
  • MiniMax-M3 focused re-review: PASS; verified prior blockers are fixed.

Consensus-driven follow-ups added after review:

  • Fail closed before provider calls when the strong model list includes a provider without configured credentials.
  • Fail closed on empty diffs with error_type: "empty_diff".
  • Fail closed when synthesis fails instead of masking it behind a successful envelope.
  • Redact raw provider logs before persistence and before embedding in JSON envelopes.
  • Align unsupported implement exit behavior and add second_opinion as a review alias.
  • Expand integration tests for no-key, missing CLI, invalid key, partial strong config, empty diff, synthesis failure, redaction, and review-only scoping.
  • Align remaining provider-count docs/assets to 19 providers.

@0xzr 0xzr merged commit be5ba66 into main Jun 17, 2026
4 checks passed
@0xzr 0xzr temporarily deployed to github-pages June 17, 2026 04:26 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant