Agent-driven integration setup: one spec-driven action tool instead of one per integration

[muddlebee]

Problem

An integration can be configured from two surfaces today:

Surface	Coverage
opensre onboard (wizard)	~32 integrations
opensre integrations setup <service>	45 integrations
Interactive shell / agent	0

Asking the agent to "set up Telegram" does not work. #4071 proposes to fix that with a hand-written configure_telegram action tool — one action tool, for one integration. Repeating that shape 45 times is not a plausible path, and each copy would re-implement prompting, validation, and persistence.

What makes a generic version possible

integrations/setup_flow.py introduced IntegrationSetupSpec — a declarative description of what an integration needs:

TELEGRAM_SETUP = IntegrationSetupSpec(
    service="telegram",
    fields=(
        SetupField(name="bot_token", label="Telegram bot token",
                   env_var="TELEGRAM_BOT_TOKEN", secret=True),
        SetupField(name="default_chat_id", label="Default chat ID",
                   env_var="TELEGRAM_DEFAULT_CHAT_ID"),
    ),
    verify=verify_telegram,
    resolve=_resolve_default_chat_id,
)

That is exactly the metadata an agent needs to drive setup conversationally: which fields to ask for, which are required, which must be masked, and what to call them. So a single configure_integration action tool can serve every integration that has a spec, instead of one action tool per integration.

This depends on specs existing, which is #4168.

Constraint to design around first: secrets in chat history

A credential typed into a chat message is persisted by that chat platform. In the local terminal REPL that is acceptable. Through the Slack or Telegram gateway it means a bot token sits in a channel's history indefinitely, outside OpenSRE's control and outside its masking layer.

SetupField.secret already marks which fields this applies to. Options, roughly in order of preference:

1. Refuse secret fields on remote surfaces. The agent collects non-secret fields, then tells the user to run opensre integrations setup <service> locally for the credential. Setup still gets easier without moving secrets through chat.
2. Gate the whole tool to the local interactive shell. Simplest to reason about; gives up remote setup entirely.
3. Accept secrets everywhere with a warning. Not recommended — the warning does not remove the message from the channel.

This should be decided before the tool is built, not retrofitted.

Scope

• A configure_integration action tool driven by the spec, not per-integration copies.
• Field collection over multiple turns: the agent asks for what is missing, accepts what the user supplies, re-asks on a validation failure. apply_setup already returns a SetupOutcome with a renderable detail, so failures come back as sentences the agent can relay.
• Execution gating. This is a mutating action that writes credentials to disk, so it needs allow_tool / execution_allowed plus side_effect_level / requires_approval / approval_reason, per surfaces/interactive_shell/AGENTS.md ("Action Selection And Execution"). feat(interactive-shell): set up Telegram from a pasted token via a gated action tool (#3933) #4071 currently has none of these.
• Surface policy per the secrets constraint above.
• Tests for the allowed, denied, and confirmation-required paths (required by the same package rule).
• Docs page plus a docs/docs.json entry.

Note: no regex/keyword intent routing around the action-agent path — surfaces/interactive_shell/AGENTS.md prohibits it. Selection goes through the action agent like any other action.

Relationship to other work

• refactor(config): move .env/keyring writing to config/ so every setup surface shares one writer #4167 — moved the .env/keyring writer to config/, making it reachable from tools/ (merged/in review).
• Migrate the remaining 44 integration setup handlers onto the shared setup flow #4168 — declares specs for the remaining 44 integrations. This issue is limited by that: the agentic tool works for whatever has a spec.
• feat(interactive-shell): set up Telegram from a pasted token via a gated action tool (#3933) #4071 — the Telegram-only action tool. Should be rebased to become the first consumer of the generic tool, or closed in favour of it.

[sanskar-singh-2403]

@muddlebee can i work on this? and the supporting issue?

[muddlebee]

Status update — this issue is gated on specs existing (#4168), so here is where that stands.

9 of 45 integrations now have an IntegrationSetupSpec: telegram (#4170), datadog/honeycomb/coralogix (#4191), groundcover/gitlab/sentry/posthog/vercel (#4194).

The running checklist lives on #4168, which is where the migration is tracked: 22 more are ready to migrate with no new spec features, and 14 are blocked on one of three gaps (conditional prompts, post-save hooks, either/or required groups).

What that means for the tool proposed here: a configure_integration action tool driven by the spec would work for those 9 today and grow as batches land. It does not need to wait for all 45 — the tool can look up the spec for a service and refuse services that have none, which also gives a clean, honest error while the migration is in flight.

The secrets-in-chat constraint in the description above is still the thing to decide before any of it gets built. SetupField.secret marks exactly which fields it applies to, so the mechanism is in place; the policy is not.

[muddlebee]

Closing to keep this work in one place — #4168 is the active issue and now carries this content.

Everything here has been folded into #4168 under "Follow-up once enough specs exist: agent-driven setup": the secrets-in-chat constraint with the three ranked options, the execution-gating requirements (allow_tool / execution_allowed, side_effect_level, requires_approval, and tests for allowed/denied/confirmation paths), the no-regex-routing rule, and the docs/docs.json requirement.

Nothing is dropped. The reason to merge rather than keep both open: this was explicitly gated on specs existing, so it was not independently actionable, and two open issues describing adjacent parts of the same effort were hard to tell apart.

Reopen or file fresh when the migration is far enough along that the tool is worth building — 9 of 45 integrations have a spec today, tracked on #4168.