Delegate research upgrade

README.md

@@ -25,9 +25,12 @@
 `madmcp`'s core idea is **delegation**: instead of an agent making 5-10
 manual tool calls to run an investigation itself, it can hand an open-ended,
 read-only investigation (or a single page + question) to Gemini, which runs
-its own tool-use loop server-side — across GitHub, Cloudflare, and Notion —
-and returns one synthesized answer. That's `delegate_gemini` and
-`Delegate_web_fetch`, described first under Connectors & tools below.
+its own tool-use loop server-side and returns one synthesized answer. This
+is split across two tools along a security boundary: `delegate_gemini`
+covers GitHub, Cloudflare, and Notion (no web access), while
+`delegate_research` covers the live web (a precision single-page mode, or
+an open-ended multi-step wide research mode) with no access to those
+internal systems. Both are described first under Connectors & tools below.
 
 On top of that, the server also gives an AI agent direct tool-level access
 to real infrastructure — GitHub, Cloudflare, Notion, Mem0, Context7, and
@@ -131,6 +134,13 @@ See **Configuration** below for the full variable reference.
 ## Connectors & tools
 
 ### ⭐ Gemini (delegation) — the flagship feature
+The two delegation tools split along a security boundary: `delegate_gemini`
+has no web access, `delegate_research` has no access to GitHub/Notion/
+Cloudflare. This means a malicious page or search result encountered
+mid-research can influence at most that run's own answer — it has no
+internal-system data to exfiltrate, because that loop never has access to
+any in the first place.
+
 `delegate_gemini` — hand an open-ended, multi-step, read-only investigation
 (e.g. "why is CI failing on PR #42", "summarize what changed in this repo over
 the last week") to Gemini instead of making 5-10 separate manual tool calls.
@@ -140,17 +150,23 @@ answer. Falls through an ordered model cascade (`GEMINI_MODEL` →
 `GEMINI_FALLBACK_MODELS`) on rate limits, with Redis-backed per-model cooldown
 so already-limited models are skipped rather than retried.
 
-Progress is checkpointed to Redis after every completed step. If the Gemini
-API call itself fails partway through (429/503/network blip), the response
-includes a `resume_run_id` and everything gathered so far instead of losing
-the run outright — pass that id back on a follow-up `delegate_gemini` call
-to continue from the last completed step (checkpoint TTL: 1 hour) rather than
-re-running, and re-paying for, steps already done.
-
-`Delegate_web_fetch` — fetch a single URL and get back Gemini's answer to a
-specific question about its content, without returning the raw page. Use this
-instead of `web_fetch` when you need a distilled answer rather than exact
-wording to copy.
+`delegate_research` — web research, in one of two mutually-exclusive modes
+selected by which args are passed:
+- **Precision mode** (`url` + `question`): fetch a single URL and get back
+  Gemini's answer to a specific question about its content, without
+  returning the raw page. Use this instead of `web_fetch` when you need a
+  distilled answer rather than exact wording to copy.
+- **Wide mode** (`task`): an open-ended, multi-step research loop —
+  Google Search grounding to find pages, `web_fetch` to read them — bounded
+  by `max_steps`, returning one synthesized answer.
+
+Progress on both `delegate_gemini` and wide-mode `delegate_research` runs is
+checkpointed to Redis after every completed step. If the underlying Gemini
+API call fails partway through (429/503/network blip), the response includes
+a `resume_run_id` and everything gathered so far instead of losing the run
+outright — pass that id back on a follow-up call to continue from the last
+completed step (checkpoint TTL: 1 hour) rather than re-running, and
+re-paying for, steps already done.
 
 Both tools can optionally log their task/question, step-by-step tool calls,
 and final answer to a Notion page under a fixed Gemini root page
@@ -224,7 +240,7 @@ All tokens are optional independently — a connector's tools fail at call time
 | `MEM0_API_KEY` | Mem0 tools (`MEM0_USER_ID` optional, defaults to `default`) |
 | `CLOUDFLARE_API_TOKEN` + `CLOUDFLARE_ACCOUNT_ID` | Cloudflare tools |
 | `CONTEXT7_API_KEY` | Context7 tools (optional — works unauthenticated at low rate limits) |
-| `GEMINI_API_KEY` | Gemini tools (`delegate_gemini`, `Delegate_web_fetch`) — required, throws if unset |
+| `GEMINI_API_KEY` | Gemini tools (`delegate_gemini`, `delegate_research`) — required, throws if unset |
 | `GEMINI_MODEL` | Primary Gemini model for delegation (default `gemini-flash-latest`) |
 | `GEMINI_FALLBACK_MODELS` | Comma-separated fallback model list used on 429s (default `gemini-3.5-flash-lite,gemini-3.1-flash-lite`) |
 | `GEMINI_NOTION_ROOT_PAGE_ID` | Notion page under which Gemini tool outputs are logged (has a working default) |

config.js

@@ -125,6 +125,14 @@ export const GEMINI_FALLBACK_MODELS = (process.env.GEMINI_FALLBACK_MODELS || "ge
   .map((s) => s.trim())
   .filter(Boolean);
 
+// Defensive ceiling on a single generateContent call -- no official guidance
+// from Google on max latency, but without SOME timeout a hung/dropped
+// connection leaves delegate.js's per-step checkpointing unable to kick in
+// at all (the call just never returns). Override via env var if this proves
+// too tight for slower multi-tool-call turns, or too loose relative to the
+// hosting platform's own request-duration limit.
+export const GEMINI_REQUEST_TIMEOUT_MS = Number(process.env.GEMINI_REQUEST_TIMEOUT_MS) || 55000;
+
 // Read/write isolation for the Gemini connector's Notion access (2026-07-25
 // plan): Gemini tools may READ any page/database reachable via the existing
 // Notion connector (Memory Index, Entity Index, Job Leads, etc.), but may

connectors/fetch/client.js

@@ -87,8 +87,9 @@ async function assertSafeUrl(urlStr) {
 
 // Strip HTML tags and collapse whitespace into readable plain text. Lives
 // here (not fetch/tools.js) so other connectors that need the same
-// HTML-to-text step server-side -- e.g. gemini/tools.js's Delegate_web_fetch,
-// which strips a fetched page before handing it to Gemini -- can reuse this
+// HTML-to-text step server-side -- e.g. gemini/tools.js's delegate_research
+// (both its precision mode and, via research.js, its wide mode), which
+// strips a fetched page before handing it to Gemini -- can reuse this
 // instead of duplicating the tag/entity-stripping regexes.
 export function htmlToText(html) {
   return html

connectors/gemini/client.js

@@ -4,20 +4,41 @@
 // Auth header: "x-goog-api-key: <api_key>"
 // ---------------------------------------------------------------------------
 
-import { GEMINI_API_KEY, GEMINI_API, GEMINI_MODEL, GEMINI_FALLBACK_MODELS } from "../../config.js";
+import { GEMINI_API_KEY, GEMINI_API, GEMINI_MODEL, GEMINI_FALLBACK_MODELS, GEMINI_REQUEST_TIMEOUT_MS } from "../../config.js";
 import { isModelCoolingDown, setModelCooldown, parseRetryDelaySeconds } from "./cooldown.js";
 
 async function callGenerateContentOnce(body, model) {
   if (!GEMINI_API_KEY) throw new Error("GEMINI_API_KEY is not set. Add it as an environment variable on the madmcp server.");
 
-  const res = await fetch(`${GEMINI_API}/models/${model}:generateContent`, {
-    method: "POST",
-    headers: {
-      "x-goog-api-key": GEMINI_API_KEY,
-      "Content-Type":   "application/json",
-    },
-    body: JSON.stringify(body),
-  });
+  const controller = new AbortController();
+  const timeout = setTimeout(() => controller.abort(), GEMINI_REQUEST_TIMEOUT_MS);
+
+  let res;
+  try {
+    res = await fetch(`${GEMINI_API}/models/${model}:generateContent`, {
+      method: "POST",
+      headers: {
+        "x-goog-api-key": GEMINI_API_KEY,
+        "Content-Type":   "application/json",
+      },
+      body: JSON.stringify(body),
+      signal: controller.signal,
+    });
+  } catch (err) {
+    // Network-level failure -- connection dropped, DNS/TLS error, or our own
+    // abort firing. None of these carry an HTTP status (err.status is
+    // undefined), so without this they'd fall through callGenerateContent's
+    // 429/503-only retry check as a hard, non-cascading failure even though
+    // they're exactly as transient as a 503 in practice. `transient: true`
+    // lets the cascade (and delegate.js's isTransientGeminiError) treat them
+    // the same way, without pretending they're a real HTTP status code.
+    const isAbort = err.name === "AbortError";
+    const wrapped = new Error(isAbort ? `Gemini request timed out after ${GEMINI_REQUEST_TIMEOUT_MS}ms (model: ${model})` : `Gemini request failed (network error, model: ${model}): ${err.message}`);
+    wrapped.transient = true;
+    throw wrapped;
+  } finally {
+    clearTimeout(timeout);
+  }
 
   const text = await res.text();
   let data;
@@ -71,7 +92,8 @@ async function callGenerateContent(body, requestedModel) {
       const isLast = i === models.length - 1;
       const isRateLimited = err.status === 429;
       const isOverloaded  = err.status === 503;
-      if ((!isRateLimited && !isOverloaded) || isLast) throw err;
+      const isNetworkTransient = err.transient === true; // timeout/dropped connection, see callGenerateContentOnce
+      if ((!isRateLimited && !isOverloaded && !isNetworkTransient) || isLast) throw err;
       if (isRateLimited) {
         // Rate-limited on this model -- record a cooldown (best-effort; never
         // blocks or throws on its own) so future calls can skip straight past
@@ -88,8 +110,8 @@ async function callGenerateContent(body, requestedModel) {
 
 // Single-turn text generation. Takes a plain prompt string (build any
 // system/user framing into it before calling) and returns the model's text
-// output. Used by Delegate_web_fetch -- a genuine one-shot "here's context,
-// answer this" call with no tool use.
+// output. Used by delegate_research's precision mode (url + question) --
+// a genuine one-shot "here's context, answer this" call with no tool use.
 export async function geminiGenerate(prompt, { model = GEMINI_MODEL, maxOutputTokens } = {}) {
   const body = {
     contents: [{ role: "user", parts: [{ text: prompt }] }],
@@ -111,7 +133,9 @@ export async function geminiGenerate(prompt, { model = GEMINI_MODEL, maxOutputTo
 }
 
 // Multi-turn call WITH function-calling support -- used by
-// connectors/gemini/delegate.js's investigation loop. Unlike geminiGenerate,
+// connectors/gemini/delegate.js's GitHub/Notion/Cloudflare investigation loop
+// AND connectors/gemini/research.js's web-only research loop (delegate_research's
+// wide mode). Unlike geminiGenerate,
 // this takes/returns the raw `contents` conversation array and the raw
 // candidate, since the caller (delegate.js) needs to inspect whether the
 // response is a functionCall (keep looping) or plain text (done), which a
@@ -126,9 +150,16 @@ export async function geminiGenerate(prompt, { model = GEMINI_MODEL, maxOutputTo
 // with functionResponse.id echoing the originating functionCall.id. See
 // delegate.js for how a turn is actually built -- don't "fix" it back to
 // role: "function" without re-checking current docs against the model in use.
-export async function geminiChat(contents, { model = GEMINI_MODEL, tools, maxOutputTokens } = {}) {
+export async function geminiChat(contents, { model = GEMINI_MODEL, tools, toolConfig, maxOutputTokens } = {}) {
   const body = { contents };
   if (tools) body.tools = tools;
+  // toolConfig is currently only ever passed as
+  // { includeServerSideToolInvocations: true } by research.js, required to
+  // combine the native googleSearch tool with a custom function declaration
+  // in the same call (see research.js's file header for the exact contract
+  // -- confirmed against Google's generateContent tool-combination docs,
+  // 2026-07-27). delegate.js never passes this: it has no built-in tools.
+  if (toolConfig) body.toolConfig = toolConfig;
   if (maxOutputTokens) body.generationConfig = { maxOutputTokens };
 
   const data = await callGenerateContent(body, model);

connectors/gemini/delegate.js

@@ -9,7 +9,7 @@
 // SCOPE: every delegated function below is READ-ONLY. Gemini is never given
 // a write-capable function here -- writes stay confined to the fixed
 // GEMINI_NOTION_ROOT_PAGE_ID path in tools.js, same isolation rule as
-// Delegate_web_fetch. This file only reaches into GitHub/Cloudflare/Notion's
+// delegate_research. This file only reaches into GitHub/Cloudflare/Notion's
 // existing client-layer functions (not the MCP tool layer) to avoid
 // round-tripping through the MCP server for its own internal calls.
 //
@@ -62,7 +62,7 @@ const HARD_MAX_STEPS = 30;
 // safety/recitation block) is a config or request problem that will
 // reproduce identically on a resume, not something retrying fixes.
 function isTransientGeminiError(err) {
-  return err?.status === 429 || err?.status === 503;
+  return err?.status === 429 || err?.status === 503 || err?.transient === true;
 }
 
 // Minimal line-based diff (LCS backtrace) -- good enough for investigation
@@ -798,12 +798,32 @@ const FUNCTION_DECLARATIONS = [{
   functionDeclarations: FUNCTIONS.map(({ name, description, parameters }) => ({ name, description, parameters })),
 }];
 
+// SCOPE NOTE (2026-07-27): this file deliberately has NO web access (no
+// web_fetch, no Google Search grounding) -- that lives entirely in
+// connectors/gemini/research.js, behind the separate delegate_research
+// tool. Keeping the two apart is a security boundary, not just a UX split:
+// this loop reads private GitHub/Notion/Cloudflare/Context7/Mem0 data, and
+// research.js's loop reads untrusted public web content -- a single loop
+// with both would let a malicious page or search result Gemini encounters
+// mid-investigation try to talk the model into leaking whatever it just
+// read from those private systems (e.g. via a crafted outbound fetch to an
+// attacker-controlled URL). Neither loop can do that, because neither ever
+// has both capabilities available at once. Do NOT re-add web_fetch or a
+// google_search tool here -- add web capability to research.js instead.
+
 const SYSTEM_PREAMBLE =
   "You are a read-only investigation agent. Use the available functions to gather whatever " +
   "information you need to answer the task fully, calling as many as necessary across multiple " +
   "turns. When you have enough information, respond with a final plain-text answer and no further " +
   "function calls. Be specific and cite what you found (file paths, commit SHAs, log entries, page " +
-  "titles) rather than speculating.";
+  "titles) rather than speculating.\n\n" +
+  "IMPORTANT -- cross-check, don't just aggregate: when the task touches more than one source " +
+  "(e.g. a GitHub PR's status vs. a Notion tracking page, or a repo file vs. what a database row " +
+  "claims), actively look for contradictions between them rather than reporting each source's claim " +
+  "in isolation. A thing that LOOKS current, open, or resolved in one source can be stale or wrong " +
+  "according to another -- if your task plan touches multiple sources for related claims, check them " +
+  "against each other before answering, and call out any discrepancy explicitly (including which " +
+  "source you consider more authoritative and why) rather than picking one silently.";
 
 // Runs the investigation loop. Returns { answer, steps, transcript, runId,
 // failed? } where transcript is a human-readable log of each function call