Skip to content

feat(clients): add native Gemini API client#13

Merged
wyenox merged 16 commits intomoly-ai:mainfrom
TigerInYourDream:feat/gemini-native-client
Mar 28, 2026
Merged

feat(clients): add native Gemini API client#13
wyenox merged 16 commits intomoly-ai:mainfrom
TigerInYourDream:feat/gemini-native-client

Conversation

@TigerInYourDream
Copy link
Copy Markdown
Contributor

Summary

Adds a native Gemini API client (GeminiClient) that talks directly to Google's Generative Language API, without going through an OpenAI-compatible shim.

Changes

  • src/clients/gemini.rs (new) — Full BotClient implementation for Gemini:

    • Model listing via GET /models with generateContent capability filtering
    • Streaming chat completions via :streamGenerateContent with SSE
    • Proper role mapping: User/Tooluser, Botmodel, Systemsystem_instruction
    • API key auth via x-goog-api-key header + key query param
    • Robust URL building that preserves existing base-URL query params
    • 5 unit tests covering model parsing, URL construction, request building, and stream parsing
  • src/clients.rs — Registers the gemini module under the api-clients feature gate

  • src/prelude.rs — Re-exports GeminiClient for ergonomic usage

Design Notes

  • Follows the same Arc<RwLock<Inner>> pattern used by OpenAiClient for thread-safe cloning
  • Uses async_stream + parse_sse (existing crate utility) for streaming
  • All async work returns futures — no internal task spawning, consistent with AITK's design philosophy
  • Cross-platform compatible (no tokio-specific APIs in the client itself)

@TigerInYourDream
Copy link
Copy Markdown
Contributor Author

TigerInYourDream commented Feb 17, 2026

This incremental update includes 3 main changes:

  1. fix(sse): parse CRLF events and skip non-data frames safely (e94d05b)

    • Fixed SSE parsing compatibility for \r\n\r\n event separators.
    • Safely skips non-data: frames to avoid Option::unwrap() panics.
  2. feat(gemini): implement native tool-calling roundtrip (76c9e54)

    • Implemented full Gemini native tool-calling roundtrip (model emits function call -> local tool execution -> function response returned -> model continues and produces final answer).
    • Aligned request/response behavior with Gemini native fields and tool mode constraints.
  3. feat(example): add gemini native tool-calls demo (49d71cc)

    • Added a runnable example at examples/gemini-tool-calls to verify native Gemini tool-calling behavior.

Validated: the example reliably triggers get_weather tool calls and returns the final answer, which confirms the native tool-calling path in this PR is working.

@TigerInYourDream TigerInYourDream marked this pull request as draft February 22, 2026 14:00
@TigerInYourDream
Copy link
Copy Markdown
Contributor Author

This incremental update includes 3 main changes:

  1. fix(sse): parse CRLF events and skip non-data frames safely (e94d05b)

    • Fixed SSE parsing compatibility for \r\n\r\n event separators.
    • Safely skips non-data: frames to avoid Option::unwrap() panics.
  2. feat(gemini): implement native tool-calling roundtrip (76c9e54)

    • Implemented full Gemini native tool-calling roundtrip (model emits function call -> local tool execution -> function response returned -> model continues and produces final answer).
    • Aligned request/response behavior with Gemini native fields and tool mode constraints.
  3. feat(example): add gemini native tool-calls demo (49d71cc)

    • Added a runnable example at examples/gemini-tool-calls to verify native Gemini tool-calling behavior.

Validated: the example reliably triggers get_weather tool calls and returns the final answer, which confirms the native tool-calling path in this PR is working.

@TigerInYourDream
Copy link
Copy Markdown
Contributor Author

CleanShot 2026-03-04 at 13 50 29@2x

@TigerInYourDream TigerInYourDream marked this pull request as ready for review March 4, 2026 05:52
Comment thread src/utils/sse.rs
Comment thread src/clients/gemini.rs Outdated
Comment thread src/clients/gemini.rs Outdated
@TigerInYourDream TigerInYourDream requested a review from wyenox March 5, 2026 03:32
…wn tool results

- Promote fallback ids to protocol ids when they arrive later in the
  stream, preferring same-index matches to avoid misassignment when
  chunk indices shift between SSE events.
- Return an error instead of silently degrading when a tool result
  references an unknown tool_call_id.
- Fix ensure_ordered_id to check the order list instead of calls_by_id.
- Collapse nested if-let per clippy, tighten comments, shorten test names.
@TigerInYourDream
Copy link
Copy Markdown
Contributor Author

TigerInYourDream commented Mar 14, 2026

Changes since last review

Core: stream tool-call id collision fix

The previous implementation used stream_index alone to correlate tool calls across SSE chunks. When indices shift between events (e.g. chunk 1 sends A at index 0 and B at index 1, chunk 2 resends only B at index 0), the old code would misassign A's fallback id to B's protocol id.

Promotion policy (3-tier):

  1. Same stream_index + same signature → promote immediately (strongest correlation, no ambiguity)
  2. Global signature search with exactly one candidate → promote (unambiguous even after index shift)
  3. Global signature search with 0 or 2+ candidates → don't promote, insert as new entry

The key design choice is tier 3: when multiple fallback calls share the same signature (same function name + args) and none has a protocol id yet, the protocol layer cannot tell them apart. Rather than guessing and potentially renaming the wrong call (state corruption), we conservatively skip the promotion. This may produce a duplicate call, but a duplicate is always safer than a wrong-assignment — downstream tool execution can handle an extra call, but cannot recover from a call whose id was silently swapped.

This limitation is inherent to the Gemini streaming protocol when functionCall.id is absent. The ambiguity resolves naturally once Gemini provides protocol ids for all calls.

Other changes

  • as_tool_parts now returns Result and errors on unknown tool_call_id instead of silently degrading to a text part
  • Fixed ensure_ordered_id checking wrong collection (calls_by_idorder)
  • Collapsed nested if let per clippy, tightened comments, trimmed redundant tests (19 → 12)

@TigerInYourDream TigerInYourDream force-pushed the feat/gemini-native-client branch 2 times, most recently from 445d950 to 097443d Compare March 14, 2026 17:07
@TigerInYourDream TigerInYourDream force-pushed the feat/gemini-native-client branch from 097443d to 3e0378b Compare March 14, 2026 17:25
@wyenox wyenox merged commit bc0534d into moly-ai:main Mar 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants