Add Gemini agent, tools and cli support, from #169 #189
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR extracts and contributes the GeminiAgent and Gemini computer-use tool from #169 and made requested changes.
Will add CLI integration for hud eval in a followup commit.
Note
Introduce a Gemini MCP agent and computer-use tool, integrate it into the CLI eval flow, and add supporting settings, types, tests, and dependency.
GeminiAgentusing Google GenAI with MCP tool execution, tool mapping, screenshot trimming, and function response formatting.GeminiAgentinhud/agents/__init__.py.GeminiComputerToolmapping Gemini computer-use functions (click_at, navigate, drag_and_drop, etc.) to executor actions with normalized coords and URL metadata.hud/tools/__init__.pyandhud/tools/computer/__init__.py.AgentType.GEMINIand integrate Gemini into agent selection, builder, and dataset/single-task flows with default model and API key checks.settings.gemini_api_keyand Gemini display/rescale defaults incomputer_settings.AgentTypewithgemini.ContentResultto carryurland emit__URL__:metadata for screenshots.wait_for_load_statetype and reuse existing CDP context/page.google-genaitopyproject.toml.GeminiAgentand CLIbuild_agentfor Gemini.Written by Cursor Bugbot for commit ff1a3ae. This will update automatically on new commits. Configure here.