feat: add screenshot capture tool #47
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Beautiful. I was literally planning on doing something similar yesterday. I will test on Windows |
Author
|
@josharagon Thanks! Really appreciate you testing on Windows - I don't have a Windows |
Add a new capture_screenshot MCP tool that captures the Roblox Studio window and returns base64-encoded JPEG data. This enables Claude to visually analyze workspaces, debug UI issues, and verify changes. Implementation details: - Cross-platform support for macOS and Windows - macOS: Uses Swift to filter for main Studio window + screencapture - Windows: Uses PowerShell with System.Drawing for screen capture - Images resized to max 1024x1024px while maintaining aspect ratio - JPEG compression at quality 85 for optimal clarity/size balance - Returns base64-encoded data (~93K chars) via MCP protocol Window selection: - macOS: Filters for windows with " - Roblox Studio" in title - Prevents capturing auxiliary windows (Output, etc.) Requirements: - macOS requires Screen Recording permission for Terminal/MCP client - Windows has no additional permission requirements Documentation: - Updated README with tool descriptions and technical details - Added TESTING.md with testing guidelines and benchmarks - Added setup instructions for macOS Screen Recording permissions 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
kevinswint
force-pushed
the
add-screenshot-capture-tool
branch
from
8135db2 to
da94594
Compare
Critical fixes: - Return Content::image instead of Content::text for proper multimodal handling - Fix Windows PowerShell script with proper Win32/RECT type definitions Improvements: - Extract shared process_screenshot() function to eliminate code duplication - Add configurable constants for image dimensions and JPEG quality - Add timeouts to external commands (Swift/PowerShell) to prevent hangs - Standardize error messages across platforms - Add -NoProfile and -ExecutionPolicy Bypass flags for Windows reliability - Add proper resource disposal in Windows script - Improve error handling with more descriptive messages - Relax Swift window matching (removed leading space requirement) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Now that images are properly returned as Content::image (not text), we can use higher resolution without hitting token limits. Bumped from 1024 to 2048 for better detail and legibility. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
4096 max dimension captures native resolution on most displays without downscaling, while still capping extremely large windows. Small windows are unaffected (no upscaling occurs). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Author
|
Hey @josharagon I've pushed some updates to PR #47:
Would love your help testing on Windows when you get a chance, thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add Screenshot Capture Tool
Summary
This PR adds a new
capture_screenshotMCP tool that enables Claude to visually analyze Roblox Studio workspaces by capturing screenshots of the Studio window.Motivation
Currently, Claude can interact with Roblox Studio through code execution (
run_code) and asset insertion (insert_model), but lacks the ability to visually inspect the workspace. This screenshot capability enables powerful new workflows:run_code)Implementation
Architecture
Platform-Specific Details
macOS:
screencapturecommand-line toolWindows:
Output Format
Testing
✅ Tested on macOS 14.x (Sonnet) with Roblox Studio
Documentation
Files Changed
Cargo.toml- Addedbase64dependencyCargo.lock- Dependency updatessrc/rbx_studio_server.rs- Screenshot tool implementation (~155 lines)README.md- Tool documentation and examples (~38 lines)Breaking Changes
None - This is a purely additive change.
Future Enhancements
Potential follow-up work (out of scope for this PR):
Checklist
Code follows project style and conventions
Cross-platform implementation (macOS + Windows)
Documentation updated (README)
Tested locally on macOS
No breaking changes
Commit message follows conventional commits format
Updates (Jan 6)
Content::imageinstead ofContent::text, so Claude processes it as a proper image rather than tokenizing base64 textprocess_screenshot()function, added named constants