feat: implementation of semanticly retrieved dynamic toolsets

[ryan-timothy-albert]

• Implements experimental dynamic tool selection through Gram-Mode header semantic_search | progressive_search
  • question should we call this mode semantic instead?
• Vector storage and semantic search of toolsets is done via pgvector, all encapsulated into the rag/search_tools.go client.
• For embedding this will be done through openrouter base client right now using the text-embedding-3-small
• Given this is experimental and a toolset cannot be marked as dynamic. Indexing is done on demand as necessary in find_tools. We do not need to reindex if we recognize there has been no change to the toolset version or tools. Even in cases where toolset indexing is necessary it is still quite fast, easily under a second (spans tracking)
• Right now we just hard delete old embeddings. There is no immediate reason to keep them around so it preserves space. The data model is setup for soft deletions

Following this we'll complete a documentation page on these experimental modes and start our own testing across large toolsets between different modes.

[changeset-bot]

🦋 Changeset detected

Latest commit: 7cbbe95

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
server	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
gram	Ready	Preview	Comment	Nov 11, 2025 8:14pm
gram-docs-redirect	Ready	Preview	Comment	Nov 11, 2025 8:14pm
gram-landing-redirect	Ready	Preview	Comment	Nov 11, 2025 8:14pm