Skip to content

Add Extended Providers experiment#148

Draft
Jameswlepage wants to merge 17 commits intoWordPress:developfrom
Jameswlepage:feature/providers
Draft

Add Extended Providers experiment#148
Jameswlepage wants to merge 17 commits intoWordPress:developfrom
Jameswlepage:feature/providers

Conversation

@Jameswlepage
Copy link
Copy Markdown
Contributor

@Jameswlepage Jameswlepage commented Dec 19, 2025

CleanShot 2025-12-19 at 09 15 39@2x

What?

Adds a new Extended Providers experiment that registers 9 additional AI provider integrations:

  • Cloudflare Workers AI
  • Cohere
  • DeepSeek
  • Fal.ai (image generation)
  • Grok (xAI)
  • Groq
  • HuggingFace
  • Ollama (local)
  • OpenRouter

Also includes provider credentials UI enhancements and a metadata registry for custom branding per provider.

Why?

The core wp-ai-client package ships with OpenAI, Anthropic, and Google. Users want access to additional providers—especially local options like Ollama and cost-effective alternatives like Groq and DeepSeek.

How?

  • Adds Extended_Providers experiment class that registers all 9 providers with the wp-ai-client registry
  • Each provider includes a Provider class, TextGenerationModel (or ImageGenerationModel for Fal.ai), and ModelMetadataDirectory
  • Adds Provider_Metadata_Registry for custom branding (icons, colors, API key URLs)
  • Adds Provider_Credentials_UI initialization to display enhanced credentials screen
  • Works around provider registration timing issue (see Add hook for registering providers before metadata collection wp-ai-client#43)

Testing Instructions

  1. Enable the Extended Providers experiment in Settings → AI → Experiments
  2. Navigate to Settings → AI and verify all 9 providers appear on the credentials screen
  3. Add an API key for at least one cloud provider (e.g., Groq, DeepSeek) and verify it saves
  4. If you have Ollama running locally, verify it appears and connects
Open WordPress Playground Preview

Adds additional AI provider integrations:
- Cloudflare Workers AI
- Cohere
- DeepSeek
- Fal.ai
- HuggingFace
- Ollama
- OpenRouter
- xAI (Grok)

Includes provider credentials UI and metadata registry.
- Initialize HTTP discovery strategy before experiments load
- Register extended providers immediately instead of on late init hook
- Initialize Provider_Credentials_UI for enhanced credentials display
- Reorder initialization: HTTP client → experiments → AI_Client

The wp-ai-client package collects providers during AI_Client::init().
Extended providers must be registered before that collection occurs.
- Add phpcs:disable/enable blocks for exception escaping false positives
- Add phpcs:ignore for DisallowMultiConstantDefinition false positives
- Update @return annotations to use fully qualified class names
- Fix variable alignment spacing issues
- Add missing translators comment for placeholder
@codecov
Copy link
Copy Markdown

codecov bot commented Dec 19, 2025

Codecov Report

❌ Patch coverage is 3.86572% with 1890 lines in your changes missing coverage. Please review.
✅ Project coverage is 36.13%. Comparing base (3b059d9) to head (c1a152b).

Files with missing lines Patch % Lines
...eriments/Extended_Providers/Extended_Providers.php 1.85% 317 Missing ⚠️
includes/bootstrap.php 10.82% 206 Missing ⚠️
includes/Admin/Provider_Metadata_Registry.php 0.00% 167 Missing ⚠️
...des/Providers/Cohere/CohereTextGenerationModel.php 0.00% 144 Missing ⚠️
...udes/Providers/Grok/GrokModelMetadataDirectory.php 0.00% 88 Missing ⚠️
...udflare/CloudflareWorkersAiTextGenerationModel.php 0.00% 82 Missing ⚠️
...udes/Providers/FalAi/FalAiImageGenerationModel.php 0.00% 79 Missing ⚠️
...des/Providers/Ollama/OllamaTextGenerationModel.php 0.00% 67 Missing ⚠️
...udes/Providers/Groq/GroqModelMetadataDirectory.php 0.00% 58 Missing ⚠️
.../Providers/Cohere/CohereModelMetadataDirectory.php 0.00% 47 Missing ⚠️
... and 29 more
Additional details and impacted files
@@              Coverage Diff               @@
##             develop     #148       +/-   ##
==============================================
- Coverage      57.48%   36.13%   -21.35%     
- Complexity       563      953      +390     
==============================================
  Files             35       65       +30     
  Lines           2907     4832     +1925     
==============================================
+ Hits            1671     1746       +75     
- Misses          1236     3086     +1850     
Flag Coverage Δ
unit 36.13% <3.86%> (-21.35%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an Extended Providers experiment that registers 9 additional AI provider integrations (Cloudflare, Cohere, DeepSeek, Fal.ai, Grok, Groq, HuggingFace, Ollama, OpenRouter) alongside the core providers (OpenAI, Anthropic, Google). It includes enhanced UI for provider credentials with custom branding, icons, and tooltips.

Key changes:

  • New Extended Providers experiment with provider class registration system
  • Enhanced credentials UI with React components for provider icons and interactive tooltips
  • Provider implementations for 9 additional AI services
  • Provider metadata registry for custom branding per provider
  • Bootstrap initialization reordered to register providers before AI Client initialization

Reviewed changes

Copilot reviewed 54 out of 54 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
webpack.config.js Adds provider-credentials entry point for admin UI assets
src/admin/provider-credentials/* React components and styles for enhanced credentials screen with provider icons and tooltips
src/admin/components/provider-icons.tsx Icon component registry mapping provider IDs to SVG icon components
src/admin/components/icons/* SVG icon components for all providers (Anthropic, OpenAI, Google, etc.)
src/admin/components/ProviderTooltipContent.tsx Tooltip component displaying provider metadata and available models
includes/bootstrap.php Reorders initialization to register extended providers before AI Client init
includes/Providers/*/ Provider implementations for OpenRouter, Ollama, HuggingFace, Groq, Grok, FalAi, DeepSeek, Cohere, and Cloudflare
includes/Experiments/Extended_Providers/* Experiment class implementing provider registration with filtering and selection UI
includes/Experiment_Loader.php Registers Extended_Providers experiment in default list
includes/Admin/Provider_Metadata_Registry.php Central registry providing structured metadata and branding for all providers
includes/Admin/Provider_Credentials_UI.php Initializes enhanced credentials UI assets
docs/experiments/extended-providers.md Documentation for Extended Providers experiment features and usage

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

*/
private static function get_initials( string $name ): string {
$parts = preg_split( '/\s+/', trim( $name ) );
if ( empty( $parts ) ) {
Copy link

Copilot AI Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The preg_split function should have its return value validated. If preg_split fails (returns false), it should be handled to prevent type errors. Consider adding error handling or using an alternative approach with explode.

Suggested change
if ( empty( $parts ) ) {
if ( false === $parts || empty( $parts ) ) {

Copilot uses AI. Check for mistakes.
Comment on lines +40 to +42
$request = new Request(
HttpMethodEnum::POST(),
rtrim( OllamaProvider::get_base_url(), '/api' ) . '/api/chat',
Copy link

Copilot AI Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The URL construction logic using rtrim to remove '/api' and then appending '/api/chat' is fragile. If the base URL doesn't end with '/api', this will result in an incorrect URL. Consider using a more robust URL construction approach or documenting the expected format of the base URL.

Suggested change
$request = new Request(
HttpMethodEnum::POST(),
rtrim( OllamaProvider::get_base_url(), '/api' ) . '/api/chat',
$base_url = rtrim( OllamaProvider::get_base_url(), '/' );
if ( preg_match( '#/api/?$#', $base_url ) === 1 ) {
$base_url = preg_replace( '#/api/?$#', '', $base_url );
}
$endpoint = $base_url . '/api/chat';
$request = new Request(
HttpMethodEnum::POST(),
$endpoint,

Copilot uses AI. Check for mistakes.
Comment on lines +32 to +34
$request = new Request(
HttpMethodEnum::GET(),
rtrim( OllamaProvider::get_base_url(), '/api' ) . '/api/tags'
Copy link

Copilot AI Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The URL construction logic using rtrim to remove '/api' and then appending '/api/tags' is fragile and duplicates the same pattern from OllamaTextGenerationModel. If the base URL doesn't end with '/api', this will result in an incorrect URL. Consider extracting this logic to a helper method in the OllamaProvider class.

Suggested change
$request = new Request(
HttpMethodEnum::GET(),
rtrim( OllamaProvider::get_base_url(), '/api' ) . '/api/tags'
$base_url = rtrim( OllamaProvider::get_base_url(), '/' );
// Ensure the base URL ends with '/api' exactly once before appending '/tags'.
if ( substr( $base_url, -4 ) !== '/api' ) {
$base_url .= '/api';
}
$request = new Request(
HttpMethodEnum::GET(),
$base_url . '/tags'

Copilot uses AI. Check for mistakes.
*/
import domReady from '@wordpress/dom-ready';
import { Popover } from '@wordpress/components';
import { useRef, useState, useEffect } from '@wordpress/element';
Copy link

Copilot AI Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused import useEffect.

Copilot uses AI. Check for mistakes.
@Jameswlepage Jameswlepage changed the title Add Extended Providers experiment with support for Cloudflare, Cohere, DeepSeek, Fal.ai, Grok, Groq, HuggingFace, Ollama, and OpenRouter Add Extended Providers experiment Dec 19, 2025
@jeffpaul
Copy link
Copy Markdown
Member

@jeffpaul
Copy link
Copy Markdown
Member

  • Reminder for us to add a screenshot gif and entry in the readme.txt for this feature before merging/releasing.

@JasonTheAdams JasonTheAdams removed their request for review December 21, 2025 05:58
@jeffpaul jeffpaul modified the milestones: 0.2.0, 0.3.0 Jan 7, 2026
@jeffpaul jeffpaul moved this from Needs review to In discussion / Needs decision in WordPress AI Planning & Roadmap Jan 7, 2026
@jeffpaul jeffpaul modified the milestones: 0.3.0, 0.4.0 Feb 3, 2026
@jeffpaul
Copy link
Copy Markdown
Member

The primary question I have here is whether the approach in includes/Providers is the "right" technical approach we'd want folks following in adding their own providers as they extend the plugin for their own needs.

cc: @felixarntz @JasonTheAdams for your input as this could be a good example for folks looking to leverage the WP AI Client as it merges to core and has the three existing providers spun out to their own plugins

@jeffpaul
Copy link
Copy Markdown
Member

jeffpaul commented Mar 4, 2026

@Jameswlepage with the advent of the Connectors screen coming to WP 7.0, perhaps the work here is best suited to crafting additional AI provider plugins for these connections and closing out this draft PR?

@Jameswlepage
Copy link
Copy Markdown
Contributor Author

I think you must have read my mind because after today's call I started jamming on that. Should have an update for this soon.

@Jameswlepage
Copy link
Copy Markdown
Contributor Author

CleanShot 2026-03-04 at 21 04 22@2x CleanShot 2026-03-04 at 21 04 12@2x

Jameswlepage added a commit to Jameswlepage/ai that referenced this pull request Mar 5, 2026
Register 9 additional AI providers (Cloudflare Workers AI, Cohere,
DeepSeek, Fal.ai, Grok, Groq, Hugging Face, Ollama, OpenRouter)
through the Extended Providers experiment. On WP 7.0+, these appear
on Settings > Connectors with custom icons, API key management via
REST API, and provider-specific settings (Cloudflare Account ID +
API Key, Ollama endpoint URL).

- Copy provider classes from feature/providers branch
- Add connectors JS module with SVG icons for all 9 providers
- Register connector settings with show_in_rest for REST API access
- Bridge stored keys to AiClient registry
- Fix PHP 8.1+ setAccessible() deprecation in bootstrap.php
- Fix Ollama URL construction (PR WordPress#148 review feedback)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Jameswlepage and others added 5 commits March 4, 2026 21:12
# Conflicts:
#	includes/Experiment_Loader.php
#	webpack.config.js
On WP 7.0+, extended providers now appear on Settings > Connectors
with custom SVG icons, API key management via REST API, and
provider-specific settings (Cloudflare Account ID + API Key,
Ollama endpoint URL).

- Add connectors JS module with SVG icons for all 9 providers
- Register connector settings with show_in_rest for REST API access
- Bridge stored keys to AiClient registry via core helpers
- Fix PHP 8.1+ setAccessible() deprecation in bootstrap.php
- Fix Ollama URL construction per PR review feedback

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Jameswlepage Jameswlepage marked this pull request as ready for review March 5, 2026 02:14
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 5, 2026

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: Jameswlepage <isotropic@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>
Co-authored-by: huzaifaalmesbah <huzaifaalmesbah@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

Jameswlepage and others added 2 commits March 4, 2026 21:42
Core's _wp_register_default_connector_settings() at init:20 auto-discovers
all registered providers and registers their API key settings, mask filters,
and key-to-registry passing. Our plugin was duplicating this work at init:10,
causing two mask filters per option. When core's
_wp_connectors_pass_default_keys_to_ai_client() ran, it only removed one
mask filter via _wp_connectors_get_real_api_key(), leaving the second active.
This meant get_option() returned masked values (e.g. "••••••fj39") which
were then set as provider authentication, overwriting correct keys.

Changes:
- Remove duplicate register_connector_settings() and pass_connector_keys_to_registry()
- Replace with register_extra_connector_settings() for Cloudflare Account ID
  and Ollama endpoint URL (settings core doesn't handle)
- Add apply_endpoint_provider_urls() for Ollama base URL filter
- Fix is_connectors_supported() to check both trunk and beta2 function names
  (_wp_connectors_get_connector_settings vs _wp_connectors_get_provider_settings)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
WP 7.0 trunk auto-discovers all registered providers and registers their
API key settings, but beta2 only handles the 3 hardcoded core providers.
Add maybe_register_api_key_settings() at init:21 that checks which settings
core already registered and only fills in the gaps. Also add
maybe_pass_keys_to_registry() at init:22 to pass stored keys for providers
that core didn't handle.

This avoids the double-mask-filter bug by checking get_registered_settings()
before registering, ensuring each setting and mask filter is added exactly once.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Jameswlepage Jameswlepage marked this pull request as draft March 5, 2026 03:21
Jameswlepage and others added 3 commits March 4, 2026 22:22
The connectors-extended.js script module is hand-written (not webpack-compiled)
and was being deleted by webpack's clean output. Moved to assets/js/ which
webpack does not manage.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…connectors

Two issues prevented extended providers from working for text generation:

1. All extended provider classes created ProviderMetadata without declaring
   an authentication method. The registry's setRequestAuthenticationForProvider()
   checks getAuthenticationMethod() and throws if null. Added
   RequestAuthenticationMethod::from('api_key') to all cloud providers
   (all except Ollama which is endpoint-based).

2. maybe_pass_keys_to_registry() was gated behind is_connectors_supported(),
   which returns false on WP versions without the Connectors screen. Restructured
   register() so key-passing and endpoint URL setup always run when the
   experiment is enabled, while Connectors-specific UI/settings only run on
   WP 7.0+.

Also adds model preference filter so extended provider models appear in
get_preferred_models_for_text_generation() for title generation etc.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cohere has renamed/versioned their models. The old IDs (command-r-plus,
command-r, command) no longer exist in the API. Updated to current model
IDs: command-r-08-2024, command-a-reasoning-08-2025, command-r7b-12-2024.

Verified end-to-end: Cohere title generation returns a successful result
using command-r-08-2024 with a real API key.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gziolo
Copy link
Copy Markdown
Member

gziolo commented Mar 5, 2026

WordPress/wordpress-develop#11175 - it’s something I still consider for WP 7.0. This way, with the AI plugin, we could feature such extended providers with a single PHP hook on the new Connectors screen.

Jameswlepage and others added 2 commits March 5, 2026 07:18
- Add ABSPATH guards to all 27 provider PHP files
- Fix PHPStan type errors: list vs array casts, null coalescing,
  missing iterable types, undefined method ignores
- Fix PHPCS: short ternary, alignment, early exit patterns,
  empty catch, unused import
- Fix ESLint: conditional hooks, unused vars, missing text domains,
  experimental API imports, prettier formatting
- Remove unused bootstrap.php import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Jameswlepage
Copy link
Copy Markdown
Contributor Author

I think we'll keep this as a draft until the connectors work is stable cc @gziolo

@jeffpaul
Copy link
Copy Markdown
Member

jeffpaul commented Mar 5, 2026

@Jameswlepage if we want to add providers to the Connectors screen, should that approach be to leverage AI Provider for XXX sorts of plugins instead of provider code within the AI Experiments plugin? Feels like the model core and its extenders should be moving to is a suite of AI Provider for XXX plugins that perhaps plugins like AI Experiments decide to include certain ones as available alongside the core 3 for whatever-reasons-the-plugin-has, yeah? Prior to the Connectors screen this sort of approach in this PR made sense, now with the Connectors screen and provider plugins I'm not so sure.

Not sure I understand the motive to keep provider code within the AI Experiments plugin at this point, so if I'm missing something please share.

@gziolo
Copy link
Copy Markdown
Member

gziolo commented Mar 13, 2026

With AI Experiments 0.5.0 (12 MAR 2026), we moved toward the native Connectors screen from WordPress core. We can pivot and take a more refined approach. My proposal is to collect vetted AI provider plugins from the Plugins Directory and inject them so they display the same way as featured providers (Anthropic, Google, OpenAI). Essentially, even when users don't have them installed, they would see the card with an on-click installation flow. I can prototype that to validate that it's already possible today for AI providers with an auth method set to API key.

@jeffpaul
Copy link
Copy Markdown
Member

collect vetted AI provider plugins from the Plugins Directory

I'm happy to see this effort continue, but we should in parallel define what criteria elevates something to be listed on the Connectors page. Does anyone have even some rudimentary thoughts that we can use to start some documentation around this? cc: @Jameswlepage @felixarntz @swissspidy

@swissspidy
Copy link
Copy Markdown
Member

Some of this I have already shared on Slack, but:

We have to consider the implications of listing third-party plugins on that screen. This will put an additional burden/responsibility on third-party plugin developers to add their plugin to that list and ensure that it's up-to-date and secure etc. They might suddenly deal with a big influx of support requests. Is this perhaps too big of an ask? Or should they be transferred to be maintained by WordPress? Probably not, as this would put unnecessary work on contributors' shoulders.

Right now the list of connectors is hardcoded. It would be much better for this data to come from the WordPress.org plugins API instead, where we can more easily swap out plugins and highlight different ones. Then the above point becomes less of a concern. It will be more akin to the Featured plugins tab.

Either way, it begs the question of how these third-party plugins are marked as such on the connectors page. Users will not realize that these aren't official WordPress offerings. Otherwise we'll get tons of support requests or security reports in Trac/HackerOne for things that we don't maintain.

@gziolo
Copy link
Copy Markdown
Member

gziolo commented Mar 13, 2026

It would be much better for this data to come from the WordPress.org plugins API instead, where we can more easily swap out plugins and highlight different ones.

That would be my favorable approach in the long term. We can exercise that in the AI plugin first and replicate it in WP core as soon as possible, given the release cadence.

@felixarntz
Copy link
Copy Markdown
Member

I think we should go with individual plugins, one for each provider.

They can live anywhere of course, but those that we consider official and supported by the WordPress project itself should live under the WordPress GitHub org.

We could probably even use that technically: Thinking of two groups:

  • We could have an area in the connectors screen that shows any plugins with a specific tag (e.g. "connector"). Those would need to be explicitly annotated as third party plugins though (just like pretty much every plugin out there).
  • As a sunset of that, any plugin that is maintained under the WordPress GitHub org would show where we currently have the three hard-coded entries.

As for naming, I think for our official plugins we should continue to use the "AI Provider for XYZ" naming convention. But it shouldn't be a formal policy, beyond the one that already exists about clearly indicating that something is not by the relevant company.

In other words, if at some point Anthropic, Google, OpenAI or others were to build their own first-party provider implementations (which honestly would be awesome!), of course they should be allowed to call it "XYZ Provider" or "XYZ AI Provider", since they are obviously the owners of that trademark.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In discussion / Needs decision

Development

Successfully merging this pull request may close these issues.

7 participants