fix(inference): probe /v1/models for OpenAI-compatible local runtimes (Closes #5053)

[M3gA-Mind]

Summary

• Local-AI model discovery now selects its probe endpoint by provider type, not by "is it localhost": genuine Ollama uses GET /api/tags, every OpenAI-compatible runtime uses GET /v1/models.
• Fixes custom OpenAI-compatible BYOK endpoints on localhost (e.g. LM Studio at http://localhost:1234/v1) that were being probed with the Ollama-only /api/tags, producing GET /v1/api/tags and an empty model list.
• Adds model_discovery_api(provider, base_url) + endpoint_is_openai_v1(base_url) selectors and routes local-AI diagnostics through them.
• Adds unit coverage for the endpoint selection and a diagnostics-level regression test.

Problem

When a custom OpenAI-compatible provider (LM Studio serving nvidia/nemotron-3-nano-omni) is configured as a BYOK endpoint on http://localhost:1234/v1, the app sends GET /v1/api/tags — an Ollama-specific listing endpoint — instead of the OpenAI-standard GET /v1/models. LM Studio logs Unexpected endpoint or method (GET /v1/api/tags), returns 200 with an incompatible payload, and model discovery silently fails so the model is unavailable in the tinyagents module.

Root cause: LocalAiService::diagnostics gated the OpenAI-compatible /v1/models probe on provider_from_config(config) == LmStudio. But provider_from_config collapses everything that isn't literally lm_studio onto Ollama — including OMLX, local-openai, and any custom BYOK endpoint whose provider tag defaulted to ollama. Those all fell through to the Ollama branch and were probed at <base>/api/tags. With an OpenAI /v1 base that yields GET /v1/api/tags. (The inference_list_models RPC path already probed /models correctly; the break was in the diagnostics discovery path that populates the selectable-model list.)

Solution

• New ModelDiscoveryApi selector in src/openhuman/inference/local/provider.rs:
  • endpoint_is_openai_v1(base_url) — true when the URL path is the OpenAI /v1 root. A genuine Ollama base is host-rooted (http://localhost:11434) with no /v1, so this separates the two API shapes by endpoint type, not by "is it localhost" (the conflation that misidentified LM Studio as Ollama).
  • model_discovery_api(provider, base_url) — returns OllamaTags only for a genuine ollama runtime on a host-rooted base; returns OpenAiModels for an explicit lm_studio/omlx slug or any OpenAI /v1 endpoint. The /v1 clause rescues a custom BYOK localhost endpoint whose provider tag still defaults to ollama.
• diagnostics() now routes any OpenAI-compatible /v1 local endpoint to the existing /v1/models diagnostics path before it can reach the Ollama /api/tags branch. Genuine Ollama (host-rooted base) is unaffected.
• Debug logging added on the reroute (no PII).

Design notes / tradeoffs: the fix keys on the OpenAI /v1 endpoint type in addition to the provider slug, because the saved provider tag is an unreliable signal (it defaults to ollama). This also incidentally fixes the same misrouting for OMLX/local-openai runtimes. Ollama's own OpenAI-compatible /v1/models works too, so even a hand-edited Ollama base ending in /v1 degrades gracefully.

Submission Checklist

If a section does not apply to this change, mark the item as N/A with a one-line reason. Do not delete items.

• Tests added or updated (happy path + at least one failure / edge case) per Testing Strategy
• Diff coverage ≥ 80% — changed lines (Vitest + cargo-llvm-cov merged via diff-cover) meet the gate enforced by .github/workflows/ci-lite.yml. Changed lines are the two selectors + the diagnostics reroute, all exercised by the new unit and regression tests.
• Coverage matrix updated — N/A: behaviour-only change (no feature row added/removed/renamed in docs/TEST-COVERAGE-MATRIX.md).
• All affected feature IDs from the matrix are listed in the PR description under ## Related — N/A: no matrix feature ID applies to this internal routing fix.
• No new external network dependencies introduced (mock backend / in-test axum mock server used; no live network).
• Manual smoke checklist updated if this touches release-cut surfaces — N/A: does not change a release-cut surface.
• Linked issue closed via Closes #NNN in the ## Related section

Impact

• Runtime/platform impact: desktop (and any core build) that uses a local OpenAI-compatible runtime (LM Studio, OMLX, local-openai, custom BYOK). No mobile/web/CLI surface change.
• Performance: none (same single probe, correct path). Security/migration/compatibility: none — genuine Ollama behaviour is byte-identical; only OpenAI-compatible /v1 endpoints change from /api/tags to /v1/models.

Related

• Closes: LM Studio / OpenAI-compatible providers: app calls Ollama-style /api/tags instead of /v1/models, breaking model discovery for nvidia/nemotron-3-nano-omni and similar #5053
• Follow-up PR(s)/TODOs: none

---

AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

• Key: N/A (GitHub-issue-driven, not Linear)
• URL: N/A

Commit & Branch

• Branch: fix/GH-5053-openai-compatible-model-discovery
• Commit SHA: 117103171

Validation Run

• pnpm --filter openhuman-app format:check — N/A: no app/src (frontend) files changed.
• pnpm typecheck — N/A: no TypeScript changed (Rust-only change).
• Focused tests: cargo test --lib -- model_discovery_api endpoint_is_openai_v1 diagnostics_openai_compatible_v1_endpoint → 4 passed.
• Rust fmt/check (if changed): cargo fmt on touched files; GGML_NATIVE=OFF cargo check --lib clean.
• Tauri fmt/check (if changed): N/A: no app/src-tauri files changed.

Validation Blocked

• command: N/A
• error: N/A
• impact: N/A

Behavior Changes

• Intended behavior change: OpenAI-compatible local runtimes (LM Studio/OMLX/local-openai/custom BYOK on a /v1 base) are probed for models at /v1/models instead of /api/tags.
• User-visible effect: local models served by LM Studio (e.g. nvidia/nemotron-3-nano-omni) now appear as selectable and no Unexpected endpoint errors are logged.

Parity Contract

• Legacy behavior preserved: genuine Ollama (host-rooted base, provider = ollama) still uses /api/tags — unchanged.
• Guard/fallback/dispatch parity checks: model_discovery_api returns OllamaTags for ollama/empty provider on a non-/v1 base (covered by model_discovery_api_uses_tags_for_genuine_ollama).

Duplicate / Superseded PR Handling

• Duplicate PR(s): none
• Canonical PR: this PR
• Resolution: N/A

Summary by CodeRabbit

• Bug Fixes

  • Improved local runtime detection for OpenAI-compatible /v1 endpoints.
  • Prevented OpenAI-compatible runtimes from being queried with Ollama-specific model APIs.
  • Improved diagnostics and installed-model detection for compatible local providers.

• Tests

  • Added coverage for endpoint detection, provider routing, and model discovery.
  • Added regression coverage for OpenAI-compatible endpoints using an Ollama-style provider setting.

[coderabbitai]

📝 Walkthrough

Walkthrough

Changes

Local model discovery routing

Layer / File(s)	Summary
Discovery API selection src/openhuman/inference/local/provider.rs	Adds endpoint-shape and provider-based selection between Ollama /api/tags and OpenAI-compatible /v1/models, with unit coverage.
Diagnostics endpoint routing src/openhuman/inference/local/service/ollama_admin/diagnostics.rs, src/openhuman/inference/local/service/ollama_admin_tests.rs	Routes /v1 endpoints through LM Studio diagnostics and verifies model discovery against a mock /v1/models endpoint.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

• tinyhumansai/openhuman#5053 — Directly addressed by routing /v1 endpoints to /v1/models.
• tinyhumansai/openhuman#5055 — Related to correcting OpenAI-compatible model discovery paths.

Suggested labels: bug, rust-core

Suggested reviewers: senamakel

Poem

A rabbit hops where models hide,
/v1/models opens wide.
No stray tags lead paths astray,
Local runtimes find their way.
Carrots cheer the routing bright!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: routing OpenAI-compatible local runtimes to probe /v1/models.

---

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1171031718

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[M3gA-Mind]

Synced onto current main (now includes #5070 feature-gate allowlist + #5003/#5025 learning-subscriber decouple). Re-running CI.

[M3gA-Mind]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[sanil-23]

The mechanism here is right and the root-cause analysis is genuinely excellent — deciding by endpoint type rather than "is it localhost" is the correct call, and it directly answers the fix suggested in #5053. My one blocking concern is scope.

The fix is discovery-only; inference still misroutes for the same config

diagnostics.rs:23 is one of 15 sites gated on provider_from_config(config) == LocalAiProvider::LmStudio. This PR corrects that gate at the diagnostics leaf, but the chat path is gated on the same expression and is unchanged.

Tracing the exact #5053 config (provider = "ollama", base_url = "http://localhost:1234/v1"):

Discovery — fixed by this PR:

provider_from_config()   -> Ollama         (line 23 gate not taken)
model_discovery_api()    -> OpenAiModels   <- NEW: /v1 endpoint wins
                         -> GET http://localhost:1234/v1/models      OK

Chat (public_infer.rs:279) — untouched:

provider_from_config()   -> Ollama         (gate not taken)
                         -> falls through to the Ollama branch
base_url = ollama_base_url_from_config(config)  -> "http://localhost:1234/v1"
                         -> POST {base_url}/api/chat
                         -> POST http://localhost:1234/v1/api/chat    BROKEN

public_infer.rs:341 builds the URL with format!("{base_url}/api/chat"), and ollama_base_url_from_config preserves the path component (it only rewrites 0.0.0.0/[::] hosts and rejects ollama.com), so the /v1 survives. That is structurally the same malformed URL as the /v1/api/tags this PR fixes — same root cause, one function over.

Worth noting the premises are identical: this PR is only necessary because the saved slug is ollama (a lm_studio slug would already route correctly at line 23). So the exact configuration that motivates this PR is the configuration for which chat stays broken. The two can't be separated.

Resulting user experience

Per #5053, LM Studio returns 200 for unrecognized endpoints (Unexpected endpoint or method ... Returning 200 anyway). So POST /v1/api/chat won't 404 cleanly — it returns 200 with a body that isn't an OllamaChatResponse, landing at public_infer.rs:367:

.map_err(|e| format!("ollama chat response parse failed: {e}"))?

Net effect: the model now appears in the picker (this PR working as intended), the user selects it, and chat fails with an error naming Ollama for a provider they configured as LM Studio. To be clear this is not a regression — nobody who had working chat loses it — but it moves the failure from setup time to use time and makes it harder to self-diagnose than the current empty-list symptom.

Blast radius

Remaining sites on the slug gate:

Site	Behavior for this config
public_infer.rs:279, :548	Chat + inference -> Ollama /api/chat
assets.rs:61, 294-337, 522, 680	Readiness/warning surface computed as Ollama
bootstrap.rs:180, 471	Bootstrap probe; lmstudio:// model URI
model_ids.rs:82, 154	Effective chat-model id resolution

Suggested direction

Rather than another inline check, apply the endpoint-type correction at the decision function so every call site inherits it:

/// Effective local provider - the saved slug corrected by endpoint type.
/// A `/v1` base is OpenAI-compatible regardless of a slug that defaulted
/// to `ollama` (GH #5053).
pub(crate) fn effective_provider_from_config(config: &Config) -> LocalAiProvider {
    match model_discovery_api(&config.local_ai.provider, &lm_studio_base_url(config)) {
        ModelDiscoveryApi::OpenAiModels => LocalAiProvider::LmStudio,
        ModelDiscoveryApi::OllamaTags => LocalAiProvider::Ollama,
    }
}

...then migrate the provider_from_config(config) == LmStudio call sites. Patching leaves means 15 sites to find individually, and whichever one gets missed reproduces this exact bug.

If that's too large for this PR, that is a completely legitimate call — but then please retarget to Refs #5053 rather than Closes, since the issue's acceptance criteria include "loads and responds to chat and agent task requests through the tinyagents module", and open a follow-up for inference dispatch.

Secondary: diagnostics payload shape changes for rerouted configs

diagnostics.rs:476-496 — lm_studio_diagnostics returns a materially different report than the Ollama branch: ollama_running: false, no context_requirement, no chat_eligibility/embedding_eligibility, no per-model eligibility/chat_capable/size/modified_at, and "provider": "lm_studio" even for an OMLX/local-openai/custom BYOK runtime.

ModelStatusSection.tsx:492 renders the status pill directly off diagnostics.ollama_running, so a working endpoint on this path displays as not running. The memory-layer context gate (MIN_CONTEXT_TOKENS, and the TAURI-RUST-4P6 chat_capable filter) also silently stops applying.

There is a narrow real regression here: provider = "omlx" on a host-rooted base previously took the Ollama branch and got the full report; it now takes the slug arm into lm_studio_diagnostics and gets the reduced shape. Worth either calling out in the description or populating context_requirement + eligibility from the native /api/v0/models data that lm_studio_context_window_for already parses.

Smaller items

• diagnostics.rs:27-46 — the routing decision is evaluated against ollama_base_url_from_config (config-first precedence) but delegates to lm_studio_diagnostics, which recomputes via lm_studio_base_url (env-first: OPENHUMAN_LM_STUDIO_BASE_URL, LM_STUDIO_BASE_URL, then config). With either env var set you can decide "OpenAI-compatible" from one URL and probe a different host. Resolve the base once and pass it down.

• diagnostics.rs:41-44 — base_url is logged raw. Neighbouring logs go through redact_ollama_base_url (ollama.rs:127) / redact_url_for_log precisely because a BYOK base can carry user:pass@; AGENTS.md requires never logging secrets. Prefer tracing::debug! with structured fields to match the rest of the file too.

• provider.rs:75-86 — endpoint_is_openai_v1 is stricter than normalize_lm_studio_base_url (lm_studio.rs:94-98), which deliberately strips /models and /chat/completions suffixes because users paste them. A base saved as http://localhost:1234/v1/models fails detection, takes the Ollama branch, and produces GET /v1/models/api/tags — the same bug class. The test at provider.rs:142 locks this in. If the strictness is deliberate, worth saying so in the doc comment.

• ollama_admin_tests.rs:781 — the new test depends on lm_studio_base_url, which reads OPENHUMAN_LM_STUDIO_BASE_URL/LM_STUDIO_BASE_URL before config, but inference_test_guard() (local/mod.rs:11-15) is only a mutex and clears nothing. A developer with either var exported gets a confusing failure. The Ollama side solved this with OllamaEnvGuard.

What's good

• Root-cause writeup names the exact collapsing behavior in provider_from_config and explains why endpoint type beats the slug — unusually clear.
• Selectors are pure and total; reusing normalize_provider keeps alias handling in one place.
• The regression test's mock serves only /v1/models with no /api/tags, so it genuinely fails on the old code rather than passing incidentally.
• Error/timeout handling on the delegated path is sound: bounded 5s probe, no unwraps, no hang path, graceful degradation into issues[]. LmStudioModelsResponse.data is #[serde(default)], so a conforming-but-empty payload yields an empty list rather than an error.
• Genuine Ollama behavior is untouched — the early return sits ahead of all Ollama work.

CI note

The 4 cancelled checks are not failures — they are duplicate PR Quality (soft) runs from the same trigger on 2c220df, cancelled by the concurrency group, with a sibling run (29755301486) green. Head commit is 13 SUCCESS / 6 SKIPPED / 0 FAILURE. No re-run needed.

---

Requesting changes on the scope question only. Happy to approve as-is under Refs #5053 with a follow-up issue for inference dispatch, if you'd prefer to land the discovery fix now.

Reviewed with Claude Code.