Skip to content

feat(providers): add DeepInfra provider and curated models (#619) - #624

Open
cagdasyurekli wants to merge 2 commits into
andrewyng:mainfrom
cagdasyurekli:feat/issue-619-deepinfra-provider
Open

feat(providers): add DeepInfra provider and curated models (#619)#624
cagdasyurekli wants to merge 2 commits into
andrewyng:mainfrom
cagdasyurekli:feat/issue-619-deepinfra-provider

Conversation

@cagdasyurekli

Copy link
Copy Markdown

Summary

This PR addresses and closes #619 by registering DeepInfra as a first-class model provider and curating its latest generation of frontier open-weight models (DeepSeek V4, GLM 5.3, Kimi K3) with verified context window metadata.

Motivation & Background

DeepInfra hosts high-performance inference endpoints for leading open-weight frontier models. While users could previously configure a generic custom endpoint, doing so caused:

  1. The desktop UI's context fill meter to show "Context meter unavailable for custom models".
  2. The context compaction engine (TurnEngine._compaction_config) to fall back to default thresholds, prematurely compacting long-context models like GLM 5.3 and Kimi K3 (which support 1,000,000 tokens).

Changes

  1. Provider Registration (coworker/providers/registry.py):

    • Registered deepinfra in DESCRIPTORS via _compat() pointing to https://api.deepinfra.com/v1/openai.
    • Default environment variable: DEEPINFRA_API_KEY.
    • Recommended model: deepseek-ai/DeepSeek-V4-Flash.
    • Pre-filled editable endpoint with help text.
  2. Curated Model Matrix (coworker/providers/matrix.py):

    • Added 8 curated entries with verified context windows and capabilities:
      • deepinfra:deepseek-ai/DeepSeek-V4-Flash: 128,000 tokens (_AGENTIC)
      • deepinfra:deepseek-ai/DeepSeek-V4-Flash-0731: 128,000 tokens (_AGENTIC)
      • deepinfra:deepseek-ai/DeepSeek-V4-Pro: 128,000 tokens (_AGENTIC)
      • deepinfra:deepseek-ai/DeepSeek-V4-Pro-0731: 128,000 tokens (_AGENTIC)
      • deepinfra:zai-org/GLM-5.3-Flash: 1,000,000 tokens (_AGENTIC_VISION)
      • deepinfra:zai-org/GLM-5.3: 1,000,000 tokens (_AGENTIC)
      • deepinfra:moonshotai/Kimi-K3: 1,000,000 tokens (_AGENTIC_VISION)
      • deepinfra:moonshotai/Kimi-K2.7-Code: 256,000 tokens (_AGENTIC)
  3. Automated Tests:

    • tests/test_token_usage.py: verified all 8 DeepInfra models export exact context windows in model_context_windows().
    • tests/test_provider_router.py: verified client builder default base URL and router prefix stripping for DeepInfra models.
    • tests/test_providers.py: verified capabilities (including vision), lockstep reseller descriptor checks, and key isolation (ensuring OPENAI_API_KEY is not leaked when DEEPINFRA_API_KEY is missing).
    • tests/test_provider_verify.py: verified test probe targets https://api.deepinfra.com/v1/openai/models with Bearer auth.

Verification

  • Provider and compaction test suites:
    pytest tests/test_token_usage.py tests/test_provider_router.py tests/test_providers.py tests/test_provider_verify.py tests/test_compaction.py tests/test_engine.py -q -> 144 passed.
  • Full regression suite: 1943 passed.
  • Zero frontend modifications required; the desktop UI automatically consumes model_context_windows from /v1/settings.

Closes #619

cagdasyurekli and others added 2 commits September 4, 2026 17:43
…#619)

Add DeepInfra as a first-class model provider and curate its latest model catalog:
- Register deepinfra in coworker/providers/registry.py via _compat() pointing to https://api.deepinfra.com/v1/openai with DEEPINFRA_API_KEY.
- Add 8 curated DeepInfra models to MATRIX in coworker/providers/matrix.py (DeepSeek V4 Flash/Pro including 0731 pinned snapshots, GLM 5.3 Flash/Pro, Kimi K3 and Kimi K2.7 Code) with verified context windows (128k, 256k, 1M).
- Enables desktop context fill meter and compaction trigger without UI changes.
- Add unit tests for routing, builder, key verification, context windows, and key isolation.

Closes andrewyng#619
deepseek-ai/DeepSeek-V4-Pro-0731 does not exist on DeepInfra; the valid
snapshot is deepseek-ai/DeepSeek-V4-Pro-0813. Verified against the live
DeepInfra model catalog. Also updates the pinned-snapshot comment which
referenced 0731 for both Flash and Pro.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Add DeepInfra provider and curated models with context window metadata

1 participant