Skip to content

docs(models): add Qwen3.5-4B page #6

Description

@xiaguan

The models/ section currently only has a Qwen3-4B page (src/content/docs/models/qwen3-4b.md and the matching sidebar entry in astro.config.mjs). Qwen3.5-4B is a supported model line in openinfer (openinfer-qwen35-4b crate, --features qwen35-4b) and should have its own page.

What's missing

  • src/content/docs/models/qwen35-4b.md
  • Sidebar entry under the Models group in astro.config.mjs

Suggested content

Mirror the structure of qwen3-4b.md, but cover the Qwen3.5-4B specifics:

  • Build & launchcargo run --release --features qwen35-4b -- --model-path models/Qwen3.5-4B
  • Build-time Python + Triton AOT — the qwen35-4b feature is the only build step that needs Python (via openinfer-kernels/tools/triton/gen_triton_aot.py); document OPENINFER_TRITON_PYTHON and the fallback chain (.venv/bin/pythonpython3python)
  • Architecture — hybrid layout: 24 linear (SLA) layers + 8 full-attention layers, and how that differs from Qwen3-4B's full attention
  • Accuracy gatescargo test --release -p openinfer-qwen35-4b --features qwen35-4b --test hf_golden_gate (with OPENINFER_TEST_MODEL_PATH=models/Qwen3.5-4B); note that exact-text JSON baselines are retired and e2e_scheduler is the liveness gate
  • Performance — benchmark snapshots when available

Context

  • Repo README confirms Qwen3.5-4B is a first-class model line.
  • AGENTS.md in the main repo has the canonical build/test commands.

Anyone with cycles, feel free to pick this up — happy to review the PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions