Add hf-best-model skill

.claude-plugin/marketplace.json

@@ -32,6 +32,12 @@
       "skills": "./",
       "description": "Add and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing scores from Artificial Analysis API, and running custom evaluations with vLLM/lighteval."
     },
+    {
+      "name": "huggingface-best",
+      "source": "./skills/huggingface-best",
+      "skills": "./",
+      "description": "Find the best AI model for any task by querying Hugging Face leaderboards and benchmarks. Recommends top models based on task type, hardware constraints, and benchmark scores."
+    },
     {
       "name": "hf-cli",
       "source": "./skills/hf-cli",

README.md

@@ -91,6 +91,7 @@ This repository contains a few skills to get you started. You can also contribut
 | Name | Description | Documentation |
 |------|-------------|---------------|
 | `hf-cli` | Execute Hugging Face Hub operations using the hf CLI. Download models/datasets, upload files, manage repos, and run cloud compute jobs. | [SKILL.md](skills/hf-cli/SKILL.md) |
+| `huggingface-best` | Find the best AI model for any task by querying Hugging Face leaderboards and benchmarks. Recommends top models based on task type, hardware constraints, and benchmark scores. | [SKILL.md](skills/huggingface-best/SKILL.md) |
 | `huggingface-community-evals` | Add and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing scores from Artificial Analysis API, and running custom evaluations with vLLM/lighteval. | [SKILL.md](skills/huggingface-community-evals/SKILL.md) |
 | `huggingface-datasets` | Explore, query, and extract data from any Hugging Face dataset using the Dataset Viewer REST API and npx tooling. Zero Python dependencies — covers split/config discovery, row pagination, text search, filtering, SQL via parquetlens, and dataset upload via CLI. | [SKILL.md](skills/huggingface-datasets/SKILL.md) |
 | `huggingface-gradio` | Build Gradio web UIs and demos in Python. Use when creating or editing Gradio apps, components, event listeners, layouts, or chatbots. | [SKILL.md](skills/huggingface-gradio/SKILL.md) |

agents/AGENTS.md

@@ -4,6 +4,7 @@ You have additional SKILLs documented in directories containing a "SKILL.md" fil
 
 These skills are:
  - hf-cli -> "skills/hf-cli/SKILL.md"
+ - huggingface-best -> "skills/huggingface-best/SKILL.md"
  - huggingface-community-evals -> "skills/huggingface-community-evals/SKILL.md"
  - huggingface-datasets -> "skills/huggingface-datasets/SKILL.md"
  - huggingface-gradio -> "skills/huggingface-gradio/SKILL.md"
@@ -20,6 +21,7 @@ IMPORTANT: You MUST read the SKILL.md file whenever the description of the skill
 <available_skills>
 
 hf-cli: `"Hugging Face Hub CLI (`hf`) for downloading, uploading, and managing models, datasets, spaces, buckets, repos, papers, jobs, and more on the Hugging Face Hub. Use when: handling authentication; managing local cache; managing Hugging Face Buckets; running or scheduling jobs on Hugging Face infrastructure; managing Hugging Face repos; discussions and pull requests; browsing models, datasets and spaces; reading, searching, or browsing academic papers; managing collections; querying datasets; configuring spaces; setting up webhooks; or deploying and managing HF Inference Endpoints. Make sure to use this skill whenever the user mentions 'hf', 'huggingface', 'Hugging Face', 'huggingface-cli', or 'hugging face cli', or wants to do anything related to the Hugging Face ecosystem and to AI and ML in general. Also use for cloud storage needs like training checkpoints, data pipelines, or agent traces. Use even if the user doesn't explicitly ask for a CLI command. Replaces the deprecated `huggingface-cli`."`
+huggingface-best: `>`
 huggingface-community-evals: `Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.`
 huggingface-datasets: `Use this skill for Hugging Face Dataset Viewer API workflows that fetch subset/split metadata, paginate rows, search text, apply filters, download parquet URLs, and read size or statistics.`
 huggingface-gradio: `Build Gradio web UIs and demos in Python. Use when creating or editing Gradio apps, components, event listeners, layouts, or chatbots.`

skills/huggingface-best/SKILL.md

@@ -0,0 +1,134 @@
+---
+name: huggingface-best
+description: >
+  Use when the user asks about finding the best, top, or recommended model for a task,
+  wants to know what AI model to use, or wants to compare models by benchmark scores.
+  Triggers on: "best model for X", "what model should I use for", "top models for [task]",
+  "which model runs on my laptop/machine/device", "recommend a model for", "what LLM should
+  I use for", "compare models for", "what's state of the art for", or any question about
+  choosing an AI model for a specific use case. Always use this skill when the user wants
+  model recommendations or comparisons, even if they don't explicitly mention HuggingFace
+  or benchmarks.
+---
+
+# HuggingFace Best Model Finder
+
+Finds the best models for a task by querying official HF benchmark leaderboards, enriching
+results with model size data, filtering for what fits on the user's device, and returning a
+comparison table with benchmark scores.
+
+---
+
+## Step 1: Parse the request
+
+Extract from the user's message:
+- **Task**: what they want the model to do (coding, math/reasoning, chat, OCR, RAG/retrieval, speech recognition, image classification, multimodal, agents, etc.)
+- **Device**: hardware constraints (MacBook M-series 8/16/32/64GB unified memory, RTX GPU with VRAM amount, CPU-only, cloud/no constraint, etc.)
+
+If device is not mentioned, skip filtering entirely and return the highest-performing models regardless of size. If the task is genuinely ambiguous, ask one clarifying question.
+
+### Device → max parameter budget
+
+When a device is specified, extract its available memory (unified RAM for Apple Silicon, VRAM for discrete GPUs) and apply:
+
+- **fp16 max params (B)** ≈ memory (GB) ÷ 2
+- **Q4 max params (B)** ≈ memory (GB) × 2
+
+Examples: 16GB → 8B fp16 / 32B Q4 — 24GB VRAM → 12B fp16 / 48B Q4 — 8GB → 4B fp16 / 16B Q4
+
+---
+
+## Step 2: Find relevant benchmark datasets
+
+Fetch the full list of official HF benchmarks:
+
+```bash
+curl -s -H "Authorization: Bearer $(cat ~/.cache/huggingface/token)" \
+  "https://huggingface.co/api/datasets?filter=benchmark:official&limit=500" | jq '[.[] | {id, tags, description}]'
+```
+
+Read the returned list and select the datasets most relevant to the user's task — match on dataset id, tags, and description. Use your judgment; don't limit yourself to 2-3. Aim for comprehensive coverage: if 5 benchmarks clearly cover the task, use all 5.
+
+---
+
+## Step 3: Fetch top models from leaderboards
+
+For each selected benchmark dataset:
+
+```bash
+curl -s -H "Authorization: Bearer $(cat ~/.cache/huggingface/token)" \
+  "https://huggingface.co/api/datasets/<namespace>/<repo>/leaderboard" | jq '[.[:15] | .[] | {rank, modelId, value, verified}]'
+```
+
+Collect model IDs and scores across all benchmarks. If a leaderboard returns an error (404, 401, etc.), skip it and note it in the output.
+
+---
+
+## Step 4: Enrich with model metadata
+
+For the top 10-15 candidate model IDs, get model infos.
+
+```bash
+# REST API
+curl -s -H "Authorization: Bearer $(cat ~/.cache/huggingface/token)" \
+  "https://huggingface.co/api/models/org/model1" | jq '{safetensors, tags, cardData}'
+
+# CLI (hf-cli)
+hf models info org/model1 --json | jq '{safetensors, tags, cardData}'
+```
+
+Extract from each response:
+- **Parameters**: `safetensors.total` → convert to B (e.g., 7_241_748_480 → "7.2B")
+- **License**: from model card tags (look for `license:apache-2.0`, `license:mit`, etc.)
+- If `safetensors` is absent, parse size from the model name (look for "7b", "8b", "13b", "70b", "72b", etc.)
+
+---
+
+## Step 5: Filter and rank
+
+**If a device was specified:**
+1. Remove models exceeding the fp16 parameter budget for the device
+2. Flag models that fit only with Q4 quantization (multiply budget by ~4 for Q4 capacity)
+3. If a highly-ranked model is slightly over budget, keep it with a "needs Q4" note — don't silently drop it
+
+**If no device was mentioned:** skip all size filtering — just rank by benchmark score.
+
+Then: rank by benchmark score (descending), keep top 5-8 models.
+
+Include proprietary models (GPT-4, Claude, Gemini) if they appear on leaderboards, but flag them as "API only / not self-hostable". If the user explicitly asked for local/open models only, exclude them.
+
+---
+
+## Step 6: Output
+
+### Comparison table
+
+```markdown
+| # | Model | Params | [Benchmark 1] | [Benchmark 2] | License | On device |
+|---|-------|--------|--------------|--------------|---------|-----------|
+| ⭐1 | [org/name](https://huggingface.co/org/name) | 7B | 85.2% | — | Apache 2.0 | Yes (fp16) |
+| 2 | [org/name](https://huggingface.co/org/name) | 13B | 83.1% | 71.5% | MIT | Q4 only |
+| 3 | [org/name](https://huggingface.co/org/name) | 70B | 90.0% | 81.0% | Llama | Too large |
+```
+
+- Link model names to `https://huggingface.co/<model_id>`
+- Use `—` for benchmarks where the model wasn't evaluated
+- Star the top recommended pick with ⭐
+- "On device" values: `Yes (fp16)`, `Q4 only`, `Too large`, `API only`
+
+### Follow-up
+
+After presenting the table, ask the user: "Would you like to run **[top recommended model]**?"
+
+If they say yes, ask whether they'd prefer to:
+- **Run locally** — ask about their device if not already known, then give appropriate setup instructions
+- **Run on HF Jobs** — point them to the HF Jobs guide: https://huggingface.co/docs/huggingface_hub/en/guides/jobs
+
+---
+
+## Error handling
+
+- **Leaderboard not found**: skip, note "leaderboard unavailable" in output
+- **Model missing from hub_repo_details**: fall back to parsing size from model name
+- **No benchmarks found for task**: use the curated fallback table above, or try `hub_repo_search` with `filters=["<task>"]` sorted by `trendingScore`
+- **All leaderboards fail**: fall back to `hub_repo_search` for popular models tagged with the task, note that results are by popularity rather than benchmark score