docs(llama_cpp_et): Pythia-410m and BLOOM-560m ports (self-converted, verified, hosting pending) - #189
Conversation
…t yet hosted Introduces the GPTNeoX and BLOOM execution families. Unlike every other port this session, neither has a usable pre-made Q8_0 GGUF at a reasonable size, so both were self-converted directly from the original safetensors via this repo's own convert_hf_to_gguf.py. Pythia-410m required fixing a real conversion bug (locally, via a standalone non-vendored wrapper script, not by editing the vendored converter): the transformers version in this environment restructures GPTNeoX's rotary_pct/rotary_emb_base into a nested rope_parameters dict, which the vendored GPTNeoXModel.set_gguf_parameters() doesn't know about yet. BLOOM converted cleanly with no changes needed. Both locally verified via sysemu: full layer ET offload (25/25 and 25/25) and a valid compute graph for each, correct param counts and architecture strings confirmed. Not yet registered in artifacts.json/benchmark_config.json: as self-converted artifacts they need hosting (GitHub Release asset, same pattern as distilbert_sst2/roberta_sst2 from an earlier PR, since there's no HF upload token), and creating that release hit a local tooling permission block this session. Recipes document full verification evidence; registration is a one-line follow-up once hosting is available.
github-actions
Bot
added
the
track: community
Per docs/MODEL_PORTS_TRACK_PLAN.md's two-stage process -- see the gemma2_2b/gpt2 commit on port-gpt2-gemma2-causal-lm for the full rationale. identity_ids "bloom"/"gptneox" pending maintainer approval (requested in issue aifoundry-org#200). Both models are self-converted (no pre-made GGUF existed); source pins the original upstream safetensors repo, not our derived artifact -- the artifact itself remains unhosted this session (recipe documents that gap separately).
github-actions
Bot
added
the
track: model-ports
|
These are still documentation/claims without the hosted immutable artifacts and registered board benchmarks. To count Pythia and BLOOM, please finish hosting, pin revision + SHA256 + size + license, add one benchmark per model, commit deterministic oracle evidence, and provide successful full-offload ET-SoC1 runs. Documentation alone cannot receive model-port credit. |
…gistration for pythia410m/bloom560m Hackathon deadline has passed. This commit is record-completeness cleanup only, not a hackathon submission -- done so this catalog is accurate for aiNekko's ET-SoC1 card review. The original converted GGUFs for both models were lost before they could be hosted, leaving these ports as documentation-only. Both were re-converted from the same pinned upstream revisions (pythia410m via the documented rope_parameters wrapper fix, bloom560m with the stock converter, no fix needed) and hosted on Hugging Face (darthceltic85/pythia-410m-gguf, darthceltic85/bloom-560m-gguf). Added missing artifacts.json entries, per-model benchmark JSON files, benchmark_config.json registration, fixed each claim's benchmark_config pointer to reference its own benchmark file instead of the top-level config, and added committed deterministic PPL oracles for both. Re-verified live: pythia410m loads with n_rot=16 (confirming the rope_parameters fix), PPL = 16.6002 +/- 2.99104. bloom560m loads with f_max_alibi_bias=8.0 (confirming ALiBi), PPL = 27.8342 +/- 5.56596.
|
Note: this update was made after the official hackathon deadline (per maintainer's own announcement in the project Discord). It is not a hackathon submission and is not seeking credit or track scoring. We're cleaning up the remaining open PRs so this catalog is accurate for aiNekko's ET-SoC1 card records. Per your review, both models were missing completed hosting and a registered board benchmark — the original converted GGUFs had been lost before they could be hosted. Both were re-converted from the same pinned upstream revisions and re-hosted on Hugging Face ( Re-verified live: pythia410m loads with |
2 participants
Adds pythia410m and bloom560m causal-LM ports to llama.cpp-et. Both self-converted (no pre-made GGUF exists); artifacts not yet hosted.