Skip to content

docs: fix stale CLI invocations in LLM inference example - #4172

Merged
zcbenz merged 1 commit into
ml-explore:mainfrom
deBrian07:docs-fix-llama-inference-cli
Aug 11, 2026
Merged

docs: fix stale CLI invocations in LLM inference example#4172
zcbenz merged 1 commit into
ml-explore:mainfrom
deBrian07:docs-fix-llama-inference-cli

Conversation

@deBrian07

Copy link
Copy Markdown
Contributor

Proposed changes## Proposed changes

The example commands in the LLM inference guide no longer run as written, tried copying and pasting them and all failed.

Both scripts in mlx-examples/llms/llama now accept flags only:

  • llama.py: --model-path (default mlx_model), --prompt, --few-shot,
    --max-tokens/-m, --write-every, --temp, --seed. No positional arguments.
  • convert.py: --torch-path, --mlx-path, --model-name, --quantize,
    --q-group-size, --q-bits, --dtype. Writes a directory (default mlx_model),
    not a single .npz file.

docs/src/examples/llama-inference.rst still passes the weights and tokenizer
positionally, and uses --num-tokens, which no longer exists. This updates the
convert.py call, the three llama.py calls, and the surrounding sentence to
match the current CLI. Prompts, sample output, and timings are unchanged.

This is only a doc change, found when following the guide, no corresponding issues.

Checklist

Put an x in the boxes that apply.

  • I have read the CONTRIBUTING document
  • I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the necessary documentation (if needed)

@deBrian07
deBrian07 force-pushed the docs-fix-llama-inference-cli branch from f67825b to 6221e3f Compare August 11, 2026 02:09

@zcbenz zcbenz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@zcbenz
zcbenz merged commit cd5b92c into ml-explore:main Aug 11, 2026
24 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants