Skip to content

mlx_lm.generate: repetition_penalty has no default (unlike other sampling params), model can loop into repeated text with no warning #1669

Description

@arjunneema09

Description
Running mlx_lm.generate with default sampling parameters, a model looped
into repeated text with no warning. Looking at generate.py on main,
other sampling parameters have explicit DEFAULT_* constants defined at
module level (DEFAULT_TEMP, DEFAULT_TOP_P, DEFAULT_MIN_P,
DEFAULT_TOP_K, etc.), but there is no DEFAULT_REPETITION_PENALTY.
SERVER.md also documents defaults for repetition_context_size (20),
presence_penalty (0.0), and frequency_penalty (0.0), but doesn't state
a default for repetition_penalty itself.

To reproduce

  1. Run mlx_lm.generate --model <model> --prompt "<prompt>" with no
    --repetition-penalty flag set.
  2. With some prompts/models, generation degenerates into repeated text
    or loops.

Expected behavior
Either a sensible default repetition penalty applied out of the box, or
at minimum a clear note in the docs/CLI help that repetition_penalty is
off by default and users should set it explicitly for longer generations.

Question
Is the lack of a default intentional (e.g. to match upstream sampling
behavior exactly)? If so, a one-line doc/help-text note would still help
first-time users avoid this.

Environment
mlx-lm version: [will confirm]
Python version: [will confirm]
OS: macOS [version], chip [M1/M2/M3/...]

Happy to submit a small docs PR to add this note if useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions