Skip to content

Enable readline for the chat prompt and persist history - #1654

Open
Vlor999 wants to merge 1 commit into
ml-explore:mainfrom
Vlor999:feat/chat-cli-readline
Open

Enable readline for the chat prompt and persist history#1654
Vlor999 wants to merge 1 commit into
ml-explore:mainfrom
Vlor999:feat/chat-cli-readline

Conversation

@Vlor999

@Vlor999 Vlor999 commented Aug 3, 2026

Copy link
Copy Markdown

Proposed changes

Fixes #1653.

corridor_input() wraps its ANSI escapes in \x01 / \x02 — readline's
markers for non-printing characters — but nothing imported readline, so those
two bytes were written to the terminal verbatim and the prompt had no history or
line editing.

This imports readline (guarded, it is absent on some platforms) and
loads/saves ~/.mlx_lm_chat_history around the chat session, on rank 0 only.

Why

Under a real PTY (reproducer in #1653):

raw 0x01 emitted raw 0x02 emitted history line editing
before yes yes no no
after no no yes, persisted yes

The markers already in corridor_input() are evidence this was the intent — they
are inert without readline.

How

  • import readline in a try/except ImportError, falling back to None. Every
    history helper no-ops when it is None, so platforms without it keep working
    exactly as they do today.
  • load_chat_history() in ChatUI.__enter__, save_chat_history() in
    __exit__, both behind the existing rank == 0 gate so distributed workers
    never touch the file.
  • History capped at 1000 entries; unreadable/unwritable paths are ignored rather
    than raising, so a bad $HOME cannot break the chat.

Scope — deliberately smaller than #1382

This replaces #1382, which bundled four things and stalled. Rebuilt on current
main against the newer ChatUI, keeping only what stands up:

from #1382 here why
readline history + line editing kept the part @nastya236 called "a great idea", and it fixes the marker leak
distributed broadcast_string dropped @nastya236 asked "did you have this issue?" and the honest answer was no — it was speculative. ChatUI.prompt() on current main is already rank-aware.
markdown prompt by default dropped @nastya236 wanted time to think; it was blocking the rest
sample_utils top_k message dropped already fixed on main

354 lines became 96.

Validation

python -m unittest tests.test_chat tests.test_generate tests.test_utils   # 44 passed
pre-commit run --files mlx_lm/cli_ui.py tests/test_chat.py                # black, isort

Five tests added: history round-trips through a file, a missing file and an
unwritable path are both non-fatal, everything no-ops when readline is None,
and rank 1 never touches the history while rank 0 does.

I checked they are not vacuous by removing the save_chat_history() call from
__exit__: the suite fails. With the change in place, all 44 pass.

Note

Ctrl+D still raises EOFError out of the chat loop rather than exiting
cleanly. That is a separate small fix and I left it out to keep this focused —
happy to send it after this lands.


Supersedes #1382. Thanks @nastya236 for the review there — the questions about
the distributed path are what led to dropping it. The demo videos from that
thread still show the history/line-editing behaviour this PR keeps.

corridor_input() wraps its ANSI escapes in \x01/\x02, the readline markers for
non-printing characters, but nothing imported readline. Without it those two
bytes were written to the terminal verbatim, and the prompt had no history or
line editing.

Import readline (guarded, it is absent on some platforms) and load/save
~/.mlx_lm_chat_history around the chat session, on rank 0 only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chat prompt emits raw readline markers (\x01/\x02) because readline is never imported

1 participant