Adding support for other LLMs especially Ollama / Llama.cpp

[jasonm23]

Hi Steve, this project looks very interesting.

Does the roadmap allow for Ollama and other LLM support?

[tazjin]

Basically everything uses an OpenAI v1/chat/messages compatible API. I think the simplest thing to do is to just switch to the OpenAI API (I looked around a bit and it doesn't seem like any Anthropic-specific features are used), and use the Anthropic models through their compatibility endpoint.

This immediately enables use of OpenAI, Gemini, DeepSeek, Qwen, GLM, local models, .... basically, of everything :)

[jasonm23]

Thanks @tazjin I'll check out https://ollama.com/blog/openai-compatibility

[conornash]

This project does not yet use the v1/chat/messages endpoint. Instead, it uses https://api.anthropic.com/v1/messages which is hardcoded and not possible to modify by a variable: https://github.com/steveyegge/efrit/blob/main/efrit-multi-turn.el#L199

I would really like to use this project but I can't even when I modify all URLs to use https://openrouter.ai which is my main LLM provider. I keep getting a username/password requirement that I don't know how to avoid. Does anyone have a working copy of this project that can use Openrouter?

[jasonm23]

@conornash The header Authorization: 'Bearer <OPENROUTER_API_KEY>' needs to be added to the request.

This will need to be integrated by using auth-source (Emacs built in)

Without digging in further I can't give you more implementation detail than that, but I expect it's enough if you know how to C-h f

[ykgoon]

I think this should be in place from day-1. Unless this is an Anthropic sponsored project?

[conornash]

My draft PR is now working with OpenRouter. #7 Please try it out and let me know if you find it useful.