how to work with llama.cpp

[scalar27]

I'm on a Mac (M1) and generally use llama.cpp with model, e.g., llama 3 instruct 8B....gguf. I want to use Piper, which I've used before and it's fast enough that it makes up for its reduced audio quality (for now!). To use AlwaysReddy, I think I have to edit config.py to set COMPLETIONS_API and COMPETION_MODEL to get this to work, right? Any help with what I need to put here? Thanks.

[scalar27]

Ok, have installed LM_STUDIO. Getting seg faults. Investigating...

[ILikeAI]

Hey there, thanks for raising the issue. What do you mean by seg issues? Jump in the discord and I can have a closer look if you like https://discord.gg/uMeEJfKN