Support for Other LLM Providers

[Manish06097]

Is their any other way to use this for the other LLM like Mistral , llama2...

[MrJarnould]

@Manish06097 currently, Instructor only works with OpenAI Function Calling. Have a look at https://github.com/PrefectHQ/marvin since Marvin works with other LLMs – this is @jxnl's recommendation.

[jxnl]

I agree. I want to mostly only support OpenAI function calling since it means that the library is not responsible for building the structured output prompts. There's so many ways to do it that I think a framework that forces you to use their prompts ends up being limiting.

I think Marvin is great, so is Guardrails and Outlines, you can always just Model(**dict) to get the Pydantic object out again.

It's definitely up for debate but my goal is to give users a great experience and focus on one provider, since I'll be building more tooling around fine-tuning function calling specifically.

When other models are being released at half-functional capabilities, I'll definitely start including them more. To do that is to probably support an OpenAI API, and then changing the base URL.

[j4k0bk]

Bump! Would love to be able to run a local Llama 2, so I avoid the cost of OpenAI, inference APIs, etc.

[jxnl]

Check out Marvin for now, I think i'd support it if there was actually some function calling support out of some of these models