[WIP] Rough sketch of Workers AI binding.

[kflansburg]

Relates to #417

Couple of notes:

• The JavaScript ai SDK actually does a lot for certain model types. Embeddings are simple, but others will be more complicated to support.
• Might want to think a bit about how this can be a more idiomatic Rust API, currently I'm copying the JS API which does a lot of duck typing.
• Not sure what the leading crates are for working with LLMs in the Rust ecosystem. Will people be using this API directly, or is there something like Langchain that we should assume they are working with (not wanting to repeat the mistake of using custom http types).

[SergioRibera]

Hi @kflansburg, just asking, why didn't you continue with this PR? I'm checking to be able to use the workers IA with Rust and I found this, I find it very useful for the moment.

Not sure what the leading crates are for working with LLMs in the Rust ecosystem.

In rust there are usually these libraries super interesting and popular (I'm probably forgetting some):

• burn
• candle

Will people be using this API directly, or is there something like Langchain that we should assume they are working with (not wanting to repeat the mistake of using custom http types).

I am not very clear on how the workers work from behind, but we could discuss how to make a more Rustacean integration.

From the @RustLangES community we are exploring the use of this feature, if needed we would like to collaborate.

[kflansburg]

This was closed in favor of #543 , but the main blocker is the number of models and input / output schemas that would need to be implemented.

Regarding Burn and Candle, those seem to be aimed at defining new model architectures and training models (similar to Torch or TensorFlow). Workers is not really the right platform for this. Instead Workers AI allows you to do inference with common open source models hosted in adjacent infrastructure.

My comment above about popular libraries was referring to libraries like LangChain which may be used to chain, prompt, or form agents using an underlying LLM API (such as Workers AI or OpenAI provide).

I do think its possible that if we feel good about the API in the other PR, we can land it with just a few models and then add models as they are needed.

[SergioRibera]

but the main blocker is the number of models and input / output schemas that would need to be implemented.

This part could be generated automatically, do you have a base with all the models and schemas?

Regarding Burn and Candle, those seem to be aimed at defining new model architectures and training models (similar to Torch or TensorFlow). Workers is not really the right platform for this. Instead Workers AI allows you to do inference with common open source models hosted in adjacent infrastructure.

My comment above about popular libraries was referring to libraries like LangChain which may be used to chain, prompt, or form agents using an underlying LLM API (such as Workers AI or OpenAI provide).

Okay, this makes sense, I got confused

I do think its possible that if we feel good about the API in the other PR, we can land it with just a few models and then add models as they are needed.

I think it's great and an excellent idea, also as it is used will arise requests, is there any way we can collaborate to have ready a first view of the Workers AI API?

[kflansburg]

I think it's great and an excellent idea, also as it is used will arise requests, is there any way we can collaborate to have ready a first view of the Workers AI API?

The proposed API is in the other PR #543