Skip to content

Releases: bosslesss/inference-labs-python

v0.1.1 - LlamaIndex integration

29 May 05:17

Choose a tag to compare

Adds inference_labs.llamaindex.InferenceLabsLLM -- a LlamaIndex CustomLLM that routes every call through the Inference Labs policy router.

pip install --extra-index-url https://inference-labs.com/pypi/simple/ "inference-labs[llamaindex]"
from inference_labs.llamaindex import InferenceLabsLLM
from llama_index.core import Settings
Settings.llm = InferenceLabsLLM(strategy="cost-first", max_cost_usd=0.01)

Full changelog: https://github.com/bosslesss/inference-labs-python/blob/main/CHANGELOG.md

v0.1.0

27 May 16:48

Choose a tag to compare

First public release. pip install inference-labs