Releases: bosslesss/inference-labs-python
Releases · bosslesss/inference-labs-python
v0.1.1 - LlamaIndex integration
Adds inference_labs.llamaindex.InferenceLabsLLM -- a LlamaIndex CustomLLM that routes every call through the Inference Labs policy router.
pip install --extra-index-url https://inference-labs.com/pypi/simple/ "inference-labs[llamaindex]"from inference_labs.llamaindex import InferenceLabsLLM
from llama_index.core import Settings
Settings.llm = InferenceLabsLLM(strategy="cost-first", max_cost_usd=0.01)Full changelog: https://github.com/bosslesss/inference-labs-python/blob/main/CHANGELOG.md