How do I get started with cascadeflow?

[saschabuehrle]

New to cascadeflow? Here's everything you need to get started!

Quick Start

Python

pip install cascadeflow[providers]

# Create agent
from cascadeflow import CascadeAgent
agent = CascadeAgent(models=[
    {"name": "claude-3-5-haiku-20241022", "provider": "anthropic", "cost": 0.0008},
    {"name": "gpt-5", "provider": "openai", "cost": 0.00125},
])

# Run query
result = agent.run("What is TypeScript?")
print(result.content)
print(f"Cost: ${result.cost:.6f}")

TypeScript

npm install @cascadeflow/core

import { CascadeAgent } from '@cascadeflow/core';

const agent = new CascadeAgent({
  models: [
    { name: 'claude-3-5-haiku-20241022', provider: 'anthropic', cost: 0.0008 },
    { name: 'gpt-5', provider: 'openai', cost: 0.00125 },
  ]
});

const result = await agent.run('What is TypeScript?');
console.log(result.content);
console.log(`Cost: $${result.cost}`);

Resources

• Installation Guide
• Architecture Overview
• Examples

Common Questions

Q: Which models should I use?
A: We recommend Claude-3.5-Haiku + GPT-5 for best results. See Presets Guide.

Q: How much can I save?
A: Most users see 40-85% cost reduction depending on their use case.

Q: Does it work with streaming?
A: Yes! Both Python and TypeScript support streaming responses.

Have more questions? Ask below!

[nixfroasty]

• CascadeFlow is an open-source framework for model cascading. It dynamically routes your LLM queries to cheaper/smaller models first, and only escalates to more expensive ones if needed.
• It uses speculative execution + quality validation: you can define thresholds for correctness, confidence, etc.
  -Supports many LLM providers: OpenAI, Anthropic, Groq, vLLM, Ollama, Hugging Face, Together, etc.
  -It has built-in cost tracking, per-query spending caps, and telemetry.
• It’s available in Python and TypeScript/JavaScript.

from cascadeflow import CascadeAgent, ModelConfig

Define cheap and expensive models

models = [
ModelConfig(name="gpt-4o-mini", provider="openai", cost=0.000375),
ModelConfig(name="gpt-4o", provider="openai", cost=0.00625),
]

agent = CascadeAgent(
models=models,
quality_threshold=0.8 # how confident the draft needs to be before accepting
)

async def run():
result = await agent.run("Explain the theory of relativity in simple terms.")
print("Answer:", result)

import asyncio
asyncio.run(run())

Integrate with LangChain

from langchain import ChatOpenAI
from cascadeflow.integrations.langchain import CascadeFlow

drafter = ChatOpenAI(model="gpt-4o-mini")
verifier = ChatOpenAI(model="gpt-4o")

cascade = CascadeFlow(
drafter=drafter,
verifier=verifier,
quality_threshold=0.8
)

Use cascade like any other LangChain model

answer = await cascade.ainvoke("What is quantum mechanics?")
print(answer)