> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# RouteLLM Integration

> Route requests between strong and weak models to optimize cost

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

agent = Agent(
    llm="router-mf-0.5",
    base_url="http://localhost:6060/v1"  # RouteLLM server
)

response = agent.chat("What is 2+2?")
print(response)
```

## Install RouteLLM

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
pip install routellm
```

## Start RouteLLM Server

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
python -m routellm.openai_server \
  --routers mf \
  --strong-model gpt-4o \
  --weak-model gpt-4o-mini \
  --port 6060
```

## Environment Variables

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
export OPENAI_API_KEY=your-api-key
```

## Router Options

| Router       | Description                        | Command                |
| ------------ | ---------------------------------- | ---------------------- |
| `mf`         | Matrix factorization (recommended) | `--routers mf`         |
| `sw_ranking` | Similarity-weighted ranking        | `--routers sw_ranking` |
| `bert`       | BERT classifier                    | `--routers bert`       |
| `causal_llm` | LLaMA classifier                   | `--routers causal_llm` |
| `random`     | Random selection                   | `--routers random`     |

## Threshold Configuration

The threshold (0.0-1.0) controls routing:

* **0.0**: Always use weak model
* **1.0**: Always use strong model
* **0.5**: Balanced (default)

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# More aggressive cost savings (use weak model more often)
agent = Agent(
    llm="router-mf-0.3",
    base_url="http://localhost:6060/v1"
)

# Higher quality (use strong model more often)
agent = Agent(
    llm="router-mf-0.7",
    base_url="http://localhost:6060/v1"
)
```

## Multi-Agent Example

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent, PraisonAIAgentTeam, Task

# All agents use RouteLLM routing
researcher = Agent(
    name="Researcher",
    role="Research analyst",
    goal="Find information",
    llm="router-mf-0.5",
    base_url="http://localhost:6060/v1"
)

writer = Agent(
    name="Writer",
    role="Content writer",
    goal="Write content",
    llm="router-mf-0.5",
    base_url="http://localhost:6060/v1"
)

task1 = Task(
    description="Research AI trends",
    agent=researcher,
    expected_output="Research summary"
)

task2 = Task(
    description="Write article based on research",
    agent=writer,
    expected_output="Article"
)

agents = PraisonAIAgents(agents=[researcher, writer], tasks=[task1, task2])
result = agents.start()
```

## Workflow Example

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent, AgentFlow

workflow = AgentFlow(
    steps=[
        Agent(
            name="Analyzer",
            role="Data analyst",
            llm="router-mf-0.5",
            base_url="http://localhost:6060/v1"
        ),
        Agent(
            name="Reporter",
            role="Report writer",
            llm="router-mf-0.5",
            base_url="http://localhost:6060/v1"
        )
    ]
)

result = workflow.run("Analyze sales data")
```

## Server with Custom Models

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Use Claude as strong model
python -m routellm.openai_server \
  --routers mf \
  --strong-model anthropic/claude-3-5-sonnet-20241022 \
  --weak-model gpt-4o-mini \
  --port 6060

# Use Ollama models
python -m routellm.openai_server \
  --routers mf \
  --strong-model ollama/llama3.1:70b \
  --weak-model ollama/llama3.1:8b \
  --port 6060
```

## Docker Deployment

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
docker run -d \
  -p 6060:6060 \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  routellm/routellm \
  --routers mf \
  --strong-model gpt-4o \
  --weak-model gpt-4o-mini
```

## Config File

Create `config.yaml`:

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
mf:
  checkpoint_path: "routellm/mf_gpt4_augmented"
```

Start with config:

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
python -m routellm.openai_server \
  --routers mf \
  --config config.yaml \
  --strong-model gpt-4o \
  --weak-model gpt-4o-mini \
  --port 6060
```

## API Compatibility

RouteLLM server is OpenAI-compatible. Use with any OpenAI client:

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:6060/v1",
    api_key="not-needed"  # Uses server's API key
)

response = client.chat.completions.create(
    model="router-mf-0.5",
    messages=[{"role": "user", "content": "Hello"}]
)
```

## Cost Savings

RouteLLM can reduce costs by up to 85% while maintaining 95% quality on benchmarks.

| Threshold | Cost Reduction | Quality Retention |
| --------- | -------------- | ----------------- |
| 0.3       | \~70%          | \~90%             |
| 0.5       | \~50%          | \~95%             |
| 0.7       | \~30%          | \~98%             |