LLM Gateways

Route agent LLM calls through unified gateways like OpenRouter and LiteLLM Proxy for access to 100+ providers

Route agent LLM calls through OpenRouter, LiteLLM Proxy, or a custom OpenAI-compatible gateway — no manual registration required.

import os
from praisonaiagents import Agent

if not os.getenv("OPENROUTER_API_KEY"):
    raise EnvironmentError("Set OPENROUTER_API_KEY in your environment")

agent = Agent(
    name="assistant",
    llm="openrouter/anthropic/claude-3.5-sonnet",
)
agent.start("Hello!")

The user picks a gateway-backed model string; the agent routes the request through the gateway to the underlying provider.

Quick Start

1. OpenRouter — one line

   import os
   from praisonaiagents import Agent
   
   if not os.getenv("OPENROUTER_API_KEY"):
       raise EnvironmentError("Set OPENROUTER_API_KEY in your environment")
   
   agent = Agent(
       name="assistant",
       llm="openrouter/anthropic/claude-3.5-sonnet",
   )
   agent.start("Hello!")

   Gateway providers auto-register on import — no register_gateway_providers() call needed.

2. LiteLLM Proxy

   import os
   from praisonaiagents import Agent
   
   os.environ["LITELLM_PROXY_BASE_URL"] = "http://localhost:4000"
   if not os.getenv("LITELLM_PROXY_API_KEY"):
       raise EnvironmentError("Set LITELLM_PROXY_API_KEY in your environment")
   
   agent = Agent(
       name="assistant",
       llm="litellm-proxy/gpt-4",
   )
   agent.start("Hello!")

3. Custom gateway (advanced)

   import os
   from praisonai.llm import create_llm_provider
   from praisonaiagents import Agent
   
   provider = create_llm_provider({
       "name": "custom-gateway",
       "model_id": "my-model",
       "config": {
           "base_url": "https://api.example.com/v1",
           "api_key": os.getenv("CUSTOM_GATEWAY_API_KEY", ""),
       },
   })
   agent = Agent(name="assistant", llm=provider)
   agent.start("Hello!")

Choose Your Gateway

Route Gateway Models by Cost

Use custom ModelProfile entries when one OpenAI-compatible gateway exposes multiple models. The router selects a model name from your metadata; LLM then sends the request to the shared endpoint.

1. Set gateway credentials

   Keep the endpoint and key outside your source code:

   export OPENAI_BASE_URL="${OPENAI_BASE_URL:?Set OPENAI_BASE_URL in your shell}"
   export OPENAI_API_KEY="${OPENAI_API_KEY:?Set OPENAI_API_KEY in your shell}"
   export GATEWAY_FAST_MODEL="${GATEWAY_FAST_MODEL:?Set GATEWAY_FAST_MODEL in your shell}"
   export GATEWAY_REASONING_MODEL="${GATEWAY_REASONING_MODEL:?Set GATEWAY_REASONING_MODEL in your shell}"

2. Describe the available models

   Replace the placeholder capabilities and costs with values published by your gateway. PraisonAI does not maintain gateway-specific pricing.

   import os
   
   from praisonaiagents import Agent
   from praisonaiagents.llm import (
       LLM,
       ModelProfile,
       ModelRouter,
       TaskComplexity,
   )
   
   gateway_url = os.environ["OPENAI_BASE_URL"]
   gateway_key = os.environ["OPENAI_API_KEY"]
   fast_model = os.environ["GATEWAY_FAST_MODEL"]
   reasoning_model = os.environ["GATEWAY_REASONING_MODEL"]
   
   models = [
       ModelProfile(
           name=f"openai/{fast_model}",
           provider="gateway",
           complexity_range=(TaskComplexity.SIMPLE, TaskComplexity.MODERATE),
           cost_per_1k_tokens=0.001,
           strengths=["speed", "summarization"],
           capabilities=["text"],
           context_window=32_000,
       ),
       ModelProfile(
           name=f"openai/{reasoning_model}",
           provider="gateway",
           complexity_range=(TaskComplexity.MODERATE, TaskComplexity.VERY_COMPLEX),
           cost_per_1k_tokens=0.01,
           strengths=["reasoning", "analysis"],
           capabilities=["text", "function-calling"],
           context_window=128_000,
       ),
   ]
   
   task = "Summarize this support conversation"
   router = ModelRouter(models=models)
   selected_model = router.select_model(task, budget_conscious=True)
   
   llm = LLM(
       model=selected_model,
       base_url=gateway_url,
       api_key=gateway_key,
   )
   agent = Agent(name="assistant", llm=llm)
   agent.start(task)

How It Works

Provider Reference

OpenRouter

Aliases: openrouter, or. Model IDs are prefixed with openrouter/ automatically unless already prefixed.

LiteLLM Proxy

Aliases: litellm-proxy, llm-proxy, litellm-gateway. Model IDs are passed through unchanged.

Custom Gateway

Aliases: custom-gateway, gateway, custom.

Explicit config["api_key"] overrides environment variables at init time.

Common Patterns

Multi-agent, different gateways:

from praisonaiagents import Agent

researcher = Agent(name="researcher", llm="openrouter/anthropic/claude-3.5-sonnet")
writer = Agent(name="writer", llm="litellm-proxy/gpt-4o-mini")

OpenRouter attribution headers:

from praisonai.llm import OpenRouterProvider
from praisonaiagents import Agent

provider = OpenRouterProvider("gpt-4", config={
    "extra_headers": {"X-Title": "My App", "HTTP-Referer": "https://myapp.com"},
})
agent = Agent(name="assistant", llm=provider)

Shorthand alias:

agent = Agent(name="assistant", llm="or/gpt-4")  # same as openrouter/gpt-4

Developer Flow

Best Practices

Related