OpenAI-Compatible Endpoints

Point PraisonAI at any /v1 host with base_url + api_key — no provider prefix needed

Point PraisonAI at any OpenAI-compatible /v1 host by setting base_url and api_key on the Agent — bare catalog model ids are fine, no provider/ prefix required.

Quick Start

1. Simplest form

   Pass base_url and api_key at the top level with a bare model id.

   from praisonaiagents import Agent
   
   agent = Agent(
       name="assistant",
       instructions="You are a helpful assistant",
       llm="deepseek-v4-flash",
       base_url="https://api.pzero.studio/v1",
       api_key="your-key-or-empty-if-public",
   )
   agent.start("Explain what an OpenAI-compatible endpoint is in one paragraph.")

2. Environment-variable variant

   Set the host once with environment variables, then keep the code clean.

   export OPENAI_API_BASE=https://api.pzero.studio/v1
   export OPENAI_API_KEY=your-key

   from praisonaiagents import Agent
   
   agent = Agent(
       name="assistant",
       instructions="You are a helpful assistant",
       llm="deepseek-v4-flash",
   )
   agent.start("Hi")

3. Dict form (equivalent)

   The dict form does the same thing when you prefer to bundle connection settings inside llm.

   from praisonaiagents import Agent
   
   agent = Agent(
       name="assistant",
       instructions="You are a helpful assistant",
       llm={
           "model": "deepseek-v4-flash",
           "api_base": "https://api.pzero.studio/v1",
           "api_key": "",
       },
   )
   agent.start("Hello")

How It Works

PraisonAI routes a bare model id plus a base_url through the OpenAI-compatible client.

Configuration Options

Set these top-level Agent parameters for any OpenAI-compatible host.

Common Patterns

Every host uses the same 5-line pattern — only base_url and the catalog id change.

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="deepseek-v4-flash",
    base_url="https://api.pzero.studio/v1",
    api_key="",
)
agent.start("Hello")

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="meta-llama/Meta-Llama-3-8B-Instruct",
    base_url="https://api.deepinfra.com/v1/openai",
    api_key="your-deepinfra-key",
)
agent.start("Hello")

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="accounts/fireworks/models/llama-v3-8b-instruct",
    base_url="https://api.fireworks.ai/inference/v1",
    api_key="your-fireworks-key",
)
agent.start("Hello")

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="meta-llama/Llama-3-8b-chat-hf",
    base_url="https://api.together.xyz/v1",
    api_key="your-together-key",
)
agent.start("Hello")

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="meta-llama/Meta-Llama-3-8B-Instruct",
    base_url="http://localhost:8000/v1",
    api_key="not-needed",
)
agent.start("Hello")

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="local-model",
    base_url="http://localhost:1234/v1",
    api_key="not-needed",
)
agent.start("Hello")

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="your-catalog-id",
    base_url="https://gateway.company.internal/v1",
    api_key="your-key",
)
agent.start("Hello")

Local prefix route (tool-call repair)

For a locally-served OpenAI-compatible server — LM Studio, vLLM, llama.cpp’s llama-server — use a dedicated prefix instead of base_url + bare model. PraisonAI recognises these prefixes and selects a LocalOpenAIAdapter that keeps the standard OpenAI message shape and adds a tool-call repair budget of 2 — a safety net for the malformed JSON tool calls small local models occasionally emit.

from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

agent = Agent(llm="lm_studio/qwen2.5-7b-instruct", tools=[get_weather])
agent.start("What is the weather in Paris?")

Every local prefix works the same way — swap the prefix, keep your tools:

from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

# LM Studio
agent = Agent(llm="lm_studio/qwen2.5-7b-instruct", tools=[get_weather])

# vLLM (self-hosted)
agent = Agent(llm="vllm/meta-llama/Llama-3.1-8B-Instruct", tools=[get_weather])

# OpenAI-compatible vLLM (LiteLLM's hosted_vllm/ spelling)
agent = Agent(llm="hosted_vllm/Qwen2.5-7B", tools=[get_weather])

# llama.cpp
agent = Agent(llm="llamacpp/qwen2.5", tools=[get_weather])

Local-server prefixes (lm_studio/, vllm/, hosted_vllm/, llamacpp/, llama_cpp/) also inherit text-based tool-call recovery: when the model emits a tool call as JSON text — common after a repair prompt on a small local model — the adapter parses it and dispatches the tool call anyway. A hosted OpenAI-compatible endpoint (a base_url pointing at a hosted API) deliberately does not get this behaviour: a hosted model returning JSON prose must never have it silently interpreted as a tool call. Recovery fires on both sync and async response paths as of PraisonAI PR #4940. Before that release, only agent.start() / agent.chat() recovered a text-emitted tool call — agent.astart() / agent.achat() would silently ignore it. The streaming path does not yet have this recovery on any adapter. See Tool Call Self-Repair → Text-based tool-call recovery.

Explicit max_tool_repairs on the Agent always wins:

from praisonaiagents import Agent

agent = Agent(llm="lm_studio/qwen", max_tool_repairs=5)  # ends up as 5

Streaming with tools keeps working on LM Studio, vLLM, and llama.cpp — the LocalOpenAIAdapter deliberately does not inherit Ollama’s stream-disabling behaviour, because these servers stream tool calls correctly.

Choosing Which Style to Use

Pick the routing style that matches what you connect to.

Which Adapter Am I Getting?

A base_url says where the server is, not what it serves — so a local URL only implies Ollama for a model a local runtime could plausibly host.

Best Practices

Related