OpenAI-Compatible Server

Drop-in OpenAI API replacement powered by PraisonAI agents

from praisonaiagents import Agent

agent = Agent(
    name="openai-server-agent",
    instructions="Serve requests via the OpenAI-compatible API.",
    llm="openai/gpt-4o-mini",
)
agent.start("Process this chat completion request.")

praisonai serve openai starts a FastAPI server that exposes OpenAI-compatible HTTP endpoints — point any existing OpenAI SDK code at it and it works as a drop-in replacement.

The user points an OpenAI SDK client at the server; chat completions route to your PraisonAI agent.

Quick Start

1. Simple Usage

   praisonai serve openai

   The server starts on http://localhost:8765 by default.

   from openai import OpenAI
   
   client = OpenAI(base_url="http://localhost:8765/v1", api_key="anything")
   
   response = client.chat.completions.create(
       model="gpt-4o-mini",
       messages=[{"role": "user", "content": "Hello!"}]
   )
   print(response.choices[0].message.content)

2. With Configuration

   # Terminal 1 — agents API
   praisonai serve agents --file agents.yaml --port 8000
   
   # Terminal 2 — OpenAI compat layer with auth
   praisonai serve openai --agents-url http://localhost:8000 --api-key $PRAISONAI_API_KEY

   import os
   from openai import OpenAI
   
   client = OpenAI(
       base_url="http://localhost:8765/v1",
       api_key=os.getenv("PRAISONAI_API_KEY"),
   )

How It Works

Endpoints

All endpoints follow the OpenAI REST specification.

Streaming

Set stream=True to receive tokens via Server-Sent Events:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8765/v1", api_key="anything")

stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Write a haiku about agents"}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

The stream ends with a data: [DONE] sentinel. Errors during streaming are also sent as SSE events:

data: {"error": {"message": "...", "type": "stream_error"}}

data: [DONE]

Tool Invocation

/v1/tools/invoke is a PraisonAI extension that calls an agent tool by name. It requires --agents-url to point to a running agents server.

curl -X POST http://localhost:8765/v1/tools/invoke \
  -H "Content-Type: application/json" \
  -d '{"tool": "search", "arguments": {"query": "PraisonAI features"}}'

Start the agents server first:

# Terminal 1 — agents API
praisonai serve agents --file agents.yaml --port 8000

# Terminal 2 — OpenAI compat layer
praisonai serve openai --agents-url http://localhost:8000

Authentication

Pass --api-key to require authentication:

praisonai serve openai --api-key my-secret-key

Clients must include the key as the Authorization header:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8765/v1",
    api_key="my-secret-key"
)

CLI Reference

praisonai serve openai [OPTIONS]

Choosing Between serve openai and serve rag --openai-compat

Error Shape

All errors follow the OpenAI error format:

{
  "error": {
    "message": "Model not found",
    "type": "api_error"
  }
}

Best Practices

praisonai serve openai --api-key $PRAISONAI_API_KEY --host 0.0.0.0

praisonai serve openai --host 0.0.0.0 --port 8765 --api-key $SECRET

Concurrency & streaming (PraisonAI #3771)

• Non-blocking handlers. All four handlers — /v1/chat/completions, /v1/completions, /v1/models, /v1/tools/invoke — offload the underlying sync provider.invoke via asyncio.to_thread. A slow provider call on one request does not stall concurrent requests on the same server.
• Simulated streaming is word-by-word. When the underlying provider doesn’t stream natively, the server splits the response by whitespace and emits one SSE data: frame per word (whitespace preserved). Previously it emitted one frame per character, which most OpenAI-SDK clients handle correctly but at multi-order-of-magnitude higher per-token overhead.
• No configuration flag — the change is unconditional and backward-compatible for real OpenAI-SDK clients.

Related