> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Realtime Agent

> Real-time voice conversations with WebSocket-based audio streaming

Build voice assistants with the `RealtimeAgent` — bidirectional audio streaming over WebSockets for live, interactive conversations.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import RealtimeAgent

agent = RealtimeAgent(name="VoiceAssistant")

await agent.aconnect()
await agent.asend_text("Hello, how can I help you?")
```

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Realtime Agent"
        User[📋 Voice/Text] --> Agent[🤖 RealtimeAgent]
        Agent --> WS[🧠 WebSocket Stream]
        WS --> Result[✅ Audio Response]
    end

    classDef input fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef process fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef output fill:#10B981,stroke:#7C90A0,color:#fff

    class User,Agent input
    class WS process
    class Result output
```

<Note>
  `RealtimeAgent` now speaks OpenAI's **GA Realtime** protocol. The default model is `gpt-realtime` (was `gpt-4o-realtime-preview`), and `RealtimeConfig.temperature` is deprecated — the GA API has no session temperature, so any value you pass is ignored. Your existing code keeps working; only the on-the-wire protocol moved. See [Migration](#migration-from-beta-realtime) below.
</Note>

The `RealtimeAgent` enables real-time voice conversations using WebSocket connections for bidirectional audio streaming. It's designed for interactive voice applications, live transcription, and real-time AI assistants.

## Quick Start

<Steps>
  <Step title="Simple Usage">
    Connect and start a conversation.

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import RealtimeAgent

    agent = RealtimeAgent(
        name="Assistant",
        instructions="You are a helpful voice assistant",
    )

    agent.connect()
    agent.send_text("Hello!")

    def on_message(message):
        print(f"Received: {message}")

    agent.on_message(on_message)
    ```
  </Step>

  <Step title="With Configuration">
    Choose a voice and tune turn detection.

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import RealtimeAgent, RealtimeConfig

    config = RealtimeConfig(
        voice="nova",
        turn_detection="server_vad",
    )

    agent = RealtimeAgent(
        name="VoiceBot",
        model="gpt-realtime",  # optional; this is the default
        realtime=config,
        instructions="Keep responses brief.",
    )

    agent.connect()
    agent.send_text("Tell me a fun fact.")
    ```
  </Step>
</Steps>

## How It Works

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant User
    participant RealtimeAgent
    participant WebSocket

    User->>RealtimeAgent: connect() + send_text/audio
    RealtimeAgent->>WebSocket: Open bidirectional stream
    WebSocket-->>RealtimeAgent: Streamed audio + text
    RealtimeAgent-->>User: on_message / on_audio callbacks
```

## Installation

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
pip install praisonaiagents[realtime]
```

<Note>
  RealtimeAgent requires the `websockets` package for WebSocket connections.
</Note>

## Basic Usage

### Simple Voice Assistant

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import RealtimeAgent

# Create agent with default settings
agent = RealtimeAgent(
    name="Assistant",
    instructions="You are a helpful voice assistant"
)

# Connect to realtime API
agent.connect()

# Send text message
agent.send_text("Hello!")

# Handle incoming messages
def on_message(message):
    print(f"Received: {message}")

agent.on_message(on_message)
```

### With Custom Configuration

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import RealtimeAgent, RealtimeConfig

config = RealtimeConfig(
    voice="alloy",
    modalities=["text", "audio"],
    turn_detection="server_vad"
)

agent = RealtimeAgent(
    name="CustomVoice",
    model="gpt-realtime",
    realtime=config,
    verbose=True
)
```

## Configuration

### RealtimeConfig Options

| Parameter | Type | Default | Description |
| - | - | - | - |
| `voice` | str | `"alloy"` | Voice (alloy, echo, fable, onyx, nova, shimmer) |
| `modalities` | list | `["text", "audio"]` | Output modality — GA accepts a single output modality; audio includes a transcript |
| `turn_detection` | str | `"server_vad"` | Turn detection mode (`server_vad`, `none`) |
| `input_audio_format` | str | `"pcm16"` | Input audio format (pcm16, g711\_ulaw, g711\_alaw) |
| `output_audio_format` | str | `"pcm16"` | Output audio format (pcm16, g711\_ulaw, g711\_alaw) |
| ~~`temperature`~~ | ~~float~~ | ~~`0.8`~~ | **Deprecated — ignored on the GA API.** Setting it emits a `DeprecationWarning`. The field is kept for back-compat; the value is never sent. |
| `max_response_output_tokens` | int | `None` | Mapped to `max_output_tokens` in the GA session |
| `instructions` | str | `None` | System instructions for the session |

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import RealtimeConfig

config = RealtimeConfig(
    voice="nova",
    turn_detection="server_vad"
)
```

## Methods

### Connection Methods

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Synchronous
agent.connect()
agent.disconnect()

# Asynchronous
await agent.aconnect()
await agent.adisconnect()
```

### Sending Data

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Send text message
agent.send_text("Hello, how are you?")

# Send audio data (bytes)
agent.send_audio(audio_bytes)
```

### Receiving Data

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Register message handler
def handle_message(message):
    print(f"Text: {message}")

agent.on_message(handle_message)

# Register audio handler
def handle_audio(audio_data):
    # Process audio bytes
    pass

agent.on_audio(handle_audio)
```

## Async Usage

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import asyncio
from praisonaiagents import RealtimeAgent

async def main():
    agent = RealtimeAgent(name="AsyncVoice")
    
    try:
        await agent.aconnect()
        
        # Send message
        await agent.asend_text("Hello!")
        
        # Listen for responses
        async for message in agent.listen():
            print(f"Response: {message}")
            
    finally:
        await agent.adisconnect()

asyncio.run(main())
```

## Voice Options

| Voice | Description |
| - | - |
| `alloy` | Neutral, balanced voice |
| `echo` | Warm, conversational |
| `fable` | Expressive, storytelling |
| `onyx` | Deep, authoritative |
| `nova` | Friendly, upbeat |
| `shimmer` | Clear, professional |

## Example: Interactive Voice Bot

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import asyncio
from praisonaiagents import RealtimeAgent, RealtimeConfig

async def voice_bot():
    config = RealtimeConfig(
        voice="nova",
        turn_detection="server_vad"
    )
    
    agent = RealtimeAgent(
        name="VoiceBot",
        model="gpt-realtime",
        realtime=config,
        instructions="You are a friendly assistant. Keep responses brief."
    )
    
    await agent.aconnect()
    print("Connected! Start speaking...")
    
    # Handle incoming audio
    def on_audio(audio):
        # Play audio through speakers
        play_audio(audio)
    
    agent.on_audio(on_audio)
    
    # Keep running
    try:
        while True:
            await asyncio.sleep(1)
    except KeyboardInterrupt:
        await agent.adisconnect()

asyncio.run(voice_bot())
```

## Error Handling

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import RealtimeAgent

agent = RealtimeAgent(name="SafeVoice")

try:
    agent.connect()
    agent.send_text("Hello")
except ConnectionError as e:
    print(f"Connection failed: {e}")
except TimeoutError as e:
    print(f"Request timed out: {e}")
finally:
    agent.disconnect()
```

## Best Practices

<AccordionGroup>
  <Accordion title="Enable turn detection for natural flow">
    Keep `turn_detection="server_vad"` so the agent knows when the user has stopped speaking. Without server-side VAD, conversations feel stilted and responses arrive at the wrong moments.
  </Accordion>

  <Accordion title="Handle disconnects gracefully">
    WebSocket connections drop. Wrap `connect()` in reconnection logic and catch `ConnectionError` so a dropped link doesn't end the session.
  </Accordion>

  <Accordion title="Buffer audio for smooth playback">
    Collect audio chunks from `on_audio` before playing them. Playing raw chunks as they arrive produces choppy, garbled speech.
  </Accordion>

  <Accordion title="Keep system prompts short">
    Long instructions slow the first response. A brief prompt keeps latency low, which matters far more in voice than in text.
  </Accordion>
</AccordionGroup>

## Migration from beta Realtime

Existing code keeps working; check these three cases.

<AccordionGroup>
  <Accordion title="I pinned llm=&#x22;gpt-4o-realtime-preview&#x22; or model=&#x22;gpt-4o-realtime-preview&#x22;">
    Switch to `model="gpt-realtime"`, or drop the argument to use the default. The preview model is still accepted when you pass it explicitly.
  </Accordion>

  <Accordion title="I set RealtimeConfig(temperature=...)">
    Remove it. The GA API has no session temperature, so the value is ignored and emits a `DeprecationWarning`.
  </Accordion>

  <Accordion title="I use websockets < 14">
    Still supported. The agent picks the right header argument (`extra_headers` on 12/13, `additional_headers` on 14+). The declared range is `websockets>=12.0`.
  </Accordion>
</AccordionGroup>

***

## Related

<CardGroup cols={2}>
  <Card icon="code" href="/docs/agents/code">
    Generate and execute code with the CodeAgent.
  </Card>

  <Card icon="eye" href="/docs/agents/vision">
    Analyze images and visual content.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.