> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Realtime Agent

> Real-time voice conversations with WebSocket-based audio streaming

Build voice assistants with the `RealtimeAgent` — bidirectional audio streaming over WebSockets for live, interactive conversations.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import RealtimeAgent

agent = RealtimeAgent(name="VoiceAssistant")

await agent.aconnect()
await agent.asend_text("Hello, how can I help you?")
```

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Realtime Agent"
        User[📋 Voice/Text] --> Agent[🤖 RealtimeAgent]
        Agent --> WS[🧠 WebSocket Stream]
        WS --> Result[✅ Audio Response]
    end

    classDef input fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef process fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef output fill:#10B981,stroke:#7C90A0,color:#fff

    class User,Agent input
    class WS process
    class Result output
```

The `RealtimeAgent` enables real-time voice conversations using WebSocket connections for bidirectional audio streaming. It's designed for interactive voice applications, live transcription, and real-time AI assistants.

## Quick Start

<Steps>
  <Step title="Simple Usage">
    Connect and start a conversation.

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import RealtimeAgent

    agent = RealtimeAgent(
        name="Assistant",
        instructions="You are a helpful voice assistant",
    )

    agent.connect()
    agent.send_text("Hello!")

    def on_message(message):
        print(f"Received: {message}")

    agent.on_message(on_message)
    ```
  </Step>

  <Step title="With Configuration">
    Choose a voice and tune turn detection.

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import RealtimeAgent, RealtimeConfig

    config = RealtimeConfig(
        voice="nova",
        turn_detection="server_vad",
    )

    agent = RealtimeAgent(
        name="VoiceBot",
        llm="gpt-4o-realtime-preview",
        realtime=config,
        instructions="Keep responses brief.",
    )

    agent.connect()
    agent.send_text("Tell me a fun fact.")
    ```
  </Step>
</Steps>

## How It Works

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant User
    participant RealtimeAgent
    participant WebSocket

    User->>RealtimeAgent: connect() + send_text/audio
    RealtimeAgent->>WebSocket: Open bidirectional stream
    WebSocket-->>RealtimeAgent: Streamed audio + text
    RealtimeAgent-->>User: on_message / on_audio callbacks
```

## Installation

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
pip install praisonaiagents[realtime]
```

<Note>
  RealtimeAgent requires the `websockets` package for WebSocket connections.
</Note>

## Basic Usage

### Simple Voice Assistant

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import RealtimeAgent

# Create agent with default settings
agent = RealtimeAgent(
    name="Assistant",
    instructions="You are a helpful voice assistant"
)

# Connect to realtime API
agent.connect()

# Send text message
agent.send_text("Hello!")

# Handle incoming messages
def on_message(message):
    print(f"Received: {message}")

agent.on_message(on_message)
```

### With Custom Configuration

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import RealtimeAgent, RealtimeConfig

config = RealtimeConfig(
    voice="alloy",
    modalities=["text", "audio"],
    turn_detection="server_vad",
    temperature=0.8
)

agent = RealtimeAgent(
    name="CustomVoice",
    llm="gpt-4o-realtime-preview",
    realtime=config,
    verbose=True
)
```

## Configuration

### RealtimeConfig Options

| Parameter                    | Type  | Default             | Description                                           |
| ---------------------------- | ----- | ------------------- | ----------------------------------------------------- |
| `voice`                      | str   | `"alloy"`           | Voice model (alloy, echo, fable, onyx, nova, shimmer) |
| `modalities`                 | list  | `["text", "audio"]` | Modalities to use (text, audio)                       |
| `turn_detection`             | str   | `"server_vad"`      | Turn detection mode (`server_vad`, `none`)            |
| `input_audio_format`         | str   | `"pcm16"`           | Input audio format (pcm16, g711\_ulaw, g711\_alaw)    |
| `output_audio_format`        | str   | `"pcm16"`           | Output audio format (pcm16, g711\_ulaw, g711\_alaw)   |
| `temperature`                | float | `0.8`               | Sampling temperature (0.0 to 2.0)                     |
| `max_response_output_tokens` | int   | `None`              | Maximum tokens for response                           |
| `instructions`               | str   | `None`              | System instructions for the session                   |

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import RealtimeConfig

config = RealtimeConfig(
    voice="nova",
    turn_detection="server_vad",
    temperature=0.6
)
```

## Methods

### Connection Methods

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Synchronous
agent.connect()
agent.disconnect()

# Asynchronous
await agent.aconnect()
await agent.adisconnect()
```

### Sending Data

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Send text message
agent.send_text("Hello, how are you?")

# Send audio data (bytes)
agent.send_audio(audio_bytes)
```

### Receiving Data

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Register message handler
def handle_message(message):
    print(f"Text: {message}")

agent.on_message(handle_message)

# Register audio handler
def handle_audio(audio_data):
    # Process audio bytes
    pass

agent.on_audio(handle_audio)
```

## Async Usage

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import asyncio
from praisonaiagents import RealtimeAgent

async def main():
    agent = RealtimeAgent(name="AsyncVoice")
    
    try:
        await agent.aconnect()
        
        # Send message
        await agent.asend_text("Hello!")
        
        # Listen for responses
        async for message in agent.listen():
            print(f"Response: {message}")
            
    finally:
        await agent.adisconnect()

asyncio.run(main())
```

## Voice Options

| Voice     | Description              |
| --------- | ------------------------ |
| `alloy`   | Neutral, balanced voice  |
| `echo`    | Warm, conversational     |
| `fable`   | Expressive, storytelling |
| `onyx`    | Deep, authoritative      |
| `nova`    | Friendly, upbeat         |
| `shimmer` | Clear, professional      |

## Example: Interactive Voice Bot

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import asyncio
from praisonaiagents import RealtimeAgent, RealtimeConfig

async def voice_bot():
    config = RealtimeConfig(
        voice="nova",
        turn_detection="server_vad"
    )
    
    agent = RealtimeAgent(
        name="VoiceBot",
        llm="gpt-4o-realtime-preview",
        realtime=config,
        instructions="You are a friendly assistant. Keep responses brief."
    )
    
    await agent.aconnect()
    print("Connected! Start speaking...")
    
    # Handle incoming audio
    def on_audio(audio):
        # Play audio through speakers
        play_audio(audio)
    
    agent.on_audio(on_audio)
    
    # Keep running
    try:
        while True:
            await asyncio.sleep(1)
    except KeyboardInterrupt:
        await agent.adisconnect()

asyncio.run(voice_bot())
```

## Error Handling

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import RealtimeAgent

agent = RealtimeAgent(name="SafeVoice")

try:
    agent.connect()
    agent.send_text("Hello")
except ConnectionError as e:
    print(f"Connection failed: {e}")
except TimeoutError as e:
    print(f"Request timed out: {e}")
finally:
    agent.disconnect()
```

## Best Practices

<AccordionGroup>
  <Accordion title="Enable turn detection for natural flow">
    Keep `turn_detection="server_vad"` so the agent knows when the user has stopped speaking. Without server-side VAD, conversations feel stilted and responses arrive at the wrong moments.
  </Accordion>

  <Accordion title="Handle disconnects gracefully">
    WebSocket connections drop. Wrap `connect()` in reconnection logic and catch `ConnectionError` so a dropped link doesn't end the session.
  </Accordion>

  <Accordion title="Buffer audio for smooth playback">
    Collect audio chunks from `on_audio` before playing them. Playing raw chunks as they arrive produces choppy, garbled speech.
  </Accordion>

  <Accordion title="Keep system prompts short">
    Long instructions slow the first response. A brief prompt keeps latency low, which matters far more in voice than in text.
  </Accordion>
</AccordionGroup>

## Related

<CardGroup cols={2}>
  <Card icon="code" href="/docs/agents/code">
    Generate and execute code with the CodeAgent.
  </Card>

  <Card icon="eye" href="/docs/agents/vision">
    Analyze images and visual content.
  </Card>
</CardGroup>