Skip to main content
Build voice assistants with the RealtimeAgent — bidirectional audio streaming over WebSockets for live, interactive conversations.
The RealtimeAgent enables real-time voice conversations using WebSocket connections for bidirectional audio streaming. It’s designed for interactive voice applications, live transcription, and real-time AI assistants.

Quick Start

1

Simple Usage

Connect and start a conversation.
2

With Configuration

Choose a voice and tune turn detection.

How It Works

Installation

RealtimeAgent requires the websockets package for WebSocket connections.

Basic Usage

Simple Voice Assistant

With Custom Configuration

Configuration

RealtimeConfig Options

Methods

Connection Methods

Sending Data

Receiving Data

Async Usage

Voice Options

Example: Interactive Voice Bot

Error Handling

Best Practices

Keep turn_detection="server_vad" so the agent knows when the user has stopped speaking. Without server-side VAD, conversations feel stilted and responses arrive at the wrong moments.
WebSocket connections drop. Wrap connect() in reconnection logic and catch ConnectionError so a dropped link doesn’t end the session.
Collect audio chunks from on_audio before playing them. Playing raw chunks as they arrive produces choppy, garbled speech.
Long instructions slow the first response. A brief prompt keeps latency low, which matters far more in voice than in text.
Generate and execute code with the CodeAgent.
Analyze images and visual content.