Realtime Agent
Real-time voice conversations with WebSocket-based audio streaming
Build voice assistants with the RealtimeAgent — bidirectional audio streaming over WebSockets for live, interactive conversations.
from praisonaiagents import RealtimeAgent
agent = RealtimeAgent(name="VoiceAssistant")
await agent.aconnect()
await agent.asend_text("Hello, how can I help you?")The RealtimeAgent enables real-time voice conversations using WebSocket connections for bidirectional audio streaming. It’s designed for interactive voice applications, live transcription, and real-time AI assistants.
Quick Start
- Simple Usage
Connect and start a conversation.
python from praisonaiagents import RealtimeAgent agent = RealtimeAgent( name="Assistant", instructions="You are a helpful voice assistant", ) agent.connect() agent.send_text("Hello!") def on_message(message): print(f"Received: {message}") agent.on_message(on_message) - With Configuration
Choose a voice and tune turn detection.
python from praisonaiagents import RealtimeAgent, RealtimeConfig config = RealtimeConfig( voice="nova", turn_detection="server_vad", ) agent = RealtimeAgent( name="VoiceBot", model="gpt-realtime", # optional; this is the default realtime=config, instructions="Keep responses brief.", ) agent.connect() agent.send_text("Tell me a fun fact.")
How It Works
Installation
pip install praisonaiagents[realtime]Basic Usage
Simple Voice Assistant
from praisonaiagents import RealtimeAgent
# Create agent with default settings
agent = RealtimeAgent(
name="Assistant",
instructions="You are a helpful voice assistant"
)
# Connect to realtime API
agent.connect()
# Send text message
agent.send_text("Hello!")
# Handle incoming messages
def on_message(message):
print(f"Received: {message}")
agent.on_message(on_message)With Custom Configuration
from praisonaiagents import RealtimeAgent, RealtimeConfig
config = RealtimeConfig(
voice="alloy",
modalities=["text", "audio"],
turn_detection="server_vad"
)
agent = RealtimeAgent(
name="CustomVoice",
model="gpt-realtime",
realtime=config,
verbose=True
)Configuration
RealtimeConfig Options
| Parameter | Type | Default | Description |
|---|---|---|---|
voice | str | "alloy" | Voice (alloy, echo, fable, onyx, nova, shimmer) |
modalities | list | ["text", "audio"] | Output modality — GA accepts a single output modality; audio includes a transcript |
turn_detection | str | "server_vad" | Turn detection mode (server_vad, none) |
input_audio_format | str | "pcm16" | Input audio format (pcm16, g711_ulaw, g711_alaw) |
output_audio_format | str | "pcm16" | Output audio format (pcm16, g711_ulaw, g711_alaw) |
~~temperature~~ | ~~float~~ | ~~0.8~~ | Deprecated — ignored on the GA API. Setting it emits a DeprecationWarning. The field is kept for back-compat; the value is never sent. |
max_response_output_tokens | int | None | Mapped to max_output_tokens in the GA session |
instructions | str | None | System instructions for the session |
from praisonaiagents import RealtimeConfig
config = RealtimeConfig(
voice="nova",
turn_detection="server_vad"
)Methods
Connection Methods
# Synchronous
agent.connect()
agent.disconnect()
# Asynchronous
await agent.aconnect()
await agent.adisconnect()Sending Data
# Send text message
agent.send_text("Hello, how are you?")
# Send audio data (bytes)
agent.send_audio(audio_bytes)Receiving Data
# Register message handler
def handle_message(message):
print(f"Text: {message}")
agent.on_message(handle_message)
# Register audio handler
def handle_audio(audio_data):
# Process audio bytes
pass
agent.on_audio(handle_audio)Async Usage
import asyncio
from praisonaiagents import RealtimeAgent
async def main():
agent = RealtimeAgent(name="AsyncVoice")
try:
await agent.aconnect()
# Send message
await agent.asend_text("Hello!")
# Listen for responses
async for message in agent.listen():
print(f"Response: {message}")
finally:
await agent.adisconnect()
asyncio.run(main())Voice Options
| Voice | Description |
|---|---|
alloy | Neutral, balanced voice |
echo | Warm, conversational |
fable | Expressive, storytelling |
onyx | Deep, authoritative |
nova | Friendly, upbeat |
shimmer | Clear, professional |
Example: Interactive Voice Bot
import asyncio
from praisonaiagents import RealtimeAgent, RealtimeConfig
async def voice_bot():
config = RealtimeConfig(
voice="nova",
turn_detection="server_vad"
)
agent = RealtimeAgent(
name="VoiceBot",
model="gpt-realtime",
realtime=config,
instructions="You are a friendly assistant. Keep responses brief."
)
await agent.aconnect()
print("Connected! Start speaking...")
# Handle incoming audio
def on_audio(audio):
# Play audio through speakers
play_audio(audio)
agent.on_audio(on_audio)
# Keep running
try:
while True:
await asyncio.sleep(1)
except KeyboardInterrupt:
await agent.adisconnect()
asyncio.run(voice_bot())Error Handling
from praisonaiagents import RealtimeAgent
agent = RealtimeAgent(name="SafeVoice")
try:
agent.connect()
agent.send_text("Hello")
except ConnectionError as e:
print(f"Connection failed: {e}")
except TimeoutError as e:
print(f"Request timed out: {e}")
finally:
agent.disconnect()Best Practices
Enable turn detection for natural flow
Keep turn_detection="server_vad" so the agent knows when the user has stopped speaking. Without server-side VAD, conversations feel stilted and responses arrive at the wrong moments.
Handle disconnects gracefully
WebSocket connections drop. Wrap connect() in reconnection logic and catch ConnectionError so a dropped link doesn’t end the session.
Buffer audio for smooth playback
Collect audio chunks from on_audio before playing them. Playing raw chunks as they arrive produces choppy, garbled speech.
Keep system prompts short
Long instructions slow the first response. A brief prompt keeps latency low, which matters far more in voice than in text.
Migration from beta Realtime
Existing code keeps working; check these three cases.
I pinned llm="gpt-4o-realtime-preview" or model="gpt-4o-realtime-preview"
Switch to model="gpt-realtime", or drop the argument to use the default. The preview model is still accepted when you pass it explicitly.
I set RealtimeConfig(temperature=…)
Remove it. The GA API has no session temperature, so the value is ignored and emits a DeprecationWarning.
I use websockets < 14
Still supported. The agent picks the right header argument (extra_headers on 12/13, additional_headers on 14+). The declared range is websockets>=12.0.