Voice Reply (Text-to-Speech)

Speak the agent's reply back as a native voice note on Telegram, Slack, WhatsApp, and Discord

Speak the agent’s plain-text reply back as a native voice note — the outbound counterpart to inbound speech-to-text, off by default and opt-in per channel.

from praisonaiagents import Agent

# 1. Author the agent exactly as before — voice is a transport concern.
agent = Agent(
    name="assistant",
    instructions="Be helpful and concise.",
)

# 2. Enable voice replies on your Telegram channel in gateway.yaml:
#
#    channels:
#      telegram:
#        platform: telegram
#        token: ${TELEGRAM_BOT_TOKEN}
#        stt:   true                    # user speaks in
#        voice: { mode: match_inbound } # bot speaks back when they do
#
# 3. Send a voice note to the bot on Telegram — you get a spoken reply.
#    Send a text message — you get a text reply.

One YAML line closes the voice loop — the bot listens and speaks.

Quick Start

1. Level 1 — Bool shorthand

   voice: true turns replies into voice notes for every message (mode: always).

   channels:
     telegram:
       platform: telegram
       token: ${TELEGRAM_BOT_TOKEN}
       voice: true    # every reply is also spoken

2. Level 2 — Mode shorthand

   A bare mode implies enabled: true. Use match_inbound to speak back only when the user sent a voice memo.

   channels:
     telegram:
       platform: telegram
       token: ${TELEGRAM_BOT_TOKEN}
       voice:
         mode: match_inbound    # speak back only when spoken to

3. Level 3 — Dict

   Tune the voice, model, speed, format, and length cap.

   channels:
     telegram:
       platform: telegram
       token: ${TELEGRAM_BOT_TOKEN}
       voice:
         enabled: true
         mode: match_inbound
         model: openai/tts-1
         voice: nova
         speed: 1.0
         format: ogg
         max_chars: 4000

4. Level 4 — TtsConfigSchema

   Build the channel config in Python with the validated schema.

   from praisonai_bot.bots._config_schema import (
       ChannelConfigSchema,
       TtsConfigSchema,
   )
   
   channel = ChannelConfigSchema(
       platform="telegram",
       token="${TELEGRAM_BOT_TOKEN}",
       voice=TtsConfigSchema(
           enabled=True,
           mode="match_inbound",
           model="openai/tts-1",
           voice="nova",
           speed=1.0,
           format="ogg",
           max_chars=4000,
       ),
   )

How It Works

The gateway synthesises the reply after the text is already sent, so voice is always a bonus — never a blocker.

Graceful degradation

Voice reply is best-effort — the text reply is always delivered first.

• Empty / whitespace text → skipped silently.
• Reply longer than max_chars → skipped, INFO log entry.
• TTS tool import fails (missing openai/litellm deps) → skipped, WARNING log entry.
• Synthesis error → skipped, ERROR log entry.
• Adapter send_voice fails → logged; the text reply already went out.

Configuration Options

Fields from TtsConfig / TtsConfigSchema.

Bool shorthand

channels:
  telegram:
    voice: true    # equivalent to {enabled: true, mode: always}

Resolution order

The effective policy is resolved from, in order:

1. config.metadata["voice"] (operator override),
2. config.metadata["tts"] (alias),
3. a direct config.voice attribute (schema-backed configs),
4. a direct config.tts attribute, then
5. the off-by-default TtsConfig().

Choosing a Mode

Pick the mode that matches how chatty your users want the bot to be.

Sentence-Level Streaming

Streaming TTS synthesises and delivers your reply sentence-by-sentence, so the user hears the first clause while the rest is still being spoken.

With stream: false (default) the whole reply is synthesised into one clip before anything plays — the user waits for full-response latency. With stream: true time-to-first-audio drops to first-sentence latency.

channels:
  telegram:
    platform: telegram
    token: ${TELEGRAM_BOT_TOKEN}
    voice:
      mode: match_inbound
      stream: true      # NEW: synthesise & deliver sentence-by-sentence

from praisonai_bot.bots._config_schema import (
    ChannelConfigSchema,
    TtsConfigSchema,
)

channel = ChannelConfigSchema(
    platform="telegram",
    token="${TELEGRAM_BOT_TOKEN}",
    voice=TtsConfigSchema(
        enabled=True,
        mode="match_inbound",
        stream=True,
    ),
)

When streaming kicks in

Streaming is used only when all three preconditions hold — otherwise the whole-file path runs.

• stream: true is set.
• The full reply is under max_chars (the cap is checked against the whole reply before the per-sentence loop, so long replies can’t sneak through as many under-cap clauses).
• The reply splits into more than one sentence. A single-sentence reply falls through to the whole-file path — streaming buys nothing there.

Failure semantics

Failures are audible-output-aware, so the user never hears a reply restart.

• Nothing played yet + the first clip fails → falls back to whole-file synth on the full text.
• Some audio already played + a clause fails → skip that clause and continue; never replays from the beginning.

When to enable

Flip stream: true when replies are usually multi-sentence and users are waiting for audio.

Per-Platform YAML

Telegram is wired today; other adapters share the same _tts helper and can adopt it without further core changes.

channels:
  telegram:
    platform: telegram
    token: ${TELEGRAM_BOT_TOKEN}
    stt: true                  # user speaks in
    voice:
      mode: match_inbound      # bot speaks back when they do
      voice: nova
      stream: true             # sentence-level delivery (opt-in)

channels:
  slack:
    platform: slack
    token: ${SLACK_BOT_TOKEN}
    app_token: ${SLACK_APP_TOKEN}
    voice:
      mode: always

channels:
  whatsapp:
    platform: whatsapp
    token: ${WHATSAPP_TOKEN}
    voice:
      mode: match_inbound

channels:
  discord:
    platform: discord
    token: ${DISCORD_BOT_TOKEN}
    voice:
      mode: always

Platform Support

The [[audio_as_voice]] manual escape hatch continues to work for pre-rendered audio on Telegram.

User Interaction Flow

A voice-in / voice-out conversation on Telegram with mode: match_inbound.

1. The user taps 🎙️ and records a voice note on Telegram.
2. Inbound STT transcribes it → the agent processes the text → produces a plain-text reply.
3. The gateway sees voice.mode: match_inbound and that the inbound was a voice memo → synthesises the reply with openai/tts-1.
4. The user receives both: the text reply and a native voice note they can play right in the chat.
5. If the user next sends a text message, they get only a text reply — no unnecessary audio.
6. When stream: true is set, the user hears the first sentence as soon as it is synthesised while the rest of the reply is still being spoken.

Best Practices

voice:
  mode: match_inbound

voice:
  mode: match_inbound
  max_chars: 600

voice:
  mode: match_inbound
  stream: true

Related