Voice Notes (Speech-to-Text)

Automatically transcribe inbound voice messages on Telegram, Slack, WhatsApp, Discord, and Signal

Voice notes sent to your gateway bot are transcribed and delivered to the agent as text — no code changes required, on by default across Telegram, Slack, WhatsApp, Discord, and Signal.

Quick Start

1. Works out of the box

   Voice notes are transcribed automatically — no config needed.

   # gateway.yaml
   channels:
     telegram:
       token: ${TELEGRAM_BOT_TOKEN}
   agent:
     name: "assistant"
     instructions: "Reply helpfully to any question, including voice notes."

   from praisonaiagents import Agent
   
   agent = Agent(
       name="assistant",
       instructions="Reply helpfully to any question, including voice notes.",
   )
   # Start the gateway — voice notes are transcribed automatically.

2. Echo the transcript back

   Mirror the recognised text back so users can verify what the bot heard.

   channels:
     telegram:
       token: ${TELEGRAM_BOT_TOKEN}
       stt:
         enabled: true          # default; set false to opt out
         echo_transcripts: true # echo the recognised text back to the user

3. Force a language

   Skip auto-detection for faster, more accurate short clips.

   channels:
     telegram:
       stt:
         language: "en"

4. Custom STT model

   Swap the default model for any LiteLLM-supported STT model.

   channels:
     telegram:
       stt:
         model: "openai/whisper-1"   # default; any LiteLLM-supported STT model

5. Opt out entirely

   Disable transcription for a channel.

   channels:
     telegram:
       stt:
         enabled: false

How It Works

A voice note is downloaded, transcribed, and handed to the agent as text.

A user records a voice note in Telegram (or Slack, WhatsApp, Discord, or Signal) and taps send. Within a second, the bot’s typing indicator appears — the audio has been downloaded and handed to whisper-1. When the transcript comes back, the agent processes it exactly as if the user had typed the text. If echo_transcripts: true, the user first sees the recognised text confirmed back to them, then the agent’s reply — helpful during onboarding so users know their voice was heard correctly.

Graceful degradation

A voice note is never silently dropped — the agent always sees at least a placeholder.

• STT disabled → the turn still reaches the agent with [Voice message received].
• Transcription fails → same placeholder.
• STT tool import fails (missing whisper/litellm deps) → same placeholder plus a WARNING in logs.

Configuration Options

The stt: block lives under any channel in gateway.yaml / bot.yaml.

channels:
  <platform>:
    stt:
      enabled: true
      echo_transcripts: false
      language: null
      model: null

Bool shorthand

channels:
  telegram:
    stt: true    # equivalent to {enabled: true}

Resolution order

The effective policy is resolved from, in order:

1. config.metadata["stt"] (operator override; dict or bool),
2. a direct config.stt attribute (schema-backed configs), then
3. the on-by-default default.

Platform Support

Discord and Signal share the same stt: block, size cap, and [Voice message received] placeholder as Telegram, Slack, and WhatsApp.

Placeholder Behaviour

When transcription is unavailable, the agent receives [Voice message received] instead of nothing.

The invariant: a voice note always reaches the agent with at least the placeholder.

Common Patterns

Voice notes fit several bot styles.

Voice-first assistant

No config needed. The user taps the mic, records a question, and the agent replies in text — the transcription happens automatically.

channels:
  telegram:
    token: ${TELEGRAM_BOT_TOKEN}

Multilingual support

Force a language for a single-language community bot. Auto-detection is skipped for faster, more accurate recognition.

channels:
  whatsapp:
    stt:
      language: "es"

Debug transcription quality

Echo the recognised text back so users can spot recognition errors and rephrase.

channels:
  slack:
    stt:
      echo_transcripts: true

Best Practices

Related