> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Voice Notes (Speech-to-Text)

> Automatically transcribe inbound voice messages on Telegram, Slack, and WhatsApp

Voice notes sent to your gateway bot are transcribed and delivered to the agent as text — no code changes required, on by default across Telegram, Slack, and WhatsApp.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Voice Note Flow"
        A[🎙️ Voice note] --> B[📥 Channel adapter]
        B --> C[📝 Transcribe]
        C --> D[🤖 Agent]
        D --> E[💬 Reply]
    end

    classDef input fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef process fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef output fill:#10B981,stroke:#7C90A0,color:#fff

    class A input
    class B,C,D process
    class E output
```

## Quick Start

<Steps>
  <Step title="Works out of the box">
    Voice notes are transcribed automatically — no config needed.

    ```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    # gateway.yaml
    channels:
      telegram:
        token: ${TELEGRAM_BOT_TOKEN}
    agent:
      name: "assistant"
      instructions: "Reply helpfully to any question, including voice notes."
    ```

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    agent = Agent(
        name="assistant",
        instructions="Reply helpfully to any question, including voice notes.",
    )
    # Start the gateway — voice notes are transcribed automatically.
    ```
  </Step>

  <Step title="Echo the transcript back">
    Mirror the recognised text back so users can verify what the bot heard.

    ```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    channels:
      telegram:
        token: ${TELEGRAM_BOT_TOKEN}
        stt:
          enabled: true          # default; set false to opt out
          echo_transcripts: true # echo the recognised text back to the user
    ```
  </Step>

  <Step title="Force a language">
    Skip auto-detection for faster, more accurate short clips.

    ```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    channels:
      telegram:
        stt:
          language: "en"
    ```
  </Step>

  <Step title="Custom STT model">
    Swap the default model for any LiteLLM-supported STT model.

    ```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    channels:
      telegram:
        stt:
          model: "openai/whisper-1"   # default; any LiteLLM-supported STT model
    ```
  </Step>

  <Step title="Opt out entirely">
    Disable transcription for a channel.

    ```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    channels:
      telegram:
        stt:
          enabled: false
    ```
  </Step>
</Steps>

<Tip>
  The `stt:` block also accepts a bare boolean as shorthand — `stt: true` is equivalent to `{enabled: true}`, and `stt: false` to `{enabled: false}`.
</Tip>

***

## How It Works

A voice note is downloaded, transcribed, and handed to the agent as text.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant User
    participant Channel as Telegram / Slack / WhatsApp
    participant Adapter as Gateway adapter
    participant STT as transcribe helper
    participant Whisper as AudioAgent.transcribe

    User->>Channel: 🎙️ Voice note
    Channel->>Adapter: Inbound media
    Adapter->>Adapter: Download to SSRF-safe media cache
    Adapter->>STT: transcribe(path, language, model)
    STT->>Whisper: stt_tool(...) → whisper-1
    Whisper-->>STT: transcript text
    STT-->>Adapter: "Hey, what's the weather?"
    Adapter->>Adapter: (optional) echo transcript back
    Adapter-->>User: Agent reply
```

A user records a voice note in Telegram (or Slack, or WhatsApp) and taps send. Within a second, the bot's typing indicator appears — the audio has been downloaded and handed to `whisper-1`. When the transcript comes back, the agent processes it exactly as if the user had typed the text. If `echo_transcripts: true`, the user first sees the recognised text confirmed back to them, then the agent's reply — helpful during onboarding so users know their voice was heard correctly.

### Graceful degradation

A voice note is never silently dropped — the agent always sees at least a placeholder.

* STT disabled → the turn still reaches the agent with `[Voice message received]`.
* Transcription fails → same placeholder.
* STT tool import fails (missing whisper/litellm deps) → same placeholder plus a `WARNING` in logs.

***

## Configuration Options

The `stt:` block lives under any channel in `gateway.yaml` / `bot.yaml`.

| Option             | Type            | Default | Description                                                    |
| ------------------ | --------------- | ------- | -------------------------------------------------------------- |
| `enabled`          | `bool`          | `True`  | Transcribe inbound audio. On by default.                       |
| `echo_transcripts` | `bool`          | `False` | Echo the recognised text back to the user.                     |
| `language`         | `Optional[str]` | `None`  | ISO language code (e.g. `"en"`, `"es"`). `None` = auto-detect. |
| `model`            | `Optional[str]` | `None`  | STT model override. `None` uses `openai/whisper-1`.            |

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  <platform>:
    stt:
      enabled: true
      echo_transcripts: false
      language: null
      model: null
```

### Bool shorthand

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  telegram:
    stt: true    # equivalent to {enabled: true}
```

### Resolution order

The effective policy is resolved from, in order:

1. `config.metadata["stt"]` (operator override; dict or bool),
2. a direct `config.stt` attribute (schema-backed configs), then
3. the on-by-default default.

***

## Platform Support

| Platform                   | Native voice-note support | Behaviour                                                                          |
| -------------------------- | ------------------------- | ---------------------------------------------------------------------------------- |
| Telegram                   | ✅                         | Downloads voice file, transcribes, feeds transcript to agent.                      |
| Slack                      | ✅                         | Downloads voice file via bot token through the SSRF-safe media cache.              |
| WhatsApp                   | ✅                         | Transcribes inbound audio, forwards transcript **plus** cached audio to the agent. |
| Discord                    | ❌                         | Voice-note API not yet wired by the adapter.                                       |
| Email / AgentMail / Linear | ❌                         | No voice-note surface.                                                             |

***

## Placeholder Behaviour

When transcription is unavailable, the agent receives `[Voice message received]` instead of nothing.

| Scenario                              | What the agent sees                                |
| ------------------------------------- | -------------------------------------------------- |
| STT enabled, transcription succeeds   | The transcript (e.g. `"Hey, what's the weather?"`) |
| STT enabled, `echo_transcripts: true` | `[Voice message]: Hey, what's the weather?`        |
| STT disabled (`enabled: false`)       | `[Voice message received]`                         |
| Transcription fails or deps missing   | `[Voice message received]`                         |

The invariant: a voice note always reaches the agent with at least the placeholder.

***

## Common Patterns

Voice notes fit several bot styles.

### Voice-first assistant

No config needed. The user taps the mic, records a question, and the agent replies in text — the transcription happens automatically.

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  telegram:
    token: ${TELEGRAM_BOT_TOKEN}
```

### Multilingual support

Force a language for a single-language community bot. Auto-detection is skipped for faster, more accurate recognition.

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  whatsapp:
    stt:
      language: "es"
```

### Debug transcription quality

Echo the recognised text back so users can spot recognition errors and rephrase.

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  slack:
    stt:
      echo_transcripts: true
```

***

## Best Practices

<AccordionGroup>
  <Accordion title="Set a language when your users speak one language">
    `language: "en"` (or `"es"`, `"fr"`, …) skips auto-detection — faster and more accurate for short clips.
  </Accordion>

  <Accordion title="Turn on echo_transcripts during onboarding">
    `echo_transcripts: true` shows users what the bot heard so they can rephrase if recognition was off.
  </Accordion>

  <Accordion title="Keep enabled: true unless you have a reason to disable">
    Even with STT off, the placeholder fallback still lets voice-note messages reach the agent. Only disable for privacy, cost, or latency reasons.
  </Accordion>

  <Accordion title="Ship your STT provider's API key">
    `openai/whisper-1` is the default, so set `OPENAI_API_KEY`. Any LiteLLM-supported STT model works via the `model:` field.
  </Accordion>
</AccordionGroup>

***

## Related

<CardGroup cols={2}>
  <Card title="Gateway Overview" icon="server" href="/docs/gateway">
    The overall gateway pattern — voice notes are one of several inbound-media types.
  </Card>

  <Card title="Audio Overview" icon="waveform-lines" href="/docs/audio/overview">
    The higher-level `AudioAgent` abstraction — different from this gateway feature.
  </Card>
</CardGroup>