Audio - PraisonAI

Transcribe audio files to text and generate speech from text using any LiteLLM-supported provider (OpenAI Whisper, Deepgram, ElevenLabs, …).

Quick Start

Give an Agent a transcription tool

from praisonaiagents import Agent
from praisonai.capabilities import transcribe

agent = Agent(
    name="MeetingSummariser",
    instructions="Transcribe the audio, then produce a bullet-point summary.",
    tools=[transcribe],
)
agent.start("Transcribe ./meeting.mp3 and summarise the decisions.")

Transcribe directly (Whisper default)

from praisonai.capabilities import transcribe

result = transcribe("./meeting.mp3")
print(result.text)

Generate speech from text

from praisonai.capabilities import speech

result = speech("Hello, world!", voice="nova")
result.save("hello.mp3")

Use the async variants

import asyncio
from praisonai.capabilities import atranscribe, aspeech

async def main():
    transcript = await atranscribe("./meeting.mp3")
    audio = await aspeech(transcript.text, voice="nova")
    audio.save("readback.mp3")

asyncio.run(main())

Switch providers without changing code

result = transcribe("./meeting.mp3", model="deepgram/nova-2", language="en")
print(result.text)

How It Works

Calls route through LiteLLM to the provider that matches your model string.

Configuration Options

`transcribe(...)` / `atranscribe(...)`

Option	Type	Default	Description
`audio`	`str \| bytes \| BinaryIO`	required	File path, bytes, or file-like object
`model`	`str`	`"whisper-1"`	Model name (e.g., `whisper-1`, `deepgram/nova-2`)
`language`	`Optional[str]`	`None`	ISO language code (e.g., `en`, `es`)
`prompt`	`Optional[str]`	`None`	Optional prompt to guide transcription
`response_format`	`str`	`"json"`	`json`, `text`, `srt`, `verbose_json`, `vtt`
`temperature`	`float`	`0.0`	Sampling temperature (0.0-1.0)
`timestamp_granularities`	`Optional[List[str]]`	`None`	List of `word` and/or `segment`
`timeout`	`float`	`600.0`	Request timeout in seconds
`api_key`	`Optional[str]`	`None`	Optional API key override
`api_base`	`Optional[str]`	`None`	Optional API base URL override
`metadata`	`Optional[Dict[str, Any]]`	`None`	Optional metadata for tracing (agent_id, session_id, etc.)

`speech(...)` / `aspeech(...)`

Option	Type	Default	Description
`text`	`str`	required	Text to convert to speech
`model`	`str`	`"tts-1"`	Model name (e.g., `tts-1`, `tts-1-hd`, `elevenlabs/...`)
`voice`	`str`	`"alloy"`	Voice name (e.g., `alloy`, `echo`, `fable`, `onyx`, `nova`, `shimmer`)
`response_format`	`str`	`"mp3"`	`mp3`, `opus`, `aac`, `flac`, `wav`, `pcm`
`speed`	`float`	`1.0`	Speed multiplier (0.25-4.0)
`timeout`	`float`	`600.0`	Request timeout in seconds
`api_key`	`Optional[str]`	`None`	Optional API key override
`api_base`	`Optional[str]`	`None`	Optional API base URL override
`metadata`	`Optional[Dict[str, Any]]`	`None`	Optional metadata for tracing

Result objects

Class	Field	Type	Default	Notes
`TranscriptionResult`	`text`	`str`	—	The transcribed text
	`duration`	`Optional[float]`	`None`	Audio duration in seconds
	`language`	`Optional[str]`	`None`	Detected/echoed language
	`segments`	`Optional[List[Dict]]`	`None`	Present when `response_format="verbose_json"`
	`words`	`Optional[List[Dict]]`	`None`	Present when `timestamp_granularities=["word"]`
	`model`	`Optional[str]`	`None`	Model used
	`metadata`	`Dict[str, Any]`	`{}`	Metadata echoed back
`SpeechResult`	`audio`	`bytes`	—	Raw audio bytes
	`content_type`	`str`	`"audio/mpeg"`	Set per `response_format`
	`model`	`Optional[str]`	`None`	Model used
	`metadata`	`Dict[str, Any]`	`{}`	Tracing metadata
	`save(path)`	method	—	Writes bytes to disk, returns path

Common Patterns

Transcribe → summarise pipeline

from praisonaiagents import Agent
from praisonai.capabilities import transcribe

transcript = transcribe("./meeting.mp3")

agent = Agent(
    name="Summariser",
    instructions="Summarise the transcript into bullet-point decisions.",
)
agent.start(transcript.text)

Multilingual dubbing

import asyncio
from praisonaiagents import Agent
from praisonai.capabilities import atranscribe, aspeech

async def dub(path):
    transcript = await atranscribe(path, language="es")
    translator = Agent(name="Translator", instructions="Translate Spanish to English.")
    english = translator.start(transcript.text)
    result = await aspeech(english, voice="nova")
    return result.save("dubbed.mp3")

asyncio.run(dub("./clip_es.mp3"))

Word-level timestamps for captions

from praisonai.capabilities import transcribe

result = transcribe(
    "./meeting.mp3",
    response_format="verbose_json",
    timestamp_granularities=["word"],
)
for word in result.words or []:
    print(word)

Best Practices

Pick the right model per provider

Use whisper-1 for OpenAI parity, deepgram/nova-2 for lower latency, and tts-1-hd when audio fidelity matters more than cost.

Set language when you know it

Passing language="en" skips detection — faster and more accurate for short clips.

Use save() on SpeechResult

SpeechResult.save("out.mp3") writes the bytes and returns the path — no manual file handling needed.

Route through metadata for tracing

Pass metadata={"agent_id": ..., "session_id": ...} so LiteLLM callbacks correlate audio calls with an Agent turn.

Capabilities Overview

All LiteLLM parity capabilities

AudioAgent

Higher-level Agent abstraction for audio

Completions

Sibling chat/text completion capability

Audio CLI

Command-line and MCP tool equivalents

Moderations CLI

Audio CLI

​Quick Start

​How It Works

​Configuration Options

​transcribe(...) / atranscribe(...)

​speech(...) / aspeech(...)

​Result objects

​Common Patterns

​Transcribe → summarise pipeline

​Multilingual dubbing

​Word-level timestamps for captions

​Best Practices

​Related

Capabilities Overview

AudioAgent

Completions

Audio CLI

Quick Start

How It Works

Configuration Options

`transcribe(...)` / `atranscribe(...)`

`speech(...)` / `aspeech(...)`

Result objects

Common Patterns

Transcribe → summarise pipeline

Multilingual dubbing

Word-level timestamps for captions

Best Practices

Related