Streaming

Stream LLM responses in real-time for better user experience

Display AI responses as they generate, creating a natural chat experience.

Diagram

Streaming Response

word by word

💬 User

🤖 Agent

⚡ Stream

📺 Display

Quick Start

  1. Enable Streaming
    rust
    use praisonai::Agent;
    
    let agent = Agent::new()
        .name("Assistant")
        .instructions("You are helpful")
        .stream(true)  // Enable streaming
        .build()?;
    
    agent.chat("Tell me a story").await?;
    // Response appears word by word
  2. Disable Streaming
    rust
    use praisonai::Agent;
    
    let agent = Agent::new()
        .name("Assistant")
        .stream(false)  // Wait for complete response
        .build()?;

How It Works

Diagram
LLMAgentUserLLMAgentUserloop[Each token]chat(“question”)Request (stream=true)StreamEvent::DeltaTextDisplay tokenStreamEvent::StreamEndComplete

Stream Events

Events emitted during streaming.

EventDescription
RequestStartAPI call initiated
HeadersReceivedHTTP headers arrived
FirstTokenFirst content (TTFT marker)
DeltaTextText content chunk
DeltaToolCallTool call in progress
ToolCallEndTool call complete
LastTokenFinal content chunk
StreamEndStream completed
ErrorError occurred

Configuration

OptionTypeDefaultDescription
streambooltrueEnable response streaming

Best Practices

Enable for chat interfaces

Streaming provides immediate feedback, improving perceived performance.

Disable for batch processing

When processing many requests, disable streaming for efficiency.


Related