Token Management

Track and limit token usage to control costs in Rust agents

Set token limits so your Rust agents stay within budget and never generate unexpectedly long responses.

Quick Start

1. Set Output Token Limit

   use praisonai::Agent;
   
   let agent = Agent::new()
       .name("Assistant")
       .instructions("You are concise")
       .max_tokens(500)  // Limit output to 500 tokens
       .build()?;
   
   let response = agent.chat("Explain quantum computing").await?;
   println!("{}", response);

2. Track Token Usage

   use praisonai::Agent;
   
   let agent = Agent::new()
       .name("Assistant")
       .instructions("You are helpful")
       .build()?;
   
   let response = agent.chat("Hello").await?;
   
   // Check usage after the call
   let usage = agent.token_usage();
   println!("Input tokens:  {}", usage.input_tokens);
   println!("Output tokens: {}", usage.output_tokens);
   println!("Total tokens:  {}", usage.total_tokens);

3. Set Context Window Limit

   use praisonai::Agent;
   
   let agent = Agent::new()
       .name("Assistant")
       .instructions("You summarize long documents")
       .max_tokens(1000)      // Output limit
       .max_context(8192)     // Context window limit
       .build()?;

How It Works

Configuration Options

Common Patterns

Budget-Aware Loops

Stop processing when cumulative cost gets too high:

use praisonai::Agent;

let agent = Agent::new()
    .name("Analyst")
    .instructions("Analyze text")
    .max_tokens(200)
    .build()?;

let questions = vec!["Q1", "Q2", "Q3"];
let token_limit = 1000u32;

for question in &questions {
    let usage = agent.token_usage();
    if usage.total_tokens >= token_limit {
        println!("Token budget exhausted after {} tokens", usage.total_tokens);
        break;
    }
    let answer = agent.chat(question).await?;
    println!("{}", answer);
}

Verbose Usage Reporting

use praisonai::Agent;

let agent = Agent::new()
    .name("Reporter")
    .instructions("Generate reports")
    .max_tokens(800)
    .verbose(true)  // Logs token usage after each request
    .build()?;

agent.chat("Write a weekly summary").await?;

Best Practices

Related