CLI Profiling

Profile agent execution from the command line

PraisonAI provides CLI commands for profiling agent performance without modifying code.

Quick Start

bash

# Quick inline profiling with timeline diagram
praisonai "What is 2+2?" --profile

# Deep profiling with call graph
praisonai "What is 2+2?" --profile --profile-deep

# Profile a query with detailed timing breakdown
praisonai profile query "What is 2+2?"

# Profile with file grouping
praisonai profile query "Hello" --show-files --limit 20

# Profile startup time
praisonai profile startup

# Profile import times
praisonai profile imports

# Run comprehensive profiling suite
praisonai profile suite --quick

Inline Profiling (–profile flag)

The simplest way to profile any command is with the --profile flag:

bash

praisonai "Your prompt here" --profile

This outputs a visual timeline diagram showing execution phases:

======================================================================
PraisonAI Profile Report
======================================================================

Run ID:     abc12345
Timestamp:  2026-01-02T05:38:01.749771Z
Method:     cli_direct
Version:    3.0.3

## Timeline Diagram

ENTER ─────────────────────────────────────────────────────────► RESPONSE
      │     imports     │  init │           network            │
      │      843ms      │  0ms  │            1414ms            │
      └─────────────────┴───────┴──────────────────────────────┘
                                                 TOTAL: 2257ms

## Execution Timeline
---------------------------------------------
  Imports                   :   843.23 ms
  Agent Init                :     0.18 ms
  Execution                 :  1413.50 ms
  ───────────────────────────────────────────
  ⏱ Time to First Response  :  2256.91 ms
  TOTAL                     :  2257.09 ms

Deep Profiling

For detailed function-level analysis with call graphs:

bash

praisonai "Your prompt" --profile --profile-deep

This adds:

Decision Trace: Agent config, model, streaming mode, tools
Top Functions: Cumulative time by function
Module Breakdown: Time grouped by module category
Call Graph: Caller/callee relationships

JSON Output

Get machine-readable profile data:

bash

praisonai "Your prompt" --profile --profile-format json

The JSON output includes the timeline diagram as a string field for easy parsing.

Commands

profile query

Profile a query execution with detailed timing breakdown:

bash

praisonai profile query "Your prompt here" [OPTIONS]

Options:

Option	Short	Description
`--model`	`-m`	Model to use
`--stream/--no-stream`		Use streaming mode
`--deep`		Enable deep call tracing (higher overhead)
`--limit`	`-n`	Top N functions to show (default: 30)
`--sort`	`-s`	Sort by: cumulative or tottime
`--show-files`		Group timing by file/module
`--show-callers`		Show caller functions
`--show-callees`		Show callee functions
`--importtime`		Show module import times
`--first-token`		Track time to first token (streaming)
`--save`		Save artifacts to path (.prof, .txt)
`--format`	`-f`	Output format: text or json

Examples:

bash

# Basic profiling with console output
praisonai profile query "Write a poem about AI"

# Profile with file grouping
praisonai profile query "Hello" --show-files --limit 15

# Save JSON report
praisonai profile query "Analyze sentiment" --format json --save=./profile_results

# Track time to first token in streaming mode
praisonai profile query "Test" --stream --first-token

# Deep call tracing with caller/callee info
praisonai profile query "Test" --deep --show-callers --show-callees

Output:

======================================================================
PraisonAI Profile Report
======================================================================

## System Information
  Timestamp:        2025-12-31T17:37:46.662247Z
  Python Version:   3.12.11
  Platform:         macOS-15.7.4-arm64-arm-64bit
  PraisonAI:        2.9.2
  Model:            default

## Timing Breakdown
  CLI Parse:              0.00 ms
  Imports:              867.21 ms
  Agent Construct:        0.06 ms
  Model Init:             0.00 ms
  Total Run:           2302.64 ms

## Per-Function Timing (Top Functions)
----------------------------------------------------------------------
Function                               Calls Cumulative (ms)    Self (ms)
----------------------------------------------------------------------
start                                      1      2302.57         0.03
chat                                       1      2302.54         0.03
_chat_completion                           1      2302.45         0.02
...
======================================================================

profile imports

Profile module import times to identify slow imports:

bash

praisonai profile imports

Output:

======================================================================
Import Time Analysis
======================================================================
Module                                        Self (μs)  Cumul (μs)
----------------------------------------------------------------------
praisonaiagents                                     624     1772006
praisonaiagents.workflows                           280     1617822
praisonaiagents.agent.agent                          30     1569219
openai                                             1163     1369693
openai.types                                       1679      786129
...
----------------------------------------------------------------------
Total import time: 1772.01 ms

profile startup

Profile CLI startup time (cold and warm):

bash

praisonai profile startup

Output:

==================================================
Startup Time Analysis
==================================================
Cold Start:       62.84 ms
Warm Start:       79.25 ms
==================================================

profile suite

Run a comprehensive profiling suite with multiple scenarios:

bash

praisonai profile suite [OPTIONS]

Options:

Option	Short	Default	Description
`--output`	`-o`	`/tmp/praisonai_profile_suite`	Output directory for results
`--iterations`	`-n`	3	Iterations per scenario
`--quick`		false	Quick mode (fewer iterations)

Examples:

bash

# Full suite (4 scenarios, 3 iterations each)
praisonai profile suite

# Quick mode (2 scenarios, 1 iteration)
praisonai profile suite --quick

# Custom output directory
praisonai profile suite --output ./my_profile_results

# More iterations for statistical significance
praisonai profile suite --iterations 5

Output:

🔬 Running Profile Suite...
   Output: /tmp/praisonai_profile_suite
   Scenarios: 4
   Iterations: 3

📊 Measuring startup times...
   Cold: 80.38ms, Warm: 84.71ms

📊 Analyzing imports...
   Top import: praisonaiagents (1908.99ms)

📊 Running scenario: simple_non_stream
   Iteration 1: 6366.58ms
   Total time: 6366.58ms (±0.00ms)

📊 Running scenario: simple_stream
   Iteration 1: 3484.61ms
   Total time: 3484.61ms (±0.00ms)

✅ Suite complete. Results saved to /tmp/praisonai_profile_suite

============================================================
Profile Suite Summary
============================================================
Startup Cold: 80.38ms
Startup Warm: 84.71ms

Top Import: praisonaiagents
  Time: 1908.99ms

Scenario Results:
  simple_non_stream: 6366.58ms (±0.00ms)
  simple_stream: 3484.61ms (±0.00ms)

✅ Full results saved to: /tmp/praisonai_profile_suite

Output Files:

suite_results.json – Machine-readable JSON with all timing data
suite_report.txt – Human-readable summary report

Advanced Usage

Deep Call Tracing

Enable deep call tracing for detailed call graph analysis:

bash

praisonai profile query "Test" --deep --show-callers --show-callees

Save Artifacts

Save profiling artifacts for later analysis:

bash

praisonai profile query "Test" --save=./profile_results

This creates:

profile_results.prof – Binary cProfile data (can be loaded with pstats)
profile_results.txt – Human-readable report

JSON Output

Get machine-readable output for CI/CD integration:

bash

praisonai profile query "Test" --format json > profile.json

Streaming with First Token Tracking

Track time to first token in streaming mode:

bash

praisonai profile query "Test" --stream --first-token

Combine with py-spy

For production-grade flamegraphs:

bash

# Install py-spy
pip install py-spy

# Record with py-spy (requires sudo on some systems)
py-spy record -o profile.svg -- python -m praisonai "Your task"

# Or for a running process
py-spy record -o profile.svg --pid <PID>

CI/CD Integration

Add profiling to your CI pipeline:

yaml

# .github/workflows/benchmark.yml
name: Performance Benchmark

on:
  push:
    branches: [main]

jobs:
  benchmark:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: pip install praisonai

      - name: Run profile suite
        run: |
          praisonai profile suite --quick --output ./benchmark_results

      - name: Upload results
        uses: actions/upload-artifact@v3
        with:
          name: benchmark-results
          path: ./benchmark_results/

Output Formats

Text Output (Default)

Human-readable format printed to terminal with timing breakdown, function stats, and response preview.

JSON Output

Machine-readable format for processing:

json

{
  "timestamp": "2025-12-31T17:37:46.662247Z",
  "metadata": {
    "python_version": "3.12.11",
    "platform": "macOS-15.7.4-arm64-arm-64bit",
    "praisonai_version": "2.9.2",
    "model": "default"
  },
  "prompt": "hi",
  "response_preview": "Hi there! How can I help...",
  "timing": {
    "cli_parse_ms": 0.0003,
    "imports_ms": 851.95,
    "agent_construction_ms": 0.05,
    "model_init_ms": 0.0001,
    "first_token_ms": 0.0,
    "total_run_ms": 5712.11
  },
  "top_functions": [...]
}

Best Practices

Use suite for comprehensive benchmarks

The suite command runs multiple scenarios with warmup:

bash

praisonai profile suite --iterations 5

Profile in production-like environment

Run benchmarks with similar data sizes and network conditions as production.

Use –show-files to identify hotspots

Group timing by file to find which modules are slowest:

bash

praisonai profile query "Test" --show-files --limit 30

Compare streaming vs non-streaming

Streaming often has faster time-to-first-token:

bash

praisonai profile query "Test" --stream --first-token
praisonai profile query "Test" --no-stream

Troubleshooting

High import times

Import times are dominated by OpenAI SDK. This is expected:

bash

praisonai profile imports

Consider lazy imports if startup time is critical.

High variance in benchmarks

Increase iterations in suite mode:

bash

praisonai profile suite --iterations 10

Deep tracing too slow

Deep tracing adds significant overhead. Use only for debugging:

bash

# Without deep tracing (faster)
praisonai profile query "Test"

# With deep tracing (slower but more detail)
praisonai profile query "Test" --deep