Telemetry & Performance Tracking

Anonymous usage metrics for agent runs — no prompts or responses collected

Telemetry tracks anonymous usage counts when agents run — no prompts, responses, or personal data.

from praisonaiagents import Agent

agent = Agent(name="Assistant", instructions="You are helpful.")
agent.start("Hello")  # telemetry counts this run automatically

The user runs an agent as usual; anonymous run metrics are recorded automatically unless telemetry is opted out.

How It Works

Quick Start

1. Install Package

   Install PraisonAI Agents:

   pip install praisonaiagents

   Note: Telemetry is included by default and automatically enabled unless disabled.

2. Simple Example

   Create simple_agent_example.py:

   from praisonaiagents import Agent, Task, AgentTeam
   
   # Create a simple agent - telemetry is automatically enabled
   agent = Agent(
       name="Assistant",
       role="Helper",
       goal="Assist with tasks",
       instructions="You are a helpful assistant."
   )
   
   # Create a task
   task = Task(
       description="Write a haiku about AI",
       expected_output="A three-line haiku poem",
       agent=agent
   )
   
   # Create and run workflow - telemetry tracks automatically
   workflow = AgentTeam(agents=[agent], tasks=[task])
   result = workflow.start()
   
   print(result)

3. Check Telemetry (Optional)

   Access telemetry metrics programmatically:

   from praisonaiagents import get_telemetry
   
   telemetry = get_telemetry()
   
   # Check metrics (stored in memory)
   metrics = telemetry.get_metrics()
   print(f"Agent executions: {metrics['agent_executions']}")
   print(f"Task completions: {metrics['task_completions']}")
   print(f"Tool calls: {metrics['tool_calls']}")
   print(f"Errors: {metrics['errors']}")

4. Disable Telemetry (Optional)

   To disable telemetry, set any of these environment variables before running:

   export PRAISONAI_TELEMETRY_DISABLED=true
   # or
   export PRAISONAI_DISABLE_TELEMETRY=true
   # or
   export DO_NOT_TRACK=true

5. Enable Telemetry Explicitly (Optional)

   To enable telemetry via environment variable (required since telemetry is off by default):

   export PRAISONAI_TELEMETRY_ENABLED=true
   # or
   export PRAISONAI_PERFORMANCE_ENABLED=true

What is Tracked

PraisonAI telemetry collects only minimal, anonymous metrics:

Disabling Telemetry

Telemetry can be disabled in multiple ways:

Environment Variables (Recommended)

Opt-out (any of these disables telemetry):

export PRAISONAI_TELEMETRY_DISABLED=true
export PRAISONAI_DISABLE_TELEMETRY=true
export PRAISONAI_PERFORMANCE_DISABLED=true
export DO_NOT_TRACK=true  # Universal standard
export OTEL_SDK_DISABLED=true  # OpenTelemetry standard

Opt-in (required since telemetry is off by default):

export PRAISONAI_TELEMETRY_ENABLED=true
# or
export PRAISONAI_PERFORMANCE_ENABLED=true

Enabling / Disabling Telemetry Programmatically

from praisonaiagents import enable_telemetry, disable_telemetry

# Turn telemetry on (no-op if an env opt-out is set)
enable_telemetry()

# Turn telemetry off — instrumentation hooks stop installing
disable_telemetry()

Langfuse + opt-out coexistence

Users now have full control over telemetry settings when using Langfuse for other applications:

# User has Langfuse credentials for an UNRELATED app, but wants OTEL off in PraisonAI:
export LANGFUSE_PUBLIC_KEY=...
export OTEL_SDK_DISABLED=true     # PraisonAI will now respect this

This addresses the issue where PraisonAI previously overwrote explicit user opt-outs when Langfuse environment variables were present.

How Telemetry Works

Telemetry in PraisonAI is automatic and minimal:

1. Automatic Integration: When you create agents and run workflows, telemetry is automatically integrated
2. Anonymous Tracking: Only counts and types are tracked, never content
3. Memory Storage: Currently metrics are only stored in memory (no network calls)
4. Session-based: Each run gets a unique anonymous session ID
5. Privacy-first: Respects DO_NOT_TRACK standard and multiple opt-out methods

What each agent run counts as

agent_executions is incremented once per logical run, no matter which entry point you called. Agent.run(), Agent.start(), and Agent.chat() all delegate to chat() internally, and telemetry now records only the outermost call — so a single agent.run("...") is one event, not two or three, and a failure inside it is one errors increment, not two.

Concurrent-execution accuracy

Concurrent chat(), run(), and start() calls on one shared Agent instance are each counted independently — you’ll see one row per top-level call in agent_executions, token totals, and duration histograms.

Nested calls (a run() that internally calls chat()) still record once — only the outermost call is counted, so a single user-facing turn produces one execution row.

Async runs produce the same LLM lifecycle spans

Async agent runs now emit the same llm_start / llm_end lifecycle events in the JSONL trace as sync runs. The trace / status output (output/status.py, output/trace.py) consumes these events, so an async --trace run records per-iteration LLM spans with cost figures.

Metrics are not lost on a failed flush

flush() resets counters only after a successful transport-level capture. If PostHog is temporarily unreachable, the batch stays intact and is retried on the next flush instead of being silently zeroed.

Shutdown never hangs on PostHog

shutdown() runs automatically at interpreter exit. It flushes pending PostHog events on a daemon thread with a hard 5-second limit:

• Flush finishes in time → PostHog’s normal cleanup runs.
• Flush still running at the deadline (wedged transport, DNS stall) → shutdown() logs PostHog flush timed out and returns early, skipping cleanup that would block on the same transport.

Interpreter exit is never blocked by a stuck telemetry flush; events still in the batch at that moment are lost for that exit. To drain earlier (e.g. at the end of a batch job), call get_telemetry().flush().

Advanced Usage

Accessing Telemetry Data

from praisonaiagents import get_telemetry

telemetry = get_telemetry()

# Check if telemetry is enabled
if telemetry.enabled:
    print("Telemetry is active")

# Get current metrics
metrics = telemetry.get_metrics()
print(f"Total agent executions: {metrics['agent_executions']}")
print(f"Total task completions: {metrics['task_completions']}")
print(f"Total tool calls: {metrics['tool_calls']}")
print(f"Total errors: {metrics['errors']}")

# Get session info
print(f"Session ID: {telemetry.session_id}")
print(f"Environment: {telemetry._environment}")

Manual Tracking (Advanced)

from praisonaiagents import get_telemetry

telemetry = get_telemetry()

# Manually track events (usually automatic)
telemetry.track_agent_execution(agent_name="MyAgent", success=True)
telemetry.track_task_completion(task_name="MyTask", success=True)
telemetry.track_tool_usage(tool_name="web_search", success=True)
telemetry.track_error(error_type="ValueError")

# Track feature usage
telemetry.track_feature_usage("custom_feature")

Integration with External Services

PostHog Integration (Coming Soon)

PraisonAI telemetry includes built-in PostHog support for anonymous analytics:

# PostHog integration is automatic when available
# Metrics are sent anonymously with session IDs only
# No configuration needed - it just works

# To verify PostHog is available:
try:
    import posthog
    print("PostHog integration available")
except ImportError:
    print("PostHog not installed - metrics stored locally only")

AgentOps Integration

For more advanced monitoring, you can use AgentOps alongside the built-in telemetry:

import os
import agentops
from praisonaiagents import Agent, Task, AgentTeam

agentops.init(api_key=os.getenv("AGENTOPS_API_KEY"))

# Use PraisonAI normally - both telemetry systems work together
agent = Agent(
    name="DataAnalyst",
    role="Analyst",
    goal="Analyze data"
)

task = Task(
    description="Analyze sales trends",
    agent=agent
)

workflow = AgentTeam(agents=[agent], tasks=[task])
result = workflow.start()

# End AgentOps session
agentops.end_session("Success")

Backward Compatibility

The telemetry module maintains compatibility with the previous interface:

from praisonaiagents import TelemetryCollector

# Legacy interface still works
collector = TelemetryCollector()
collector.start()

with collector.trace_agent_execution("MyAgent"):
    # Agent execution code
    pass

collector.stop()

Best Practices

# Development - verbose metrics
import os
os.environ['LOGLEVEL'] = 'DEBUG'

# Production - disable if needed
os.environ['PRAISONAI_TELEMETRY_DISABLED'] = 'true'

Troubleshooting

Next Steps

Related