LangSmith

Monitor and debug your AI agents with LangSmith traces

Monitor agent execution, LLM calls, tool usage, and token costs in LangSmith.

Diagram

LangSmith Observability

🤖 Agent

📡 Auto-Trace

💬 LLM Calls

🔧 Tool Calls

📊 Token Usage

🔍 LangSmith Dashboard

Quick Start

  1. Install Dependencies
    bash
    pip install praisonaiagents praisonai-tools opentelemetry-sdk opentelemetry-exporter-otlp
  2. Set Environment Variables
    bash
    export LANGSMITH_API_KEY=lsv2_xxx
    export LANGSMITH_PROJECT=my-project
  3. Run Your Agent
    python
    from praisonai_tools.observability import obs
    from praisonaiagents import Agent
    
    obs.init(provider="langsmith")
    
    agent = Agent(
        name="Assistant",
        instructions="You are a helpful assistant.",
        model="gpt-4o-mini",
    )
    
    response = agent.chat("What is AI?")
    print(response)

How It Works

Diagram
LangSmithLLMAgentUserLangSmithLLMAgentUserobs auto-traces agent startobs auto-traces LLM callobs auto-traces agent endShows inputs, outputs,tokens, cost, latencyagent.chat(“What is AI?”)Send prompt + messagesResponse + token usage“AI is…”Export trace (async)
What Gets TracedDetails
Agent lifecycleStart/end timing, agent name, role
LLM callsInput messages, output, model, token usage
Tool callsTool name, arguments, results
Token usagePrompt tokens, completion tokens, total
ErrorsStack traces, error messages

Configuration Options

OptionEnvironment VariableDescription
api_keyLANGSMITH_API_KEYYour LangSmith API key
projectLANGSMITH_PROJECTProject name (default: "default")
endpointLANGSMITH_ENDPOINTAPI endpoint (default: https://api.smith.langchain.com)
tracingLANGSMITH_TRACINGSet to true to enable (auto-detected)

Common Patterns

Single Agent

python
from praisonai_tools.observability import obs
from praisonaiagents import Agent

obs.init(provider="langsmith")

agent = Agent(
    name="Assistant",
    instructions="You are a helpful assistant.",
    model="gpt-4o-mini",
)

response = agent.chat("What is AI?")
print(response)

Multi-Agent Team

python
from praisonai_tools.observability import obs
from praisonaiagents import Agent, Task, PraisonAIAgents

obs.init(provider="langsmith", project_name="my-team")

researcher = Agent(
    name="Researcher",
    instructions="Search for information.",
    model="gpt-4o-mini",
)

writer = Agent(
    name="Writer",
    instructions="Write clear summaries.",
    model="gpt-4o-mini",
)

task1 = Task(description="Research AI trends", agent=researcher)
task2 = Task(description="Summarize findings", agent=writer)

agents = PraisonAIAgents(agents=[researcher, writer], tasks=[task1, task2])
agents.start()

With Tools

python
from praisonai_tools.observability import obs
from praisonaiagents import Agent, tool

obs.init(provider="langsmith")

@tool
def search_web(query: str) -> str:
    """Search the web for information."""
    return f"Results for: {query}"

agent = Agent(
    name="Researcher",
    instructions="Search and answer questions.",
    tools=[search_web],
    model="gpt-4o-mini",
)

response = agent.chat("Latest AI news")
print(response)

Explicit Tracing

python
from praisonai_tools.observability import obs
from praisonaiagents import Agent

obs.init(provider="langsmith", auto_instrument=False)

agent = Agent(
    name="Assistant",
    instructions="You are helpful.",
    model="gpt-4o-mini",
)

with obs.trace("chat-session"):
    response = agent.chat("What is AI?")
    print(response)

What You See in LangSmith

Diagram

LangSmith Trace View

🔍 Trace: agent.Assistant.chat

🤖 Agent: Assistant

💬 LLM: gpt-4o-mini

🔧 Tool: search_web

📊 Tokens: 50 in / 120 out

💰 Cost: $0.001

⏱️ Latency: 1.2s


Diagnostics & Verification

Check your setup with the built-in doctor:

Python

python
from praisonai_tools.observability import obs

obs.init(provider="langsmith")
results = obs.doctor()
print(results)
json
{
    "enabled": true,
    "provider": "langsmith",
    "connection_status": true,
    "connection_message": "LangSmith API key configured"
}

CLI

bash
# Health check
python -m praisonai_tools.observability.cli doctor

# Health check (JSON)
python -m praisonai_tools.observability.cli doctor --json

# Verify traces in LangSmith (requires LANGSMITH_API_KEY)
python -m praisonai_tools.observability.cli verify --project "My First App"

# Verify with JSON output
python -m praisonai_tools.observability.cli verify --project "My First App" --json

PraisonAI Branding

Every agent and workflow span automatically includes PraisonAI branding in LangSmith metadata:

Metadata KeyValueDescription
praisonai.version0.2.20SDK version used
praisonai.frameworkpraisonaiFramework identifier

Workflow spans also capture structured input (agent names, task descriptions) and output.


Best Practices

Use project names to organize traces

Set LANGSMITH_PROJECT or pass project_name to group traces by environment or feature.

python
obs.init(provider="langsmith", project_name="production-chatbot")
Use auto-instrumentation for most cases

Auto-instrumentation traces everything automatically. Only use explicit obs.trace() when you need custom trace boundaries or additional metadata.

Set environment variables in .env files

Keep API keys out of code. Use .env files or your deployment platform’s secret management.

bash
# .env
LANGSMITH_API_KEY=lsv2_xxx
LANGSMITH_PROJECT=my-project
Monitor token usage for cost control

LangSmith traces include token counts for every LLM call. Use this to identify expensive operations and optimize prompts.


Related