eval

AI Agent PraisonAI Agents Evaluation Framework. Provides comprehensive evaluation capabilities for AI agents with zero performance impact when not in use through lazy loading. Evaluator Types:

AccuracyEvaluator: Compare output against expected output using LLM-as-judge
PerformanceEvaluator: Measure runtime and memory usage
ReliabilityEvaluator: Verify expected tool calls are made
CriteriaEvaluator: Evaluate against custom criteria
ContextEvaluator: Score context budget compliance and multi-agent handoff fidelity
ComparisonEval: Side-by-side comparison of two agent outputs
SafetyEval: Detect harmful, biased, or inappropriate outputs
LoopEvaluator: Score loop health (convergence, wasted iterations, doom-loop guards)
EvalSuite: Orchestrator for running multiple evaluations

Example:

>>> from praisonaiagents.eval import AccuracyEvaluator
>>> evaluator = AccuracyEvaluator(
...     agent=my_agent,
...     input_text="What is 2+2?",
...     expected_output="4"
... )
>>> result = evaluator.run(print_summary=True)

Import

from praisonaiagents import eval

Constants

Name	Value
`_LAZY_IMPORTS`	`{'BaseEvaluator': ('praisonaiagents.eval.base', 'BaseEvaluator'), 'AccuracyEvaluator': ('praisonaiagents.eval.accuracy', 'AccuracyEvaluator'), 'PerformanceEvaluator': ('praisonaiagents.eval.performanc...`

Guide

Reference

eval • AI Agent SDK

eval

Import

Constants

Evaluation Concept

LLM as Judge

Evaluation Loop

​eval

​Import

​Constants

​Related Documentation

Evaluation Concept

LLM as Judge

Evaluation Loop

eval

Import

Constants

Related Documentation