Evaluation CLI
CLI commands for evaluation in PraisonAI TypeScript
The praisonai-ts CLI provides the eval command for agent evaluation.
Quick Start
- Simple Usage
bash praisonai-ts eval accuracy --input "2+2" --expected "4" - With Configuration
bash praisonai-ts eval performance --iterations 10 --json
Accuracy Evaluation
# Run accuracy evaluation
praisonai-ts eval accuracy --input "2+2" --expected "4"
# With multiple iterations
praisonai-ts eval accuracy --input "What is 2+2?" --expected "4" --iterations 3 --jsonPerformance Evaluation
# Run performance benchmark
praisonai-ts eval performance --iterations 10
# With warmup runs
praisonai-ts eval performance --iterations 50 --warmup 5 --jsonReliability Evaluation
# Check tool call reliability
praisonai-ts eval reliability --expected-tools "calculator,web_search"SDK Usage
For programmatic evaluation:
import { AccuracyEval, PerformanceEval, ReliabilityEval } from 'praisonai';
// Accuracy evaluation
const accuracy = new AccuracyEval({
agent: myAgent,
input: "What is 2+2?",
expectedOutput: "4",
numIterations: 3
});
const result = await accuracy.run();
// Performance evaluation
const perf = new PerformanceEval({
func: () => agent.chat("Hello"),
numIterations: 50,
warmupRuns: 10
});
const perfResult = await perf.run();For more details, see the Evaluation SDK documentation.