Recipe Cycle Testing

The Recipe Cycle Testing workflow helps you iteratively improve your AI agent recipes.

Quick Start

Create a Recipe

praisonai recipe create "Research AI agent frameworks"

Run with Trace

praisonai recipe run my-recipe --save --name run1

Judge the Output

praisonai recipe judge run1 --yaml my-recipe/agents.yaml

Apply Fixes

praisonai recipe apply judge_plan.yaml --confirm

Repeat

Run cycles 2-5 until score reaches target threshold.

Common Errors & Fixes

Tool Result Truncation

Error: Tool 'tavily_search' result was truncatedCause: Large search results exceed context windowFix:

Use --chunked flag when judging
Reduce max_results in tool calls
Add instructions to summarize results

Low Task Achievement Score

Error: Task Achievement: 5.0/10Cause: Agent output doesn’t match expected formatFix:

Add clearer expected_output in steps
Include specific format requirements in action
Add verification instructions to backstory

Hallucination Detected

Error: Hallucination: 3.0/10Cause: Agent generates unverified informationFix:

Add instruction to cite sources
Use include_answer: true in tavily_search
Add verification step in backstory

Missing Goal

Error: Recipe Goal: Not specifiedCause: No goal field in agents.yamlFix: Add a clear, measurable goal:

goal: Research and compare top 5 AI frameworks

Tool-Specific Fixes

tavily_search
search_web
write_file

Issue: Results truncated or incomplete

# Fix: Add specific instructions
agents:
  researcher:
    backstory: |
      Expert researcher. Always verify completeness
      of search results before finalizing output.
      Cite sources for all claims.

Issue: Fallback to DuckDuckGo with limited results

# Fix: Set TAVILY_API_KEY for better results
metadata:
  requires:
    env:
      - TAVILY_API_KEY

Issue: Approval required but not configured

# Fix: Add approve field
approve:
  - write_file

Cycle Results Example

Cycle	Score	Fixes Applied	Key Issue
1	7.5/10	2	Tool truncation warnings
2	6.67/10	6	Hallucination detection
3	4.5/10	5	Output format issues
4	4.5/10	5	Verification needed
5	4.5/10	-	Stable, needs manual tuning

Cycle 1 achieved 7.5/10 with improved truncation detection. Subsequent cycles may vary based on LLM responses and tool result sizes.

If scores plateau, manual intervention is needed. Review the backstory and action prompts directly.

Best Practices

Start Simple

Use 1 agent for simple tasks. Add agents only when needed.

Be Specific

Use concrete values in actions, not variables.

Set Goals

Always include a measurable goal field.

Verify Output

Add verification instructions to agent backstory.

CLI Reference

# Create recipe
praisonai recipe create "goal" [--no-optimize]

# Run with trace
praisonai recipe run <name> --save --name <trace-id>

# Judge trace
praisonai recipe judge <trace-id> --yaml <file> [--chunked]

# Apply fixes
praisonai recipe apply <plan.yaml> --confirm

# Optimize (automated cycles)
praisonai recipe optimize <path> --iterations 5 --threshold 8.0

Getting Started

Install

Core Concepts

Guides

Features

Models

Databases

Observability

Memory

Telemetry & Analytics

Knowledge

RAG

Persistence

Tools

Other Features

Developers

Configuration

Best Practices

Getting Started (No Code)

Recipe Cycle Testing

Quick Start

Common Errors & Fixes

Tool-Specific Fixes

Cycle Results Example

Best Practices

Start Simple

Be Specific

Set Goals

Verify Output

CLI Reference

​Quick Start

​Common Errors & Fixes

​Tool-Specific Fixes

​Cycle Results Example

​Best Practices

Start Simple

Be Specific

Set Goals

Verify Output

​CLI Reference

Quick Start

Common Errors & Fixes

Tool-Specific Fixes

Cycle Results Example

Best Practices

CLI Reference