Praison AI

Recipe Workflow

Run, judge, and improve agent recipes with automated evaluation and fixes

Overview

The Recipe Workflow provides a complete cycle for running, evaluating, and improving agent configurations.

Diagram

Improve

Evaluate

Run

–save

iterate

recipe run

History auto

Replay Trace

recipe runs

recipe judge

Plan

recipe apply

Updated YAML

Commands

Run a Recipe

bash
praisonai recipe run <recipe-name> [OPTIONS]
OptionDescription
--save, -sSave replay trace for debugging
--name, -n NAMECustom trace name
--var KEY=VALUESet variables
--debugEnable debug output
--verbose, -vVerbose output

Example:

bash
praisonai recipe run ai-topic-gatherer --save --name my-trace --var topic="AI 2026"

Judge a Trace

bash
praisonai recipe judge <trace-id> [OPTIONS]
OptionDescription
--yaml FILEYAML for context-aware evaluation
--output FILEOutput plan file
--model MODELJudge model (default: gpt-4o-mini)

Example:

bash
praisonai recipe judge my-trace --yaml agents.yaml --output plan.yaml

Apply Improvements

bash
praisonai recipe apply <plan-file> [OPTIONS]
OptionDescription
--confirm / -cApply without confirmation
--dry-runPreview only
--fix-ids IDSApply specific fixes
--no-backupSkip backup

Example:

bash
# Preview changes
praisonai recipe apply plan.yaml --dry-run

# Apply with confirmation
praisonai recipe apply plan.yaml --confirm

Complete Workflow

  1. Run Recipe with Trace
    bash
    praisonai recipe run ai-wordpress-post-generator \
  --save --name wp-test \
  --var topic="AI News"
  2. Judge the Execution

    Find the run ID with praisonai recipe runs, then pass it to recipe judge.

    bash
    praisonai recipe judge wp-test \
  --yaml agents.yaml \
  --output improvements.yaml
  3. Review Suggestions
    bash
    praisonai recipe apply improvements.yaml --dry-run
  4. Apply Improvements
    bash
    praisonai recipe apply improvements.yaml --confirm
  5. Re-run and Verify
    bash
    praisonai recipe run ai-wordpress-post-generator \
  --save --name wp-test-v2 \
  --var topic="AI News"

Evaluation Metrics

The judge evaluates each agent on 6 criteria:

Diagram
17%17%17%17%17%17%Agent Evaluation CriteriaTask AchievementContext UtilizationOutput QualityInstruction FollowingHallucination DetectionError Handling
MetricDescription
Task AchievementDid the agent complete its goal?
Context UtilizationWas context effectively used?
Output QualityIs output high quality?
Instruction FollowingDid agent follow instructions?
Hallucination10=accurate, 1=fabricated
Error HandlingHow well were errors handled?

Fix Types

The judge generates actionable fixes:

TypeDescription
append_instructionAdd guidance to instructions
add_expected_outputSpecify output format
modify_context_configAdjust context settings
suggestionGeneral improvement

Example Output

Judge Report

============================================================
  LLM JUDGE REPORT: wp-test
============================================================

  Overall Score: 7.86/10
  Agents Evaluated: 6

  AGENT SCORES:
    Research Agent:
      Task Achievement: 8.0/10
      Context Utilization: 7.0/10
      Output Quality: 8.0/10
      Instruction Following: 9.0/10
      Hallucination: 8.0/10
      Error Handling: 10.0/10
      Overall: 8.33/10

  CONTEXT FLOW:
    Research Agent → Writer Agent: 10.0/10 ✓
    Writer Agent → Publisher: 10.0/10 ✓

  RECOMMENDATIONS:
    - Research Agent: Include specific citations
    - Writer Agent: Ensure all sections complete
============================================================

Plan Preview

============================================================
  PREVIEW: Changes to be applied
============================================================

  [MEDIUM] fix_abc123
    Agent: Research Agent
    Type: append_instruction
    Path: agents.Research Agent.instructions
    New: IMPROVEMENT: Include specific citations...

  [LOW] fix_def456
    Agent: Writer Agent
    Type: append_instruction
    New: IMPROVEMENT: Ensure sections complete...

============================================================

Best Practices

History is Automatic

Every run is recorded to history; add --save only for full replay traces

 Name Your Traces

Use --name for easy identification

 Preview First

Always --dry-run before applying

 Iterate

Run → Judge → Apply → Repeat

Viewing Traces

Use the replay commands to inspect traces:

bash
# List all traces
praisonai replay list

# View trace details
praisonai replay context my-trace --dump

# Full content (no truncation)
praisonai replay context my-trace --dump --full

# Statistics
praisonai replay context my-trace --stats

Related