Skip to main content
Loop Evaluator scores how healthy a loop run was — did it converge, were iterations wasted, and did a doom-loop guard fire — on top of EvaluationLoop’s output-quality score. A loop can succeed (score ≥ 8.0) while burning max iterations, triggering doom-loop recovery, or costing 5× the expected tokens. EvaluationLoop scores output quality; LoopEvaluator scores loop health on top of it — no extra LLM calls.

​
Quick Start

1

Score an already-run loop

Run an EvaluationLoop, then score its health.
from praisonaiagents import Agent
from praisonaiagents.eval import EvaluationLoop, LoopEvaluator

agent = Agent(name="writer", instructions="Write clear explanations.")
loop = EvaluationLoop(agent=agent, criteria="Explanation is clear", threshold=8.0, max_iterations=5)

loop_result = loop.run("Explain how HTTPS works.")

health = LoopEvaluator().run(loop_result)
health.print_summary()   # ✅ HEALTHY / iterations / waste / doom-loop
2

Tolerate wasted iterations

Set an explicit threshold and allow one wasted iteration.
health = LoopEvaluator(threshold=8.0, max_wasted_iterations=1).run(loop_result)
assert health.passed
3

Pass structured guard events

Feed doom-loop guard events. The evaluator is duck-typed — dicts or objects both work.
# Any of these shapes is recognised:
guard_events = [
    {"type": "doom_loop", "reason": "repeated tool call"},   # generic marker
    # or a DoomLoopEvent-style object with a .loop_type
    # or a DoomLoopResult-style dict {"is_loop": True}
]

health = LoopEvaluator().run(loop_result, guard_events=guard_events)
assert health.doom_loop_fired is True
assert not health.passed
Compose with the full suite — add this evaluator to an EvalSuite to gate CI on Context, Loop, Harness, and Accuracy in one pass.

​
How It Works

LoopEvaluator derives health from a completed EvaluationLoopResult — it never re-runs the loop, so it makes zero LLM calls and is safe in CI.
InputSourceUsed For
score_historyEvaluationLoopResultConvergence, per-iteration deltas
thresholdConstructor or loop resultConvergence check
modeEvaluationLoopResultOptimize vs review convergence
guard_eventsdoom_loop.py / loop_detection_pluginDoom-loop detection

​
Mode-Aware Convergence

Convergence is computed differently in optimize vs review mode — the health verdict never disagrees with the loop’s own verdict.
  • optimize mode: convergence = first iteration whose score ≥ threshold. wasted_iterations = num_iterations − iterations_to_success.
  • review mode: EvaluationLoop runs every iteration and derives success from the final score, so a transient earlier high score is not treated as convergence. LoopEvaluator defers to loop_result.success (or score_history[-1] >= threshold). wasted_iterations is always 0.

​
Configuration Options

Constructor parameters for LoopEvaluator.
OptionTypeDefaultDescription
thresholdOptional[float]NoneScore threshold for convergence. Falls back to the threshold on the EvaluationLoopResult.
max_wasted_iterationsint0Maximum wasted iterations tolerated before the loop is unhealthy.
nameOptional[str]NoneOptional name for this evaluation run.
save_results_pathOptional[str]NoneOptional path to persist the result JSON.
verboseboolFalsePrint a rich summary table after run().
run(loop_result, guard_events=None) takes a completed EvaluationLoopResult (or any object exposing score_history, threshold, total_duration_seconds, mode, success) and an optional list of guard events.

Eval SDK Reference

Full auto-generated Python reference for the eval module

​
LoopHealthResult Fields

run() returns a LoopHealthResult. These are the fields returned by to_dict().
FieldTypeDescription
iterations_to_successint1-based iteration at which threshold was first met; otherwise total iterations.
wasted_iterationsintIterations that ran after threshold was first met (always 0 in review mode).
doom_loop_firedboolWhether any guard event indicated a doom-loop.
guard_interventionsintTotal number of guard events observed.
total_duration_sfloatTotal wall-clock duration in seconds.
score_delta_per_iterationList[float]Per-iteration delta (len == num_iterations − 1, rounded to 4dp).
convergedboolWhether the loop reached its threshold.
success / passedboolconverged AND not doom_loop_fired AND wasted_iterations <= max_wasted_iterations. passed aliases success.
thresholdfloatThe threshold used (constructor, else loop result, else 8.0).
reasoningstrHuman-readable explanation of the verdict.
metadataDict[str, Any]Includes num_iterations.
to_dict(), to_json(), and print_summary() are all provided.

​
Recognised Guard-Event Shapes

Guard events are duck-typed — the evaluator detects a doom-loop without coupling to any specific class.
ShapeTrigger
DoomLoopResult-styleis_loop bool is truthy
DoomLoopEvent-stylepresence of a loop_type (a DoomLoopType enum)
Explicit flagdoom_loop / doom_loop_fired boolean is truthy
Generic markertype / event field matches a known string (below)
Accepted type / event string values (case-insensitive): doom_loop, doom-loop, doomloop, repetition, loop_detected, repeated_action, repeated_failure, no_progress, circular_plan, resource_exhaustion, repeated_output.

​
Common Patterns

​
CI Gate on Loop Health

Fail the build when a loop wastes iterations or a doom-loop fires. 
import sys
from praisonaiagents.eval import LoopEvaluator

health = LoopEvaluator(max_wasted_iterations=0).run(loop_result)
if not health.passed:
    print(f"Loop unhealthy: {health.reasoning}")
    sys.exit(1)

​
Combine Quality and Health

Score output quality (EvaluationLoop) and loop health (LoopEvaluator) on the same run — they are complementary. 
from praisonaiagents import Agent
from praisonaiagents.eval import EvaluationLoop, LoopEvaluator

agent = Agent(name="analyst", instructions="Analyze systems thoroughly.")
loop = EvaluationLoop(agent=agent, criteria="Analysis is thorough", threshold=8.0)

loop_result = loop.run("Analyze the auth flow")

print(f"Quality: {loop_result.final_score}/10, success={loop_result.success}")

health = LoopEvaluator().run(loop_result)
print(f"Health: passed={health.passed}, wasted={health.wasted_iterations}")

​
Wire Real Doom-Loop Guards

Plug in structured DoomLoopEvents (or loop_detection_plugin events) as guard_events. 
from praisonaiagents.escalation.doom_loop import DoomLoopEvent  # emits .loop_type
from praisonaiagents.eval import LoopEvaluator

# guard_events collected during the agent run
health = LoopEvaluator().run(loop_result, guard_events=guard_events)
if health.doom_loop_fired:
    print(f"Doom-loop fired: {health.reasoning}")

​
Best Practices

0 is the strictest setting — any post-threshold iteration marks the loop unhealthy. Use it for optimize-mode CI gates; use ≥1 for exploratory review-mode runs.
That is intentional, not a bug — review mode runs all iterations by design, so post-threshold iterations are not “waste”.
A dict or an object works. type, event, loop_type, is_loop, and explicit doom_loop flags are all recognised — see the accepted string values above.
The evaluator only consumes an existing EvaluationLoopResult. No external deps, no network — safe in offline CI.

Evaluation Suite

Run all four evaluators as one CI gate

Evaluation Loop

The loop this evaluator scores

Loop Detection

The guard whose events feed this evaluator

Loop Guard

Per-turn tool-call guard, complementary safety net

Judge

The underlying judge used by EvaluationLoop

CHL Engineering

The Context / Harness / Loop rubric this evaluator scores against.