> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Loop Evaluator

> Score loop health — convergence, wasted iterations, and doom-loop guard events

Loop Evaluator scores how healthy a loop run was — did it converge, were iterations wasted, and did a doom-loop guard fire — on top of `EvaluationLoop`'s output-quality score.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Loop Health Scoring"
        SH[📈 Score History]:::input --> LE[💓 LoopEvaluator]:::process
        GE[🛡️ Guard Events]:::input --> LE
        LE --> HR[✅ LoopHealthResult]:::output
    end

    classDef input fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef process fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef output fill:#10B981,stroke:#7C90A0,color:#fff

    class SH,GE input
    class LE process
    class HR output
```

A loop can **succeed** (score ≥ 8.0) while burning max iterations, triggering doom-loop recovery, or costing 5× the expected tokens. `EvaluationLoop` scores output quality; `LoopEvaluator` scores loop health on top of it — no extra LLM calls.

## Quick Start

<Steps>
  <Step title="Score an already-run loop">
    Run an `EvaluationLoop`, then score its health.

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent
    from praisonaiagents.eval import EvaluationLoop, LoopEvaluator

    agent = Agent(name="writer", instructions="Write clear explanations.")
    loop = EvaluationLoop(agent=agent, criteria="Explanation is clear", threshold=8.0, max_iterations=5)

    loop_result = loop.run("Explain how HTTPS works.")

    health = LoopEvaluator().run(loop_result)
    health.print_summary()   # ✅ HEALTHY / iterations / waste / doom-loop
    ```
  </Step>

  <Step title="Tolerate wasted iterations">
    Set an explicit threshold and allow one wasted iteration.

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    health = LoopEvaluator(threshold=8.0, max_wasted_iterations=1).run(loop_result)
    assert health.passed
    ```
  </Step>

  <Step title="Pass structured guard events">
    Feed doom-loop guard events. The evaluator is duck-typed — dicts or objects both work.

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    # Any of these shapes is recognised:
    guard_events = [
        {"type": "doom_loop", "reason": "repeated tool call"},   # generic marker
        # or a DoomLoopEvent-style object with a .loop_type
        # or a DoomLoopResult-style dict {"is_loop": True}
    ]

    health = LoopEvaluator().run(loop_result, guard_events=guard_events)
    assert health.doom_loop_fired is True
    assert not health.passed
    ```
  </Step>
</Steps>

<Tip>
  Compose with the full suite — add this evaluator to an [`EvalSuite`](/docs/features/eval-suite) to gate CI on Context, Loop, Harness, and Accuracy in one pass.
</Tip>

***

## How It Works

`LoopEvaluator` **derives** health from a completed `EvaluationLoopResult` — it never re-runs the loop, so it makes zero LLM calls and is safe in CI.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant Loop as EvaluationLoop
    participant Result as EvaluationLoopResult
    participant Eval as LoopEvaluator
    participant Health as LoopHealthResult

    Loop->>Result: run() → score_history, threshold, mode
    Result->>Eval: run(loop_result, guard_events)
    Eval->>Eval: Derive convergence + waste + doom-loop
    Eval-->>Health: iterations, waste, doom_loop_fired, passed
```

| Input | Source | Used For |
| - | - | - |
| `score_history` | `EvaluationLoopResult` | Convergence, per-iteration deltas |
| `threshold` | Constructor or loop result | Convergence check |
| `mode` | `EvaluationLoopResult` | Optimize vs review convergence |
| `guard_events` | `doom_loop.py` / `loop_detection_plugin` | Doom-loop detection |

***

## Mode-Aware Convergence

Convergence is computed differently in `optimize` vs `review` mode — the health verdict never disagrees with the loop's own verdict.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Start{Loop mode?}:::config
    Start -->|optimize| O[First score ≥ threshold<br/>= convergence]:::process
    O --> OW[wasted = iterations − convergence point]:::output
    Start -->|review| R[Defer to loop_result.success<br/>or final score ≥ threshold]:::process
    R --> RW[wasted = 0 always]:::output

    classDef config fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef process fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef output fill:#10B981,stroke:#7C90A0,color:#fff
```

* **`optimize` mode**: convergence = first iteration whose score ≥ threshold. `wasted_iterations = num_iterations − iterations_to_success`.
* **`review` mode**: `EvaluationLoop` runs every iteration and derives success from the *final* score, so a transient earlier high score is not treated as convergence. `LoopEvaluator` defers to `loop_result.success` (or `score_history[-1] >= threshold`). `wasted_iterations` is always `0`.

***

## Configuration Options

Constructor parameters for `LoopEvaluator`.

| Option | Type | Default | Description |
| - | - | - | - |
| `threshold` | `Optional[float]` | `None` | Score threshold for convergence. Falls back to the threshold on the `EvaluationLoopResult`. |
| `max_wasted_iterations` | `int` | `0` | Maximum wasted iterations tolerated before the loop is unhealthy. |
| `name` | `Optional[str]` | `None` | Optional name for this evaluation run. |
| `save_results_path` | `Optional[str]` | `None` | Optional path to persist the result JSON. |
| `verbose` | `bool` | `False` | Print a rich summary table after `run()`. |

`run(loop_result, guard_events=None)` takes a completed `EvaluationLoopResult` (or any object exposing `score_history`, `threshold`, `total_duration_seconds`, `mode`, `success`) and an optional list of guard events.

<Card title="Eval SDK Reference" icon="code" href="/docs/sdk/reference/praisonaiagents/modules/eval">
  Full auto-generated Python reference for the `eval` module
</Card>

***

## LoopHealthResult Fields

`run()` returns a `LoopHealthResult`. These are the fields returned by `to_dict()`.

| Field | Type | Description |
| - | - | - |
| `iterations_to_success` | `int` | 1-based iteration at which threshold was first met; otherwise total iterations. |
| `wasted_iterations` | `int` | Iterations that ran after threshold was first met (always `0` in review mode). |
| `doom_loop_fired` | `bool` | Whether any guard event indicated a doom-loop. |
| `guard_interventions` | `int` | Total number of guard events observed. |
| `total_duration_s` | `float` | Total wall-clock duration in seconds. |
| `score_delta_per_iteration` | `List[float]` | Per-iteration delta (`len == num_iterations − 1`, rounded to 4dp). |
| `converged` | `bool` | Whether the loop reached its threshold. |
| `success` / `passed` | `bool` | `converged AND not doom_loop_fired AND wasted_iterations <= max_wasted_iterations`. `passed` aliases `success`. |
| `threshold` | `float` | The threshold used (constructor, else loop result, else `8.0`). |
| `reasoning` | `str` | Human-readable explanation of the verdict. |
| `metadata` | `Dict[str, Any]` | Includes `num_iterations`. |

`to_dict()`, `to_json()`, and `print_summary()` are all provided.

***

## Recognised Guard-Event Shapes

Guard events are duck-typed — the evaluator detects a doom-loop without coupling to any specific class.

| Shape | Trigger |
| - | - |
| `DoomLoopResult`-style | `is_loop` bool is truthy |
| `DoomLoopEvent`-style | presence of a `loop_type` (a `DoomLoopType` enum) |
| Explicit flag | `doom_loop` / `doom_loop_fired` boolean is truthy |
| Generic marker | `type` / `event` field matches a known string (below) |

Accepted `type` / `event` string values (case-insensitive): `doom_loop`, `doom-loop`, `doomloop`, `repetition`, `loop_detected`, `repeated_action`, `repeated_failure`, `no_progress`, `circular_plan`, `resource_exhaustion`, `repeated_output`.

***

## Common Patterns

### CI Gate on Loop Health

Fail the build when a loop wastes iterations or a doom-loop fires.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import sys
from praisonaiagents.eval import LoopEvaluator

health = LoopEvaluator(max_wasted_iterations=0).run(loop_result)
if not health.passed:
    print(f"Loop unhealthy: {health.reasoning}")
    sys.exit(1)
```

### Combine Quality and Health

Score output quality (`EvaluationLoop`) and loop health (`LoopEvaluator`) on the same run — they are complementary.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent
from praisonaiagents.eval import EvaluationLoop, LoopEvaluator

agent = Agent(name="analyst", instructions="Analyze systems thoroughly.")
loop = EvaluationLoop(agent=agent, criteria="Analysis is thorough", threshold=8.0)

loop_result = loop.run("Analyze the auth flow")

print(f"Quality: {loop_result.final_score}/10, success={loop_result.success}")

health = LoopEvaluator().run(loop_result)
print(f"Health: passed={health.passed}, wasted={health.wasted_iterations}")
```

### Wire Real Doom-Loop Guards

Plug in structured `DoomLoopEvent`s (or `loop_detection_plugin` events) as `guard_events`.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.escalation.doom_loop import DoomLoopEvent  # emits .loop_type
from praisonaiagents.eval import LoopEvaluator

# guard_events collected during the agent run
health = LoopEvaluator().run(loop_result, guard_events=guard_events)
if health.doom_loop_fired:
    print(f"Doom-loop fired: {health.reasoning}")
```

***

## Best Practices

<AccordionGroup>
  <Accordion title="Use max_wasted_iterations=0 for optimize-mode CI gates">
    `0` is the strictest setting — any post-threshold iteration marks the loop unhealthy. Use it for optimize-mode CI gates; use `≥1` for exploratory review-mode runs.
  </Accordion>

  <Accordion title="Review mode always reports wasted_iterations=0">
    That is intentional, not a bug — review mode runs all iterations by design, so post-threshold iterations are not "waste".
  </Accordion>

  <Accordion title="Guard events are duck-typed">
    A dict *or* an object works. `type`, `event`, `loop_type`, `is_loop`, and explicit `doom_loop` flags are all recognised — see the accepted string values above.
  </Accordion>

  <Accordion title="Zero LLM calls, safe offline">
    The evaluator only consumes an existing `EvaluationLoopResult`. No external deps, no network — safe in offline CI.
  </Accordion>
</AccordionGroup>

***

## Related

<CardGroup cols={2}>
  <Card title="Evaluation Suite" icon="scale-balanced" href="/docs/features/eval-suite">
    Run all four evaluators as one CI gate
  </Card>

  <Card title="Evaluation Loop" icon="rotate" href="/docs/eval/evaluation-loop">
    The loop this evaluator scores
  </Card>

  <Card title="Loop Detection" icon="rotate" href="/docs/features/doom-loop-detection">
    The guard whose events feed this evaluator
  </Card>

  <Card title="Loop Guard" icon="shield-halved" href="/docs/features/loop-guard">
    Per-turn tool-call guard, complementary safety net
  </Card>

  <Card title="Judge" icon="gavel" href="/docs/eval/judge">
    The underlying judge used by EvaluationLoop
  </Card>

  <Card title="CHL Engineering" icon="ruler-combined" href="/docs/concepts/chl-engineering">
    The Context / Harness / Loop rubric this evaluator scores against.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.