Praison AI

Run Until • AI Agent SDK

run_until: Run agent iteratively until output meets quality criteria.

run_until

Method

</div>

This is a method of the Agent class in the agent module.

Run agent iteratively until output meets quality criteria.

This method implements the “Ralph Loop” pattern: run agent → judge output → improve based on feedback → repeat until threshold met.

When `goal is provided, delegates to the tool-using goal loop (:meth:run_goal`) so the acceptance-criteria completion judge gates a real tool-iteration loop (rather than re-generating a whole answer).

Signature

python
def run_until(prompt: str, criteria: str, threshold: float, max_iterations: int, mode: str, on_iteration: Optional[Callable[[Any], None]], verbose: bool, goal: Optional[str], goal_criteria: Optional[Any], judge_model: Optional[str]) -> 'EvaluationLoopResult'

Parameters

ParamField str

The prompt to send to the agent

ParamField str

Evaluation criteria for the Judge (e.g., “Response is thorough”)

ParamField float

Score threshold for success (default: 8.0, scale 1-10)

ParamField int

Maximum iterations before stopping (default: 5)

ParamField str

“optimize” (stop on success) or “review” (run all iterations)

ParamField Optional

Optional callback called after each iteration

ParamField bool

Enable verbose logging

ParamField Optional[str]

Optional goal text — enables the tool-using goal loop

ParamField Optional

Optional structured GoalCriteria for the goal loop

ParamField Optional[str]

Optional independent judge model for the goal loop

Returns

ResponseField 'EvaluationLoopResult'

when `goal is provided this delegates to the tool-using goal loop and returns an :class:~praisonaiagents.agent.autonomy.AutonomyResult instead (success/output/completion_reason), which is a different shape from the default EvaluationLoopResult`.

Usage

python
agent = Agent(name="analyzer", instructions="Analyze systems")
    result = agent.run_until(
        "Analyze the auth flow",
        criteria="Analysis is thorough and actionable",
        threshold=8.0,
    )
    print(result.final_score)  # 8.5
    print(result.success)      # True

Uses

  • GoalCriteria
  • run_goal
  • EvaluationLoop
  • loop.run

Source

View on GitHub

praisonaiagents/agent/agent.py at line 5697