Runtime Resolution

General runtime-resolution subsystem for resolving agent runtimes at turn-time

The runtime-resolution subsystem maps agent and model references to concrete runtime instances at turn-time, enabling dynamic model switching and custom routing logic.

from praisonaiagents import Agent

researcher = Agent(
    name="Researcher",
    instructions="Research the topic and summarise it",
    llm="gpt-4o-mini",
)
writer = Agent(
    name="Writer",
    instructions="Write a polished article",
    llm="gpt-4o-mini",
    handoffs=[researcher],
)

researcher.llm = "claude-3-sonnet"  # picked up on the next turn
writer.start("Research and write about ocean currents")

The user changes model or runtime references on an agent; the resolver picks the concrete runtime on the next turn.

Quick Start

1. Simple Usage

   Update the target model at any time — the next invocation reads the live llm value.

   from praisonaiagents import Agent
   
   analyst = Agent(name="Analyst", instructions="Analyse data", llm="gpt-4o-mini")
   coordinator = Agent(name="Coordinator", instructions="Coordinate", handoffs=[analyst])
   
   coordinator.start("Quick summary of Q1 sales")
   
   analyst.llm = "gpt-4o"
   coordinator.start("Full analysis of Q1 vs Q2 with trend forecasting")

2. With Configuration

   Clear the runtime cache after credential rotation or register a custom resolver:

   from praisonaiagents import Agent
   from praisonaiagents.runtime import clear_runtime_cache, set_global_resolver
   from praisonaiagents.runtime.resolve import RuntimeResolver, LLMRuntimeWrapper
   from praisonaiagents.llm.llm import LLM
   
   clear_runtime_cache()  # force fresh resolution
   
   class MyResolver(RuntimeResolver):
       def supports_model(self, model_ref: str) -> bool:
           return model_ref.startswith("my-model-")
   
       def resolve(self, agent_id, model_ref, session_ctx, **kwargs):
           llm = LLM(model="gpt-4o-mini", api_base="https://my.endpoint/v1")
           return LLMRuntimeWrapper(llm=llm, model_ref=model_ref, agent_id=agent_id)
   
   set_global_resolver(MyResolver())
   
   agent = Agent(name="MyAgent", instructions="Help users", llm="my-model-fast")
   agent.start("Hello!")

How It Works

The subsystem reads the agent’s current llm (or model) attribute at invocation time, not at construction time.

Configuration Options

SessionContext

Passed to resolve_runtime to scope caching and track depth.

Cache constants

Cache keys use the format "{session_id}:{agent_id}:{model_ref}" — caches are session-isolated so different conversations never share runtimes.

Background cleanup thread (PR #5219)

The cleanup daemon runs every _cleanup_interval seconds to evict expired entries, but it is not started at module import. As of PR #5219 (fixes #5162) it starts lazily on the first cache write — inside resolve_runtime, under _runtime_cache_lock — so two concurrent first-writers cannot each spawn a redundant daemon.

Common Patterns

Mid-conversation model swap

from praisonaiagents import Agent

analyst = Agent(name="Analyst", instructions="Analyse data", llm="gpt-4o-mini")
coordinator = Agent(name="Coordinator", instructions="Coordinate", handoffs=[analyst])

# First few turns use gpt-4o-mini
coordinator.start("Quick summary of Q1 sales")

# Upgrade to a more capable model for a detailed report
analyst.llm = "gpt-4o"
coordinator.start("Full analysis of Q1 vs Q2 with trend forecasting")

Force cache refresh

from praisonaiagents.runtime import clear_runtime_cache

# After rotating API keys or changing model config
clear_runtime_cache()

agent.start("Continue with the updated model settings")

Introspect resolved runtimes

from praisonaiagents.runtime import get_runtime_cache

cache = get_runtime_cache()
total = sum(len(entries) for entries in cache.values())
print(f"Active runtimes: {total} across {len(cache)} sessions")

Best Practices

Public API

All names are importable from praisonaiagents.runtime:

from praisonaiagents.runtime import (
    resolve_runtime,
    SessionContext,
    RuntimeProtocol,
    get_runtime_cache,
    clear_runtime_cache,
    set_global_resolver,
)

resolve_runtime

def resolve_runtime(
    agent_id: str,
    model_ref: str,
    session_ctx: SessionContext,
    **kwargs,
) -> AgentRuntimeProtocol:
    ...

The main entry point. Checks the TTL cache first; creates and caches a new runtime if none exists or the entry expired.

RuntimeProtocol / AgentRuntimeProtocol

Protocols that custom runtimes must satisfy. Only relevant when building a custom resolver.

class RuntimeProtocol(Protocol):
    def execute(self, prompt: str, **kwargs) -> Any: ...
    async def aexecute(self, prompt: str, **kwargs) -> Any: ...
    @property
    def model_ref(self) -> str: ...
    @property
    def provider(self) -> str: ...

class AgentRuntimeProtocol(RuntimeProtocol):
    @property
    def supports_streaming(self) -> bool: ...
    @property
    def supports_tools(self) -> bool: ...

Related