Reclaim Stray Sandboxes

Find and stop sandboxes left behind by a crashed or killed run

Sandboxes normally self-destruct when your run ends. A crashed script, a SIGKILL, or a laptop that slept before teardown can leave one behind β€” burning your Docker daemon, your E2B credits, or your Modal quota. Two commands find and reclaim them.

Diagram

πŸ’₯ Crashed run

πŸ“¦ Stray sandbox

⚑ managed ps

⚑ managed stop

βœ… Reclaimed

Quick Start

  1. See what is still running
    bash
    praisonai managed ps

    If nothing is up:

    text
    No running sandboxes.

    If something is up:

    text
    INSTANCE ID                PROVIDER   STATUS     UPTIME   IMAGE
    ------------------------------------------------------------------------------
    docker_5afaa3bfa704        docker     running    42s      python:3.12-slim
    
    1 running. Stop with: praisonai managed stop <instance-id>
  2. Reclaim it
    bash
    # One
    praisonai managed stop docker_5afaa3bfa704
    
    # All (across every provider)
    praisonai managed stop --all
    
    # All from one provider only
    praisonai managed stop --all --provider docker
  3. Confirm it is gone
    bash
    praisonai managed ps
    # β†’ No running sandboxes.

How It Works

Diagram
Provider (Docker/E2B/…)praisonai managedUserProvider (Docker/E2B/…)praisonai managedUsermanaged pslist by praisonai=managed labelrunning instancestable of sandboxesmanaged stop <id>shut down instancestoppedreclaimed

Docker containers are found cross-process by the praisonai=managed label and the praisonai_<id> name, so containers started by an earlier script are visible from a fresh CLI.


Behaviour change: the silent leak on shutdown

Two collaborating fixes close the leak:

  1. SyncComputeProvider._offload() falls back to inline execution when the executor is gone β€” it catches a RuntimeError whose message contains "interpreter shutdown" or "no running event loop" and calls the sync function directly (blocking is fine: there is no event loop left to starve). Unrelated RuntimeErrors still propagate.
  2. _shutdown_orphan() in tools_placement.py now logs at warning level (was debug), so a genuine teardown failure is visible instead of buried. SharedCompute.shutdown() re-raises the underlying exception after clearing instance_id, so the caller can log it.

If you ran an older version, reclaim any accumulated leaks with praisonai managed ps / managed stop (and docker container prune for stopped-but-not-removed containers).


…and the run_on= finalizer that was never registered (PR #4109)

The PR #4107 fix above only fires when a finalizer actually runs. On ComputeManagedAgent β€” the backend behind run_on= β€” no finalizer had ever been registered, so the release path was never invoked and the instance survived the whole process.

Your run_on= instance is now reclaimed when your script exits β€” no code change:

python
from praisonaiagents import Agent

agent = Agent(name="builder", instructions="You build things.", run_on="docker")
agent.start("Write a script that prints the first 10 primes, then run it")
# When the script exits, the container is reclaimed automatically.

Verified before/after: containers left after a run_on= script exits went from 1 β†’ 0; an ordinary run is still 42 β†’ 42.

The mechanism is a weakref.finalize registered inside _ensure() right after provision() succeeds:

python
# praisonai/integrations/compute_managed_agent.py (inside _ensure())
self._finalizer = weakref.finalize(self, _release, provider, self._instance, self._place)
  • It runs on garbage collection and at interpreter exit β€” not one or the other.
  • It deliberately does not close over the backend self β€” closing over self would keep the backend alive and prevent the very collection it waits for.

_release(provider, instance_id, place) is a module-level function that shuts the instance down safely on any thread. As of PR #5348 it routes through the wrapper’s shared AsyncBridge instead of spinning a fresh loop per teardown:

  • Shared bridge, never a fresh loop. It submits the shutdown coroutine to praisonai._async_bridge.current_bridge().submit(_shutdown()) β€” one long-lived background loop, never an asyncio.run loop per teardown and never a per-teardown ThreadPoolExecutor.
  • Fire-and-forget during live GC. The future is not .result()-ed on the caller’s thread, so a GC-triggered release inside async workflow code never pins the caller’s event loop for the cloud round-trip. Completion is logged via add_done_callback.
  • Blocking (bounded) at interpreter exit. When sys.is_finalizing() is True, _release blocks on fut.result(timeout=_RELEASE_EXIT_TIMEOUT) so the cloud instance is actually reclaimed before the bridge’s own atexit teardown cancels the in-flight shutdown or its daemon loop thread is killed. sys.is_finalizing() is only true during shutdown, so live GC never takes the blocking path.
  • Bridge unavailable β†’ logged as leaked. If the bridge is poisoned or already gone at interpreter exit, the instance is logged as leaked at warning level instead of raising.

Environment Variables

Env VarDefaultDescription
PRAISONAI_COMPUTE_RELEASE_EXIT_TIMEOUT10Seconds _release() will block on the shared bridge at interpreter exit before giving up. Long enough for a normal provider shutdown round-trip, short enough that a hung provider cannot stall process exit indefinitely. Read once at import; non-numeric values fall back to 10.0. Only affects the interpreter-exit path β€” live-GC releases stay fire-and-forget.
bash
# Give a slow cloud provider more room to reclaim on exit
export PRAISONAI_COMPUTE_RELEASE_EXIT_TIMEOUT=30

PR #4107 made the release path survive interpreter shutdown; PR #4109 made sure the release path is actually invoked on run_on=; PR #5348 routed that release through the shared bridge and added the bounded exit-time wait. Together they close the container-leak story end to end for both sandbox= and run_on=.


When a provider can’t be queried

SituationBehaviour
Provider not installed / not configured (no E2B key, Docker daemon down)Skipped silently. Absence of credentials is a normal state.
Explicit --provider foo for an unknown providerExits 1 with a clear message.
Provider is available but listing failsSurfaced under errors: in --json, printed at the bottom of the text output, exit code 1. stop --all never reports a clean sweep when some providers could not be queried.

From a script

bash
# JSON output for scripting
praisonai managed ps --json
json
{
  "sandboxes": [
    {
      "provider": "docker",
      "instance_id": "docker_5afaa3bfa704",
      "status": "running",
      "endpoint": "docker://5afaa3bfa704",
      "created_at": 1755253800.0,
      "metadata": {"image": "python:3.12-slim", "name": "praisonai_5afaa3bfa704"}
    }
  ],
  "errors": []
}

Exit code is 0 when the query is clean, 1 if any provider errored.


Best Practices

Run managed ps after any crash

Any time a script that used run_on= (or per-agent compute=) crashed or you killed it, run managed ps before starting a new run. Costs stack quietly.

Wire stop –all into your exit hooks

For long-running dev sessions, wire praisonai managed stop --all into your shell’s exit trap or your IDE’s on-close hook so nothing is left behind when you close the laptop.

local is not scanned β€” on purpose

managed ps scans docker, e2b, modal, daytona, flyio, tenki. local runs on this machine and has nothing to reclaim.

Since PR #4071, this list is derived from the compute registry rather than hardcoded, so a compute backend contributed by a plugin is listed and stopped automatically β€” no extra wiring. local, subprocess, sandlock, ssh, native, and novita are excluded on purpose. See Bringing your own compute place.


Related