Computer Use Tools

Let agents take screenshots and control mouse, keyboard, and scroll — safely gated behind a human approval callback.

Computer Use tools let an agent take screenshots and control the mouse, keyboard, and scroll — gated by a human approval callback so nothing runs without your say-so.

Quick Start

1. Simple Usage

   Read-only tools never require approval, so an observer agent works out of the box.

   from praisonaiagents import Agent
   from praisonaiagents.tools import computer_screenshot, computer_screen_size
   
   agent = Agent(
       name="observer",
       instructions="Describe what's on screen.",
       tools=[computer_screenshot, computer_screen_size],
   )
   agent.start("What size is the screen and what's on it?")

2. With Configuration

   Register an approval callback, then add the control tools. Every click, type, key, and scroll passes through the callback first.

   from praisonaiagents import Agent
   from praisonaiagents.tools import (
       computer_screenshot, computer_click, computer_type,
       computer_key, computer_scroll, computer_move,
       set_computer_approval,
   )
   
   set_computer_approval(lambda action: input(f"Approve {action}? (y/n) ") == "y")
   
   agent = Agent(
       name="assistant",
       instructions="You can control the computer to help the user.",
       tools=[
           computer_screenshot, computer_click, computer_type,
           computer_key, computer_scroll, computer_move,
       ],
   )
   agent.start("Take a screenshot and describe what's on screen")

How It Works

Configuration Options

Every tool returns a string — errors are reported as messages, never raised.

Tools:

Approval Callback:

set_computer_approval sets a process-wide singleton behind a lock. Call it once and every later tool call in the process uses the same gate. Pass None to revoke — control actions then default back to deny.

Common Patterns

Read-only observer — no approval callback, so the agent can look but not touch.

from praisonaiagents import Agent
from praisonaiagents.tools import computer_screenshot, computer_screen_size

agent = Agent(
    name="observer",
    instructions="Report the screen size and describe what's visible.",
    tools=[computer_screenshot, computer_screen_size],
)
agent.start("Describe the current screen")

Selective auto-approve — allow only screenshots-to-disk and cursor moves, deny clicks and typing. Pattern-match on the action string.

from praisonaiagents import Agent
from praisonaiagents.tools import (
    computer_screenshot, computer_move, computer_click,
    computer_type, set_computer_approval,
)

def allow_safe(action: str) -> bool:
    return action.startswith("screenshot_save(") or action.startswith("move(")

set_computer_approval(allow_safe)

agent = Agent(
    name="safe-driver",
    instructions="Save screenshots and move the cursor to point things out.",
    tools=[computer_screenshot, computer_move, computer_click, computer_type],
)
agent.start("Save a screenshot to /tmp/shot.png and move the cursor to it")

Interactive approval — prompt a human for every control action during local development.

from praisonaiagents import Agent
from praisonaiagents.tools import (
    computer_screenshot, computer_click, computer_type, set_computer_approval,
)

set_computer_approval(lambda action: input(f"Approve {action}? (y/n) ") == "y")

agent = Agent(
    name="assistant",
    instructions="Help the user by controlling the screen.",
    tools=[computer_screenshot, computer_click, computer_type],
)
agent.start("Open the search box and type 'hello'")

Best Practices

Related