Browser Agent Module

AI-powered browser automation using Chrome Extension and PraisonAI agents

from praisonaiagents import Agent

agent = Agent(name="browser-agent", instructions="Browse the web and extract information.")
agent.start("Go to https://example.com and summarise the main content.")

The user states a browsing goal; the Chrome extension and bridge execute actions until the task completes.

Control web browsers with AI agents through a Chrome Extension connected to PraisonAI.

Install

Browser automation lives in the standalone Tier-2 package praisonai-browser.

pip install praisonai-browser            # core
pip install "praisonai-browser[server]"  # FastAPI bridge server
pip install "praisonai-browser[playwright]"  # cross-browser Playwright engine
pip install "praisonai-browser[all]"     # everything

Quick Start

1. Install a browser engine (once)

   playwright install chromium
   # or all engines:
   playwright install
   # Docker / CI (adds OS-level deps):
   playwright install chromium --with-deps

2. Start the bridge server

   praisonai browser start --port 8765 --model gpt-4o

3. Load the Chrome extension

   Open chrome://extensions, enable Developer mode, and load unpacked from praisonai-chrome-extension/dist.

4. Run a goal

   Open the developer extension’s side panel with Ctrl+Shift+P, or run from CLI:

   praisonai browser run "Go to google and search praisonai"

Architecture

Flow: Chrome Extension ↔ WebSocket ↔ Bridge Server ↔ PraisonAI Agent

The system consists of:

• Chrome Extension: Captures page state and executes actions via CDP
• Session entry points: A run can start from either the CLI (praisonai browser run "…") or the developer extension’s side panel (opened with Ctrl+Shift+P). Both send start_session to the bridge; only one may hold the extension at a time — check /health extension_busy before starting a second run. As of PR #4313 the CLI warns instead of silently discarding engine-only options (--vision, --screenshots, --record-video, --max-retries, --headless, --extension, --no-record) when they are set on the default extension engine — see the Engine-only options warning.
• Bridge Server: FastAPI WebSocket server that routes messages to agents
• BrowserAgent: PraisonAI agent that decides actions based on observations
• SessionManager: SQLite-based persistence for session history
• Hybrid Mode: Falls back to on-device Gemini Nano if server unavailable

Session Flow

Session States

Smart Features

Click Fallbacks

When clicks fail, the agent automatically tries:

1. Viewport click using getBoundingClientRect() + scrollIntoView()
2. JavaScript click via element.click()
3. Focus + Enter for buttons

Goal Context & Self-Correction

Every observation sent to the LLM includes:

• Original goal: Always visible to prevent drift
• Action history: Last 5 actions with success/failure status
• Progress notes: Summary of steps completed

Failure Communication

When actions fail, the LLM receives explicit feedback:

⛔ LAST ACTION FAILED!
   Error: All click methods failed for: a.MV3Tnb
   → You MUST try a DIFFERENT approach!

This enables the agent to self-correct and find alternate paths.

CLI Commands

Run Browser Agent

Execute a goal directly from CLI with live progress display:

praisonai browser run "Go to google and search praisonai"
praisonai browser run "Find flights to Paris" --model gpt-4o
praisonai browser run "task" --debug  # Show all WebSocket messages

Options:

• --url, -u: Start URL (default: https://www.google.com)
• --model, -m: LLM model (default: gpt-4o-mini)
• --timeout, -t: Timeout in seconds (default: 120)
• --debug, -d: Debug mode – show all events

Example Output:

🚀 Starting browser agent
   Goal: Go to google and search praisonai
   Model: gpt-4o-mini

Session: 4a703667

Step 0: ▶ TYPE → textarea#APjFqb
        📍 https://www.google.com/

Step 1: ▶ CLICK

Step 2: ▶ CLICK
        📍 https://www.google.com/search?q=praisonai

✅ Task completed!

Scripting run

run returns a stable exit code, so CI and cron jobs can branch on the failure type:

praisonai browser run "Go to google and search praisonai"
case $? in
  0) echo "done" ;;
  2) echo "fix the environment: start the bridge / connect the extension" ;;
  *) echo "task did not complete" ;;
esac

See Browser CLI → Exit codes for the full reference.

If the bridge server is not running

Your browser agent won’t start and you see a friendly bridge-unreachable message. The fix is one command in a second terminal, not a code change.

# Terminal A (leave it running):
#   praisonai browser start --port 8765
#
# Terminal B (your agent):
#   praisonai browser run "Go to google and search praisonai"
#
# Or, from Python, react to the specific error code:
result = run_agent(...)  # or the equivalent programmatic call
if result.get("error") == "bridge_unreachable":
    print("Start the bridge with: praisonai browser start")

run connects to the local bridge server, not the site in --url. When that server is down it prints:

Cannot connect to PraisonAI Browser bridge at ws://localhost:8765/ws

The bridge server is not running. In a separate terminal, start it with:

  praisonai browser start --port 8765

Then verify it is up:
  curl http://localhost:8765/health

Note: this is the local bridge server, not your target site (--url).

Recover with the two-terminal setup, then point to Start Server for the fix:

1. Terminal A — start the bridge

   praisonai browser start --port 8765

   Leave this terminal open; the server blocks while it listens.

2. Terminal B — run the agent again

   praisonai browser run "Go to google and search praisonai"

Launch Browser with Goal

Launch Chrome with the extension and optionally run a goal:

# Just launch Chrome with extension
praisonai browser launch

# Launch and run goal
praisonai browser launch "Go to google and search AI"

# With specific engine
praisonai browser launch "Search for AI" --engine cdp
praisonai browser launch "Search for AI" --engine extension

Options:

• --url, -u: Start URL (default: https://www.google.com)
• --model, -m: LLM model (default: gpt-4o-mini)
• --max-steps: Maximum steps (default: 20)
• --engine: Automation engine: extension, cdp, auto (default: auto)
• --debug, -d: Debug mode with detailed logging
• --record-video: Record video of browser session
• --profile: Enable performance profiling
• --deep-profile: Enable deep profiling with cProfile

Performance Profiling

Track execution time per step to identify bottlenecks:

praisonai browser launch "Go to google, search for AI" --profile

Example Output:

📊 Performance Profile
──────────────────────────────────────────────────────────────────────
Total Time: 16.4s | Steps: 3 | Avg: 5.5s/step

Step |    LLM | Screen | Action | Verify | Stable |  Total
──────────────────────────────────────────────────────────────────────
   0 |   0.0s |   0.0s |   0.0s |   0.0s |   0.0s |   5.1s
   1 |   0.0s |   0.0s |   0.0s |   0.0s |   0.0s |   1.5s
   2 |   0.0s |   0.0s |   0.0s |   0.0s |   0.0s |   3.6s
──────────────────────────────────────────────────────────────────────
Total |   0.0s |   0.0s |   0.0s |   0.0s |   0.0s |  16.4s

Bottlenecks: LLM 0% | Verify 0% | Stable 0%

For deep function-level profiling (cProfile):

praisonai browser launch "goal" --deep-profile

Tab Management

praisonai browser tabs              # List all tabs
praisonai browser tabs --new https://google.com  # Open new tab
praisonai browser tabs --close TAB_ID    # Close tab
praisonai browser tabs --focus TAB_ID    # Focus tab

Navigate

praisonai browser navigate "https://github.com"
praisonai browser navigate "https://praison.ai/docs" --tab TAB_ID

Execute JavaScript

praisonai browser execute "document.title"
praisonai browser execute "document.querySelectorAll('a').length"

Page Inspection (New)

Inspect browser pages without the extension:

# List all open pages
praisonai browser pages

# Get DOM tree
praisonai browser dom <PAGE_ID>

# Read page content as text
praisonai browser content <PAGE_ID>

# Capture console logs
praisonai browser console <PAGE_ID>

# Execute JavaScript
praisonai browser js <PAGE_ID> "document.title"

Automation Engines

Choose different execution engines with --engine:

# Extension mode (default) - requires extension
praisonai browser run "task" --engine extension

# CDP mode - direct Chrome control, no extension needed
praisonai browser run "task" --engine cdp

# Playwright mode - cross-browser, headless support
praisonai browser run "task" --engine playwright

Screenshot

praisonai browser screenshot -o page.png
praisonai browser screenshot --fullpage -o full.png

Start Server

praisonai browser start [OPTIONS]

Options:

• --port, -p: Port to listen on (default: 8765)
• --host, -H: Host to bind to (default: 0.0.0.0)
• --model, -m: LLM model (default: gpt-4o-mini)
• --max-steps: Maximum steps per session (default: 20)
• --verbose, -v: Enable verbose logging

“Already running” behaviour

Starting the server on a port that is already in use exits cleanly instead of failing. Before binding, the CLI probes the port with _port_in_use(host, port). If a server is already there, it prints a friendly message and exits with code 0:

Bridge server already running on port 8765
  Health: http://127.0.0.1:8765/health
  Leave that terminal open; use a second terminal for commands.

If the port slips through the pre-check and binding fails at runtime, the CLI catches the OSError — WinError 10048 on Windows or errno.EADDRINUSE on Linux/macOS — prints the same message, and still exits 0. Any other OSError prints Error: <message> and exits 1.

Windows notes

Startup banners are ASCII (no emoji) and the console script forces stdout/stderr to UTF-8, so cp1252 Windows consoles run praisonai browser start without a UnicodeEncodeError. This works out of the box — no code page changes required. See PraisonAI issue #3094 for background.

Binding to IPv6

Pass an IPv6 literal to --host and it works:

praisonai browser start --host ::1 --port 8765

The duplicate-start check resolves the address family with getaddrinfo, so IPv6 hosts like ::1 are probed on the correct family instead of being silently missed.

List Sessions

praisonai browser sessions [OPTIONS]

Options:

• --status, -s: Filter by status (running, completed, failed)
• --limit, -l: Maximum sessions to show

View History

praisonai browser history <SESSION_ID>

Clear Sessions

praisonai browser clear --status completed --yes

Reload Extension

Reload the Chrome extension after making changes:

praisonai browser reload
praisonai browser reload --port 9222  # Custom Chrome debug port

Health Diagnostics

Your agent is trying to drive the browser, but nothing happens. One command tells you which layer broke:

# In another terminal:
#   praisonai browser doctor extension
#
# Reads the bridge's ground-truth signal (extension_connections in /health).
# CDP :9222 is checked too, but shown as info only — it is empty on your
# daily "Work" Chrome, and that is normal.

Run health checks for the browser automation system:

praisonai browser doctor                          # Run all checks
praisonai browser doctor server                   # Check bridge server
praisonai browser doctor chrome                   # Check Chrome debugging
praisonai browser doctor extension --server-port 8765  # Check bridge connection (Chrome 137+ safe) + CDP :9222 (optional)
praisonai browser doctor db                       # Check session database

Example Output:

Browser Health Check

✅ Server: ok
   Connections: 1
   Sessions: 0

✅ Chrome: Chrome/131.0.6778.85
   WebSocket: ws://127.0.0.1:9222/devtools/browser/...

✅ Extension connected to bridge (1 connection(s))
ℹ️ Extension not in CDP Chrome on port 9222 (normal for daily 'Work' Chrome).

✅ Session database
   Path: ~/.praisonai/browser_sessions.db
   Sessions: 42
   Steps: 387

Python API

Import the public classes from praisonai_browser.

from praisonai_browser import BrowserServer, BrowserAgent

# Start server
server = BrowserServer(port=8765, model="gpt-4o")
server.start()  # Blocks until Ctrl+C / SIGTERM

# Or create agent directly
agent = BrowserAgent(model="gpt-4o")
action = agent.process_observation({
    "task": "Search for AI frameworks",
    "url": "https://google.com",
    "title": "Google",
    "elements": [{"selector": "#search", "tag": "input", "text": ""}]
})

Run Return Shape

When run cannot reach the bridge, it returns a structured result so scripts can detect the failure without parsing text:

{"status": "error", "error": "bridge_unreachable"}

Other failures keep their existing generic error output — behaviour on reachable-but-failing runs is unchanged.

Session Management

from praisonai_browser import SessionManager

manager = SessionManager()

# Create session
session = manager.create_session("Find best restaurants")
print(session["session_id"])

# List sessions
sessions = manager.list_sessions(status="running")

# Get session details with steps
details = manager.get_session(session_id)
for step in details["steps"]:
    print(f"Step {step['step_number']}: {step['action']}")

Hybrid Mode (Extension)

The Chrome Extension supports hybrid mode:

1. Bridge Mode: Connect to PraisonAI server for cloud LLMs
2. Built-in Mode: Use Chrome’s Gemini Nano on-device

If the bridge server is unavailable, it automatically falls back to built-in AI.

Keyboard Shortcuts

These are the developer extension’s bindings.

Supported Actions

Error Detection & Recovery (v1.3+)

The agent automatically detects and recovers from errors:

Detected Errors

• Garbled/duplicated text in input fields
• Wrong page navigation (user or browser interference)
• Failed actions (click not working, submit didn’t fire)
• Blocking elements (popups, consent dialogs, login walls)

Recovery Actions

When errors are detected, the agent will:

1. Set error_detected: true with description
2. Report input_field_value showing actual text visible
3. Use clear_input to fix garbled input
4. Use navigate to return to correct URL if off-track

Step Timestamps

Debug mode now shows elapsed time for each step:

praisonai browser launch "goal" --debug

Output:

[+0.0s] Step 1: type → #APjFqb = "search term" (done=False)
   📝 Input field shows: "search term"
   📊 Progress: 50% [✓ on track]

[+2.3s] Step 2: submit → #APjFqb (done=False)
   📊 Progress: 75% [✓ on track]

[+4.1s] Step 3: done (done=True)
   📊 Progress: 100% [✓ on track]

Performance Optimized

Action delays have been optimized for faster execution:

• Click: 200ms (was 500ms)
• Submit: 300ms (was 500ms)
• Search: 400ms (was 1000ms)

WebSocket Protocol

Connect to ws://localhost:8765/ws and send/receive JSON messages:

// Start session
{"type": "start_session", "goal": "Find flights to Paris", "model": "gpt-4o"}

// Send observation
{"type": "observation", "session_id": "...", "task": "...", 
 "url": "...", "elements": [...]}

// Receive action
{"type": "action", "action": "click", "selector": "#search", 
 "thought": "Clicking search button"}

Environment Variables

Best Practices

Related