Browser Agent Deep Dive

Comprehensive guide to browser automation modes, APIs, and integration patterns

from praisonaiagents import Agent

agent = Agent(name="browser-agent", instructions="Browse the web and complete tasks in Chrome.")
agent.start("Search for PraisonAI documentation and summarise the install steps.")

The user describes a web goal; the browser agent drives Chrome via extension, CDP, or Playwright.

Browser Agent Deep Dive

This guide explains how PraisonAI browser automation works under the hood, covering all execution modes, APIs, and integration patterns.

Package Layout

Heavy browser code lives in the standalone Tier-2 package praisonai_browser; the agents tier keeps only lightweight protocol types.

Install with pip install praisonai-browser. Import from praisonai_browser; from praisonai.browser import … still works via the shim.

Environment Variables

Environment variables tune the bridge server’s security posture and extension-readiness behaviour before a run starts.

The skip only triggers on the truthy values above. PRAISONAI_BROWSER_SKIP_EXTENSION_WAIT=false (or 0, no, off, or any inherited value) leaves the readiness check active.

Testing without a live extension

1. Mock the websocket layer

   Patch the websocket layer (unittest.mock, pytest-asyncio, etc.) so the bridge server is never reached.

2. Enable the skip

   import os
   
   os.environ["PRAISONAI_BROWSER_SKIP_EXTENSION_WAIT"] = "1"

   Any of 1, true, yes, on (case-insensitive) enables the skip. Anything else — including false / 0 — leaves the readiness check active.

3. Run against the mock

   The readiness poll used to block on the aiohttp /health loop before the mock got a chance. With the skip set, the run hits the mock immediately.

Quick Start

1. Extension mode (default)

   praisonai browser run "Search for PraisonAI on Google"

2. CDP mode (headless)

   # Start Chrome with remote debugging, then:
   praisonai browser run "Search for AI" --engine cdp --record-video

3. Playwright mode

   praisonai browser run "Search for AI" --engine playwright --headless

Architecture Overview

Instruction Flow (Extension Mode)

This diagram shows exactly how your instruction flows through the system:

Key Points

1. User → CLI: User provides goal via praisonai browser run "goal"
2. CLI → Server: CLI connects over WebSocket, sends start_session
3. Server → Extension: Server forwards start_automation to Chrome extension
4. Extension → Page: Extension uses CDP to capture page state (DOM, screenshot)
5. Extension → Server: Sends observation with page details
6. Server → Agent: Passes observation to BrowserAgent
7. Agent → LLM: Agent builds prompt with goal + context, gets action from LLM
8. Server → Extension: Forwards action (click, type, scroll, etc.)
9. Extension → Page: Executes action via CDP
10. Loop: Repeats until agent returns done: true or timeout

Execution Modes

Engine × Option Compatibility

praisonai browser run accepts the same 14 options on every engine, but most only reach the wire on --engine cdp. On the default extension engine only goal, model, and max-steps are forwarded; playwright and hybrid additionally honour url but not the CDP-only extras (--vision, --screenshots, --record-video, --max-retries, --no-record). As of PR #4313 the CLI warns instead of silently discarding an engine-only flag.

See the full Engine × Option matrix in the CLI reference for the authoritative per-flag breakdown.

Comparison Table

Extension Mode (Default)

How It Works

Backend APIs

CLI Usage

# Basic usage (extension mode is default)
praisonai browser run "Search for PraisonAI on Google"

# With options
praisonai browser run "task" \
    --url https://google.com \
    --model gpt-4o \
    --timeout 120 \
    --debug

Programmatic Usage

from praisonai_browser import BrowserServer, BrowserAgent

# Start server
server = BrowserServer(port=8765, model="gpt-4o-mini")
server.start_background()  # Runs in thread

# Create agent (requires extension to be connected)
agent = BrowserAgent(model="gpt-4o-mini", session_id="my-session")

# Agent is called via WebSocket, not directly
# Extension sends observations → Server calls agent → Extension executes actions

Limitations

CDP Mode

How It Works

Backend APIs

Retry & alternative-selector strategies

When _execute_action reports success=false, error="selector matched no element: …", CDPBrowserAgent.run() retries with three alternative selectors before propagating the failure to the LLM. See Browser CLI → Retry & selector fallback for the full flow diagram.

Fixed in PR #4312: before the fix, missing selectors returned success=true, so the retry loop was unreachable for the one failure it was written to handle.

Prerequisites

# Start Chrome with remote debugging
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
    --remote-debugging-port=9222 \
    --user-data-dir=/tmp/chrome-debug

CLI Usage

# Use CDP mode explicitly
praisonai browser run "Search for AI" --engine cdp

# CDP-specific commands (no extension needed)
praisonai browser pages              # List all tabs
praisonai browser dom <PAGE_ID>      # Get DOM tree
praisonai browser content <PAGE_ID>  # Read page text
praisonai browser js <PAGE_ID> "document.title"  # Execute JS
praisonai browser console <PAGE_ID>  # Capture logs

Programmatic Usage

from praisonai_browser import CDPBrowserAgent
from praisonai_browser.cdp_utils import get_pages, execute_js, get_dom

# Direct CDP control
async def example():
    # List pages
    pages = await get_pages(port=9222)
    for page in pages:
        print(f"{page.id}: {page.title}")

    # Execute JavaScript
    result = await execute_js(pages[0].id, "document.title")
    print(f"Title: {result}")

    # Get DOM
    dom = await get_dom(pages[0].id, depth=3)

    # Read page content
    from praisonai_browser.cdp_utils import read_page
    content = await read_page(pages[0].id)

# Run full automation
async def automate():
    agent = CDPBrowserAgent(
        port=9222,
        model="gpt-4o-mini",
        headless=False,
    )
    result = await agent.run("Search for PraisonAI on Google")
    print(result)

Sync Wrappers

from praisonai_browser.cdp_utils import (
    get_pages_sync,
    execute_js_sync,
    get_dom_sync,
    read_page_sync,
    get_console_sync,
)

# Synchronous usage
pages = get_pages_sync(port=9222)
title = execute_js_sync(pages[0].id, "document.title")

Playwright Mode

How It Works

Backend APIs

Prerequisites

# Install Playwright
pip install playwright

# Install browsers
playwright install chromium
playwright install firefox  # Optional
playwright install webkit   # Optional

CLI Usage

# Use Playwright mode
praisonai browser run "Search for AI" --engine playwright

# Headless mode
praisonai browser run "task" --engine playwright --headless

Programmatic Usage

from praisonai_browser.playwright_agent import PlaywrightBrowserAgent

async def example():
    agent = PlaywrightBrowserAgent(
        model="gpt-4o-mini",
        browser_type="chromium",  # or "firefox", "webkit"
        headless=True,
    )

    result = await agent.run(
        goal="Search for PraisonAI on Google",
        start_url="https://google.com",
    )
    print(result)

Page Inspection Commands

These commands work via CDP (Chrome must be running with --remote-debugging-port=9222).

CLI Reference

Python API

from praisonai_browser.cdp_utils import (
    get_pages,      # async: List[PageInfo]
    get_dom,        # async: Dict (DOM tree)
    read_page,      # async: str (page text)
    get_console,    # async: List[Dict] (log entries)
    execute_js,     # async: Any (JS result)
    wait_for_element,  # async: bool
)

# Synchronous versions also available:
# get_pages_sync, get_dom_sync, read_page_sync, etc.

Agent Memory & Sessions

Session Isolation

Each browser session now uses Agent’s built-in session management:

from praisonai_browser import BrowserAgent

# Session is auto-generated or can be provided
agent = BrowserAgent(
    model="gpt-4o-mini",
    session_id="my-unique-session",  # For memory isolation
)

# Reset for new session (clears chat_history)
agent.reset(new_session_id="new-session-id")

Memory Flow

Common Patterns

Sequential Tasks (CDP Mode)

import asyncio
from praisonai_browser import CDPBrowserAgent

async def run_multiple_tasks():
    agent = CDPBrowserAgent(port=9222, model="gpt-4o-mini")

    tasks = [
        "Search for AI on Google",
        "Go to GitHub and search for praisonai",
        "Navigate to praison.ai/docs",
    ]

    for task in tasks:
        result = await agent.run(task)
        print(f"✅ {task}: {result['status']}")
        await asyncio.sleep(2)  # Pause between tasks

asyncio.run(run_multiple_tasks())

Headless Screenshot

from praisonai_browser.playwright_agent import PlaywrightBrowserAgent

async def take_screenshot():
    agent = PlaywrightBrowserAgent(
        headless=True,
        browser_type="chromium",
    )

    result = await agent.run(
        "Go to praison.ai/docs and take a screenshot",
        start_url="https://praison.ai/docs",
    )

Page Scraping

from praisonai_browser.cdp_utils import get_pages, read_page, execute_js

async def scrape_page():
    pages = await get_pages()

    # Find the right page
    target = next(p for p in pages if "google" in p.url.lower())

    # Get text content
    content = await read_page(target.id)

    # Or execute custom JS
    links = await execute_js(
        target.id,
        """
        Array.from(document.querySelectorAll('a'))
            .map(a => ({href: a.href, text: a.textContent.trim()}))
            .slice(0, 10)
        """
    )
    return links

Troubleshooting

Debug CLI Commands

# Check server health
praisonai browser doctor

# List all sessions with step counts
praisonai browser sessions --limit 10

# View session history (step-by-step)
praisonai browser history <session_id>

# Reload extension after code changes
praisonai browser reload

# Run with debug mode
praisonai browser run "goal" --debug

Session Tracking

Both agent-side and server-side sessions are tracked in the same SQLite database:

# Database location
~/.praisonai/browser_sessions.db

# Tables:
#   sessions: session_id, goal, status, started_at, current_url
#   steps: session_id, step_number, observation, action, thought, created_at

Writing Your Own Bridge Client

Only sockets that connect to ws://127.0.0.1:8765/ws without an Origin header can claim a role with register.

Re-registering as cli is blocked once the socket has a session in flight. Details: Role Registration.

Extension Mode Issues

CDP Mode Issues

Playwright Mode Issues

Chrome Extension Console Logs

1. Go to chrome://extensions
2. Find “PraisonAI Browser Agent”
3. Click “Service worker” link to open DevTools
4. Check Console for [PraisonAI] and [Bridge] logs

Manual Extension Load (Chrome 137+ / Windows)

Chrome 137+ on Windows blocks the automated --load-extension flag praisonai browser launch uses, so the debug profile opens empty. Load the extension into your daily Chrome once and it just works.

# Try the automated launch first — it may still work depending on Chrome build
praisonai browser launch

When auto-load fails, launch prints the exact Load unpacked path plus a curl verification step instead of a generic error. Follow the recipe below.

1. Open your daily Chrome

   Use your normal Chrome (Work profile) — not a fresh debug profile.

   chrome://extensions

2. Enable Developer mode

   Toggle Developer mode on (top-right).

3. Load the dist folder

   Click Load unpacked and select the dist path printed by praisonai browser launch (typically ~/.praisonai/browser/extension/dist).

4. Confirm the bridge connection

   Open the PraisonAI side panel and confirm it connects to ws://127.0.0.1:8765/ws.

   If you launched with --no-server, start the bridge first:

   praisonai browser start

   Check /health directly (expect extension_connections >= 1):

   curl http://127.0.0.1:8765/health

   A connected extension returns:

   {
     "status": "ok",
     "connections": 1,
     "extension_connections": 1,
     "sessions": 0,
     "extension_busy": false,
     "active_session_id": null
   }

   extension_busy and active_session_id (added in PraisonAI #3095 hardening) show whether a session currently owns the extension — a healthy-and-idle bridge reports extension_busy: false. Servers older than that commit omit both fields; treat missing as false / null.

5. Verify

   praisonai browser doctor extension
   # expect: ✅ Extension connected to bridge (1 connection(s))

Common Log Patterns

# Successful click:
[Bridge] Clicking element: #submit

# Failed click:
[Bridge] Click failed: All click methods failed for: #submit

# Session cleanup:
[PraisonAI] Cleaning up previous session (tab 12345)...

# Observation sent:
[PraisonAI] Step 0: https://google.com

Action Verification

All actions return { success, error }:

• success=true: CDP operation completed
• success=false: Error message indicates what failed

The error is sent in the next observation, allowing the LLM to retry or try alternative actions.

Selector misses on click / type / clear_input show as error: "selector matched no element: <selector>" and trigger the alternative-selector strategies before reaching the LLM.

Best Practices

Related