Browser Automation via CDP Bridge

praisonai-browser developer extension that connects to the bridge server for browser automation with multi-layer persistence

Chrome Extension

A Chrome Extension that enables AI-powered browser automation through the Chrome DevTools Protocol (CDP) with multi-layer persistence for reliable connections.

Quick Start

1. Install

   pip install praisonaiagents selenium

2. Create a browser automation agent

   from praisonaiagents import Agent
   
   agent = Agent(
       name="BrowserAgent",
       instructions="Automate browser tasks and extract web content.",
   )
   
   agent.start("Navigate to example.com and extract the main heading")

Architecture Overview

The extension uses a multi-layer architecture to ensure reliable browser automation: The user asks for a browser task; the Chrome extension and agent automate the page via CDP.

Multi-Layer Connection Strategy

Chrome Manifest V3 service workers terminate after 30 seconds of inactivity, which can kill WebSocket connections. We use a three-layer approach to maintain persistent connections:

Layer 1: Offscreen Document (PRIMARY)

The offscreen document is the primary method for maintaining WebSocket connections:

// Service worker creates offscreen document
await chrome.offscreen.createDocument({
    url: 'offscreen.html',
    reasons: [chrome.offscreen.Reason.BLOBS],
    justification: 'Persistent WebSocket connection for bridge server',
});

Why Offscreen Document?

• Lives longer than service workers
• Can maintain WebSocket connections indefinitely
• Auto-reconnects when bridge server restarts
• Sends heartbeats to keep connection alive

Layer 2: Service Worker Bridge (FALLBACK)

If the offscreen document fails, the service worker maintains its own WebSocket:

// Falls back to service worker bridge
bridgeClient = getBridgeClient({
    serverUrl: 'ws://localhost:8765/ws',
    maxReconnectAttempts: 3,
});

When Used:

• Offscreen document creation fails
• Chrome doesn’t support offscreen API
• Temporary connection until offscreen is ready

Layer 3: Content Script Keep-Alive (BACKUP)

Content scripts open persistent ports to keep the service worker alive:

// Content script opens keep-alive port
const port = chrome.runtime.connect({ name: 'keepAlive' });

// Ping every 4.5 minutes (Chrome's 5-minute limit)
setInterval(() => port.postMessage({ type: 'ping' }), 270000);

Internal Message Flow

When using extension mode, messages flow through multiple components:

Message Types

Registering with the Bridge

The offscreen document must send register on open when it connects without an Origin header.

ws.addEventListener("open", () => {
  ws.send(JSON.stringify({ type: "register", client: "extension" }));
});

This applies to the unpacked developer extension that bridges to the Python server. The Chrome Web Store build is standalone, has no WebSocket, and is unaffected. See Role Registration.

Why This Architecture?

The offscreen document holds the actual WebSocket connection because:

• Service workers may terminate after 30s of inactivity
• Offscreen documents persist longer
• All outgoing messages route through offscreen

The background script’s bridgeClient handles:

• Parsing incoming messages
• Calling appropriate handlers (e.g., onStartAutomation)
• Forwarding outgoing messages to offscreen

Browser Automation Engines

Extension Mode (PRIMARY)

Extension mode uses the WebSocket bridge for AI-driven browser automation:

praisonai browser launch "search for AI" --engine extension

How it works:

1. Bridge server starts on localhost:8765
2. Chrome launches with extension loaded
3. Extension connects to bridge via WebSocket
4. AI agent sends actions, extension executes them

Advantages:

• ✅ Full browser context (cookies, sessions)
• ✅ Access to all tabs
• ✅ User-like interactions

CDP Mode (FALLBACK)

CDP mode uses direct Chrome DevTools Protocol for reliable automation:

praisonai browser launch "search for AI" --engine cdp

How it works:

1. Chrome launches with --remote-debugging-port
2. Python connects directly via CDP
3. No extension needed
4. Direct browser control

Advantages:

• ✅ Always reliable
• ✅ No extension required
• ✅ Faster for simple tasks

Browser Subagent

The browser subagent is a specialized AI agent that handles browser interactions:

# Browser subagent usage
result = browser_subagent(
    Task="Navigate to google.com and search for AI",
    TaskName="Google Search",
    RecordingName="search_demo"
)

What Is It?

The browser subagent is a secondary agent with browser-specific tools:

• click – Click elements
• type – Enter text
• navigate – Go to URLs
• scroll – Scroll pages
• screenshot – Capture screenshots
• wait – Wait for elements

How It’s Used

1. Main agent delegates browser tasks to browser subagent
2. Browser subagent uses CDP to control Chrome
3. Results returned to main agent

Is It a Fallback?

No – The browser subagent is the main agent for browser control. It’s used by both:

• Extension mode (via WebSocket bridge)
• CDP mode (direct control)

Technology Stack

Project Structure

praisonai-chrome-extension/
├── src/
│   ├── background/     # Service worker (Layer 2)
│   │   └── index.ts    # Bridge connection, CDP, agents
│   ├── offscreen/      # Offscreen document (Layer 1)
│   │   └── index.ts    # Persistent WebSocket
│   ├── content/        # Content scripts (Layer 3)
│   │   └── index.ts    # Keep-alive ports
│   ├── sidepanel/      # Side panel UI
│   ├── cdp/            # CDP client library
│   ├── bridge/         # WebSocket bridge client
│   └── ai/             # AI modules (Gemini Nano)
├── dist/               # Built extension
└── manifest.json       # Extension manifest

Installation

From Source

1. Clone and build:

git clone https://github.com/MervinPraison/praisonai-chrome-extension.git
cd praisonai-chrome-extension
npm install
npm run build

1. Load in Chrome:
2. Open chrome://extensions
3. Enable “Developer mode”
4. Click “Load unpacked”
5. Select the dist folder

Via CLI

The CLI automatically loads the extension:

praisonai browser launch "search for AI" --engine extension

Manual Load Recovery (Chrome 137+ / Windows)

The CLI’s auto-load can silently fail on Chrome 137+ / Windows, which blocks automated --load-extension. When that happens, launch prints a manual “Load unpacked” flow — load the extension into your daily “Work” Chrome instead.

1. Open chrome://extensions

   Use your normal Chrome (Work profile) and enable Developer mode.

2. Load unpacked

   Click Load unpacked and select the dist/ folder.

3. Verify the connection

   praisonai browser doctor extension
   # expect: ✅ Extension connected to bridge (1 connection(s))

Keyboard Shortcuts

These are the developer extension’s bindings.

Mac users: Use Cmd instead of Ctrl

Configuration

// Extension settings (storage.local)
{
    "bridgeServerUrl": "ws://localhost:8765/ws",
    "preferBridge": true,
    "fallbackToBuiltIn": true,
    "maxSteps": 20
}

Profile Persistence

By default, PraisonAI uses a persistent Chrome profile at ~/.praisonai/browser_profile. This provides:

• ✅ Consistent Extension ID: Same ID across runs for reliable automation
• ✅ Session Persistence: Cookies, login state, and history preserved
• ✅ Faster Startup: No need to re-initialize extension settings

Default Behavior (Persistent)

# Uses ~/.praisonai/browser_profile by default
praisonai browser launch "search for AI" --engine extension

Custom Profile Path

# Use a custom profile directory
praisonai browser launch "search for AI" --chrome-profile /path/to/profile

Temporary Profile

For isolated testing or clean sessions, use --temp-profile:

# Uses a temporary profile (deleted after run)
praisonai browser launch "search for AI" --temp-profile

Troubleshooting

Extension Not Connecting

If you see “Timeout after 30s” errors:

1. Check bridge server:

curl http://localhost:8765/health

1. Check extension console:
2. Go to chrome://extensions
3. Find “PraisonAI Browser Agent”
4. Click “Inspect views: service worker”
5. Look for connection errors

1. Use CDP fallback:

praisonai browser launch "search for AI" --engine cdp

CLI says “No automation steps in 30s”

The praisonai browser run first-step watchdog fired because start_automation was not confirmed delivered to an extension. Verify the bridge sees an extension and that no other client holds it:

curl http://127.0.0.1:8765/health    # expect extension_connections >= 1

Use the side panel or the CLI, not both at once. For extension-free automation, add --engine cdp.

“Receiving End Does Not Exist” Error

This occurs when the offscreen document isn’t ready. The extension now:

• Waits 1 second after creating offscreen document
• Retries 3 times with 500ms delay
• Falls back to service worker bridge

Chrome Instance Management

The CLI only manages its own Chrome instance:

• ✅ Only kills the Chrome window it launched
• ✅ Does NOT kill your manually opened Chrome windows
• ✅ Reuses existing session when possible

Permissions

The developer extension (which connects to the bridge server) requires:

Development

npm run dev      # Development build with watch
npm run build    # Production build
npm run test     # Run tests
npm run lint     # Check code style

Best Practices

Related