Skip to main content
Firecrawl lets an agent scrape and crawl websites into clean, LLM-ready markdown.

Overview

Firecrawl is a powerful web scraping API that converts websites into clean, LLM-ready markdown or structured data.

Installation

pip install "praisonai[tools]"

Environment Variables

export FIRECRAWL_API_KEY="${FIRECRAWL_API_KEY:?Set FIRECRAWL_API_KEY in your shell}"
Get your API key from Firecrawl.

How It Works

Quick Start

1

Simple Usage

from praisonai_tools import FirecrawlTool

# Initialize
firecrawl = FirecrawlTool()

# Scrape a page
result = firecrawl.scrape("https://example.com")
print(result)
2

With Configuration

Use the same tool with an agent — see Usage with Agent below, or pass env vars and options from the sections above.

Usage with Agent

from praisonaiagents import Agent
from praisonai_tools import FirecrawlTool

agent = Agent(
    name="WebScraper",
    instructions="You are a web scraping assistant. Use Firecrawl to extract content.",
    tools=[FirecrawlTool()]
)

response = agent.chat("Scrape the content from https://praison.ai/docs")
print(response)

Available Methods

scrape(url)

Scrape a single URL and get markdown content.
from praisonai_tools import FirecrawlTool

firecrawl = FirecrawlTool()
result = firecrawl.scrape("https://example.com")

# Returns:
# {
#     "url": "https://example.com",
#     "markdown": "# Example Domain\n\nThis domain is...",
#     "metadata": {...}
# }

crawl(url, limit=10)

Crawl a website and get multiple pages.
results = firecrawl.crawl("https://docs.example.com", limit=5)

# Returns list of scraped pages

Configuration Options

firecrawl = FirecrawlTool(
    api_key="your_key",           # Optional: defaults to FIRECRAWL_API_KEY
    formats=["markdown", "html"], # Output formats
    only_main_content=True        # Extract only main content
)

Function-Based Usage

from praisonai_tools import firecrawl_scrape

# Quick scrape without instantiating class
result = firecrawl_scrape("https://example.com")

CLI Usage

# Set API key
export FIRECRAWL_API_KEY=your_key

# Use with praisonai
praisonai --tools FirecrawlTool "Scrape the content from https://example.com"

Error Handling

from praisonai_tools import FirecrawlTool

firecrawl = FirecrawlTool()
result = firecrawl.scrape("https://example.com")

if "error" in result:
    print(f"Error: {result['error']}")
else:
    print(f"Content: {result['markdown'][:500]}")

Common Errors

ErrorCauseSolution
FIRECRAWL_API_KEY not configuredMissing API keySet environment variable
firecrawl not installedMissing dependencyRun pip install firecrawl-py
Rate limitedToo many requestsUpgrade plan or add delays

Best Practices

FirecrawlTool() defaults to the FIRECRAWL_API_KEY env var. Set it in your shell or .env rather than passing api_key= inline.
crawl(url, limit=10) follows links up to the limit. Keep it small so the agent does not pull an entire site into context.
Firecrawl returns HTTP 429 when the plan quota is exceeded. Check for an error key in the result and back off or fall back to another scraper.

Crawl4AI

Open-source crawler

Spider

Fast web crawler

Jina

Reader API