Overview
Firecrawl is a powerful web scraping API that converts websites into clean, LLM-ready markdown or structured data.Installation
Environment Variables
How It Works
Quick Start
Usage with Agent
Available Methods
scrape(url)
Scrape a single URL and get markdown content.crawl(url, limit=10)
Crawl a website and get multiple pages.Configuration Options
Function-Based Usage
CLI Usage
Error Handling
Common Errors
| Error | Cause | Solution |
|---|---|---|
FIRECRAWL_API_KEY not configured | Missing API key | Set environment variable |
firecrawl not installed | Missing dependency | Run pip install firecrawl-py |
Rate limited | Too many requests | Upgrade plan or add delays |
Best Practices
Let FIRECRAWL_API_KEY come from the environment
Let FIRECRAWL_API_KEY come from the environment
FirecrawlTool() defaults to the FIRECRAWL_API_KEY env var. Set it in your shell or .env rather than passing api_key= inline.Limit crawl scope
Limit crawl scope
crawl(url, limit=10) follows links up to the limit. Keep it small so the agent does not pull an entire site into context.Handle rate limits
Handle rate limits
Firecrawl returns HTTP 429 when the plan quota is exceeded. Check for an
error key in the result and back off or fall back to another scraper.Related Tools
Crawl4AI
Open-source crawler
Spider
Fast web crawler
Jina
Reader API

