Skip to main content
AgentQL lets an agent extract structured data from any webpage using a natural-language prompt.

​
Overview

AgentQL is a tool that allows you to extract structured data from webpages using AI Agents. 
pip install langchain_agentql langchain-community

export AGENTQL_API_KEY="${AGENTQL_API_KEY:?Set AGENTQL_API_KEY in your shell}"

from praisonaiagents import Agent, AgentTeam
from langchain_agentql.tools import ExtractWebDataTool
from dotenv import load_dotenv

import os

os.environ["AGENTQL_API_KEY"] = os.getenv('AGENTQL_API_KEY')

def extract_web_data_tool(url, query):
    agentql_tool = ExtractWebDataTool().invoke(
        {
            "url": url,
            "prompt": query,
        },)
    return agentql_tool

# Create agent with web extraction instructions
orchestration_agent = Agent(
    instructions="""Extract All 37 products from the url https://www.colorbarcosmetics.com/bestsellers along with its name, overview, description, price and additional information by recursively clicking on each product""",
    tools=[extract_web_data_tool]
)

# Initialize and run agents
agents = AgentTeam(agents=[orchestration_agent])
agents.start()

​
How It Works

​
Getting Started

1

Simple Usage

  1. Get your AgentQL API key from AgentQL Dashboard
  2. Set the API key in your environment variables
  3. Install the required dependencies
  4. Use the example code to start extracting structured data
2

With Configuration

Use the same tool with an agent — see Usage with Agent below, or pass env vars and options from the sections above.

​
Best Practices

Set AGENTQL_API_KEY in your shell or .env and read it with os.getenv. Never commit the key in code.
AgentQL extracts exactly what the prompt asks for. Name every field you need (name, price, description) so the returned schema is predictable and token-efficient.
Pages change and requests can time out. Wrap ExtractWebDataTool().invoke(...) in try/except so the agent can retry or report a clean message instead of crashing.

Firecrawl

Web scraping

Crawl4AI

AI web crawler

GitHub

Repository automation