Building an Agentic Web Research Pipeline with Browser-Use and Structured Outputs

Introduction

AI agents capable of autonomously browsing the web, extracting data, and returning structured results are transforming how we gather and process online information. Browser-Use — an open-source Python library with over 79,000 GitHub stars — lets AI agents control a real web browser the same way a human does: opening pages, clicking buttons, typing into forms, and reading rendered content GitHub. Combined with Pydantic models for structured output validation, you can build a reliable, type-safe research pipeline that goes from a natural-language query to a validated dataset without writing a single CSS selector or XPath expression.

This guide walks through constructing exactly such a pipeline end-to-end, covering environment setup, schema design, agent configuration, and production best practices.

Prerequisites

Before diving in, make sure you have:

• Python 3.11+ — Browser-Use relies on modern async patterns and Playwright bindings PyPI
• An LLM API key — OpenAI, Anthropic, Google Gemini, or any OpenAI-compatible provider. The library ships wrapper classes like ChatOpenAI, ChatAnthropic, and ChatBrowserUse Docs: Supported Models
• Playwright browsers — installed via the Playwright CLI after the pip install

Step 1: Installation and Project Setup

Create a fresh project and install the dependencies:

mkdir research-pipeline
cd research-pipeline
uv venv --python 3.12
source .venv/bin/activate
uv pip install browser-use playwright python-dotenv
playwright install chromium

Browser-Use uses Playwright under the hood to drive a Chromium instance Docs: Quickstart. The playwright install chromium command downloads the browser binary.

Next, create a .env file with your LLM provider key:

OPENAI_API_KEY=sk-...

Or for Anthropic:

ANTHROPIC_API_KEY=sk-ant-...

Step 2: Define Your Data Schema with Pydantic

The core idea is to tell the agent what shape the final answer should take. Browser-Use accepts a Pydantic BaseModel as the output_model_schema parameter. The agent’s LLM is instructed to return JSON matching this schema, and Browser-Use validates the response automatically Docs: Output Format.

For a web research pipeline that compiles a report from multiple sources, define a nested model:

from pydantic import BaseModel, Field
from typing import List
from datetime import datetime


class Source(BaseModel):
    """A single reference source used in the research."""
    title: str = Field(description="The page title or article headline")
    url: str = Field(description="Full URL of the source")
    relevance: str = Field(description="Why this source is relevant to the query")


class ResearchFinding(BaseModel):
    """A specific finding extracted from a source."""
    claim: str = Field(description="The extracted fact or claim")
    source_url: str = Field(description="URL where this claim was found")
    confidence: str = Field(description="high / medium / low")


class ResearchReport(BaseModel):
    """Final structured research report."""
    query: str = Field(description="The original research query")
    summary: str = Field(description="2-3 paragraph synthesis of findings")
    findings: List[ResearchFinding] = Field(description="Key findings from research")
    sources: List[Source] = Field(description="All sources consulted")
    generated_at: str = Field(
        default_factory=lambda: datetime.now().isoformat()
    )

The Field(description=...) annotations are critical: they become part of the prompt that tells the LLM what data to extract in each field Pydantic Docs. Well-written descriptions dramatically improve output quality.

Step 3: Build the Agent with Structured Output

Now wire everything together. The Agent class from browser_use is the main entry point GitHub: AGENTS.md.

import asyncio
import os
from dotenv import load_dotenv
from browser_use import Agent, Browser, BrowserConfig
from langchain_openai import ChatOpenAI

load_dotenv()


async def main():
    # Choose your LLM
    llm = ChatOpenAI(
        model="gpt-4o",
        temperature=0.1,  # Lower temperature = more deterministic
    )

    # Optional: configure a persistent browser to stay logged in
    browser = Browser(
        config=BrowserConfig(
            headless=False,  # Set True for server deployment
            disable_security=True,
        )
    )

    task = """
    1. Go to https://news.ycombinator.com
    2. Find the top 5 stories and click each one
    3. For each story page, extract the key claims and arguments
    4. Compile findings into the ResearchReport format
    """

    agent = Agent(
        task=task,
        llm=llm,
        browser=browser,
        output_model_schema=ResearchReport,  # <-- The Pydantic model
        use_vision=True,                     # Screenshot-based reasoning
        max_failures=3,                      # Retry on errors
    )

    history = await agent.run()
    result = history.structured_output()  # Returns ResearchReport instance

    # Use the typed result
    print(f"Summary: {result.summary}")
    print(f"Sources found: {len(result.sources)}")
    for finding in result.findings:
        print(f"  - {finding.claim} ({finding.confidence})")


asyncio.run(main())

Key points about this setup:

• output_model_schema — Pass the Pydantic class (not an instance). Browser-Use converts it to JSON Schema and injects it into the system prompt DeepWiki: Custom Output Formats.
• history.structured_output() — This property returns the validated Pydantic object. If validation fails, the agent retries automatically GitHub: AGENTS.md.
• use_vision=True — The agent takes screenshots and uses vision-language capabilities. This dramatically improves accuracy on JavaScript-rendered pages CodeCut Tutorial.

Step 4: Using the Controller for Custom Tools

For advanced pipelines where you need the agent to call custom Python functions during its research loop, use the Controller class. This is especially useful when you want the agent to save intermediate results or call an API mid-research DEV Community.

from browser_use import Controller, ActionResult

controller = Controller(output_model=ResearchReport)


@controller.action("Save a finding to the research log")
def save_finding(claim: str, source_url: str, confidence: str):
    # Custom logic — e.g., write to a database
    print(f"[LOG] {claim} (confidence: {confidence})")
    return ActionResult(extracted_content=f"Saved: {claim}")


agent = Agent(
    task=task,
    llm=llm,
    controller=controller,
    output_model_schema=ResearchReport,
)

The Controller lets you register typed actions with Pydantic-validated parameters, giving you full control over what the agent can do beyond basic browsing Apify SDK Guide.

Step 5: Building a Multi-Source Research Pipeline

A real research pipeline visits multiple sites and synthesizes findings. Here’s an expanded example that searches HN, a tech blog, and a documentation site:

pipeline_task = """
You are a research agent. Complete these steps in order:

1. Go to Hacker News (https://news.ycombinator.com) and find the top 3 stories about AI agents
2. Click each story and extract the main claims
3. Go to https://simonwillison.net and search for "browser agent"
4. Extract key arguments from the top 2 results
5. Go to https://docs.browser-use.com and read the quickstart and output-format pages
6. Compile everything into a structured ResearchReport

Important: Do NOT make up data. If you cannot access a page, skip it and note it in your report.
"""

agent = Agent(
    task=pipeline_task,
    llm=llm,
    output_model_schema=ResearchReport,
    use_vision=True,
    max_failures=5,
    max_actions_per_step=5,  # Limit actions per LLM call
)

history = await agent.run(max_steps=50)  # Max total steps
report: ResearchReport = history.structured_output()

# Export to JSON
with open("research_report.json", "w") as f:
    f.write(report.model_dump_json(indent=2))

The model_dump_json() method comes from Pydantic and gives you clean, validated JSON ready for downstream consumption Pydantic Docs.

Configuration Tips and Best Practices

1. Write Explicit Task Descriptions

Browser-Use agents infer intent from natural language. A vague task like “research AI agents” produces inconsistent results. Be specific about steps, URLs, and what data to extract Decodo Guide. Break complex tasks into numbered substeps.

2. Use initial_actions for Speed

Skip the homepage navigation by pre-loading a URL:

agent = Agent(
    task=task,
    llm=llm,
    initial_actions=[{"go_to_url": {"url": "https://news.ycombinator.com"}}],
)

This runs without an LLM call, saving tokens and time Docs: All Parameters.

3. Tune Vision Settings

use_vision=True gives higher accuracy but costs more tokens and runs slower because each step sends a screenshot to the LLM. For text-heavy sites, you can disable vision and rely on DOM extraction:

agent = Agent(task=task, llm=llm, use_vision=False)

4. Validate with Pydantic Field Constraints

Add validation to catch extraction errors early:

from pydantic import Field, field_validator


class ResearchFinding(BaseModel):
    claim: str = Field(min_length=10, description="Extracted fact or claim")
    confidence: str = Field(pattern=r"^(high|medium|low)$")

    @field_validator("claim")
    @classmethod
    def claim_must_be_substantial(cls, v):
        if len(v.split()) < 3:
            raise ValueError("Claim too short — likely a parsing error")
        return v

Pydantic’s field_validator runs after the LLM fills the schema, catching low-quality extractions before they enter your pipeline Pydantic Docs.

5. Monitor Token Usage

Enable cost tracking to keep your pipeline budget-aware:

agent = Agent(
    task=task,
    llm=llm,
    output_model_schema=ResearchReport,
    calculate_cost=True,
)
history = await agent.run()
print(f"Cost: ${history.total_cost:.4f} | Tokens: {history.total_input_tokens}")

The calculate_cost=True flag makes the agent track per-step token usage CodeCut Tutorial.

6. Use a Persistent Browser Profile

For research sites that require login (e.g., GitHub, Medium, Gmail), persist the browser session:

browser = Browser(
    config=BrowserConfig(
        headless=False,
        user_data_dir="./browser_profile",
    )
)

The first run logs in manually; subsequent runs reuse the session cookies Diffused Creations Guide.

7. Handle Errors Gracefully

Set max_failures to control retry behavior. Combine with max_steps in run() to prevent runaway agents:

agent = Agent(
    task=task,
    llm=llm,
    output_model_schema=ResearchReport,
    max_failures=3,
)
history = await agent.run(max_steps=100)

When the agent exceeds max_failures consecutive errors, it stops and returns whatever structured data it has collected so far GitHub: AGENTS.md.

Putting It All Together

Here’s the complete runnable script:

import asyncio
from dotenv import load_dotenv
from pydantic import BaseModel, Field
from typing import List
from browser_use import Agent
from langchain_openai import ChatOpenAI

load_dotenv()


class ResearchFinding(BaseModel):
    claim: str = Field(description="Extracted fact or claim")
    source_url: str = Field(description="URL of the source")
    confidence: str = Field(description="high/medium/low")


class ResearchReport(BaseModel):
    query: str = Field(description="The research question")
    summary: str = Field(description="Synthesis of findings")
    findings: List[ResearchFinding]
    sources: List[str] = Field(description="List of source URLs")


async def main():
    llm = ChatOpenAI(model="gpt-4o", temperature=0.0)
    agent = Agent(
        task="Research what Browser-Use is. Visit browser-use.com, the GitHub repo, "
             "and docs. Summarize what it does, how it works, and key features.",
        llm=llm,
        output_model_schema=ResearchReport,
        use_vision=True,
        calculate_cost=True,
    )
    history = await agent.run()
    report: ResearchReport = history.structured_output()
    print(report.model_dump_json(indent=2))
    print(f"\nCost: ${history.total_cost:.4f}")


asyncio.run(main())

When you run this, the agent will autonomously navigate three sites, extract information, and return a validated ResearchReport with no manual parsing — no BeautifulSoup, no RegEx, no XPath.

Conclusion

Browser-Use combined with Pydantic structured outputs gives you a type-safe, self-healing web research pipeline that adapts to website changes automatically. Instead of writing brittle selectors that break when a site redesigns, you describe the intent and let the LLM figure out the DOM traversal. The Pydantic schema acts as a contract between the agent and your application, ensuring every research run produces data you can trust.

The ecosystem is evolving rapidly — Browser-Use now supports multi-tab browsing, custom tool registration via the Controller, persistent profiles, and integration with every major LLM provider Stackademic Article. Whether you are building a competitive intelligence scraper, a market research aggregator, or an automated due-diligence bot, this pattern of agentic browsing + structured extraction is the cleanest path from unstructured web data to typed, production-ready datasets.

---

🏆 Arena Match — Generated by deepseek-v4-preview via Nexum (8.0/10). All 5 subagents ran on the same model (delegate_task limitation — scores reflect generation variance, not model comparison).

📖 Related Reads

• CodeIntel Log — code quality, debugging, and software engineering benchmarks
• ToolBrain — tool reviews, LLM comparisons, and AI workflow guides

Cross-links automatically generated from NiteAgent.

← Back to all posts