Skip to main content

How to analyze Browse AI data with Claude

M
Written by Melissa Shires

Claude's large context window (up to 200K tokens) makes it ideal for analyzing scraped data at scale. You can feed entire datasets, multi-page documents, or collections of competitor pages into a single conversation and ask Claude to find patterns, compare information, generate reports, or serve as a knowledge base you can query in natural language.

πŸ“– Looking to add AI-generated fields to individual records (like summaries, categories, or sentiment scores)? See our data enrichment guide instead. This article focuses on analyzing your data as a whole to extract insights, build context, and generate reports.

This guide covers three ways to analyze Browse AI data with Claude:

Method

Best for

Difficulty

Zapier / Make

Automated report generation, no-code analysis

Easy

Webhooks + Claude API

Real-time analysis with custom logic

Intermediate

API polling + batch analysis

Large-scale analysis, knowledge bases, multi-source reports

Intermediate

What you can do

Use case

Example

Competitive analysis

Scrape competitor product pages, ask Claude to compare features, pricing, and positioning

Market research reports

Scrape industry listings, generate a structured market overview with trends and gaps

Knowledge base / context

Scrape documentation or site content, use it as a reference Claude can answer questions about

Trend analysis

Scrape the same source over time, ask Claude to identify what changed and what it means

Content auditing

Scrape your own site pages, have Claude assess quality, consistency, and gaps

Multi-source synthesis

Combine data from multiple robots, ask Claude to produce a unified briefing

Method 1: Zapier / Make (no-code)

Use Zapier or Make to automatically send scraped data to Claude for analysis and route the output to your preferred destination.

Zapier setup

  1. Create a new Zap with Browse AI as the trigger. Select the New Successful Task Run event.

  2. Add a Claude (Anthropic) action step. Select Send Message.

  3. Connect your Anthropic API key from console.anthropic.com.

  4. Write an analysis prompt and map in your Browse AI fields. For example:

You are a market research analyst. Analyze the following scraped product data and produce a brief market intelligence report covering:1. Price positioning (budget, mid-range, premium)2. Common features across listings3. Gaps or opportunities you notice4. A 2-3 sentence executive summaryProduct Name: {{product_name}}Price: {{price}}Description: {{description}}URL: {{url}}

  1. Route Claude's analysis to your preferred destination: email, Slack, Google Docs, Notion, or a spreadsheet.

Make (Integromat) setup

  1. Create a scenario with Browse AI > Watch Successful Task Runs as the trigger.

  2. Add an Anthropic (Claude) > Create a Message module with your analysis prompt.

  3. Add a destination module for the output (Google Docs, Slack, email, etc.).

βœ… Tip: For analysis tasks, use claude-sonnet-4-6 or claude-opus-4-6 for better reasoning. Unlike enrichment (where speed matters most), analysis benefits from Claude's stronger models that can identify subtle patterns and produce more nuanced insights.

Method 2: Webhooks + Claude API (real-time analysis)

For custom analysis pipelines with full control over prompting, context, and output formatting.

Prerequisites

  • An Anthropic API key from console.anthropic.com

  • A server or cloud function to receive webhooks

  • pip install anthropic flask

Example: Competitive analysis on arrival

This endpoint receives scraped competitor data, analyzes it with Claude, and posts the analysis to a Slack channel:

import json, requestsfrom flask import Flask, request, jsonifyfrom anthropic import Anthropicapp = Flask(__name__)client = Anthropic(api_key="your-anthropic-api-key")SLACK_WEBHOOK_URL = "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"@app.route("/webhook/browse-ai/analyze", methods=["POST"])def analyze_competitor():    event = request.json    if event.get("event") != "taskFinishedSuccessfully":        return jsonify({"status": "ignored"}), 200    task = event.get("task", {})    captured = task.get("capturedTexts", {})    lists = task.get("capturedLists", {}).get("items", [])    # Build context from all captured data    context = "Captured fields:\n" + json.dumps(captured, indent=2)    if lists:        context += f"\n\nCaptured table ({len(lists)} rows):\n"        context += json.dumps(lists[:50], indent=2)  # Cap at 50 rows    # Ask Claude to analyze    message = client.messages.create(        model="claude-sonnet-4-6",        max_tokens=2048,        system="You are a competitive intelligence analyst. Provide clear, actionable insights.",        messages=[{            "role": "user",            "content": f"""Analyze this competitor data scraped from the web. Produce a briefing with:- Key findings (bullet points)- How this compares to typical market positioning- Recommended actions for our team- Any red flags or opportunitiesData:{context}"""        }]    )    analysis = message.content[0].text    # Post to Slack    requests.post(SLACK_WEBHOOK_URL, json={        "text": f"*Competitor Analysis*\nRobot: {task.get('robotId')}\n\n{analysis}"    })    return jsonify({"status": "analyzed"}), 200if __name__ == "__main__":    app.run(port=5000)

⚠️ Browse AI does not support webhook signature verification. To verify that requests are coming from Browse AI, allowlist the IP address 3.228.254.190. See our webhook IP allowlisting guide.

Method 3: API polling + batch analysis

The most powerful method for analysis. Poll Browse AI to collect data from one or multiple robots, then feed it all into Claude's large context window for comprehensive analysis.

Example: Multi-source market report

This script pulls data from multiple Browse AI robots and generates a unified market report:

import requests, jsonfrom anthropic import AnthropicBROWSE_AI_API_KEY = "your-browse-ai-api-key"client = Anthropic(api_key="your-anthropic-api-key")# Define the robots to pull data fromROBOTS = {    "competitor_pricing": "robot-id-1",    "industry_news": "robot-id-2",    "job_listings": "robot-id-3",}def get_latest_data(robot_id):    """Fetch the most recent successful task from a robot."""    response = requests.get(        f"https://api.browse.ai/v2/robots/{robot_id}/tasks",        headers={"Authorization": f"Bearer {BROWSE_AI_API_KEY}"},        params={"page": 1}    )    tasks = response.json().get("result", {}).get("robotTasks", {}).get("items", [])    for task in tasks:        if task.get("status") == "successful":            return task    return Nonedef build_analysis_context():    """Collect data from all robots into a single context."""    context_parts = []    for label, robot_id in ROBOTS.items():        task = get_latest_data(robot_id)        if task:            data = {                "captured_texts": task.get("capturedTexts", {}),                "captured_lists": task.get("capturedLists", {}).get("items", [])[:100]            }            context_parts.append(f"## {label}\n{json.dumps(data, indent=2)}")    return "\n\n".join(context_parts)def generate_report():    """Generate a market intelligence report from all sources."""    context = build_analysis_context()    message = client.messages.create(        model="claude-opus-4-6",        max_tokens=4096,        system="""You are a senior market research analyst. Write structured, data-driven reports with clear sections and actionable recommendations.""",        messages=[{            "role": "user",            "content": f"""Using the following data collected from multiple web sources, generate a comprehensive market intelligence report.Include these sections:1. Executive Summary (3-5 sentences)2. Market Overview (key trends, pricing patterns)3. Competitive Landscape (major players, positioning)4. Hiring Signals (what job listings reveal about strategy)5. Opportunities and Threats6. Recommended Actions (prioritized)Data sources:{context}"""        }]    )    return message.content[0].text# Generate and save the reportreport = generate_report()with open("market_report.md", "w") as f:    f.write(report)print("Report generated: market_report.md")

Example: Building a knowledge base from scraped content

Use Browse AI to scrape website content (documentation, help articles, product pages), then use that content as a knowledge base you can query with Claude:

import requests, jsonfrom anthropic import AnthropicBROWSE_AI_API_KEY = "your-browse-ai-api-key"client = Anthropic(api_key="your-anthropic-api-key")def load_knowledge_base(robot_id):    """Load all scraped content from a robot's task history."""    all_content = []    page = 1    while True:        response = requests.get(            f"https://api.browse.ai/v2/robots/{robot_id}/tasks",            headers={"Authorization": f"Bearer {BROWSE_AI_API_KEY}"},            params={"page": page}        )        tasks = response.json().get("result", {}).get("robotTasks", {}).get("items", [])        if not tasks:            break        for task in tasks:            if task.get("status") == "successful":                texts = task.get("capturedTexts", {})                if texts:                    all_content.append(texts)        page += 1    return all_contentdef query_knowledge_base(knowledge_base, question):    """Ask Claude a question using scraped content as context."""    context = json.dumps(knowledge_base, indent=2)    message = client.messages.create(        model="claude-sonnet-4-6",        max_tokens=2048,        system="""You are a helpful research assistant. Answer questions using ONLY the provided reference data. If the data doesn't contain enough information to answer, say so clearly. Cite specific entries when possible.""",        messages=[{            "role": "user",            "content": f"""Reference data (scraped from the web):{context}Question: {question}"""        }]    )    return message.content[0].text# Load scraped content as knowledge basekb = load_knowledge_base("your-robot-id")print(f"Loaded {len(kb)} pages into knowledge base")# Query itanswer = query_knowledge_base(kb, "What pricing tiers do competitors offer?")print(answer)

βœ… Tip: Claude's context window can hold roughly 150,000 words. If your scraped dataset exceeds this, prioritize the most relevant pages, or split the analysis into focused chunks and ask Claude to synthesize the results at the end.

Choosing the right model for analysis

Model

Best for

Context window

claude-haiku-4-5-20251001

Quick summaries, simple Q&A over data

200K tokens

claude-sonnet-4-6

Most analysis tasks, reports, comparisons

200K tokens

claude-opus-4-6

Complex multi-source analysis, nuanced research reports

200K tokens

Next steps

  • Enrich your data first: Add AI-generated fields like categories and sentiment before analysis. See our enrichment guide.

  • Automate the pipeline: Set up monitors to trigger analysis automatically when data changes. See our automation guide.

  • Browse AI API reference: See the full Browse AI API documentation for all available endpoints.

Did this answer your question?