Claude's large context window (up to 200K tokens) makes it ideal for analyzing scraped data at scale. You can feed entire datasets, multi-page documents, or collections of competitor pages into a single conversation and ask Claude to find patterns, compare information, generate reports, or serve as a knowledge base you can query in natural language.
π Looking to add AI-generated fields to individual records (like summaries, categories, or sentiment scores)? See our data enrichment guide instead. This article focuses on analyzing your data as a whole to extract insights, build context, and generate reports.
This guide covers three ways to analyze Browse AI data with Claude:
Method | Best for | Difficulty |
Zapier / Make | Automated report generation, no-code analysis | Easy |
Webhooks + Claude API | Real-time analysis with custom logic | Intermediate |
API polling + batch analysis | Large-scale analysis, knowledge bases, multi-source reports | Intermediate |
What you can do
Use case | Example |
Competitive analysis | Scrape competitor product pages, ask Claude to compare features, pricing, and positioning |
Market research reports | Scrape industry listings, generate a structured market overview with trends and gaps |
Knowledge base / context | Scrape documentation or site content, use it as a reference Claude can answer questions about |
Trend analysis | Scrape the same source over time, ask Claude to identify what changed and what it means |
Content auditing | Scrape your own site pages, have Claude assess quality, consistency, and gaps |
Multi-source synthesis | Combine data from multiple robots, ask Claude to produce a unified briefing |
Method 1: Zapier / Make (no-code)
Use Zapier or Make to automatically send scraped data to Claude for analysis and route the output to your preferred destination.
Zapier setup
Create a new Zap with Browse AI as the trigger. Select the New Successful Task Run event.
Add a Claude (Anthropic) action step. Select Send Message.
Connect your Anthropic API key from console.anthropic.com.
Write an analysis prompt and map in your Browse AI fields. For example:
You are a market research analyst. Analyze the following scraped product data and produce a brief market intelligence report covering:1. Price positioning (budget, mid-range, premium)2. Common features across listings3. Gaps or opportunities you notice4. A 2-3 sentence executive summaryProduct Name: {{product_name}}Price: {{price}}Description: {{description}}URL: {{url}}
Route Claude's analysis to your preferred destination: email, Slack, Google Docs, Notion, or a spreadsheet.
Make (Integromat) setup
Create a scenario with Browse AI > Watch Successful Task Runs as the trigger.
Add an Anthropic (Claude) > Create a Message module with your analysis prompt.
Add a destination module for the output (Google Docs, Slack, email, etc.).
β
Tip: For analysis tasks, use claude-sonnet-4-6 or claude-opus-4-6 for better reasoning. Unlike enrichment (where speed matters most), analysis benefits from Claude's stronger models that can identify subtle patterns and produce more nuanced insights.
Method 2: Webhooks + Claude API (real-time analysis)
For custom analysis pipelines with full control over prompting, context, and output formatting.
Prerequisites
An Anthropic API key from console.anthropic.com
A server or cloud function to receive webhooks
pip install anthropic flask
Example: Competitive analysis on arrival
This endpoint receives scraped competitor data, analyzes it with Claude, and posts the analysis to a Slack channel:
import json, requestsfrom flask import Flask, request, jsonifyfrom anthropic import Anthropicapp = Flask(__name__)client = Anthropic(api_key="your-anthropic-api-key")SLACK_WEBHOOK_URL = "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"@app.route("/webhook/browse-ai/analyze", methods=["POST"])def analyze_competitor(): event = request.json if event.get("event") != "taskFinishedSuccessfully": return jsonify({"status": "ignored"}), 200 task = event.get("task", {}) captured = task.get("capturedTexts", {}) lists = task.get("capturedLists", {}).get("items", []) # Build context from all captured data context = "Captured fields:\n" + json.dumps(captured, indent=2) if lists: context += f"\n\nCaptured table ({len(lists)} rows):\n" context += json.dumps(lists[:50], indent=2) # Cap at 50 rows # Ask Claude to analyze message = client.messages.create( model="claude-sonnet-4-6", max_tokens=2048, system="You are a competitive intelligence analyst. Provide clear, actionable insights.", messages=[{ "role": "user", "content": f"""Analyze this competitor data scraped from the web. Produce a briefing with:- Key findings (bullet points)- How this compares to typical market positioning- Recommended actions for our team- Any red flags or opportunitiesData:{context}""" }] ) analysis = message.content[0].text # Post to Slack requests.post(SLACK_WEBHOOK_URL, json={ "text": f"*Competitor Analysis*\nRobot: {task.get('robotId')}\n\n{analysis}" }) return jsonify({"status": "analyzed"}), 200if __name__ == "__main__": app.run(port=5000)
β οΈ Browse AI does not support webhook signature verification. To verify that requests are coming from Browse AI, allowlist the IP address 3.228.254.190. See our webhook IP allowlisting guide.
Method 3: API polling + batch analysis
The most powerful method for analysis. Poll Browse AI to collect data from one or multiple robots, then feed it all into Claude's large context window for comprehensive analysis.
Example: Multi-source market report
This script pulls data from multiple Browse AI robots and generates a unified market report:
import requests, jsonfrom anthropic import AnthropicBROWSE_AI_API_KEY = "your-browse-ai-api-key"client = Anthropic(api_key="your-anthropic-api-key")# Define the robots to pull data fromROBOTS = { "competitor_pricing": "robot-id-1", "industry_news": "robot-id-2", "job_listings": "robot-id-3",}def get_latest_data(robot_id): """Fetch the most recent successful task from a robot.""" response = requests.get( f"https://api.browse.ai/v2/robots/{robot_id}/tasks", headers={"Authorization": f"Bearer {BROWSE_AI_API_KEY}"}, params={"page": 1} ) tasks = response.json().get("result", {}).get("robotTasks", {}).get("items", []) for task in tasks: if task.get("status") == "successful": return task return Nonedef build_analysis_context(): """Collect data from all robots into a single context.""" context_parts = [] for label, robot_id in ROBOTS.items(): task = get_latest_data(robot_id) if task: data = { "captured_texts": task.get("capturedTexts", {}), "captured_lists": task.get("capturedLists", {}).get("items", [])[:100] } context_parts.append(f"## {label}\n{json.dumps(data, indent=2)}") return "\n\n".join(context_parts)def generate_report(): """Generate a market intelligence report from all sources.""" context = build_analysis_context() message = client.messages.create( model="claude-opus-4-6", max_tokens=4096, system="""You are a senior market research analyst. Write structured, data-driven reports with clear sections and actionable recommendations.""", messages=[{ "role": "user", "content": f"""Using the following data collected from multiple web sources, generate a comprehensive market intelligence report.Include these sections:1. Executive Summary (3-5 sentences)2. Market Overview (key trends, pricing patterns)3. Competitive Landscape (major players, positioning)4. Hiring Signals (what job listings reveal about strategy)5. Opportunities and Threats6. Recommended Actions (prioritized)Data sources:{context}""" }] ) return message.content[0].text# Generate and save the reportreport = generate_report()with open("market_report.md", "w") as f: f.write(report)print("Report generated: market_report.md")
Example: Building a knowledge base from scraped content
Use Browse AI to scrape website content (documentation, help articles, product pages), then use that content as a knowledge base you can query with Claude:
import requests, jsonfrom anthropic import AnthropicBROWSE_AI_API_KEY = "your-browse-ai-api-key"client = Anthropic(api_key="your-anthropic-api-key")def load_knowledge_base(robot_id): """Load all scraped content from a robot's task history.""" all_content = [] page = 1 while True: response = requests.get( f"https://api.browse.ai/v2/robots/{robot_id}/tasks", headers={"Authorization": f"Bearer {BROWSE_AI_API_KEY}"}, params={"page": page} ) tasks = response.json().get("result", {}).get("robotTasks", {}).get("items", []) if not tasks: break for task in tasks: if task.get("status") == "successful": texts = task.get("capturedTexts", {}) if texts: all_content.append(texts) page += 1 return all_contentdef query_knowledge_base(knowledge_base, question): """Ask Claude a question using scraped content as context.""" context = json.dumps(knowledge_base, indent=2) message = client.messages.create( model="claude-sonnet-4-6", max_tokens=2048, system="""You are a helpful research assistant. Answer questions using ONLY the provided reference data. If the data doesn't contain enough information to answer, say so clearly. Cite specific entries when possible.""", messages=[{ "role": "user", "content": f"""Reference data (scraped from the web):{context}Question: {question}""" }] ) return message.content[0].text# Load scraped content as knowledge basekb = load_knowledge_base("your-robot-id")print(f"Loaded {len(kb)} pages into knowledge base")# Query itanswer = query_knowledge_base(kb, "What pricing tiers do competitors offer?")print(answer)
β Tip: Claude's context window can hold roughly 150,000 words. If your scraped dataset exceeds this, prioritize the most relevant pages, or split the analysis into focused chunks and ask Claude to synthesize the results at the end.
Choosing the right model for analysis
Model | Best for | Context window |
| Quick summaries, simple Q&A over data | 200K tokens |
| Most analysis tasks, reports, comparisons | 200K tokens |
| Complex multi-source analysis, nuanced research reports | 200K tokens |
Next steps
Enrich your data first: Add AI-generated fields like categories and sentiment before analysis. See our enrichment guide.
Automate the pipeline: Set up monitors to trigger analysis automatically when data changes. See our automation guide.
Browse AI API reference: See the full Browse AI API documentation for all available endpoints.
