Webhooks: Set up guide | Browse AI Help Center

Webhooks are HTTP callbacks that Browse AI sends to your server immediately when specific events occur. Instead of repeatedly checking the API to see if something has finished (polling), Browse AI pushes notifications directly to you in real-time.

Webhooks vs polling:

Instant notifications: Get data the moment it's available
Reduced API calls: No need to constantly check task status
Better performance: Lower latency and more efficient resource usage
Real-time automation: Build truly responsive data pipelines

Available webhook events

Browse AI supports several webhook event types to match different use cases:

Task completion events:

taskFinished: Any task completion (success or failure)
taskFinishedSuccessfully: Only successful task completions
taskFinishedWithError: Only failed task completions

Data change events:

taskCapturedDataChanged: When monitoring detects data changes

Export events:

tableExportFinishedSuccessfully: When table exports complete (Beta)

Step 1: Choose your webhook destination

Before setting up webhooks in Browse AI, decide where you want to receive the notifications.

Option A: Use an existing tool's webhook URL (easiest)

Many popular tools provide webhook URLs that you can use directly:

Zapier:

Create a new Zap with "Webhooks by Zapier" as the trigger
Choose "Catch Hook"
Copy the provided webhook URL

Make.com:

Create a new scenario with "Webhooks" module
Choose "Custom webhook"
Copy the provided webhook URL

Slack:

Go to your Slack workspace settings
Create an "Incoming Webhook" app
Copy the webhook URL to post directly to a Slack channel

Discord:

Go to your Discord server settings
Create a webhook for a specific channel
Copy the webhook URL

Microsoft Teams:

Create an "Incoming Webhook" connector
Configure it for your desired channel
Copy the webhook URL

Other tools:

Most team communication and automation tools offer webhook integrations
CRM systems (HubSpot, Salesforce)
Project management tools (Notion, Airtable)
Email marketing platforms (Mailchimp, ConvertKit)

Option B: Build your own webhook endpoint

If you need custom processing logic, build your own endpoint (see the Python/Node.js examples in the next section).

Example webhook endpoint (Python/Flask):

from flask import Flask, request, jsonify
import json

app = Flask(__name__)

@app.route('/browse-ai-webhook', methods=['POST'])
def handle_webhook():
    try:
        # Get the webhook payload
        payload = request.get_json()
        
        # Verify it's from Browse AI (optional but recommended)
        # You can check for specific fields or implement signature verification
        
        # Process the webhook based on event type
        if payload.get('event') == 'taskFinished':
            handle_task_completion(payload)
        elif payload.get('event') == 'taskCapturedDataChanged':
            handle_data_change(payload)
            
        return jsonify({'status': 'success'}), 200
        
    except Exception as e:
        print(f"Webhook error: {e}")
        return jsonify({'error': 'Internal error'}), 500

def handle_task_completion(payload):
    task_id = payload['task']['id']
    robot_id = payload['task']['robotId']
    status = payload['task']['status']
    
    print(f"Task {task_id} completed with status: {status}")
    
    # Your business logic here
    if status == 'successful':
        # Process the extracted data
        extracted_data = payload['task']['capturedTexts']
        save_to_database(extracted_data)
    else:
        # Handle failed tasks
        error_message = payload['task'].get('userFriendlyError', 'Unknown error')
        log_failure(task_id, error_message)

def handle_data_change(payload):
    print("Data change detected in monitoring!")
    # Send alert, update dashboard, etc.
    send_slack_notification(payload)

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

Example webhook endpoint (Node.js/Express):

const express = require('express');
const app = express();

app.use(express.json());

app.post('/browse-ai-webhook', (req, res) => {
    try {
        const payload = req.body;
        
        // Process webhook based on event type
        if (payload.event === 'taskFinished') {
            handleTaskCompletion(payload);
        } else if (payload.event === 'taskCapturedDataChanged') {
            handleDataChange(payload);
        }
        
        res.status(200).json({ status: 'success' });
        
    } catch (error) {
        console.error('Webhook error:', error);
        res.status(500).json({ error: 'Internal error' });
    }
});

function handleTaskCompletion(payload) {
    const { id, status, capturedTexts } = payload.task;
    
    console.log(`Task ${id} completed with status: ${status}`);
    
    if (status === 'successful') {
        // Process extracted data
        processExtractedData(capturedTexts);
    }
}

function handleDataChange(payload) {
    console.log('Data change detected!');
    // Send notifications, update systems, etc.
}

app.listen(5000, () => {
    console.log('Webhook server running on port 5000');
});

Step 2: Set up webhooks

Once you have your webhook URL you can integrate it with your approved robot in Browse AI. You can do this either via the dashboard or using our API.

Dashboard setup (recommended):

Go to your approved robot → Integrate tab
Find the Webhooks section → Add webhook
Enter your webhook URL (from Zapier, Slack, etc.)
Select the event type
Click Save

Available webhook events

Event Type	Event Name	When It Triggers	Best For
task is finished	`taskFinished`	Any task completion (success or failure)	General task monitoring, logging all activity
task is finished successfully	`taskFinishedSuccessfully`	Only successful task completions	Data processing pipelines, successful extractions only
task is finished with an error	`taskFinishedWithError`	Only failed task completions	Error monitoring, failure alerts
monitoring check is finished successfully with changes detected	`taskCapturedDataChanged`	When monitoring detects data changes	Real-time alerts, competitive intelligence
table export is finished	`tableExportFinishedSuccessfully`	When table exports complete	Bulk data processing, large dataset modifications

Most common use cases:

taskFinishedSuccessfully: Perfect for data pipelines - only get notified when you have new data to process
taskCapturedDataChanged: Ideal for monitoring - only get notified when something actually changes
taskFinished: Good for comprehensive logging and debugging - see all task activity
taskFinishedWithError: Essential for error monitoring and support

API setup (for automation):

If you need to create webhooks programmatically or manage many robots:

curl -X POST "https://api.browse.ai/v2/robots/ROBOT_ID/webhooks" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "hookUrl": "https://your-server.com/browse-ai-webhook",
    "eventType": "taskFinishedSuccessfully"
  }'

Benefits of API setup:

Automate webhook creation across multiple robots
Integrate webhook management into your systems
Programmatically manage webhook lifecycles

Step 3: Understanding webhook payloads

Different events send different payload structures.

Task completion webhook (taskFinished):

{
  "event": "taskFinished",
  "task": {
    "id": "task-uuid-here",
    "robotId": "robot-uuid-here",
    "status": "successful",
    "capturedTexts": {
      "title": "Product Name",
      "price": "$99.99",
      "availability": "In Stock"
    },
    "capturedScreenshots": [
      {
        "name": "product_image",
        "url": "https://screenshot-url.jpg"
      }
    ],
    "finishedAt": 1678795867879,
    "runByTaskMonitorId": null,
    "runByAPI": true
  }
}

Data change webhook (taskCapturedDataChanged):

{
  "event": "taskCapturedDataChanged",
  "task": {
    "id": "task-uuid-here",
    "robotId": "robot-uuid-here", 
    "status": "successful",
    "capturedTexts": {
      "price": "$89.99",
      "availability": "Limited Stock"
    },
    "runByTaskMonitorId": "monitor-uuid-here",
    "changedFields": ["price", "availability"]
  }
}

Table export webhook (tableExportFinishedSuccessfully):

{
  "event": "tableExportFinishedSuccessfully",
  "robotId": "robot-uuid-here",
  "export": {
    "id": "export-uuid-here",
    "format": "csv",
    "downloadUrl": "https://download-url.com/export.csv",
    "recordCount": 15000,
    "finishedAt": 1678795867879
  }
}

Step 4: Set up webhooks for different events

You can set up multiple webhooks for different purposes.

Monitor for data changes:

curl -X POST "https://api.browse.ai/v2/robots/ROBOT_ID/webhooks" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "hookUrl": "https://hooks.slack.com/services/your-slack-webhook",
    "eventType": "taskCapturedDataChanged"
  }'

Get notified of any failures:

curl -X POST "https://api.browse.ai/v2/robots/ROBOT_ID/webhooks" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "hookUrl": "https://your-server.com/failure-alert",
    "eventType": "taskFinishedWithError"
  }'

Bulk operation completion:

curl -X POST "https://api.browse.ai/v2/robots/ROBOT_ID/webhooks" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "hookUrl": "https://hooks.zapier.com/hooks/catch/bulk-complete",
    "eventType": "tableExportFinishedSuccessfully"
  }'

How to run your robot

API Guide: Getting started

API Guide: Retrieving and managing scraped data

API Guide: Setting up automated monitoring

API Guide: Bulk operations and large-scale data extraction