Webhooks are HTTP callbacks that Browse AI sends to your server immediately when specific events occur. Instead of repeatedly checking the API to see if something has finished (polling), Browse AI pushes notifications directly to you in real-time.

Webhooks vs polling:

Instant notifications: Get data the moment it's available
Reduced API calls: No need to constantly check task status
Better performance: Lower latency and more efficient resource usage
Real-time automation: Build truly responsive data pipelines

Available webhook events

Browse AI supports several webhook event types to match different use cases:

Task completion events:

taskFinished: Any task completion (success or failure)
taskFinishedSuccessfully: Only successful task completions
taskFinishedWithError: Only failed task completions

Data change events:

taskCapturedDataChanged: When monitoring detects data changes

Export events:

tableExportFinishedSuccessfully: When table exports complete (Beta)

Step 1: Choose your webhook destination

Before setting up webhooks in Browse AI, decide where you want to receive the notifications.

Option A: Use an existing tool's webhook URL (easiest)

Many popular tools provide webhook URLs that you can use directly:

Zapier:

Create a new Zap with "Webhooks by Zapier" as the trigger
Choose "Catch Hook"
Copy the provided webhook URL

Make.com:

Create a new scenario with "Webhooks" module
Choose "Custom webhook"
Copy the provided webhook URL

Slack:

Go to your Slack workspace settings
Create an "Incoming Webhook" app
Copy the webhook URL to post directly to a Slack channel

Discord:

Go to your Discord server settings
Create a webhook for a specific channel
Copy the webhook URL

Microsoft Teams:

Create an "Incoming Webhook" connector
Configure it for your desired channel
Copy the webhook URL

Other tools:

Most team communication and automation tools offer webhook integrations
CRM systems (HubSpot, Salesforce)
Project management tools (Notion, Airtable)
Email marketing platforms (Mailchimp, ConvertKit)

Option B: Build your own webhook endpoint

If you need custom processing logic, build your own endpoint (see the Python/Node.js examples in the next section).

💡 Need to allowlist our IP? Browse AI webhooks are sent from the IP address:

3.228.254.190

If your infrastructure requires IP allowlisting, add this address to your allowlist. For more details, see Webhook IP address for allowlisting.

Example webhook endpoint (Python/Flask):

from flask import Flask, request, jsonify
import json

app = Flask(__name__)

@app.route('/browse-ai-webhook', methods=['POST'])
def handle_webhook():
    try:
        payload = request.get_json()
        
        # Verify the request is from Browse AI by checking the source IP
        # Browse AI sends webhooks from 3.228.254.190
        # See: https://help.browse.ai/en/articles/13400027-webhooks-ip-address-for-allowlisting
        
        if payload.get('event') == 'taskFinished':
            handle_task_completion(payload)
        elif payload.get('event') == 'taskCapturedDataChanged':
            handle_data_change(payload)
            
        return jsonify({'status': 'success'}), 200
    except Exception as e:
        print(f"Webhook error: {e}")
        return jsonify({'error': 'Internal error'}), 500

def handle_task_completion(payload):
    task_id = payload['task']['id']
    robot_id = payload['task']['robotId']
    status = payload['task']['status']
    print(f"Task {task_id} completed with status: {status}")
    if status == 'successful':
        extracted_data = payload['task']['capturedTexts']
        save_to_database(extracted_data)
    else:
        error_message = payload['task'].get('userFriendlyError', 'Unknown error')
        log_failure(task_id, error_message)

def handle_data_change(payload):
    print("Data change detected in monitoring!")
    send_slack_notification(payload)

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

Example webhook endpoint (Node.js/Express):

const express = require('express');
const app = express();
app.use(express.json());

app.post('/browse-ai-webhook', (req, res) => {
    try {
        const payload = req.body;
        if (payload.event === 'taskFinished') {
            handleTaskCompletion(payload);
        } else if (payload.event === 'taskCapturedDataChanged') {
            handleDataChange(payload);
        }
        res.status(200).json({ status: 'success' });
    } catch (error) {
        console.error('Webhook error:', error);
        res.status(500).json({ error: 'Internal error' });
    }
});

function handleTaskCompletion(payload) {
    const { id, status, capturedTexts } = payload.task;
    console.log(`Task ${id} completed with status: ${status}`);
    if (status === 'successful') { processExtractedData(capturedTexts); }
}

function handleDataChange(payload) {
    console.log('Data change detected!');
}

app.listen(5000, () => {
    console.log('Webhook server running on port 5000');
});

Step 2: Set up webhooks

Once you have your webhook URL you can integrate it with your approved robot in Browse AI. You can do this either via the dashboard or using our API.

Dashboard setup (recommended):

Go to your approved robot → Integrate tab
Find the Webhooks section → Add webhook
Enter your webhook URL (from Zapier, Slack, etc.)
Select the event type
Click Save

API setup:

You can also add webhooks programmatically using the Browse AI API:

curl -X POST "https://api.browse.ai/v2/robots/YOUR_ROBOT_ID/webhooks" \   -H "Authorization: Bearer YOUR_API_KEY" \   -H "Content-Type: application/json" \   -d '{"url":"https://your-server.com/webhook","event":"taskFinished"}'

Benefits of API setup:

Automate webhook registration across multiple robots
Easily update or rotate webhook URLs
Manage webhooks as part of your infrastructure-as-code

Step 3: Understanding webhook payloads

When a webhook fires, Browse AI sends a JSON payload to your endpoint. Here are examples for each event type:

taskFinished payload:

{   "event": "taskFinished",   "task": {     "id": "task_abc123",     "robotId": "robot_xyz789",     "status": "successful",     "capturedTexts": {       "Product Name": "Example Product",       "Price": "$29.99",       "Availability": "In Stock"     },     "createdAt": 1680000000,     "finishedAt": 1680000060   } }

taskCapturedDataChanged payload:

{   "event": "taskCapturedDataChanged",   "task": {     "id": "task_abc123",     "robotId": "robot_xyz789",     "status": "successful",     "capturedTexts": {       "Price": "$24.99"     },     "previousCapturedTexts": {       "Price": "$29.99"     }   } }

tableExportFinishedSuccessfully payload:

{   "event": "tableExportFinishedSuccessfully",   "export": {     "id": "export_def456",     "robotId": "robot_xyz789",     "downloadUrl": "https://api.browse.ai/exports/export_def456/download",     "rowCount": 150,     "format": "csv"   } }

Step 4: Set up webhooks for different events

You can configure different webhook URLs for different event types. Here are curl examples for each:

Monitor for any task completion:

curl -X POST "https://api.browse.ai/v2/robots/YOUR_ROBOT_ID/webhooks" \   -H "Authorization: Bearer YOUR_API_KEY" \   -H "Content-Type: application/json" \   -d '{"url":"https://your-server.com/task-webhook","event":"taskFinished"}'

Monitor for data changes:

curl -X POST "https://api.browse.ai/v2/robots/YOUR_ROBOT_ID/webhooks" \   -H "Authorization: Bearer YOUR_API_KEY" \   -H "Content-Type: application/json" \   -d '{"url":"https://your-server.com/change-webhook","event":"taskCapturedDataChanged"}'

Monitor for table exports:

curl -X POST "https://api.browse.ai/v2/robots/YOUR_ROBOT_ID/webhooks" \   -H "Authorization: Bearer YOUR_API_KEY" \   -H "Content-Type: application/json" \   -d '{"url":"https://your-server.com/export-webhook","event":"tableExportFinishedSuccessfully"}'

How to run your robot

API Guide: Getting started

API Guide: Retrieving and managing scraped data

API Guide: Setting up automated monitoring

API Guide: Bulk operations and large-scale data extraction

Webhooks: Set up guide