Skip to main content

API guide: Creating and retraining robots with Robot Drafts (Beta)

Create and retrain Browse AI robots from code with the Robot Drafts API, from your own scripts, your app or an AI agent, then run them with the rest of the API.

M
Written by Melissa Shires

Beta. The Robot Drafts API may change before it is generally available. The API reference always has the current details.

With the Robot Drafts API you can create a robot from code, without opening the dashboard. You send a webpage and tell the robot what you're after, AI reads the page and works out the data to extract, you check the preview, and you save it as a robot. From there, the robot works like any other: run tasks, set up monitors and add webhooks with the endpoints you already use.

You can call it from anything that can make an HTTP request:

  • Your own scripts, for one-off jobs or scheduled pipelines.

  • Your app or backend, for example to let your own users add a site to track.

  • An AI agent like Claude Code, which can create, check and save robots for you while it works on a bigger task.

This guide walks through one common use case end to end: tracking a competitor's prices. If you haven't used the Browse AI API before, start with API guide: Getting started to create your API key.

How robot drafts work

A robot draft is a robot in training. Every draft goes through the same steps:

  1. Create the draft with a webpage (originUrl) and what you want from it: plain-language instructions, a schema of the columns you want, or both.

  2. The preview runs. AI reads the page and extracts sample data. This starts right away and takes a short while.

  3. Check the preview. Poll the draft until its status is ready, partially_ready or failed, then look at latestPreview.

  4. Refine it if needed. Update the instructions or schema, and a new preview runs.

  5. Save it. Saving a ready draft creates the robot and starts its first task.

Endpoint

What it does

POST /v2/robot-drafts

Create a draft and start its preview

GET /v2/robot-drafts/{draftId}

Get the draft's status and its preview data

PATCH /v2/robot-drafts/{draftId}

Change the page, instructions, schema or name, which starts a new preview

POST /v2/robot-drafts/{draftId}/save

Create the robot (or a new version of it) and start its first task

GET /v2/robot-drafts

List your drafts, newest first

One robot per site. A robot isn't tied to one page. Once you have a robot for a site, run it with a different originUrl to get the same data from another page on that site. You don't need a new robot for every page.

Example: track a competitor's prices

Say you want to track the prices on a competitor's product category page, every day. In the examples below, replace YOUR_SECRET_API_KEY with your API key and the example URL with the page you want to track.

Step 1: Create the draft

You can send instructions, a schema, or both.

Instructions only. Tell the robot what you're after, and the preview works out the columns from the page. This is the quickest way to start when you're not sure exactly which fields the page has.

curl -X POST "https://api.browse.ai/v2/robot-drafts" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "originUrl": "https://competitor.example.com/collections/standing-desks",
    "instructions": "Get every product on this page with its name, current price, original price if it is on sale, and link."
  }'

With a schema. When you already know the columns you need, for example because your app expects them, send a schema. It's a JSON Schema: each property is a column, and its description holds that column's instructions. A list ("type": "array") captures repeating items like product cards.

curl -X POST "https://api.browse.ai/v2/robot-drafts" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "originUrl": "https://competitor.example.com/collections/standing-desks",
    "name": "Competitor standing desk prices",
    "instructions": "Only include standing desks, not accessories.",
    "schema": {
      "type": "object",
      "properties": {
        "Products": {
          "type": "array",
          "description": "Every product card on the page",
          "maxItems": 200,
          "items": {
            "type": "object",
            "properties": {
              "Name": { "type": "string" },
              "Price": {
                "type": "string",
                "description": "The current price as shown, with the currency symbol"
              },
              "Original Price": {
                "type": "string",
                "description": "The crossed-out price if the product is on sale, otherwise empty"
              },
              "Link": {
                "type": "string",
                "description": "Link to the product page"
              }
            }
          }
        }
      }
    }
  }'

The response includes the draft's id. Its status is in_progress while the preview runs.

A few things to know about the request:

  • name is optional. If you leave it out, a name is suggested for you.

  • country is optional. Send a 3-letter country code (like GBR) for sites that show different content in different countries. If you leave it out, it's picked from the site's domain.

  • maxItems sets how many items the robot extracts from a list on each task. Previews show a sample of up to 50 items, whatever you set.

  • A column can also be a screenshot: { "type": "string", "contentMediaType": "image/png" }.

Step 2: Wait for the preview

There are no webhooks for drafts, so poll the draft every few seconds until the preview finishes:

curl -X GET "https://api.browse.ai/v2/robot-drafts/DRAFT_ID" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY"

Check result.status:

Status

What it means

What to do

in_progress

The preview is still running.

Poll again.

ready

Every column was found.

Check latestPreview, then save.

partially_ready

The preview finished, but some columns weren't found. latestPreview.missingColumns lists them.

Update the schema or instructions. A partially ready draft can't be saved.

failed

The preview couldn't finish. userFriendlyError says why.

Fix the cause, then send an update to run the preview again.

saved

The draft was saved. robotId points to the robot.

Nothing. Saved drafts can't be changed.

Step 3: Check the preview

When the draft is ready, result.schema holds the columns and result.latestPreview.data holds the sample data the preview extracted:

{
  "statusCode": 200,
  "messageCode": "success",
  "result": {
    "id": "8a0c3b1e-5f8e-4f4b-9f39-2a5d6d5a3c11",
    "status": "ready",
    "name": "Competitor standing desk prices",
    "originUrl": "https://competitor.example.com/collections/standing-desks",
    "latestPreview": {
      "missingColumns": [],
      "data": {
        "Products": [
          {
            "Name": "Oak standing desk",
            "Price": "$549.00",
            "Original Price": "$649.00",
            "Link": "https://competitor.example.com/products/oak-standing-desk"
          }
        ]
      }
    }
  }
}

This is a review step. Look at whether the columns and values are what you need before you save.

Step 4: Refine the draft (if needed)

To change what the robot extracts, update the draft. Sending originUrl, instructions or schema starts a new preview, so go back to step 2 afterwards.

curl -X PATCH "https://api.browse.ai/v2/robot-drafts/DRAFT_ID" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "schema": {
      "type": "object",
      "properties": {
        "Products": {
          "type": "array",
          "description": "Every product card on the page",
          "maxItems": 200,
          "items": {
            "type": "object",
            "properties": {
              "Name": { "type": "string" },
              "Price": { "type": "string", "description": "The current price as shown, with the currency symbol" },
              "Original Price": { "type": "string", "description": "The crossed-out price if the product is on sale, otherwise empty" },
              "Link": { "type": "string", "description": "Link to the product page" },
              "Rating": { "type": "string", "description": "The star rating, like 4.5" }
            }
          }
        }
      }
    }
  }'

Keep in mind:

  • Sending schema replaces the whole schema. Include every column you want to keep, and send maxItems again, or the list goes back to 50.

  • Sending new instructions without a schema starts over. The columns are worked out again from the new instructions. To keep your columns, send the schema with the instructions.

  • Changing only name renames the draft without a new preview.

Step 5: Save the draft

When the draft is ready, save it:

curl -X POST "https://api.browse.ai/v2/robot-drafts/DRAFT_ID/save" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY"

Saving creates the robot and starts its first task. The response includes the saved draft, the new robot (with its id and input parameters) and the taskId of that first task.

If the draft isn't ready yet, saving is refused with draft_not_savable and a message that says why.

Step 6: Get the data

From here, the robot is a regular robot. Follow its first task with the task endpoints:

curl -X GET "https://api.browse.ai/v2/robots/ROBOT_ID/tasks/TASK_ID" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY"

When the task's status is successful, your columns are in the task: text columns in capturedTexts, lists in capturedLists and screenshots in capturedScreenshots, under the same names you saw in the preview.

To get the same data from another page on the site, run the same robot with a different originUrl:

curl -X POST "https://api.browse.ai/v2/robots/ROBOT_ID/tasks" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inputParameters": {
      "originUrl": "https://competitor.example.com/collections/office-chairs"
    }
  }'

See API guide: Retrieving and managing scraped data for more on reading task results.

Step 7: Check prices every day and get notified

Add a monitor to run the robot on a schedule:

curl -X POST "https://api.browse.ai/v2/robots/ROBOT_ID/monitors" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Daily standing desk prices",
    "schedule": "FREQ=DAILY;INTERVAL=1;BYHOUR=9",
    "inputParameters": {
      "originUrl": "https://competitor.example.com/collections/standing-desks"
    },
    "notifyOnCapturedTextChange": true,
    "notifyOnCapturedScreenshotChange": false,
    "capturedScreenshotNotificationThreshold": 10
  }'

Send notifyOnCapturedScreenshotChange and capturedScreenshotNotificationThreshold even if you don't want screenshot alerts. Creating a monitor without them is refused with bad_request.

Then add a webhook so your app hears about changes as they happen:

curl -X POST "https://api.browse.ai/v2/robots/ROBOT_ID/webhooks" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "hookUrl": "https://your-app.example.com/webhooks/browse-ai",
    "eventType": "taskCapturedDataChanged"
  }'

Use taskFinishedSuccessfully instead if you want every result, not only the changes. See API guide: Setting up automated monitoring for more monitor options.

The whole flow in one script

Here's the same flow as a Python script you can run as is or adapt for your app. It creates the draft, waits for the preview, saves the robot, waits for its first task and prints the products.

import os
import timeimport requestsAPI = "https://api.browse.ai/v2"
HEADERS = {"Authorization": f"Bearer {os.environ['BROWSE_AI_API_KEY']}"}# 1. Create the draft
draft = requests.post(f"{API}/robot-drafts", headers=HEADERS, json={
    "originUrl": "https://competitor.example.com/collections/standing-desks",
    "name": "Competitor standing desk prices",
    "instructions": "Get every product on this page with its name, current price, "
                    "original price if it is on sale, and link.",
}).json()["result"]# 2. Wait for the preview to finish
while draft["status"] == "in_progress":
    time.sleep(10)
    draft = requests.get(f"{API}/robot-drafts/{draft['id']}", headers=HEADERS).json()["result"]# 3. Check the preview
if draft["status"] != "ready":
    raise SystemExit(f"Draft is {draft['status']}: "
                     f"{draft['userFriendlyError'] or draft['latestPreview']['missingColumns']}")
print("Columns:", list(draft["schema"]["properties"]))# 4. Save it: creates the robot and starts its first task
saved = requests.post(f"{API}/robot-drafts/{draft['id']}/save", headers=HEADERS).json()["result"]
robot_id, task_id = saved["robot"]["id"], saved["taskId"]# 5. Wait for the first task and print the data
while True:
    task = requests.get(f"{API}/robots/{robot_id}/tasks/{task_id}", headers=HEADERS).json()["result"]
    if task["status"] in ("successful", "failed"):
        break
    time.sleep(10)for product in task["capturedLists"].get("Products", []):
    print(product)

The column and list names depend on what the preview found, so with instructions only, check draft["schema"] for the list name before you rely on it. Send a schema when your code needs fixed names.

Using it from an AI agent like Claude Code

An AI agent that can make HTTP requests or run code can do all of the steps above for you. Give the agent your API key (for example as the BROWSE_AI_API_KEY environment variable, never pasted into a shared prompt) and the API reference, then give it the job. For example:

Using the Browse AI API, create a robot that tracks the prices on https://competitor.example.com/collections/standing-desks. Use the Robot Drafts endpoints, show me the preview data before you save it, then set up a daily monitor and a webhook to https://your-app.example.com/webhooks/browse-ai.

The agent creates the draft, polls it, reads the preview, refines the instructions or schema if columns are missing, and saves the robot once you're happy with the preview. Asking it to show you the preview before saving keeps the review step with you.

Retraining a robot

When a site changes or you need different data, you can retrain a robot from code too. Create a draft with the robot's robotId:

curl -X POST "https://api.browse.ai/v2/robot-drafts" \
  -H "Authorization: Bearer YOUR_SECRET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "originUrl": "https://competitor.example.com/collections/standing-desks",
    "robotId": "ROBOT_ID",
    "instructions": "Skip products that are out of stock."
  }'

  • The draft starts with the robot's current columns.

  • With only instructions, those columns are kept and the instructions apply on top of them.

  • With a schema, the schema replaces the robot's columns. Columns you leave out are removed from the robot when you save.

  • Preview, refine and save it the same way. Saving writes a new version of the same robot, which keeps its ID, and starts its first task right away.

Robots that can't be retrained this way

If a robot can't be retrained through the API, creating the draft is refused with robot_not_eligible_for_retraining, and the message says why and where to retrain it instead.

  • Robots created from a Prebuilt Robot can't be retrained.

  • Robots that log in with a password or use another saved secret can be retrained only in Robot Studio or the Chrome extension.

  • Robots trained in Robot Studio or the Chrome extension can't be retrained through the API if they log in with saved browser storage, or if they click, type or move between pages before extracting data. Robots that log in with cookies can be retrained.

Robots created from a robot draft, through the API or in Table Studio, can always be retrained unless they use a password or other saved secret.

Credits and limits

  • Training: previews and saving don't use credits while your plan's AI budget lasts. Once the budget is used up for the billing cycle, previews may use credits, or training is limited and you'll need to contact support to keep training robots, depending on your plan.

  • The first task: saving starts the robot's first task, which uses credits like any other task.

  • Plan limits: a new robot counts toward your plan's robot and website limits. If there aren't enough credits, or you've reached a limit, nothing is created, and you can save again once it's fixed.

  • If your plan doesn't include AI robot training, creating a draft is refused with training_disabled.

Good to know

  • Column suggestions aren't made for drafts created through the API. Send instructions, a schema, or both.

  • Formula columns can't be set through a draft.

  • Drafts stay in your draft list after they're saved, with robotId pointing to the robot.

  • Saved drafts can't be changed. To change the robot again, create a new draft with its robotId.

  • For every field, error and response, see the Robot Drafts API reference.

Did this answer your question?