Skip to main content

How to use monitors to automate data extraction

Learn how to automate data extraction by setting up monitors that track website changes and notify you when updates occur.

M
Written by Melissa Shires

What are monitors?

Monitors are automated tracking systems that watch for changes on specific web pages.

When you add a monitor to your robot, Browse AI automatically checks the page according to your schedule and notifies you when changes occur. You can add a monitor to any robot.

Setting up monitors

Each robot can have multiple monitors configured, each with its own URL and inputs. To add one, open the robot's Monitor tab and click Create New Monitor.

Here's what you can monitor:

  • Individual product pages for price changes

  • Category pages for new items

  • Search results for new listings

  • Competitor pages for content updates

  • Directory pages for new entries

Configuring monitor settings

When setting up a monitor, you can customize:

  • Monitoring frequency (every so many minutes, hours, days, weeks or months, or an advanced RRULE schedule; the fastest schedule depends on your plan)

  • Email notifications when the data changes (webhooks and integrations are set up separately, on the robot's Integrate tab)

  • Screenshot sensitivity (how big a visual change must be to count)

💡 Timezone note: Monitor schedules run in the timezone set on the monitor, which starts as your own timezone. To choose a time and a different timezone, clear Anytime next to At around.

Using monitors with integrations

Integrations are set up on the robot's Integrate tab and receive the data from your monitor checks. Connect your monitors directly to your workflow by:

  • Sending data to your preferred tools through our thousands of integrations

  • Creating custom webhooks for direct data delivery

  • Setting up automated actions when changes are detected

  • Storing historical data for trend analysis

Best practices for reliable monitoring

To ensure your monitors work effectively:

  • Test your robot thoroughly before setting up monitoring

  • Turn on email notifications for the monitors you want to hear about

  • Regularly review your monitors to ensure they're still needed

  • Keep your extraction patterns up to date

Did this answer your question?