NOTE
The following help article is about using the Browse AI Chrome extension, but the recommended method is to use Robot Studio.
Learn more in How to build a robot using Robot Studio.
It's easy to build & train a robot to extract data from any website. You can watch this video below to learn more or scroll down to read the text version of the tutorial.
Watch & learn
Full guide
To train a new robot, go to your dashboard click on Build New Robot, and select between extracting structured data or monitoring site changes.
If you only need to extract data infrequently or on a one-time basis, you should choose the "Extract Structured Data" option. On the other hand, if you need to extract data on a regular basis, such as daily or hourly, you should select the "Monitor Site Changes" option.
On the following page, enter the URL of the webpage from which you want to extract data. For example, we will demonstrate how to extract the top 10 products of the day from Product Hunt.
If this is the first time you are training a robot, you will be prompted to install the Browse AI Extension when you click on "Use Chrome Extension."
Check out this brief video to see the entire process:
Once you complete the process above, the training mode will begin, and you can now start training your robot.
Remember: The robot will repeat every action that you take during the training mode, so it is best to only perform scrolls and clicks when they are necessary.
To begin, click on the robot and choose between capturing a list, capturing text on the page, or capturing a screenshot. Since we want to extract the top 20 products from ProductHunt, we will select "Capture List" here.
After clicking on "Capture List," you can now select the list you want to extract by simply clicking on it. As you move your mouse, you will see the robot highlight different sections of the page. When you see the robot highlight the data items and divide them, click on the list itself to select it.
Go ahead and click on the data points you need. When you've got them all, hit Enter to give each one a name.
Tip: You can use the Esc key to undo any changes to the data point names.
Once you have finished naming the fields, you can review the captured list as a table:
After confirming that you have captured all the necessary data, give your list a name, set the maximum number of products you want to extract, and select the pagination type. Since Product Hunt loads more products as you scroll down the page, we will select "Scroll down to load more items." You can read more about different types of pagination in this article.
If everything looks good, click "Capture List," click on the robot, and select "Finish recording."
Congratulations! you trained your first robot! Now name your robot and click save!
On the next page, review your extracted data. If everything looks good, click on "Yes, looks good".
What else can you do?
After training your robot to extract structured data, you have multiple other options to explore:
To download the data as a spreadsheet file, go to the "History" tab and find the task you just ran. Then, click "Download CSV" to save the data to your computer.
Run it again to extract fresh data by going to the Run Task tab.
Setting up a new Monitor to run this robot on your schedule and inform you if anything changes.