When scraping large lists of items with one Robot, you almost always have to handle pagination. With Browse AI, all you need to do is to select the type of pagination you see on the page.
When you build a Robot to capture a list of items, you get to select the specific data you need from each item and label them. Once you have labeled the last selection, a summary list will appear to review.
- Make sure to name the list you have captured.
- You can scroll vertically and horizontally to preview the data that you just selected.
There is a drop-down menu at the bottom-left that you can use if you want the robot to load more items by "scrolling down" or clicking on a "Next Page" or "Load More" button, depending on how the website displays large lists of items.
If you choose to select the "Next page" button, make sure you select a button that says "Next" or shows a next arrow (>). If you instead choose the button that always links to page number 2, the robot will not be able to proceed past page 2.
Once you are happy with the results and done gathering all the wanted information, go ahead and click on Capture List.