Understanding pagination
Pagination is how websites display large amounts of data across multiple pages. When scraping data, you'll need to tell your robot how to navigate through these pages to collect complete datasets.
Common pagination methods include:
Next page buttons: clicking on arrows or next links to move to another page
Load more buttons: clicking buttons that add more content to the current page
Infinite scrolling: scrolling down to trigger loading additional content
Page number links: clicking on specific page numbers
Setting up pagination in Robot Studio (recommended)
After selecting your data points:
Robot Studio will often automatically detect the appropriate pagination method. Simply review the suggested method and confirm it if it's accurate.
If Robot Studio doesn't automatically detect the pagination, or if the suggested method is incorrect, you'll need to manually select the appropriate method from the options provided.
Review the suggested method and confirm if it's accurate.
If Robot Studio doesn't detect the correct method, you can manually select from:
Click "Next" to navigate to the next page
Click "Load more" to add more items
Scroll down to load more items
No more items to load (when you've reached the end)
Setting up pagination in the Chrome Extension
Use the Capture List action to select the items you want to extract
After labeling your data points, a summary list will appear for review
Name your list and configure how many rows of data to extract
Choose the appropriate pagination method:
If using Next page or Load more, carefully select the button or link that advances to the next page
Make sure to choose the element that dynamically moves to the next page, not just a static link
Click the Capture List button to complete the setup
See:
Watch & learn