How can I create a workflow connecting two robots?
Browse AI's workflow feature allows you to connect two robots to work together automatically. This feature solves a common data extraction challenge: getting data from both list pages and detail pages efficiently.
We commonly refer to this as deep scraping.
How it works
- Robot A extracts a list of items, including a URL to its detail page (e.g. products, jobs, etc)
- Robot B then extracts the data from each item (e.g. prices, descriptions, salaries, etc)
You simply set up your two robots and connect them in sequence. The system handles the rest, making data extraction from multiple pages seamless and efficient.
Choose Your Learning Path
- 🎥 Watch & learn - Follow along with our video tutorial
- 🚀 Quick start - Just the essential steps to get going
- 📖 Full guide - Complete instructions with screenshots and examples
🎥 Watch & learn
💡You can view the video below in fullscreen by clicking on the icon in the top right corner of the player.
🚀 Quick start
Before creating the workflow, you’ll need two robots:
- one robot that extracts a list of items, including a URL to its detail page
- a second robot that extracts the data from those detail pages
Example: Robot A extracts a list of products from an eCommerce website (title, price, link to product). Robot B extracts detailed information from the product page (description, stock, SKU, etc).
When both robots are created:
- From your Dashboard, click on the Workflows tab
- Click Add New Workflow
- Name the workflow something descriptive
- In the Robot A tab
- From the dropdown, select the robot that extracts the list of items
- Click the Next Step button
- In the Robot B tab
- From the dropdown, select the robot that extracts item details
- In the Origin URL dropdown, select the item from Robot A that links to the detailed item page
- Click the Next Step button
- In the Filter tab choose one of these options:
- Always
- Only if robot A finds changes while monitoring
- Only if robot A finds new items while monitoring
- Only if robot A finds new or changed items while monitoring
- Click the Save and Enable Workflow button
- On the Overview tab of your workflow, you can:
- Click the Run Task button to trigger Robot A to run, which kick starts the workflow
- Click the Create New Monitor button to add a monitoring schedule to Robot A, so it runs automatically
- Edit Robot A, the Filter, or Robot B by clicking the Edit button on the right side of those items
- See Robot B’s extracted data by clicking the View Data in Tables button
- Add data integrations to Robot B by clicking the + icon then selecting Add other integrations
- On the Settings tab of your workflow, you can:
- Rename the workflow by clicking into the box, changing the text, then clicking the Save button
- Delete the workflow by clicking the Delete Workflow button
📖 Full guide
Let’s connect two robots to create a workflow:
- Robot A: extract a list of companies from Y Combinator’s Companies page
- Robot B: extract company details from a single company page on Y Combinator
From your Dashboard, (1) click on the Workflows tab, then (2) Click Add New Workflow
Name the workflow something descriptive, like Extract YC Company Details.
Robot A: From the dropdown, select the robot that extracts the list of items. In this case it’s the robot called Extract Companies | Y Combinator. Then click the Next Step button.
Robot B: From the dropdown, select the robot that extracts item details. In this case it’s the robot called Extract Company Details | Y Combinator. From the Origin URL dropdown, select the item from Robot A that links to the detailed item page (in this case, Company URL). Then click the Next Step button.
Filter: Choose one of the following options, to decide when the workflow will run:
- a) Always (will run every time Robot A runs)
- b) Only if robot A finds changes while monitoring
- c) Only if robot A finds new items while monitoring
- d) Only if robot A finds new or changed items while monitoring
Save: After you’ve chosen the workflow Filter, click the Save and Enable Workflow button:
Overview: On this tab you can do the following.
- 1) Click the Run Task button to trigger Robot A to run, which kick starts the workflow
- 2) Click the Create New Monitor button to add a monitoring schedule to Robot A, so it runs automatically
- 3) Edit Robot A, the Filter, or Robot B by clicking the Edit button on the right side of those items
- 4) See Robot B’s extracted data by clicking the View Data in Tables button
- 5) Add data integrations to Robot B by clicking the + icon then selecting Add other integrations
Settings: On this tab you can do the following.
- 1) Rename the workflow by clicking into the box, changing the text, then clicking the Save button
- 2) Delete the workflow by clicking the Delete Workflow button
