Skip to main content

How to automate data extraction from multiple search queries

You can extract data from up to 50,000 input parameters automatically using Bulk Run.

M
Written by Melissa Shires
Updated today

Bulk Run with input parameters lets you execute hundreds or thousands of different searches automatically using a single robot. Upload a CSV file with all your search terms and extract up to 50,000 searches at once.

What are input parameters?

Input parameters are variables created when you type text into form fields during robot training. They make your robot flexible - train once with one search term, then run the same robot with thousands of different terms without retraining.

Example: Train your robot searching for "laptop", and the search term becomes a parameter you can change to "tablet", "smartphone", or any other term when running the robot.

Prerequisites

Before you can perform bulk searches, you need:

βœ… A trained robot with search functionality

  • Robot must fill a search box during training

  • Search term must be an input parameter

βœ… A list of search terms

  • Needs be formatted as CSV for the upload

  • Maximum 50,000 rows per bulk run

Step-by-step guide

πŸ“˜ This guide assumes you've already trained and approved a robot to extract data from a search query.

Step 1: Prepare your CSV file

Create a CSV with column names matching your robot's parameters:

Basic search (single parameter):

searchTerm laptop tablet smartphone wireless headphones gaming keyboard

Advanced search (multiple parameters):

searchTerm,location,minPrice,maxPrice laptop,New York,500,1500 tablet,Los Angeles,200,800 smartphone,Chicago,300,1000

πŸ’‘ Column names must match parameter names exactly (case-sensitive). If you're unsure, you can download a sample CSV.

Step 2: Upload and run bulk search

  1. Go to your robot β†’ Click "Run Task" tab

  2. Click "Bulk Run Tasks" button

  3. Upload your CSV file

  4. Review the preview to ensure parameters mapped correctly

  5. Confirm to start the bulk run

  6. Browse AI will perform each search and extract results

Practical examples

Example 1: E-commerce price research

  • Scenario: Compare prices across 100 products

  • Robot trains on: Product search β†’ Extract name, price, availability

CSV file

searchTerm 
iPhone 15 Pro
Samsung Galaxy S24
iPad Air MacBook Pro 14
Sony WH-1000XM5
[... 95 more products]
  • Result: Table with all products and their current prices

Example 2: Job market analysis

  • Scenario: Research salaries for roles across cities

  • Robot trains on: Job search with location β†’ Extract title, salary, company

CSV file:

jobTitle,location Software Engineer,San Francisco Software Engineer,Austin Software Engineer,New York Data Scientist,San Francisco Data Scientist,Austin Product Manager,San Francisco [... more combinations]
  • Result: Salary data comparison across roles and locations

Example 3: Competitor monitoring

  • Scenario: Track product listings for multiple brands

  • Robot trains on: Brand search β†’ Extract product listings

CSV file:

brandName,category 
Nike,Running Shoes
Adidas,Running Shoes
Puma,Running Shoes
Nike,Basketball Shoes
Adidas,Basketball Shoes

Result: Complete inventory snapshot by brand and category

Example 4: Lead generation

  • Scenario: Find businesses in different cities

  • Robot trains on: Directory search β†’ Extract business details

CSV file:

businessType,city,state 
restaurants,Austin,TX
restaurants,Dallas,TX
dentists,Austin,TX
lawyers,Houston,TX
accountants,Austin,TX

Result: Business contact database organized by type and location

Advanced techniques

Using wildcards and variations

Include search variations to catch more results:

searchTerm 
laptop
laptops
laptop
computer
notebook
computer
portable computer

πŸ’‘ You can use an LLM (ChatGPT, Claude, etc.) to generate a list for you.

Combining with other parameters

You can use both Origin URL and search based on how the site you're extracting from is structured.

originUrl,searchTerm 
https://site.com/electronics,laptop
https://site.com/electronics,tablet
https://site.com/furniture,desk
https://site.com/furniture,chair

Building search terms programmatically

Generate comprehensive search lists using python or LLMs.

Python example:

products = ['laptop', 'tablet', 'phone'] 
brands = ['Apple', 'Samsung', 'Dell']
searches = [f"{brand} {product}" for brand in brands for product in products]

# Creates: Apple laptop, Apple tablet, Samsung laptop, etc.

LLM example (ChatGPT, Claude, etc.):

``` Generate a CSV list of search terms combining these elements: - Product types: laptop, tablet, smartphone - Brands: Apple, Samsung, Dell, HP, Lenovo - Years: 2023, 2024 Format: brand product year

Monitoring and management

  • Track progress via robot's History tab.

  • Check success rate in usage reports.

  • Set up notifications for completion.

  • Review failed searches for reprocessing.

Integrations and automations with other features

Scheduling bulk searches

Combine with monitoring to check for updates for specific input parameters.

Workflows with search

Connect search results to other robots to extract data across multiple pages.

  • Robot A: Search and get result URLs

  • Robot B: Extract details from each URL

API automation

Trigger bulk searches programmatically using the API.

POST /robots/{robotId}/bulk-run {   "data": [     {"searchTerm": "laptop", "location": "NYC"},     {"searchTerm": "tablet", "location": "LA"}   ] }

Limitations

Cannot run a bulk search with:

  • Dropdown selections (fixed during training)

  • Checkboxes or radio buttons

  • Image uploads

  • Interactive filters (sliders, maps)

Workarounds:

  • Create separate robots for different filter combinations

  • Use direct URL parameters if available

  • Chain multiple robots for complex searches

Did this answer your question?