✅ In order to automatically create a historical database, enable monitoring for your robot.
Historical data includes all previous extractions with identical parameters:
Same Origin URL
Same input parameters (search terms, filters, etc.)
Multiple extraction dates
For example, if you monitor Amazon prices daily for "laptop", you'll have one row for each day's extraction - that's your historical data.
Show Historical Data
In Tables, you have the ability to show or hide historical data using the checkmark in the top right corner.
☑ Show Historical Data (checked)
Displays:
All extraction runs ever performed.
Multiple versions of the same data.
Complete timeline of changes.
Every attempt, including duplicates.
Perfect for:
Tracking price changes over time.
Monitoring inventory fluctuations.
Analyzing content evolution.
Building trend reports.
☐ Show Historical Data (unchecked)
Shows only:
Most recent extraction for each unique parameter set.
Current state of your data.
Latest successful run per URL.
Deduplicated view without repetition.
Perfect for:
Current data snapshots.
Clean exports without duplicates.
Status dashboards.
Quick data reviews.
Example - product page monitoring
Imagine monitoring a product page daily for a week.
With historical data ON:
7 rows (one per day)
See price progression: $99 → $95 → $89 → $92 → $87 → $85 → $79
Identify trends and patterns
With historical data OFF:
1 row (latest extraction)
See current price: $79
Clean, current view
Working with historical data effectively
Identifying extraction versions
Each historical entry includes:
Extract Date - When the data was captured
Status - Success or failure of that run
Origin URL - What page was scraped
Input parameters - What variations were used
Common historical data patterns
Daily monitoring
Same URL extracted every day
Historical view shows progression
Hide historical for current snapshot
Bulk operations
Multiple URLs extracted once
Historical data rarely needed
Each URL typically has one entry
A/B testing
Same URL with different parameters
Historical tracks all variations
Essential for comparison
Combining historical toggle with filters
View this week's price changes
☑ Show Historical Data
Filter: Extract Date = This week
Filter: Status = Successful
Result: Complete price history for the week
Latest data from all sources
☐ Show Historical Data
No filters needed
Result: Current state across all URLs
Failed extractions over time
☑ Show Historical Data
Filter: Status = Failed
Result: Pattern of failures to investigate
Historical data and exports
💡 Enabling the historical checkbox applies when you export data.
Exporting with historical ON:
Includes all versions
Larger file sizes
Complete audit trail
Time-series analysis ready
Exporting with historical OFF:
Only latest data
Smaller, cleaner files
Current state snapshot
No duplicates
Use cases for historical data
Price monitoring
Track product prices over time:
Daily extractions build price history
Identify best times to buy
Spot pricing patterns
Alert on significant changes
Content tracking
Monitor website updates:
Track article revisions
Document policy changes
Archive terms of service
Prove content timing
Inventory analysis
Watch stock levels:
Build availability patterns
Predict restocking cycles
Identify fast-moving items
Plan purchasing decisions
Competitive intelligence
Monitor competitor changes:
Track pricing strategies
Document feature updates
Analyze marketing messages
Build competitive timeline
Managing large historical datasets
Use filters strategically
Date ranges to focus on recent data
Status filters to exclude failures
Origin URL filters for specific sources
Export in chunks
Monthly exports for archiving
Filtered exports for analysis
Recent-only for reporting
Plan extraction frequency
Daily for volatile data
Weekly for stable content
Monthly for archival purposes