What sites does Browse AI work on?

Our success rate for newly created custom robots is about 90%. The remaining 10% fail because of the following reasons.

Captcha

We support solving several types of captcha (e.g., ReCaptcha, hCaptcha), but not all of them. For example, we currently do not solve custom captchas.

Bot detection

Some sites have strong bot detection mechanisms. Browse AI tries to mimic human behavior and uses a browser session just like a regular user, with the same delays, pauses, and scrolling. We also rotate through IP addresses (usually located in the same country as the user who set up the robot) and almost never use the same IP address twice.

However, no matter what we do, if your robot has to log into the website (either using your session cookies or your login credentials), the website can always detect two things:

1. This user is logging in from at least two different IP addresses (your local IP address and Browse AI's IPs)
2. If your robot is running a lot of tasks, it can seem suspicious.

For these reasons, logged-in robots have a higher chance of getting detected whenever you use cloud-based web automation. The only way to work around this would be to run the automation on your local machine, which is currently not supported. Just a heads up, MFA (multi-factor authentication), or 2FA (two-factor authentication), is not supported as well.

A/B tests on websites

When a site has an A/B test going on if the variation you trained your robot on is not the same variation that the robot sees when it runs a task, it might fail or extract the wrong data. The robot can adjust to some variations, but not all of them. 

Sites rendering virtual lists

A small portion of sites renders virtual lists. Virtual lists are lists where as you scroll down, only the visible list items are rendered, and anything that is not visible does not get rendered at all. We have not made the software compatible with these lists yet, but it is something we want to support in the future.

iFrames on websites

We do not support interacting with or extracting data from iFrames yet. A workaround can be inspecting the iFrame to find its URL. Some iFrames have a URL that can be used to load that iFrame directly. Then, you can train a new robot that loads that iFrame and interacts with it or extracts data from it.

In conclusion

All this being said, given that there are billions of websites out there, there are always edge cases to be found that typically come from inaccessible code or patterns that are against the standards of those websites. Whenever we come across a new edge case, we try to update the software to be able to handle that edge case and any similar issue on other sites.

Did you just create a new custom robot that failed for a reason other than the above?

Make sure you use the "Report" button to report it to us. We will investigate and get back to you.