In Selenium, CAPTCHA is a common hurdle when automating web testing workflows. Designed to distinguish between human users and bots, CAPTCHAs present tasks like identifying objects in images, solving puzzles, or entering distorted text, which are straightforward for humans but difficult for automated systems. While CAPTCHAs are effective for preventing abuse, they can disrupt Selenium scripts, making it challenging to fully automate processes on CAPTCHA-protected websites.
How to automatically bypass CAPTCHA with Selenium? →
In this article, we’ll walk you through the steps to automatically bypass CAPTCHA in Selenium, ensuring smooth execution of your test or scraping scripts.
Prerequisites
Before setting up the process to bypass CAPTCHA in Selenium, ensure the following prerequisites are in place:
- Install Selenium: Ensure that Python is installed on your system, then install the Selenium Python library using pip:
pip install selenium Additionally, upgrade to Selenium 4 or later for built-in WebDriver support:
pip install --upgrade selenium- CAPTCHA-Solving Service: You’ll need a third-party service like 2Captcha or Anti-Captcha to solve CAPTCHA. Obtain an API key by registering on one of these platforms.
- Proxies (Optional): Use a proxy service like Webshare to avoid detection and bypass restrictions. Webshare offers 10 free shared datacenter proxies with 1GB bandwidth per month, along with options for rotating and static proxies. Rotating proxies can significantly lower the amount of CAPTCHA requests that are triggered - saving you money on expensive CAPTCHA solving third-party services. We even have a Selenium proxy guide to help you get started.
- Install Additional Python Libraries: For integrating CAPTCHA-solving services, you may need libraries like 2captcha-python. Install using the below command:
pip install 2captcha-python How to automatically bypass CAPTCHA with Selenium?
CAPTCHAs are often a barrier when automating web interactions. This method demonstrates how to bypass CAPTCHA using Selenium and the 2Captcha service with a simple example.
Here's how you can integrate it step by step.
Step 1: Import required modules
Start by importing all necessary libraries for Selenium, twocaptcha, and time management.
from selenium.webdriver.common.by import By
from twocaptcha import TwoCaptcha
from selenium import webdriver
import timeStep 2: Launch the browser and navigate to the target page
Use Selenium to open a browser instance and navigate to the page with the CAPTCHA you want to bypass.
driver = webdriver.Chrome() # Ensure you have the ChromeDriver installed
url = "https://2captcha.com/demo/normal" # Demo page for testing
driver.get(url)Step 3: Solve the CAPTCHA using 2captcha
Locate the CAPTCHA image, extract its URL, and pass it to the solver.normal() method of 2Captcha. Replace Your_2Captcha_API_key with your actual API key.
imgResults = driver.find_elements(By.XPATH, "//img[contains(@class,'_2hXzbgz7SSP0DXCyvKWcha')]")
solver = TwoCaptcha('Your_2Captcha_API_key') # Initialize the solver
result = solver.normal(imgResults[0].get_attribute("src")) # Solve the CAPTCHA
print("Solved CAPTCHA: " + str(result))Step 4: Fill in the CAPTCHA solution and submit the form
Find the CAPTCHA input field and submit button, then use the solution returned by 2Captcha to complete the process.
captchafield = driver.find_element(By.XPATH, "//input[contains(@class,'_26Pq0m_qFk19UXx1w0U5Kv')]")
captchafield.send_keys(result["code"]) # Enter the CAPTCHA solution
button = driver.find_element(By.XPATH, "//button[contains(@class, 'l2z7-tVRGe-3sq5kU4uu5 _2xjDiWmBxfqem8nGQMmGci _2HIb5VBFp6Oi5_JoLdEcl6 _2vbG_IBm-DpI5KeEAHJkRy')]")
button.click() # Submit the form
time.sleep(10) # Allow time for the page to loadStep 5: Verify the result
Finally, confirm that the CAPTCHA has been bypassed by checking the success message on the page:
messagefield = driver.find_element(By.XPATH, "//p[contains(@class,'_2WOJoV7Dg493S8DW_GobSK')]")
print("Result: " + messagefield.text) # Output: Captcha is passed successfully!Apart from using 2Captcha, you can also leverage the selenium-stealth Python package to avoid detection when scraping with Selenium. This package helps mimic human-like behavior, making your automated traffic appear more manual and reducing the likelihood of encountering CAPTCHAs or being blocked.
Fixing common issues
When automating CAPTCHA bypass with Selenium, there are several common issues you may encounter.
Headless mode detection
Problem: Some websites can detect when Selenium is running in headless mode and may block access or treat the request as a bot. This can happen even when you set a custom user-agent.
Solution: Use a headless browser that mimics human behavior more effectively. Tools like undetected-chromedriver help bypass detection. You can also set a custom user-agent along with other arguments like window size and enabling certain features to avoid detection.
from undetected_chromedriver.v2 import Chrome, ChromeOptions
options = ChromeOptions()
options.add_argument('--headless')
options.add_argument('user-agent=your_custom_user_agent')
driver = Chrome(options=options)User-agent resetting
Problem: When you set a custom user-agent in Selenium, it may only apply to the initial page load. Subsequent page requests might reset the user-agent to its default value.
Solution: Ensure that the user-agent is set consistently for each request or use the set_preference method to ensure the setting persists across all page loads.
options = webdriver.ChromeOptions()
options.add_argument('user-agent=your_custom_user_agent')
driver = webdriver.Chrome(options=options)
driver.get("https://example.com")
# Continue with your automationIssues with headless mode in custom user-agent
Problem: When using Selenium with Chrome in headless mode, some users report issues like the browser failing to launch correctly even after setting a custom user-agent.
Solution: Ensure that the --headless flag is used correctly. Sometimes, specifying the window size or enabling the GPU feature may resolve the issue.
options = webdriver.ChromeOptions()
options.add_argument('user-agent=your_custom_user_agent')
options.add_argument('--headless')
options.add_argument('--window-size=1920x1080') # Specify window size
driver = webdriver.Chrome(options=options)Wrapping up: bypass CAPTCHA with Selenium
Bypassing CAPTCHA in Selenium can significantly enhance your automation tasks, but it requires careful handling of web driver settings, custom user-agent configurations, and integration with CAPTCHA-solving services. By following the steps outlined and addressing common issues such as headless mode detection, user-agent persistence, and JavaScript execution problems, you can streamline your automation process.
