Web Scraping

How to Scrape eBay: Listings, Reviews & Product Data

A practical Python guide to scraping eBay listings, prices, sellers, and product reviews, plus the legal limits, common challenges, and proxy setup to consider at scale.

How to Scrape Amazon (Without Getting Blocked)

A practical Python guide to scraping Amazon product, search, and review data at scale without getting blocked, using Webshare rotating proxies.

How to Scrape Google Lens

How to Scrape Google AI Overviews

How to Scrape Google Maps

How to Scrape Google Images

How to Scrape Google Shopping

Learn how to extract structured data from Google Shopping using Python, Playwright and Webshare Residential Proxies.

How to Scrape Google Knowledge Graph

Learn how to extract structured data from Google Knowledge Graph using Python and Playwright. Step-by-step guide with code to export entities to JSON/CSV.

How to Scrape Google News

Learn how to scrape Google News using Python, Playwright & Webshare proxies

How to Scrape Google Related Searches and 'People also search for'?

Learn How to Scrape Google Related Searches and 'People also search for' using Python, Playwright & Webshare proxies

How to Scrape Google Search Ads?

Learn how to scrape Google Search Ads results using Python, Playwright & Webshare proxies

How to Scrape Google SERP?

Learn how to scrape organic Google Search (SERP) results using Python, Playwright & Webshare proxies

How to Scrape Google Scholar?

Learn how to scrape Google Scholar using Python, Playwright & Webshare Proxy

How to Scrape Google Finance?

Learn how to scrape Google Finance using Python

How to Scrape Google Flights?

Learn how to scrape Flights data such as flights, flight duration, ticket prices and emissions .

How to Scrape Google Play? 3 Methods

Learn how to scrape Google Play reviews, apps and search results using Python and proxies

How to Scrape Google Play Movies & TV? [2026]

Learn how to scrape Google Play Movies & TV using Python and Beautiful Soup with a Proxy setup

How to Scrape Google Play Books Search & Products

In this guide, we demonstrate a two-step approach to scraping Google Play Books: first, collecting search results, and second, enriching individual book pages with additional details all using Webshare proxies.

Web Scraping on AWS Free Tier with Scrapy

Learn to setup a sample scraping project on AWS Free Tier with Scrapy

Proxy in AIOHTTP: 3 Effective Setup Methods Explained

Learn all that you need to successfully setup a proxy in AIOHTTP for Python

How to Bypass CAPTCHA with Playwright Automatically

Learn how to bypass CAPTCHA with Playwright automatically

Proxy in Axios: 3 Effective Setup Methods Explained

Learn to setup proxy in Axios in 3 easy and effective methods with code examples.

How to Bypass CAPTCHA with Selenium Automatically

Learn how to bypass CAPTCHA automatically in Selenium using 2Captcha solving service.

Octoparse Proxy Integration: 2 Methods Explained

Learn to integrate datacenter or residential proxy on Octoparse

Proxy in Node.js: 5 Common Setup Methods Explained

Learn to setup a proxy server in Node.js choosing one the 5 common setup methods.

Proxy in Linux: Terminal and Squid Methods Explained

Learn to setup proxy in Linux using Terminal or Squid method.

Proxy in Playwright: 3 Effective Setup Methods Explained

Learn to set up proxies in Playwright in 3 different methods, best fit for your use-case.

Proxy in Ubuntu: HTTP and SOCKS5 Methods Explained

Learn to setup forward proxy in Ubuntu using CLI or Squid methods.

Wget with Proxy: HTTP, SOCKS5 & Rotating Methods Explained

Learn to setup proxies on wget - a popular command-line utility, commonly used on Linux.

Proxy in Selenium: 3 Setup Methods Explained 

Learn to setup proxies in Selenium for enhanced web scraping, automation and testing.

Proxy in cURL: 3 Effective Setup Methods Explained

Learning proxy configurations in cURL enables seamless web scraping operations, enhancing anonymity and overcoming access restrictions.

Web Scraping in Scrapy: Working Example [5 Minutes]

A quick 5-minute guide to quickly set up a Scrapy project, create a spider, and start scraping data from a website.

Rotating Proxies in Scrapy: 2 Methods Explained

Learn two setup rotating proxies in Residential or Datacenter Proxies for Scrapy

Proxy Rotator in Python Requests: 2 Methods Explained

Implementing proxy rotation in Python Requests is helpful for effective web scraping, enhancing anonymity and reducing the risk of IP bans.

Proxy with Python Requests: 3 Setup Methods Explained

Incorporating proxies into your Python Requests workflow greatly enhances web scraping efficiency and reliability

Login in Puppeteer: Examples, Session Cookies & More

Using Puppeteer learn to login to Wikipedia, Facebook, Twitter or Gmail with Google single sign-on.

Downloading Files in Puppeteer: 5 Methods Explained

Downloading files using Puppeteer is a powerful way to automate web interactions and streamline the acquisition of resources from webpages.

Replay in Puppeteer: Create Session Recordings

The Puppeteer Replay library provides a high-level API that allows developers to automate and test web experiences efficiently.

Debugging Puppeteer: Setup, Tips & Examples

Learn the various ways of debugging in Puppeteer along with the tools and techniques associated with it.

Check Element in Puppeteer: How to Check if it Exists or Is Active?

Learn to check if element exists via Puppeteer and how to check whether they're in an active state.

How to Intercept Requests in Puppeteer?

Intercepting requests with Puppeteer offers powerful capabilities for testing and ensuring the robustness of web applications.

How to Wait For Button to be Enabled in Puppeteer?

This article guides you through the process of Puppeteer wait for button to be enabled on a webpage.

Upload File in Puppeteer: Automation Examples

With Puppeteer's convenient uploadFile() method, you can easily upload files to a site - we'll show you how.

Bypass or Solve CAPTCHA in Puppeteer: Working Examples

In this article, we explore strategies for bypassing CAPTCHA challenges in Puppeteer automation.

Puppeteer in Docker: How to Run Examples & Error Troubleshoot

We’ve explored the process of running Puppeteer in Docker, covering essential steps, optimizing performance, and troubleshooting common errors.

What is Puppeteer? Popular Node.js Library Explained

We explore what Puppeteer is, how it works, and what it can be used for.

Viewport in Puppeteer: How to Manipulate Default Size

We discuss changing viewport sizes, troubleshooting common issues like setViewport failures, and ensuring content visibility in Puppeteer.

How To Execute JavaScript on Page in Puppeteer: Examples

In this article, we explored the capabilities of Puppeteer in executing Javascript on web pages, focusing on a live example.

'networkidle0' vs. 'networkidle2' in Puppetteer

Understanding the distinctions between networkidle0 and networkidle2 in Puppeteer is crucial for optimizing web automation tasks.

Get Text From Element in Puppeteer: Examples

We discuss several scenarios where you would need to extract text and how to do it with Puppeteer using the methods available with real-life examples.

Using Puppeteer on AWS Lambda for Scraping

Puppeteer with AWS Lambda gives a powerful and cost-effective way to run large-scale web scraping and other web automation tasks without managing servers or infrastructure.

Scroll in Puppeteer: Scroll to Bottom, Top, or Into View

Fundamentals and advanced tips for effective scrolling in Puppeteer.

Fill & Submit Form in Puppeteer

Learn to fill & submit forms in Puppeteer.

Puppeteer vs. Playwright

Learn the differences and similarities between Puppeteer and Playwright.

How to scrape websites using Puppeteer?

Basic web scraping techniques using Puppeteer and Node.js.

Convert HTML to PDF Using Puppeteer

In this article, we covered two handy methods for turning HTML into PDFs with Puppeteer and Node.js.

Puppeteer vs. Cypress

Puppeteer excels in web scraping, while Cypress focuses on comprehensive end-to-end testing for web applications.

Puppeteer vs. Selenium

Learn more about differences and similarities between Puppeteer and Selenium.

Cookies in Puppeteer: How to Accept, Save, Load or Clear Them

In this guide, we’ll cover the following key aspects of cookie management in Puppeteer: How to Accept, Save, Load or Clear Cookies for Puppeteer web automation tasks.

5 Puppeteer Alternatives For Scraping & Application Testing

Let's cover 5 alternatives for Puppeteer depending on your goal - web scraping or application testing.

Downloading Images in Puppeteer: 6 Methods Explained

In this article, we explored six methods using Puppeteer for downloading images.

Input in Puppeteer: How to Enter, Retrieve or Clear Values?

Mastering input handling in Puppeteer is fundamental for efficient browser automation or scraping form data.

Timeout in Puppeteer: Setup Methods & Error Fixes

We discuss various Puppeteer timeout setup methods, troubleshooting common timeout errors, and strategies to enhance script performance.

Puppeteer Cluster: Basic & Advanced Setup Explained

If you are someone looking to perform mass scraping from various websites on Puppeteer - this article is for you.

Guide to Puppeteer Extra: Best Plugins For Scraping Ranked

In this article, we explore Puppeteer Extra and its selection of plugins, each designed to enhance the web scraping operations.

waitForNavigation in Puppeteer: Basic and Advanced Configuration

In this guide, we discuss the waitForNavigation function in Puppeteer, a method for ensuring precise navigation synchronization during web automation.

How to Use Puppeteer Stealth For Advanced Scraping?

Learn how Puppeteer Stealth goes beyond regular Puppeteer, enhancing stealth measures for advanced scraping tasks with code examples.

Proxy in Pyppeteer: Learn the Perfect Scraping Setup

In this article, we discuss several methods for setting up proxies in Pyppeteer to enhance web scraping capabilities. Whether you need a simple static proxy or a sophisticated residential proxy, Pyppeteer offers the flexibility to cater to your scraping requirements.

Page Reload in Puppeteer: Methods & Errors Explained

In this article, we cover various methods to reload web pages using Puppeteer from the simple page.reload() to executing client-side Javascript and which of the methods you should use for your case. Furthermore, we cover common page reload errors, offering solutions like extending timeouts and employing try-catch blocks.

End-to-End Testing with Puppeteer: Getting Started

Get started with E2E testing in Puppeteer: 4 simple tests explained.

How to Get HTML in Puppeteer?

In this guide, we discussed how to effectively work with Puppeteer to capture the page source HTML, extract specific elements, and handle Javascript-reliant web pages.

How to Take Screenshot in Puppeteer: Complete Guide

Puppeteer is a powerful tool for web scraping, and test automation, offering a high degree of flexibility and control over the screenshot-taking process.

Pyppeteer Tutorial: Installation & Code Examples

Pyppeteer is a robust and efficient tool for browser task automation and scraping of web data. Pyppeteer enables users to use a more "Pythonic" approach to web automation compared to Puppeteer and Selenium.

iFrame in Puppeteer: Guide For Developers

To access and extract critical data for scraping, developers need to know how to navigate, manipulate, and interact with iframes. This blog post will teach exactly that.

Click in Puppeteer: Guide to Master Puppeteer’s Clicking Methods

In this article, we’ll explain how to configure clicking in Puppeteer in 3 different methods, using Selector, XPath and Text search.

Wait For Page to Load in Puppeteer: 4 Methods Compared

In this article, we explored four commonly-used methods: load, domcontentloaded, networkidle0, and networkidle2 to determine when the page is ready for interaction.

waitForSelector in Puppeteer: Basic and Advanced Configuration

In this article, we’ll explore both the fundamental details and advanced configuration of waitForSelector in Puppeteer. From waiting for a single element to appear on a web page to modifying the timeout and handling multiple selectors, we’ll discuss all the details.

Get Element in Puppeteer: Mastering Class, ID and Text Methods

In this article, we will discuss Puppeteer’s methods for precisely targeting and manipulating elements on web pages. We’ll explore three essential techniques: selecting elements by class, by ID, and by their text content.

Proxy in Puppeteer: 3 Effective Setup Methods Explained

In this article, we’ll explain how to setup proxies in Puppeteer in 3 different methods, using Static, Proxy List and Rotating proxies.

User-Agent in Puppeteer: Random & Custom Methods Explained

In this article, we’ll dive deeper into the user-agent manipulation in Puppeteer, using random or custom user-agent setups.