Web scraping infrastructure for modern teams

Generative AI search API with real-time web research. Citations and customizable sources. Multiple tiers.

Low-cost Google SERP API by Traject Data. Clean data for SEO apps. As low as $0.15/1K at volume.

Multi-engine SERP API. Google, Bing, Baidu, YouTube, Amazon. Real-time structured JSON output.

All-in-one scraping API with AI extraction. 100% success rate on Google SERP. Formerly Scrapinghub.

Semantic search API powered by embeddings. Sub-350ms latency. Reduces AI hallucinations with factual web data.

AI-native search API built for LLMs and RAG. Real-time factual results with citations. SOC 2 certified.

Independent search API with 35B+ pages indexed. Only viable alternative after Bing API retirement. No ads, privacy-focused.

S3-compatible storage with ZERO egress fees. $15/TB/month. Perfect for frequently accessed scraped data.

Serverless SQL queries on S3. Query scraped data files directly without loading. Pay per query.

Serverless ETL service. Data catalog, crawlers, Spark-based transforms. Native S3 and Redshift integration.

Budget-friendly proxy provider. 30M+ IPs, free tier available. Datacenter and residential proxies.

Affordable residential and datacenter proxies. Static residential IPs available. Good for small-medium scraping projects.

65M+ residential and datacenter proxies. Pay-as-you-go pricing. Rotating and sticky sessions for scraping.

Cloud platform for web services, background workers, and cron jobs. Simple deployment for scraping services.

Deploy apps from GitHub with zero config. Auto-scaling containers for scraping workloads with built-in databases.

Amazon serverless compute. Run scrapers triggered by events or schedules. Integrates with S3, SQS for scraping pipelines.

Edge serverless platform. Run JavaScript/TypeScript at the edge with low latency. Great for lightweight scrapers and proxies.

A Python library to interact with ScrapingBee's API for headless browsers and proxy rotation

Official DataForSEO python client

Bright Data's python SDK, use it to call bright data's scrape and search tools. bypass any Bot-detection or Captcha and extract data from any website in seconds.

Python 3 package for easy integration with the API of capsolver captcha solving service to solver recaptcha, geetest and solve any other captchas. Capsolver.

a Python client library for SerpApi.

The Apify SDK for Python is the official library for creating Apify Actors in Python. It provides useful features like actor lifecycle management, local storage emulation, and actor event handling.

git mirror for Beautiful Soup 4.3.2

Serverless Python platform with native Playwright support. Run headless browsers at scale with built-in scheduling.

Python 3 package for easy integration with the API of 2captcha captcha solving service to bypass recaptcha, сloudflare turnstile, funcaptcha, geetest and solve any other captchas.

Async-first undetected browser automation based on nodriver. Clean API with Docker support for scalable scraping.

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

A scalable frontier for web crawlers

Deploy containers globally on edge servers. Run long-running scrapers with persistent storage and global distribution.

Undetected Playwright fork. Patches runtime to evade bot detection while maintaining full Playwright API compatibility.

Code-first ELT engine built on Singer. GitOps workflows, 300+ connectors, integrates with dbt.

Open-source super proxy aggregator. Manage multiple proxy providers in one place. Built-in fingerprinting and rotation.

Build production-ready AI agents in Python and TypeScript. Linux Foundation project.