crawling-tool

List of libraries, tools and APIs for web scraping and data processing.

SpideyX a multipurpose Web Penetration Testing tool with asynchronous concurrent performance with multiple mode and configurations.

📱 A utility for downloading Android apps from the Google Play Store and Xiaomi App Store (the Chinese App Store).

🚀 OFFICIAL STARTER TEMPLATE FOR BOTASAURUS SCRAPING FRAMEWORK 🤖

An declarative and easy to use web crawler and scraper in C#

Web crawling & scraping framework for Node.js on top of headless Chrome browser

🕷️ Easily scrap the web for torrent and media files.

Firetastic - a tool for automating interaction with FireTV apps

AI Scraper : scrap and extract data from website in any format (CSV, JSON, HTML...) using Selenium or Crawl4ai, and using Ollama or Sambanova API, and using Streamlit for UI as chatbot

🚀 THIS WEB SCRAPING TEMPLATE PROVIDES YOU WITH A GREAT STARTING POINT WHEN CREATING WEB SCRAPING BOTS. 🤖

List of best web crawlers to extract data from the web. Find web crawling tools for different needs.

A Website Spider/Crawler, Python 3.x

This python script crawls course title, ratings, description and instructors from coursera.org

A web Crowler design , basic setup c++

Recon Crawling Endpoint and Parameter Scan Tool

Google AI Mode scraper API in Python and Node using ScrapingBee. Pull the AI-generated answer, citations, and sources as JSON.

Firetastic - a tool for automating interaction with FireTV apps

By scraping leads from Google Maps, you can build a database of potential customers who have shown interest in products or services related to your business. This data can be used for targeted marketing campaigns, email outreach, or sales prospecting.

Google popular times API alternative: place opening hours, ratings and review counts from Google Maps via ScrapingBee.

Web scraping API examples in Python, Node, Go, Java, PHP, Ruby, cURL and CLI. Proxies, JS rendering, AI extraction, 9 scrapers.