Implementation of the MIME Sniffing standard (https://mimesniff.spec.whatwg.org/)
A command-line utility for scraping Wayback Machine snapshots from archive.org.
Handy tools to maintain persistent meta values between requests in Scrapy spiders
Spidermon is a framework to build monitors for Scrapy spiders.
spider-feeder is a library to help loading inputs to scrapy spiders.
Provides a simple api interface with a scrapy backen for requesting complicated scraping tasks in an...
Run a Scrapy spider programmatically from a script or a Celery task - no project required.
JavaScript support for Scrapy using Splash