Skip to content

@scrapy-plugins Scrapy Plugins

Change the repository type filter

All

    Repositories list

    33 repositories

    • scrapy-playwright

      Public
      🎭 Playwright integration for Scrapy
      pythonpython3scrapyhacktoberfestchrome-headlesspython-asyncioheadless-browserjavascript-rendererfirefox-headlessplaywright
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •150•1.3k•17•2•Updated Dec 11, 2025Dec 11, 2025

    • scrapy-zyte-api

      Public
      Zyte API integration for Scrapy
      plugincrawlerproxyscrapingscrapy
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •22•39•17•4•Updated Nov 9, 2025Nov 9, 2025

    • scrapy-crawl-maps

      Public
      Scrapy plugin that allows defining the logic of a spider using a directed graph defined in JSON format.
      BSD 3-Clause "New" or "Revised" License
      •0•3•0•1•Updated May 21, 2025May 21, 2025

    • scrapy-spider-metadata

      Public
      Python
      •
      MIT License
      •3•7•3•1•Updated Mar 24, 2025Mar 24, 2025

    • scrapy-zyte-smartproxy

      Public
      Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
      plugincrawlerproxyscrapingscrapycrawler-detection
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •90•368•7•1•Updated Mar 24, 2025Mar 24, 2025

    • scrapy-deltafetch

      Public
      Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls
      hacktoberfesthacktoberfest2021
      Python
      •48•276•13•4•Updated Feb 26, 2025Feb 26, 2025

    • scrapy-feedexporter-sftp

      Public
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •4•7•0•0•Updated Feb 12, 2025Feb 12, 2025

    • scrapy-splash

      Public
      Scrapy+Splash for JavaScript integration
      scrapyheadless-browsers
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •452•3.2k•62•7•Updated Feb 11, 2025Feb 11, 2025

    • zyte-spidermon

      Public
      Python
      •1•1•0•1•Updated Sep 5, 2024Sep 5, 2024

    • scrapy-settings-log

      Public
      An extension that allows a user to display all or some of their scrapy spider settings at runtime.
      Python
      •
      GNU General Public License v3.0
      •4•1•0•0•Updated Aug 20, 2024Aug 20, 2024

    • scrapy-feedexporter-dropbox

      Public
      Scrapy feed exporter for Dropbox
      Python
      •3•2•0•0•Updated Jul 11, 2024Jul 11, 2024

    • scrapy-crawlera-fetch

      Public archive
      Scrapy Downloader Middleware for Crawlera Fetch API
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •10•9•5•5•Updated Jun 5, 2024Jun 5, 2024

    • scrapy-feedexporter-azure-storage

      Public
      Python
      •6•4•0•3•Updated May 18, 2024May 18, 2024

    • scrapy-feedexporter-onedrive

      Public
      Export to OneDrive
      Python
      •1•3•0•0•Updated Feb 23, 2024Feb 23, 2024

    • scrapy-djangoitem

      Public
      Scrapy extension to write scraped items using Django models
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •87•503•10•3•Updated Oct 15, 2023Oct 15, 2023

    • scrapy-feedexporter-google-sheets

      Public
      Python
      •1•2•0•0•Updated Aug 15, 2023Aug 15, 2023

    • scrapy-incremental

      Public
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •1•3•0•0•Updated May 1, 2023May 1, 2023

    • scrapy-feedexporter-google-drive

      Public
      Python
      •3•4•1•0•Updated Jan 4, 2023Jan 4, 2023

    • scrapy-statsd

      Public
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •3•7•1•2•Updated Oct 19, 2022Oct 19, 2022

    • scrapy-dotpersistence

      Public
      A scrapy extension to sync `.scrapy` folder to an S3 bucket
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •8•18•2•1•Updated Mar 28, 2022Mar 28, 2022

    • scrapy-magicfields

      Public
      Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •6•56•0•0•Updated Mar 16, 2022Mar 16, 2022

    • scrapy-pagestorage

      Public
      A scrapy extension to store requests and responses information in storage service
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •6•27•3•0•Updated Mar 11, 2022Mar 11, 2022

    • scrapy-snowflake-stage-exporter

      Public
      Snowflake database loading utility with Scrapy integration
      etlsnowflakescrapy
      Python
      •2•6•0•0•Updated Dec 3, 2021Dec 3, 2021

    • scrapy-headless

      Public
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •8•29•2•2•Updated Apr 28, 2021Apr 28, 2021

    • scrapy-jsonschema

      Public
      Scrapy schema validation pipeline and Item builder using JSON Schema
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •12•45•1•0•Updated Mar 26, 2021Mar 26, 2021

    • scrapy-jsonrpc

      Public
      Scrapy extension to control spiders using JSON-RPC
      Python
      •71•300•7•2•Updated Aug 26, 2019Aug 26, 2019

    • scrapy-hcf

      Public
      Scrapy spider middleware to use Scrapinghub's Hub Crawl Frontier as a backend for URLs
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •6•5•1•1•Updated Aug 28, 2018Aug 28, 2018

    • scrapy-monkeylearn

      Public
      A Scrapy pipeline to categorize items using MonkeyLearn
      Python
      •13•38•0•0•Updated Apr 28, 2017Apr 28, 2017

    • scrapy-splitvariants

      Public
      Scrapy spider middleware to split an item into multiple items using a multi-valued key
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •1•21•0•0•Updated Feb 8, 2017Feb 8, 2017

    • scrapy-streaming

      Public
      Python
      •5•19•2•0•Updated Oct 12, 2016Oct 12, 2016