Scrapy Add-on Management
Hi! My name’s Jakob, and I was lucky enough to be selected for this year’s Google Summer of Code. Here, I will blog about my progress and about the journey of making my first larger contribution to the...
View ArticleIntroducing the Technical Details
In my previous post, I introduced the Scrapy framework and its use cases, and touched that it provides a broad variety of hooks and mechanisms to extend its functionality. My GSoC project is to ease...
View ArticleTowards an Add-on Framework
Last time, we learned that most Scrapy extension hooks are controlled via dictionary-like settings variables. We allowed updating these settings from different places without having to worry about...
View ArticleMeet the Add-on Manager
Previously, I introduced the concept of Scrapy add-ons and how it will improve the experience of both users and developers. Users will have a single entry-point to enabling and configuring add-ons...
View ArticleThe add-on system in action
In my earlier posts, I have talked mostly about the motivation and the internal implementation of Scrapy’s add-on system. Here, I want to talk about how the add-on framework looks in action, i.e. how...
View ArticleThe base add-on class
Things are going smoothly for my Summer of Code. As the add-on system changed its orientation from a focus on user-friendliness more into the direction of being precise and fitting into the existing...
View ArticleThe party's over :(
Alright, that’s it. The “firm pencils down” date has come and gone last Friday. As a result, my two major pull requests, the per-key priorities for Scrapy’s dictionary-like settings and the add-on...
View Article
