How to deploy custom docker images for your web crawlers (opens in new tab)

[UPDATE]: Please see this article for an up-to-date version. What if you could have complete control over your environment? Your crawling environment, that is... One of the many benefits of our upgraded production environment, Scrapy Cloud 2.0, is that you can customize your crawler runtime environment via Docker images. It’s like a superpower that allows you […]