本文介绍了使用 Feed Exporter 将项目从 Scrapyd 保存到 Amazon S3的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
在亚马逊 S3 上使用 Scrapy 相当简单，你设置:
Using Scrapy with amazon S3 is fairly simple, you set:
- FEED_URI = 's3://MYBUCKET/feeds/%(name)s/%(time)s.jl'
- FEED_FORMAT = 'jsonlines'
- AWS_ACCESS_KEY_ID = [访问密钥]
- AWS_SECRET_ACCESS_KEY = [密钥]
一切正常.
但 Scrapyd 似乎覆盖了该设置并将项目保存在服务器上(在网站中有一个链接)
But Scrapyd seems to override that setting and saves the items on the server (with a link in the web site)
添加items_dir ="设置似乎没有任何改变.
Adding the "items_dir =" setting doesn't seem to change anything.
什么样的设置使它起作用?
What kind of setting makes it work?
可能相关的额外信息 - 我们正在使用 Scrapy-Heroku.
Extra info that might be relevant - we are using Scrapy-Heroku.
推荐答案
我也遇到了同样的问题.从 scrapyd.conf 文件中删除 items_dir= 对我有用.
I also faced the same problem.Removing the items_dir= from scrapyd.conf file worked for me.
这篇关于使用 Feed Exporter 将项目从 Scrapyd 保存到 Amazon S3的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！