scrapyd schedule.json setting 传入多个值

使用案例：

import requests

adder='http://127.0.0.1:6800'

data = {

        'project':'v1',

        'version':'12379',

        'setting':['ROBOTSTXT_OBEY=True','CONCURRENT_REQUESTS=32']

}

resp = requests.post(adder,data=data)

　　

 

问题解决思路：

版本1.2文档中：

• • setting (string, optional) - a Scrapy setting to use when running the spider

setting 是字符串，那么多个字符串该如何呢。

通过源码 site-packages/scrapyd/webservice.py

class Schedule(WsResource):

    def render_POST(self, txrequest):

        args = native_stringify_dict(copy(txrequest.args), keys_only=False)

        settings = args.pop('setting', [])

        settings = dict(x.split('=', 1) for x in settings)

        args = dict((k, v[0]) for k, v in args.items())

        project = args.pop('project')

        spider = args.pop('spider')

        version = args.get('_version', '')

        spiders = get_spider_list(project, version=version)

        if not spider in spiders:

            return {"status": "error", "message": "spider '%s' not found" % spider}

        args['settings'] = settings

        jobid = args.pop('jobid', uuid.uuid1().hex)

        args['_job'] = jobid

        self.root.scheduler.schedule(project, spider, **args)

        return {"node_name": self.root.nodename, "status": "ok", "jobid": jobid}

　可以发现 setting 在这里是list对象，且里面元素是字符，字符串必须包含=符号。

以此我们尝试传入setting:['DEBUG=True','HOST_NAME=axdda']

结果真的可以。对于比较复杂的设置，下次有需求再研究吧。