Scrapy:将参数传递给 cmdline.execute()

问题描述

我知道如何在从命令行运行爬虫蜘蛛时传递参数.但是，当我尝试使用 scrapy 的 cmdline.execute() 从脚本以编程方式运行它时遇到问题.

I know how to pass arguments when running a scrapy spider from the command line. However, I'm having problems when trying to run it programatically from a script using scrapy's cmdline.execute().

我需要传递的参数是我之前格式化为字符串的列表，就像这样:

The arguments I need to pass are lists that I previously formatted as strings, just like this:

numbers = "one,two,three,four,five"
colors = "red,blue,black,yellow,pink"

cmdline.execute('scrapy crawl myspider -a arg1='+numbers+' -a arg2='+colors)

蜘蛛是...

    class MySpider(Spider):

        name = "myS"

        def __init__(self, arg1, arg2):
            super(MySpider, self).__init__()

#Rest of the code

但是，当我运行它时，出现此错误:

However, when I run it I get this error:

  Traceback (most recent call last):
  File "C:/Users/ME/projects/script.py", line 207, in run
    cmdline.execute("scrapy crawl myS -a arg1="+numbers+" -a data="+colors)
  File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 123, in execute
    cmdname = _pop_command_name(argv)
  File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 57, in _pop_command_name
    del argv[i]
TypeError: 'str' object doesn't support item deletion

有什么想法吗?

操作系统:Windows7；Python 版本:2.7.8

OS: Windows7;Python version: 2.7.8

推荐答案

execute() 函数需要一个参数列表，而不是一个字符串.试试这个:

The execute() function expects a list of arguments, not a string. Try this:

cmdline.execute([
    'scrapy', 'crawl', 'myspider',
    '-a', 'arg1='+numbers, '-a', 'arg2='+colors])

这篇关于Scrapy:将参数传递给 cmdline.execute()的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！