我正在尝试使用scrapy填写POST表单，以尝试预订火车票。
我以为FormRequest类可以完成此操作，但是我无法处理javascript表单。 Scrapy搜寻器不返回任何内容。
我使用的文件足以发送表格。
import scrapy
from scrapy.item import Item, Field
from scrapy.http import FormRequest
from scrapy.spider import BaseSpider
class SncfItem(Item):
title = Field()
link = Field()
desc = Field()
class SncfSpider(scrapy.Spider):
name = "sncf"
allowed_domains = ["voyages-sncf.com"]
start_urls = (
'http://www.voyages-sncf.com/billet-train',
)
def parse(self, response):
yield FormRequest.from_response(response,
formname='saisie',
formdata={'ORIGIN_CITY': 'Gare de Lyon (Paris)',
'DESTINATION_CITY': 'Lyon Part-Dieu',
'OUTWARD_DATE': '03.06.2015'},
callback=self.parse1)
def parse1(self, response):
print response.status
如果我在使用mySpider时错过了一步，谁能告诉我？
任何帮助将不胜感激。
谢谢
scrapy crawl sncf -o items.xml -t xml
2015-06-01 23:13:54+0200 [sncf] INFO: Spider opened
2015-06-01 23:13:54+0200 [sncf] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2015-06-01 23:13:54+0200 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6024
2015-06-01 23:13:54+0200 [scrapy] DEBUG: Web service listening on 127.0.0.1:6081
2015-06-01 23:13:55+0200 [sncf] DEBUG: Crawled (200) <GET http://www.voyages-sncf.com/billet-train> (referer: None)
2015-06-01 23:13:56+0200 [sncf] DEBUG: Redirecting (302) to <GET http://www.voyages-sncf.com/billet-train> from <POST http://www.voyages- sncf.com/vsc/train-ticket/>
2015-06-01 23:13:56+0200 [sncf] DEBUG: Crawled (200) <GET http://www.voyages-sncf.com/billet-train> (referer: http://www.voyages- sncf.com/billet-train)
200
2015-06-01 23:13:56+0200 [sncf] INFO: Closing spider (finished)
最佳答案
由于日期格式无效，您将获得重定向。
我在Scrapy shell中重播了以下请求：
$ scrapy shell http://www.voyages-sncf.com/billet-train
.... a few log messages later, I get a shell with the response...
>>> # first I recreate the FormRequest per your code:
>>> fr = FormRequest.from_response(response,
formname='saisie',
formdata={'ORIGIN_CITY': 'Gare de Lyon (Paris)',
'DESTINATION_CITY': 'Lyon Part-Dieu',
'OUTWARD_DATE': '03.06.2015'})
>>> # checked the url and method:
>>> fr.url
'http://www.voyages-sncf.com/vsc/train-ticket/'
>>> fr.method
'POST'
>>> fetch(fr) # execute the request
>>> view(response) # opened the result in browser
查看结果后，我看到了该日期的验证错误消息，说：“格式错误，格式错误。请使用”。