因此,我的问题来自网页中的特定标签,该标签隐藏了我要剪贴的内容。
Here the link of the page
让我们向您展示我确切希望取消的内容。
由于无法插入图片,因为我错过了它的声誉,因此给了我一些指向图片主机的链接。
您可以在其中看到,“ COTES”标签内容实际上隐藏在html渲染中,正如您在红框中看到的那样。
但是我注意到,如果在< div id="pariCotesTab" class="tab" style="display: none;"
中将属性style
更改为style="display: block;"
,则会显示隐藏的html部分:you can see it in the second image。
当我使用scrapy shell https://www.zeturf.fr/fr/course/2018-10-19/R1C1-vincennes-prix-klymene/turf
并尝试获取“ COTES”选项卡的元素时,例如In [1]: response.xpath("//td[@class='cote-simplegagnant cote-reference']/text()").extract()
,它不返回任何Out 1: []
,这是合乎逻辑的,但它阻止了我。
那么如何在刮y的过程中将属性style
更改为style="display: block;"
?并在“ COTES”标签中获取要剪贴的内容。
我想提一下,我更喜欢不要使用splash,因为在docker安装方面这对我来说真是一个地狱。我不会使用selenium,因为我想剪贴大约1000页,甚至更多。我在前一个项目中使用了硒,而使用任何浏览器进行网络驱动只是浪费时间。那首先是一个网络测试器,而不是一个网络爬虫程序,而不是一个网络爬虫模块。
Scrapy版本:1.5.0,Python版本:2.7.9
最佳答案
您的问题不是您想像的那样。
Scrapy不在乎使用哪种CSS样式(除非您在选择器中明确使用它们),而不必关心页面源代码中存在的样式。
页面的源包含以下内容:
<th class="cote-simplegagnant cote-reference"></th>
如您所见,您选择的
th
为空,稍后将由javascript填充。仔细查看源代码,您可以找到包含所需信息的
script
标记:cotesInfos: {"referenceDateTime":{"date":"2018-10-19 19:30:00.000000","timezone_type":3,"timezone":"Europe\/Paris"},"meilleureCoteSG":{"reference":3.3,"live":3.6},"displayedColumns":{"hasSG":1,"hasSP":1,"hasZC":1,"hasZS":1},"1":{"odds_single":27.2,"odds_couillon":11.7,"odds_zeshow":29,"odds":{"reference":36.4,"SG":27.2,"SPMin":5,"SPMax":9.1,"ZC":11.7,"ZS":29},"oddsprogress_single":"moins"},"2":{"odds_single":13.3,"odds_couillon":13.6,"odds_zeshow":11.4,"odds":{"reference":14.5,"SG":13.3,"SPMin":2.3,"SPMax":4.1,"ZC":13.6,"ZS":11.4}},"3":{"odds_single":3.7,"odds_couillon":7.2,"odds_zeshow":8,"odds":{"reference":6.8,"SG":3.7,"SPMin":1.2,"SPMax":1.8,"ZC":7.2,"ZS":8},"oddsprogress_single":"moins"},"4":{"odds_single":274.1,"odds_couillon":19.6,"odds_zeshow":40.9,"odds":{"reference":168.9,"SG":274.1,"SPMin":13.5,"SPMax":41.7,"ZC":19.6,"ZS":40.9},"oddsprogress_single":"plus"},"5":{"odds_single":20.2,"odds_couillon":9,"odds_zeshow":13.1,"odds":{"reference":16,"SG":20.2,"SPMin":2.9,"SPMax":5.2,"ZC":9,"ZS":13.1},"oddsprogress_single":"plus"},"6":{"odds_single":9.4,"odds_couillon":11.7,"odds_zeshow":12.6,"odds":{"reference":4.8,"SG":9.4,"SPMin":3.2,"SPMax":5.8,"ZC":11.7,"ZS":12.6},"oddsprogress_single":"plus"},"7":{"odds_single":32.3,"odds_couillon":9.8,"odds_zeshow":11.4,"odds":{"reference":27.9,"SG":32.3,"SPMin":5.1,"SPMax":9.2,"ZC":9.8,"ZS":11.4},"oddsprogress_single":"plus"},"8":{"odds_single":78.2,"odds_couillon":16.3,"odds_zeshow":34.8,"odds":{"reference":109.3,"SG":78.2,"SPMin":8,"SPMax":14.7,"ZC":16.3,"ZS":34.8},"oddsprogress_single":"moins"},"9":{"odds_single":7.1,"odds_couillon":9.9,"odds_zeshow":9.5,"odds":{"reference":11.2,"SG":7.1,"SPMin":1.5,"SPMax":2.5,"ZC":9.9,"ZS":9.5},"oddsprogress_single":"moins"},"10":{"odds_single":3.6,"odds_couillon":18.9,"odds_zeshow":2.9,"odds":{"reference":3.3,"SG":3.6,"SPMin":1.6,"SPMax":2.7,"ZC":18.9,"ZS":2.9}},"11":{"odds_single":16.4,"odds_couillon":9.6,"odds_zeshow":13.1,"odds":{"reference":14.4,"SG":16.4,"SPMin":3.4,"SPMax":6,"ZC":9.6,"ZS":13.1},"oddsprogress_single":"plus"},"12":{"odds_single":21.3,"odds_couillon":6.7,"odds_zeshow":10,"odds":{"reference":23.3,"SG":21.3,"SPMin":3.8,"SPMax":6.8,"ZC":6.7,"ZS":10}},"13":{"odds_single":40.9,"odds_couillon":21,"odds_zeshow":27.8,"odds":{"reference":20.1,"SG":40.9,"SPMin":5.8,"SPMax":10.6,"ZC":21,"ZS":27.8},"oddsprogress_single":"plus"},"14":{"odds_single":34.8,"odds_couillon":10.8,"odds_zeshow":20.4,"odds":{"reference":22.2,"SG":34.8,"SPMin":5.2,"SPMax":9.5,"ZC":10.8,"ZS":20.4},"oddsprogress_single":"plus"}}
关于python - 如何在抓取过程中通过div更改div标签中样式属性的值?,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/53069244/