我正在尝试从CNN文章中提取元标记

import httplib2
from bs4 import BeautifulSoup

http = httplib2.Http()
status, response = http.request(http://www.cnn.com/2016/08/09/health/chagas-sleeping-sickness-leishmaniasis-drug/index.html)
soup = BeautifulSoup(response)
print(soup.select('body > div.pg-right-rail-tall.pg-wrapper.pg__background__image > article > meta'))


我正在尝试将其缩小到仅此输出

<meta content="health" itemprop="articleSection"><meta content="2016-08-09T12:10:24Z" itemprop="dateCreated"><meta content="2016-08-09T12:10:24Z" itemprop="datePublished"><meta content="2016-08-09T12:10:24Z" itemprop="dateModified"><meta content="http://www.cnn.com/2016/08/09/health/chagas-sleeping-sickness-leishmaniasis-drug/index.html" itemprop="url"><meta content="Meera Senthilingam, for CNN" itemprop="author"><meta content="Could one discovery take on three deadly parasites?  - CNN.com" itemprop="headline"><meta content="Three seemingly different diseases infect 20 million people each year: Chagas disease, leishmaniasis and African sleeping sickness. But one drug could be developed to fight all three." itemprop="description"><meta content="sleeping sickness, disease, drug, drug development, chagas disease, leishmaniasis, Novartis, health, Could one discovery take on three deadly parasites?  - CNN.com" itemprop="keywords"><meta content="http://i2.cdn.turner.com/cnnnext/dam/assets/150812101743-chagas-bug-large-tease.jpg" itemprop="image"><meta content="http://i2.cdn.turner.com/cnnnext/dam/assets/150812101743-chagas-bug-large-tease.jpg" itemprop="thumbnailUrl"><meta content="Could one discovery take on three deadly parasites? " itemprop="alternativeHeadline">


但由于某种原因,BeautifulSoup.select()方法返回的HTML大约是我想要的100倍。我非常感谢有关如何解决此问题的任何建议。

最佳答案

问题出在解析器/ html,lxml和html5lib给您您想要的东西。

soup = BeautifulSoup(response,"lxml")


要么:

 soup = BeautifulSoup(response,"html5lib")


如果您没有安装lxml或html5lib,则可以使用pip安装html5lib。lxml取决于您的操作系统,因为它有一些依赖性,因此涉及更多,但是绝对值得安装。

您还可以简化选择:

soup.select('div.pg-right-rail-tall.pg-wrapper.pg__background__image meta')

关于python - 尝试从新闻文章中提取元数据,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/38907114/

10-16 16:09
查看更多