我试图使用Requests然后Beautifulsoup从网页中提取一些数据。我首先获取带有请求的html代码，然后在Beautifulsoup中“放入”：
from bs4 import BeautifulSoup
import requests
result = requests.get("https://XXXXX")
#print(result.status_code)
#print(result.headers)
src = result.content
soup = BeautifulSoup(src, 'lxml')
然后，我选择了一些代码段：
tags = soup.findAll('ol',{'class':'activity-popup-users'})
print(tags)
这是我得到的一部分：
<div class="account js-actionable-user js-profile-popup-actionable " data-emojified-name="" data-feedback-token="" data-impression-id="" data-name="The UN Times" data-screen-name="TheUNTimes" data-user-id="3787869561">
我现在想要的是提取
data-user-id=之后的数据，该数据由""之间的数字组成。然后，我希望将这些数据输入某种计算表中。我是一个绝对的初学者，我要粘贴在教程或文档中其他地方找到的代码。
非常感谢您的时间...
编辑：
所以这是我尝试的：
from bs4 import BeautifulSoup
import requests
result = requests.get("https://XXXX")
src = result.content
soup = BeautifulSoup(src, 'html.parser')
tags = soup.findAll('ol',{'class':'activity-popup-users'})
print(tags['data-user-id'])
这就是我得到的：
TypeError: list indices must be integers or slices, not str
所以我尝试了：
from bs4 import BeautifulSoup
import requests
result = requests.get("https://XXXX")
src = result.content soup = BeautifulSoup(src, 'html.parser')
#tags = soup.findAll('a',{'class':'account-group js-user-profile-link'})
tags = soup.findAll('ol',{'class':'activity-popup-users'})
tags.attrs
#print(tags['data-user-id'])
并得到：
File "C:\Users\XXXX\element.py", line 1884, in __getattr__
"ResultSet object has no attribute '%s'. You're probably treating a list of items like a single item. Did you call find_all() when you meant to call find()?" % key
AttributeError: ResultSet object has no attribute 'attrs'. You're probably treating a list of items like a single item. Did you call find_all() when you meant to call find()?
最佳答案
您可以通过将标签视为属性值字典来获取标签的任何属性值。
Read the BeautifulSoup documentation on attributes.
tag['data-user-id']
例如
html="""
<div class="account js-actionable-user js-profile-popup-actionable " data-emojified-name="" data-feedback-token="" data-impression-id="" data-name="The UN Times" data-screen-name="TheUNTimes" data-user-id="3787869561">
"""
from bs4 import BeautifulSoup
soup=BeautifulSoup(html,'html.parser')
tag=soup.find('div')
print(tag['data-user-id'])
输出量
3787869561
编辑以包含OP的问题更改：
from bs4 import BeautifulSoup
import requests
result = requests.get("http://twitter.com/RussiaUN/media")
src = result.content
soup = BeautifulSoup(src, 'html.parser')
divs = soup.find_all('div',class_='account')
#just print
for div in divs:
print(div['data-user-id'])
#write to a file
with open('file.txt','w') as f:
for div in divs:
f.write(div['data-user-id']+'\n')
输出：
255471924
2154112404
408696260
1267887043
475954041
3787869561
796979978
261711504
398068796
1174451010
...
关于python - 尝试从网页中提取一些数据(抓取初学者)，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/54792663/