我不确定是否由于mechanize
而无法获取完整表
这有效:
from bs4 import BeautifulSoup
import requests
page = 'http://www.airchina.com.cn/www/jsp/airlines_operating_data/exlshow_en.jsp'
r = requests.get(page)
r.encoding = 'utf-8'
soup = BeautifulSoup(r.text)
div = soup.find('div', class_='mainRight').find_all('div')[1]
table = div.find('table', recursive=False)
for row in table.find_all('tr', recursive=False):
for cell in row('td', recursive=False):
print cell.text.split()
但这不是:
import mechanize
from bs4 import BeautifulSoup
import requests
URL='http://www.airchina.com.cn/www/jsp/airlines_operating_data/exlshow_en.jsp'
control_year=['2006', '2007', '2008', '2009', '2010', '2011', '2012', '2013', '2014']
control_month=['01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12']
br = mechanize.Browser()
r=br.open(URL)
br.select_form("exl")
control_m = br.form.find_control('month')
control_y = br.form.find_control('year')
br[control_m.name]=['06']
br[control_y.name]=['2012']
response = br.submit()
soup = BeautifulSoup(response,'html.parser')
#div = soup.find('div', class_='mainRight')
div = soup.find('div', class_='mainRight').find_all('div')[1]
table = div.find('table', recursive=False)
for row in table.find_all('tr', recursive=False):
for cell in row('td', recursive=False):
print cell.text.strip()
即使在萤火虫中我看到所有的
mechanize
和tr
,使用td
的代码只会产生以下内容Jun 2012
% change vs Jun 2011
% change vs May 2012
Cumulative Jun 2012
% cumulative change
最佳答案
将两者结合时,它可以正常工作,因此可能与您使用的html.parser
有关。
import mechanize
from bs4 import BeautifulSoup
URL = ('http://www.airchina.com.cn/www/jsp/airlines_operating_data/'
'exlshow_en.jsp')
control_year = ['2006', '2007', '2008', '2009', '2010', '2011', '2012', '2013',
'2014']
control_month = ['01', '02', '03', '04', '05', '06', '07', '08', '09', '10',
'11', '12']
br = mechanize.Browser()
r = br.open(URL)
br.select_form("exl")
control_m = br.form.find_control('month')
control_y = br.form.find_control('year')
br[control_m.name] = ['06']
br[control_y.name] = ['2012']
response = br.submit()
soup = BeautifulSoup(response)
div = soup.find('div', class_='mainRight').find_all('div')[1]
table = div.find('table', recursive=False)
for row in table.find_all('tr', recursive=False):
for cell in row('td', recursive=False):
print cell.text.split()
关于python - Python beautifulsoup无法获取完整表,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/23213367/