如何使用BeautifulSoup遍历网站的每个页面以进行Web抓取

推荐答案首先，BeautifulSoup与获取网页没有任何关系-您可以自己获取网页，然后将其提供给bs4进行处理.First, BeautifulSoup doesn't have anything to do with GETing web pages - you get the webpage yourself, then feed it to bs4 for processing.您链接的页面的问题在于它是javascript-仅在浏览器(或任何其他javascript VM)中正确显示.The problem with the page you linked is that it's javascript - it only renders correctly in a browser (or any other javascript VM). @Fabricator处在正确的轨道上-您需要观察开发人员控制台，并查看ajax请求js将其发送到服务器的内容.在这种情况下，还要看看查询字符串参数，其中包括一个名为CurrentPage的参数-可能是您要关注的参数.@Fabricator is on the right track - you'll need to watch the developer console and see what the ajax requests the js is sending to the server. In this case, also take a look at the query string params, which include a param called CurrentPage - that's probably the one you want to focus on. 这篇关于如何使用BeautifulSoup遍历网站的每个页面以进行Web抓取的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！ 上岸，阿里云！