仅使用python请求下载文档的一部分

本文介绍了仅使用python请求下载文档的一部分的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！问题描述 29岁程序员，3月因学历无情被辞！我正在使用python-requests编写一个web scraper。I'm writing a web scraper using python-requests.每个页面超过1MB，但我需要提取的实际数据很早就在文档的流程，所以我浪费时间下载大量不必要的数据。Each page is over 1MB, but the actual data I need to extract is very early on in the document's flow, so I'm wasting time downloading a lot of unnecessary data.如果可能，我想在文档中出现所需数据后立即停止下载源代码，以节省时间。If possible I would like to stop the download as soon as the required data appears in the document's source code, in order to save time.例如，我只想提取abcDiv中的文本，文档的其余部分是无用的：For example, I only want to extract the text in the "abc" Div, the rest of the document is useless:<html><head><title>My site</title></head><body><div id="abc">blah blah...</div><p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris fermentum molestie ligula, a pharetra eros mollis ut.</p><p>Quisque auctor volutpat lobortis. Vestibulum pellentesque lacus sapien, quis vulputate enim mollis a. Vestibulum ultrices fermentum urna ac sodales.</p><p>Nunc sit amet augue at dolor fermentum ultrices. Curabitur faucibus porttitor vehicula. Lorem ipsum dolor sit amet, consectetur adipiscing elit.</p><p>Etiam sed leo at ipsum blandit dignissim ut a est.</p></body></html>目前我只是这样做：r = requests.get(URL)推荐答案你想在这里使用的是范围 HTTP标题。What you want to use here is called Range HTTP Header.参见： http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html （特别是Range上的位）。另请参阅自定义标题示例：from requests import geturl = "http://download.thinkbroadband.com/5MB.zip"headers = {"Range": "bytes=0-100"} # first 100 bytesr = get(url, headers=headers) 这篇关于仅使用python请求下载文档的一部分的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！上岸，阿里云！