Python 请求，如何限制接收大小、传输速率和/或总时间?

问题描述

我的服务器执行外部请求，我想限制失败的请求可能造成的损害.我希望在以下情况下取消请求:

My server does external requests and I'd like to limit the damage a failing request can do. I'm looking to cancel the request in these situations:

• 请求的总时间超过一定限制(即使数据仍在到达)
• 接收到的总大小超过了某个限制(我需要在接受更多数据之前取消)
• 传输速度低于某个水平(尽管如果可以提供总时间限制，我可以没有它)

注意我不是在请求中寻找 timeout 参数，因为这只是不活动的超时.我找不到与总超时或限制总大小的方法有关的任何事情.一个示例显示了 HTTPAdapter 上的 maxsize 参数，但没有记录.

Note I am not looking for the timeout parameter in requests, as this is a timeout only for inactivity. I'm unable to find anything to do with a total timeout, or a way to limit the total size. One example shows a maxsize parameter on HTTPAdapter but that is not documented.

如何使用 requests 实现这些要求?

How can I achieve these requirements using requests?

推荐答案

您可以尝试设置 stream=True，然后在读取数据时超出时间或大小限制时中止请求块.

You could try setting stream=True, then aborting a request when your time or size limits are exceeded while you read the data in chunks.

截至 requests 2.3.0 版 超时也适用于流请求，因此您需要做的就是允许初始连接和每个迭代步骤超时:

As of requests release 2.3.0 the timeout applies to streaming requests too, so all you need to do is allow for a timeout for the initial connection and each iteration step:

r = requests.get(..., stream=True, timeout=initial_timeout)
r.raise_for_status()

if int(r.headers.get('Content-Length')) > your_maximum:
    raise ValueError('response too large')

size = 0
start = time.time()

for chunk in r.iter_content(1024):
    if time.time() - start > receive_timeout:
        raise ValueError('timeout reached')

    size += len(chunk)
    if size > your_maximum:
        raise ValueError('response too large')

    # do something with chunk

根据需要调整超时.

对于 requests 发布 2.3.0(包括此更改)您无法使 r 超时.iter_content() 产量；在块中间停止响应的服务器仍然会占用连接.您必须将上述代码包装在一个额外的 超时函数中 及早切断长时间运行的响应.

For requests releases < 2.3.0 (which included this change) you could not time out the r.iter_content() yield; a server that stops responding in the middle of a chunk would still tie up the connection. You'd have to wrap the above code in an additional timeout function to cut off long-running responses early.

这篇关于Python 请求，如何限制接收大小、传输速率和/或总时间?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！