本文介绍了使用boto3时S3连接超时的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我正在使用boto3与S3一起操作.如果由于网络问题我的应用程序无法访问S3,则连接将挂起,直到最终超时.我想设置一个较低的连接超时.我遇到了此PR 的botocore,它允许设置超时:

I am using boto3 to operate with S3. If my application is unable to reach S3 due to a network issue, the connection will hang until eventually it times out. I would like to set a lower connection timeout. I came across this PR for botocore that allows setting a timeout:

$ sudo iptables -A OUTPUT -p tcp --dport 443 -j DROP

from botocore.client import Config
import boto3

config = Config(connect_timeout=5, read_timeout=5)

s3 = boto3.client('s3', config=config)

s3.head_bucket(Bucket='my-s3-bucket')

这将引发ConnectTimeout,但仍然需要花费太长时间才能出错:

This throws a ConnectTimeout, but it still takes too long to error out:

ConnectTimeout: HTTPSConnectionPool(host='my-s3-bucket.s3.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<botocore.awsrequest.AWSHTTPSConnection object at 0x2ad5dd0>, 'Connection to my-s3-bucket.s3.amazonaws.com timed out. (connect timeout=5)'))

同时调整连接和读取超时不会影响连接响应的速度.

Tweaking both the connect and read timeouts doesn't impact how quickly the connection responds.

推荐答案

boto3的默认行为是多次尝试重试连接并在两次之间以指数方式退回,这可能使您陷入困境.我在以下方面取得了不错的成绩:

You are probably getting bitten by boto3's default behaviour of retrying connections multiple times and exponentially backing off in between. I had good results with the following:

from botocore.client import Config
import boto3

config = Config(connect_timeout=5, retries={'max_attempts': 0})
s3 = boto3.client('s3', config=config)

这篇关于使用boto3时S3连接超时的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

07-18 02:35
查看更多