在AWS Boto3上传中获取进度回调

问题描述

对于原始boto的上传，这里有一个很好的问题和答案:

There's a great question and answer for the original boto uploads here:

如何上传文件使用boto到S3存储桶中的目录

具有回调:

k = Key(bucket)
k.key = 'my test file'
k.set_contents_from_filename(testfile,
    cb=percent_cb, num_cb=10)

虽然我看到boto3包需要回调:

While I see the boto3 package takes a callback:

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.upload_fileobj

我看不到num_cb参数的等效项.如何使用boto3获取 upload_fileobj 的进度表?

I don't see the equivalent of the num_cb argument. How can I get a progress meter for upload_fileobj using boto3?

s3.upload_fileobj(data, 'mybucket', 'mykey')

推荐答案

如果您不需要限制调用回调的次数，(并且没有办法使用upload_fileobj进行此操作)，
1.显示百分比

If you don't need to limit the number of calling callback, (and there is no way to do it with upload_fileobj),
1. show percentage

import os
import boto3

class Test:
    def __init__(self):
        self.total = 0
        self.uploaded = 0
        self.s3 = boto3.client('s3')

    def upload_callback(self, size):
        if self.total == 0:
            return
        self.uploaded += size
        print("{} %".format(int(self.uploaded / self.total * 100)))

    def upload(self, bucket, key, file):
        self.total = os.stat(file).st_size

        with open(file, 'rb') as data:
            self.s3.upload_fileobj(
                data, bucket, key, Callback=self.upload_callback)

1. 使用进度条

import os
import boto3
import progressbar


class Test2:
    def __init__(self):
        self.s3 = boto3.client('s3')

    def upload_callback(self, size):
        self.pg.update(self.pg.currval + size)

    def upload(self, bucket, key, file):
        self.pg = progressbar.progressbar.ProgressBar(
            maxval=os.stat(file).st_size)
        self.pg.start()

        with open(file, 'rb') as data:
            self.s3.upload_fileobj(
                data, bucket, key, Callback=self.upload_callback)

这篇关于在AWS Boto3上传中获取进度回调的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！