如何在AWS S3上附加到json文件

问题描述

我需要在AWS S3上附加到json文件，在EC2实例上运行python代码.

I need to append to a json file on aws S3, python code is running on an EC2 instance.

在本地设置中，我可以轻松地执行以下操作:

In a local setting I can easily do this as follows:

import json

#example data
json_data = {"id": "123", "name": "XYZ", "transaction": [20.0, 30.0]}
# path
local_path = '/home/ubuntu/test.json'

with open(local_path, 'a', encoding='utf-8-sig') as file:
    json.dump(json_data, file)
    file.write('\n')
    file.close()

在EC2上，我可以按以下方式连接到S3:

On EC2 I can connect to S3 as follows:

import boto
s3_open = boto.connect_s3(host='s3.eu-central-1.amazonaws.com')

我定义了S3的路径:

s3_path = 's3://my-bucket/test.json'

如何使用上述逻辑将其附加到此文件?

How can I append to this file using the logic described above?

推荐答案

S3没有附加操作，您可以执行的操作是下载文件，自己附加，然后将其上传为同一对象的新版本.

S3 does not have an append action what you can do is download the file, append it yourself then upload it as a new version of the same object.

遵循这些原则.

import json
import boto3

s3_path = '/some/s3/path'
bucket = 'somebucket'
local_data = {"id": "123", "name": "XYZ", "transaction": [20.0, 30.0]}

s3 = boto3.client('s3')
resp=s3.get_object(Bucket=bucket, Key=s3_path)
data=resp.get('Body')

json_data = json.loads(data)
json_data.append(local_data)
s3.put_object(Bucket=bucket, Key=s3_path, Body=json.dumps(json_data).encode())

这篇关于如何在AWS S3上附加到json文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！