使用IO库将字符串变量作为txt文件加载到s3/从s3加载

问题描述

我下面有旧代码，该文件使用IO库将文件gzip压缩并存储为json到S3中(因此文件不会在本地保存).我在转换相同方法(即使用IO库作为缓冲区)以创建.txt文件并将其推入S3并稍后检索时遇到麻烦.我知道如何创建txt文件并将其推送到s3中也是，但是不知道如何在此过程中使用IO.

我想存储在文本值中的值只是一个字符串值为'test'的变量

目标:使用IO库并将字符串变量作为文本文件保存到S3中，并能够再次将其下拉.

x = 'test'
inmemory = io.BytesIO()
    with gzip.GzipFile(fileobj=inmemory, mode='wb') as fh:

        with io.TextIOWrapper(fh, encoding='utf-8',errors='replace') as wrapper:
            wrapper.write(json.dumps(x, ensure_ascii=False,indent=2))
    inmemory.seek(0)
    s3_resource.Object(s3bucket, s3path + '.json.gz').upload_fileobj(inmemory)
    inmemory.close()

任何人都喜欢的与IO库和写入文件有关的任何文档，因为实际的文档(f = io.StringIO(一些初始文本数据") 等等. https://docs.python.org/3/library/io.html )就目前的水平而言，这还不够.

解决方案

重复.

为简洁起见，事实证明，有一种方法覆盖putObject调用，以便它可以接受一串文本而不是一个文件.

原始帖子用Java回答，但是此额外的线程对于特定于Python的答案应该足够了./p>

I have old code below that gzips a file and stores it as json into S3, using the IO library ( so a file does not save locally). I am having trouble converting this same approach (ie using IO library for a buffer) to create a .txt file and push into S3 and later retrieve. I know how to create txt files and push into s3 is as well, but not how to use IO in the process.

The value I would want to be stored in the text value would just be a variable with a string value of 'test'

Goal: Use IO library and save string variable as a text file into S3 and be able to pull it down again.

x = 'test'
inmemory = io.BytesIO()
    with gzip.GzipFile(fileobj=inmemory, mode='wb') as fh:

        with io.TextIOWrapper(fh, encoding='utf-8',errors='replace') as wrapper:
            wrapper.write(json.dumps(x, ensure_ascii=False,indent=2))
    inmemory.seek(0)
    s3_resource.Object(s3bucket, s3path + '.json.gz').upload_fileobj(inmemory)
    inmemory.close()

Also any documentation with that anyone likes with specific respect to the IO library and writing to files, because the actual documentation ( f = io.StringIO("some initial text data") ect..https://docs.python.org/3/library/io.html ) It just did not give me enough at my current level.

解决方案

Duplicate.

For sake of brevity, it turns out there's a way to override the putObject call so that it takes in a string of text instead of a file.

The original post is answered in Java, but this additional thread should be sufficient for a Python-specific answer.

这篇关于使用IO库将字符串变量作为txt文件加载到s3/从s3加载的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！