如何将我的AWS CLI命令转换为AWS Java API代码?

问题描述

我有一段在HTTP Server中经常调用的代码.我在管道上得到响应，并使用输出String进行播放.但是显然，将其用作bash脚本的AWS CLI管道可能不是很有效.

I have a piece of code that is called frequently in a HTTP Server. I get the responses on pipes and play with the output String. But apparently using it as AWS CLI pipes as bash scripts may not be very efficient.

与hese命令等效的AWS JAVA API是什么?如何转换它们?并提高效率，访问延迟等吗?

What would be the AWS JAVA API equivalent of hese commands? How to convert them? And does it improve efficiency, access latency, etc?

public void run() {
    String command = "aws --endpoint-url=" + S3server + " s3 cp s3://file-store/" + id + "/files/" + id + ".txt -";
    processBuilder.command("sh", "-c", command + " ; true");
    Process process = processBuilder.start();
    //String command2 = "aws --endpoint-url=https://s3-ourserver.mystore.mycompany.net s3api head-object --bucket file-store --key FILES/3762/files/mytext.scc  --query 'ContentLength'";

    BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
    BufferedReader readerErrors = new BufferedReader(new InputStreamReader(process.getErrorStream()));

    String line;
    while ((line = reader.readLine()) != null) {
        response += line + "\n";
    }
    //...send the response back via HTTP
    OutputStream os = he.getResponseBody();
    os.write(response.toString().getBytes());
    os.close();
}

推荐答案

从V2 Java SDK中的S3读取内容非常简单，就像这样:

A super simple read from S3 in the V2 Java SDK would look something like:

Region region = Region.US_WEST_2;
String bucketName = "file-store";

S3Client s3Client = S3Client.builder().region(region).build();

...

GetObjectRequest getObjectRequest = GetObjectRequest.builder()
                                         .bucket(bucketName)
                                         .key(id + "/files/" + id + ".txt")
                                         .build();
OutputStream os = he.getResponseBody();
s3Client.getObject(getObjectRequest, ResponseTransformer.toOutputStream(outputStream));
os.close();

这将从S3下载文件，并将字节直接写入HTTP响应中的输出流.如果您超级频繁地执行此操作，则应在执行读取操作的地方(即构造函数)之外创建 S3Client 并重新使用.

This downloads a file from S3 and writes the bytes directly to the output stream in the HTTP response. If you're doing this super frequently then the S3Client should be created outside of where you're doing the read (i.e. a constructor) and reused.

现在，这比您显示的速度快吗?我强烈希望它会实现，但是我没有一种方法可以在您正在运行的负载下对其进行测试.

Now, is this faster than what you're showing? I strongly expect that it will be but I don't have a way of testing it at the load you're running.

另一个选择可能是创建预设URL 客户端会请求，然后他们可以直接从S3下载文件，而无需通过您的服务器.我不确定这是否适用于您的用例.

One other option might be to create a presigned URL that a client would request and then they could download the file directly from S3 and not go through your server. I'm not sure if that would work for your use case.

编辑我强烈建议您深入研究AWS Java SDK文档，以及您的用例 S3示例.但是在那之前，下面是一个获取文件的简单程序以及两种获取内容长度的不同方法.这个使用本地凭据-您的用例可能有所不同.

EDITI'd strongly encourage you to dig into the AWS Java SDK docs and, for your use case, the S3 Examples. But until that time, below is a simple program to get a file and two different ways to get the content length. This one uses local credentials - your use case may be different.

import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.core.exception.SdkClientException;
import software.amazon.awssdk.core.sync.ResponseTransformer;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.GetObjectResponse;
import software.amazon.awssdk.services.s3.model.HeadObjectRequest;
import software.amazon.awssdk.services.s3.model.HeadObjectResponse;
import software.amazon.awssdk.services.s3.model.NoSuchKeyException;
import software.amazon.awssdk.services.s3.model.S3Exception;

import java.io.FileOutputStream;
import java.io.OutputStream;


public class DownloadObjectV2 {

    public static void main(String[] args) {
        Region region = Region.US_EAST_1;
        String bucketName = "the-bucket-name";
        String fileObjKeyName = "file.txt";
        String fileName = "/tmp/file.txt";

        long startTime = System.currentTimeMillis();

        try {
            S3Client s3 = S3Client.builder()
                    .region(region)
                    .credentialsProvider(ProfileCredentialsProvider.create("profile-name"))
                    .build();

            // get meta data about the object without downloading it
            HeadObjectRequest headObjectRequest = HeadObjectRequest.builder()
                    .bucket(bucketName)
                    .key(fileObjKeyName)
                    .build();
            HeadObjectResponse headObjectResponse = s3.headObject(headObjectRequest);
            System.out.println("object is " + headObjectResponse.contentLength() + " bytes long from head call");

            // get the object, save it to a file
            GetObjectRequest getObjectRequest = GetObjectRequest.builder()
                    .bucket(bucketName)
                    .key(fileObjKeyName)
                    .build();

            OutputStream outputStream = new FileOutputStream(fileName);
            GetObjectResponse getObjectResponse = s3.getObject(getObjectRequest, ResponseTransformer.toOutputStream(outputStream));

            // meta data is also available from the GetObjectResponse
            System.out.println("object is " + getObjectResponse.contentLength() + " bytes long from get call");
        }
        catch( NoSuchKeyException nske ) { // bad key name
            System.out.println( "the key does not exist" );
        }
        catch( S3Exception s3e ) { // errors like bad credentials
            System.out.println( "a general s3 exception occurred: ");
            s3e.printStackTrace();
        }
        catch( SdkClientException sce ) { // other errors
            System.out.println( "a sdk exception occurred: ");
            sce.printStackTrace();
        }
        catch( Exception e ) {
            e.printStackTrace();
        }

        System.out.println( "run took " + (System.currentTimeMillis() - startTime) + "ms");
        System.exit(0);
    }
}

这篇关于如何将我的AWS CLI命令转换为AWS Java API代码?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！