在Lambda中创建AmazonS3Client时出现OutOfMemoryError

问题描述

我有一个AWS Lambda函数，该函数仅配置有128MB的内存，由SNS触发(它本身由S3触发)，并将从S3下载文件.

I have an AWS Lambda function, configured with only 128MB of memory, is triggered by SNS (which is itself triggered by S3) and will download the file from S3.

在我的职务上，我有以下内容:

In my function, I have the following:

public class LambdaHandler {

    private final AmazonS3Client s3Client = new AmazonS3Client();

    public void gdeltHandler(SNSEvent event, Context context) {
        System.out.println("Starting");
        System.out.println("Found " + eventFiles.size() + " event files");
    }

我已经注释掉了所有逻辑，并从这篇文章中排除了所有逻辑，因为我遇到了OutOfMemoryError，该错误与创建AmazonS3Client对象无关.当我拿出那个物体时，我没有得到错误.上面的确切代码会导致OutOfMemoryError.

I've commented out and excluded from this post all of the logic because I am getting an OutOfMemoryError which I have isolated to the creation of the AmazonS3Client object. When I take that object out, I don't get the error. The exact above code results in the OutOfMemoryError.

我为该功能分配了128MB的内存，这真的不足以简单地获取凭证并实例化AmazonS3Client对象吗?

I assigned 128MB of memory to the function, is that really not enough to simply grab the credentials and instantiate the AmazonS3Client object?

我尝试提供AmazonS3Client构造函数

I've tried giving the AmazonS3Client constructor

new EnvironmentVariableCredentialsProvider()

以及

new InstanceProfileCredentialsProvider()

具有相似的结果.

创建AmazonS3Client对象是否仅需要更多内存?

Does the creation of the AmazonS3Client object simply require more memory?

下面是堆栈跟踪:

当我尝试提供InstanceProfileCredentialsProvider或EnvironmentVariableCredentialsProvider时，得到以下堆栈跟踪:

When I try providing the InstanceProfileCredentialsProvider or EnvironmentVariableCredentialsProvider, I get the following stack trace:

编辑1 如果我将分配给该功能的内存增加到192MB，它就可以正常工作，尽管这很奇怪，但报告仅使用cloudwatch日志中的59MB内存.我只是失去了其余的记忆吗?

EDIT 1 If I increase the memory allocated to the function to even 192MB, it works just fine, though strangely enough, reports only using 59MB of memory in the cloudwatch logs. Am I simply losing the rest of the memory?

推荐答案

在Lambda函数中使用AWS Java SDK时，我一直在观察这一点.创建任何AWS客户端(同步或异步)时，您似乎都可能会脱离元空间.

I have been observing this when using AWS Java SDK within the Lambda function.It would seem when creating any of the AWS clients (Sync or Async) you may get out of Metaspace.

我认为这是由于Amazon Client在实例化时执行的事情所致，包括AmazonHttpClient的创建以及请求处理程序链的动态加载(AmazonEc2Client#init() private方法的一部分).

I believe this is due to things that the Amazon Client is performing upon instantiation, including AmazonHttpClient creation as well as dynamic loading of request handler chains (part of AmazonEc2Client#init() private method).

报告的内存使用情况可能是针对堆本身的，但可能不包括元空间. AWS论坛上有几个主题，但AWS对此没有任何回应.

It is possible that the reported memory usage is for Heap itself, but may not include Metaspace. There are a few threads on AWS Forums but no responses from AWS on the matter.

这篇关于在Lambda中创建AmazonS3Client时出现OutOfMemoryError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！