Webflux上传大文件导致Java堆空间

问题描述

可能没有多少开发人员面临与我相同的问题.
但是我想分享已经解决了将近1个月的解决方案.
我使用Kubernetes和docker-compose，此Webflux服务(容器)的内存限制设置为1g mem_limit:1g 我不允许增加内存限制.

Probably not many developers are facing the problem the same as me.
But I want to share the solution which I had been solved for almost 1 month.
I use Kubernetes and docker-compose, this Webflux service (container) is set the memory limit 1g mem_limit: 1g I am not allowed to increase the memory limit.

coRouter 已用作容器中的控制器.

coRouter has been used as a controller which is in the container.

@Configuration
class WebController(
) {

    @Bean
    fun endpoints() = coRouter {
        contentType(MediaType.MULTIPART_FORM_DATA).nest {
            POST("/uploadFile") { requestParm ->
                requestParm.awaitMultipartData().multipartFormData["file"]?.first()
            }
        }
    }
}

如果我使用API​​上传文件 http://localhost:9004/uploadFile 100MB，10个文件(使用JS Ajax)
它将产生 java.lang.OutOfMemoryError:Java堆空间，因为它没有足够的内存.
每当您调用 requestParm.awaitMultipartData()时，它将把这些字节流式传输到内存中.

If I upload files using the APIhttp://localhost:9004/uploadFile 100MB, 10 files (using JS Ajax)
It will produce java.lang.OutOfMemoryError: Java heap space because it doesn't have enough memory.
Whenever you call requestParm.awaitMultipartData() it will stream those bytes into memory.

推荐答案

我意识到，我当前的代码将直接将上传文件存储在内存中.有两种方法可以解决此问题.

I realized that, my current code will store the upload files in memory directly.There are 2 ways to solve this problem.

• 第一种方法，在路由器或控制器上添加 @EnableWebFlux

• 1st way, add @EnableWebFlux on your Router or Controller

@Configuration
@EnableWebFlux
class WebController(
) { }

• 第二种方式，从 DelegatingWebFluxConfiguration 扩展到路由器或控制器
  参考:https://docs.spring.io/spring-framework/docs/current/reference/html/web-reactive.html#webflux-config-advanced-java
• 2nd way, extending from DelegatingWebFluxConfiguration to your Router or Controller
  Reference: https://docs.spring.io/spring-framework/docs/current/reference/html/web-reactive.html#webflux-config-advanced-java

@Configuration
class WebController(
): DelegatingWebFluxConfiguration() { }

为什么可以解决上传大文件的问题?

Why does it solve the uploading large file problem?

以上两种方法将自动使用晚餐类的方法
configureHttpMessageCodecs(configurer:ServerCodecConfigurer)

https://docs.spring.io/spring-framework/docs/current/reference/html/web-reactive.html#webflux-codecs-multipart

These 2 ways above will be automatically used a supper class's method
configureHttpMessageCodecs(configurer: ServerCodecConfigurer)

https://docs.spring.io/spring-framework/docs/current/reference/html/web-reactive.html#webflux-codecs-multipart

如您所见， ServerCodecConfigurer 将默认使用DefaultPartHttpMessageReader.

As you can see ServerCodecConfigurer will use DefaultPartHttpMessageReader by default.

DefaultPartHttpMessageReader 具有此重要属性 MaxInMemorySize (如果内存已满，则存储在磁盘中)

DefaultPartHttpMessageReader has this important properties MaxInMemorySize (if memory full, store in disk)

https://docs.spring.io/spring-framework/docs/5.3.1/javadoc-api/org/springframework/http/codec/multipart/DefaultPartHttpMessageReader.html#setMaxInMemorySize-int-

用于上传文件的默认内存为256K
如果内存已满，它将把这些文件临时保存到磁盘上.


The default memory for uploading files is 256K
It will save those files to disk for temporary if the memory is full.


您可以自定义内存以存储大小文件，也可以

You can custom the memory to store size files also Spring WebFlux: set max file(s) size for uploading files

这篇关于Webflux上传大文件导致Java堆空间的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！