我正在开发一个Web应用程序，该应用程序需要一个zip文件，由用户上传，然后在服务器上将其解压缩并处理文件。当zip文件不是太大（20-25MB）时，它的工作原理就像一种魅力。但是，如果该文件大约或超过（50MB），它将产生OutOfMemoryError。
我试图通过添加来增加Java最大内存分配池export CATALINA_OPTS="-Xmx1024M"到tomcat7中的startup.sh，但错误仍然存在。
AFAIK，问题在于解压缩.zip文件。 top显示tomcat在提取50MB文件期间使用800MB内存。有什么解决方案可以在有效使用可用内存的情况下实现约200MB的上传？
解压缩的代码如下：
package user;
import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
public class unzip {
public void unzipFile(String filePath, String oPath)
{
FileInputStream fis = null;
ZipInputStream zipIs = null;
ZipEntry zEntry = null;
try {
fis = new FileInputStream(filePath);
zipIs = new ZipInputStream(new BufferedInputStream(fis));
while((zEntry = zipIs.getNextEntry()) != null){
try{
byte[] tmp = new byte[8*1024];
FileOutputStream fos = null;
String opFilePath = oPath+zEntry.getName();
System.out.println("Extracting file to "+opFilePath);
fos = new FileOutputStream(opFilePath);
int size = 0;
while((size = zipIs.read(tmp)) != -1){
fos.write(tmp, 0 , size);
}
fos.flush();
fos.close();
}catch(Exception ex){
}
}
zipIs.close();
fis.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
错误代码如下：
HTTP Status 500 - javax.servlet.ServletException: java.lang.OutOfMemoryError: Java heap space
type Exception report
message javax.servlet.ServletException: java.lang.OutOfMemoryError: Java heap space
description The server encountered an internal error that prevented it from fulfilling this request.
exception
org.apache.jasper.JasperException: javax.servlet.ServletException: java.lang.OutOfMemoryError: Java heap space
org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:549)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:455)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
root cause
javax.servlet.ServletException: java.lang.OutOfMemoryError: Java heap space
org.apache.jasper.runtime.PageContextImpl.doHandlePageException(PageContextImpl.java:916)
org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:845)
org.apache.jsp.Upload_jsp._jspService(Upload_jsp.java:369)
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
root cause
java.lang.OutOfMemoryError: Java heap space
org.apache.commons.io.output.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:322)
org.apache.commons.io.output.DeferredFileOutputStream.getData(DeferredFileOutputStream.java:213)
org.apache.commons.fileupload.disk.DiskFileItem.getSize(DiskFileItem.java:289)
org.apache.jsp.Upload_jsp._jspService(Upload_jsp.java:159)
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
note The full stack trace of the root cause is available in the Apache Tomcat/7.0.52 (Ubuntu) logs.
Apache Tomcat/7.0.52 (Ubuntu)
令人惊讶的是，catalina.out文件上没有关于此异常的任何内容。
提前致谢。
Upload.jsp中DiskFileItem的编辑代码
//necessary imports go here
File file ;
int maxFileSize = 1000 * 1000 * 1024;
int maxMemSize = 1000 * 1024;
ServletContext context = pageContext.getServletContext();
String filePath = context.getInitParameter("file-upload");
String contentType = request.getContentType();
if(contentType != null)
{
if ((contentType.indexOf("multipart/form-data") >= 0))
{
DiskFileItemFactory factory = new DiskFileItemFactory();
factory.setSizeThreshold(maxMemSize);
factory.setRepository(new File("/tmp/"));
ServletFileUpload upload = new ServletFileUpload(factory);
upload.setSizeMax( maxFileSize );
try{
List fileItems = upload.parseRequest(request);
Iterator i = fileItems.iterator();
while (i.hasNext ())
{
FileItem fi = (FileItem)i.next();
if ( !fi.isFormField () )
{
String fieldName = fi.getFieldName();
String fileName = fi.getName();
if(fileName.endsWith(".zip")||fileName.endsWith(".pdf")||fileName.endsWith(".doc")||fileName.endsWith(".docx")||fileName.endsWith(".ppt")||fileName.endsWith(".pptx")||fileName.endsWith(".html")||fileName.endsWith(".htm")||fileName.endsWith(".epub")||fileName.endsWith(".djvu"))
{
boolean isInMemory = fi.isInMemory();
long sizeInBytes = fi.getSize();
new File(filePath+fileName).mkdir();
filePath = filePath+fileName+"/";
file = new File( filePath + fileName.substring( fileName.lastIndexOf("/"))) ;
fi.write(file);
String fileExtension = FilenameUtils.getExtension(fileName);
if(fileExtension.equals("zip"))
{
System.out.println("In zip.");
unzip mfe = new unzip();
mfe.unzipFile(filePath+fileName,filePath);
File zip = new File(filePath+fileName);
zip.delete();
}
File corePath = new File(filePath);
int count=0;
//some more processing
}
}
}
}
catch(Exception e)
{
//exception handling goes here
}
}
}
最佳答案
该问题不在您发布的邮政编码中。根本原因在于：
java.lang.OutOfMemoryError: Java heap space
org.apache.commons.io.output.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:322)
org.apache.commons.io.output.DeferredFileOutputStream.getData(DeferredFileOutputStream.java:213)
org.apache.commons.fileupload.disk.DiskFileItem.getSize(DiskFileItem.java:289)
您注意到
ByteArrayOutputStream.toByteArray吗？因此，似乎您正在写的ByteArrayOutputStream增长太多。请找到并张贴使用此ByteArrayOutputStream的代码，因为您的邮政编码未使用该代码更新：
从您发布的代码看来，您的代码还可以。但是
FileItem.getSize()调用有一些讨厌的事情：283 public long getSize() {
284 if (size >= 0) {
285 return size;
286 } else if (cachedContent != null) {
287 return cachedContent.length;
288 } else if (dfos.isInMemory()) {
289 return dfos.getData().length;
290 } else {
291 return dfos.getFile().length();
292 }
293 }
如果文件项的数据存储在内存中-它会调用
getData()，后者会调用toByteArray()209 public byte[] [More ...] getData()
210 {
211 if (memoryOutputStream != null)
212 {
213 return memoryOutputStream.toByteArray();
214 }
215 return null;
216 }
依次分配一个新数组：
317 public synchronized byte[] toByteArray() {
318 int remaining = count;
319 if (remaining == 0) {
320 return EMPTY_BYTE_ARRAY;
321 }
322 byte newbuf[] = new byte[remaining];
//Do stuff
333 return newbuf;
334 }
因此，在短时间内，您的内存消耗是普通内存的两倍。
我建议您：
将
maxMemSize设置为不再8-32 Kb给JVM进程更多的内存：例如
-Xmx2g确保您没有不必要地引用
FileItem，因为在当前配置中它们会占用大量内存。如果OOM再次发生，请进行堆转储。您可以使用
-XX:+HeapDumpOnOutOfMemoryError JVM标志为您自动创建一个堆转储。然后，您可以使用堆转储分析器（例如Eclipse MAT）来检查谁在分配这么多的内存以及在哪里分配了内存。