我读取大文本文件的Java程序内存不足，任何人都可以帮忙解释原因吗？

本文介绍了我读取大文本文件的Java程序内存不足，任何人都可以帮忙解释原因吗？的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述 我有一个包含2000万行文字的大文本文件。当我使用以下程序读取文件时，它工作得很好，事实上我可以读取更大的文件，没有内存问题。 public static void main（String [] args）throws IOException { File tempFile = new File（temp.dat）; String tempLine = null; BufferedReader br = null; int lineCount = 0; try { br = new BufferedReader（new FileReader（tempFile））; while（（tempLine = br.readLine（））！= null）{ lineCount + = 1; } } catch（例外e）{ System.out.println（br error：+ e.getMessage（））; } finally { br.close（）; System.out.println（lineCount +从文件中读取的行）; } } 但是如果我需要在此文件中附加一些记录在阅读它之前，BufferedReader消耗了大量的内存（我刚刚使用Windows任务管理器监控这个，我知道的不是很科学，但它证明了这个问题）。修改后的程序如下，与第一个程序相同，只是我首先在文件中附加一条记录。 public static void main（String [] args）throws IOException { File tempFile = new File（temp.dat）; PrintWriter pw = null; try { pw = new PrintWriter（new BufferedWriter（new FileWriter（tempFile，true）））; pw.println（）; } catch（例外e）{ System.out.println（pw error：+ e.getMessage（））; }最后{ pw.close（）; } String tempLine = null; BufferedReader br = null; int lineCount = 0; try { br = new BufferedReader（new FileReader（tempFile））; while（（tempLine = br.readLine（））！= null）{ lineCount + = 1; } } catch（例外e）{ System.out.println（br error：+ e.getMessage（））; } finally { br.close（）; System.out.println（lineCount +从文件中读取的行）; } } Windows任务管理器的屏幕截图，其中大凸起在行中显示了运行程序的第二个版本时的内存消耗。 所以我能够在不耗尽内存的情况下读取此文件。但是我有更大的文件，有超过5000万条记录，当我对它们运行这个程序时遇到内存不足的异常？有人可以解释为什么程序的第一个版本适用于任何大小的文件，但第二个程序的行为如此不同并以失败告终？我在Windows 7上运行： java版本1.7.0_05 Java（TM）SE运行时环境（版本1.7.0_05- b05） Java HotSpot（TM）客户端VM（版本23.1-b03，混合模式，共享）解决方案您可以使用 VM-Options启动Java-VM -XX：+ HeapDumpOnOutOfMemoryError 这会将堆转储写入文件，可以对其进行分析以查找泄漏嫌疑人 使用'+'添加一个选项和一个' - '来删除一个选项。 如果您正在使用Eclipse Java Memory Analyzer插件 MAT 通过对泄漏嫌疑人等进行一些不错的分析来运行虚拟机的堆转储。 I have a large text file with 20 million lines of text. When I read the file using the following program, it works just fine, and in fact I can read much larger files with no memory problems.public static void main(String[] args) throws IOException { File tempFile = new File("temp.dat"); String tempLine = null; BufferedReader br = null; int lineCount = 0; try { br = new BufferedReader(new FileReader(tempFile)); while ((tempLine = br.readLine()) != null) { lineCount += 1; } } catch (Exception e) { System.out.println("br error: " +e.getMessage()); } finally { br.close(); System.out.println(lineCount + " lines read from file"); }}However if I need to append some records to this file before reading it, the BufferedReader consumes a huge amount of memory (I have just used Windows task manager to monitor this, not very scientific I know but it demonstrates the problem). The amended program is below, which is the same as the first one, except I am appending a single record to the file first.public static void main(String[] args) throws IOException { File tempFile = new File("temp.dat"); PrintWriter pw = null; try { pw = new PrintWriter(new BufferedWriter(new FileWriter(tempFile, true))); pw.println(" "); } catch (Exception e) { System.out.println("pw error: " + e.getMessage()); } finally { pw.close(); } String tempLine = null; BufferedReader br = null; int lineCount = 0; try { br = new BufferedReader(new FileReader(tempFile)); while ((tempLine = br.readLine()) != null) { lineCount += 1; } } catch (Exception e) { System.out.println("br error: " +e.getMessage()); } finally { br.close(); System.out.println(lineCount + " lines read from file"); }}A screenshot of Windows task manager, where the large bump in the line shows the memory consumption when I run the second version of the program.So I was able to read this file without running out of memory. But I have much larger files with more than 50 million records, which encounter an out of memory exception when I run this program against them? Can someone explain why the first version of the program works fine on files of any size, but the second program behaves so differently and ends in failure? I am running on Windows 7 with:java version "1.7.0_05"Java(TM) SE Runtime Environment (build 1.7.0_05-b05)Java HotSpot(TM) Client VM (build 23.1-b03, mixed mode, sharing) 解决方案 you can start a Java-VM with VM-Options-XX:+HeapDumpOnOutOfMemoryErrorthis will write a heap dump to a file, which can be analysed for finding leak suspectsUse a '+' to add an option and a '-' to remove an option.If you are using Eclipse the Java Memory Analyzer Plugin MAT to get Heap-Dumps from running VMs with some nice analyses for Leak Suspects etc. 这篇关于我读取大文本文件的Java程序内存不足，任何人都可以帮忙解释原因吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！