在运行时使用JDK编译器时发生内存泄漏

问题描述

我试图在程序中添加javaeditor以在运行时扩展程序。除非广泛使用该程序（我模拟了1000-10000个编译器执行），否则一切正常。内存使用量不断增加，似乎有内存泄漏。

I am trying to add a javaeditor to my program to extend the program at run time. It all works fine, except when using the program extensively (I simulated 1000-10000 compiler executions). The memory usage rises and rises, it looks like there is a memory leak.

在我的程序中，类被加载，构造函数被执行，并且类被卸载（没有剩余实例，并且当我将指针设置为null时，classLoader变得无效）。我使用JConsole分析了该过程，当执行垃圾收集器时，将卸载类。

In my program, the class gets loaded, the constructor gets executed and the class gets unloaded (no remaining instance and the classLoader becomes invalid as I set the pointer to null). I analyzed the process with JConsole, the classes get unloaded when the garbage collector is executed.

我在内存分析器中打开了heapdum，似乎是java.net.FactoryURLClassLoader内部的问题（在com.sun.tools.javac.util中）。列表对象）。由于（com.sun.tools.javac）是JDK的一部分，而不是JRE的一部分，并且SystemToolClassLoader是FactoryURLClassLoader对象，因此我将泄漏定位在此处。当我第一次执行编译器时，SystemToolClassLoader中已加载类的数量从1增加到521，但此后保持不变。

I did a heapdum opened it in memory analyzer, the problem seems to be inside of java.net.FactoryURLClassLoader (in a com.sun.tools.javac.util.List Object). Since (com.sun.tools.javac) is part of the JDK and not in the JRE and the SystemToolClassLoader is an FactoryURLClassLoader Object, I would locate the leak somewhere there. The number of loaded classes in the SystemToolClassLoader rises from 1 to 521 when I execute the compiler the first time but stays the same afterwards.

所以我不知道泄漏在哪里，有没有办法重置SystemToolClassLoader？我该如何更精确地定位泄漏。

So I have no idea where the leak is , is there a way to reset the SystemToolClassLoader? How could I locate the leak more precisely.

编辑：好的，我发现它也发生在一个非常简单的示例中。所以它似乎是编译的一部分，我不需要加载类或实例化它：

import java.io.File;
import java.io.IOException;
import java.util.Arrays;

import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;


public class Example {

public static void main(String[] args)
{
    for (int i =0; i<10000;i++){
        try {
            System.out.println(i);
            compile();
        } catch (InstantiationException | IllegalAccessException
                | ClassNotFoundException | IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }

}

public static void compile() throws IOException, InstantiationException, IllegalAccessException, ClassNotFoundException
{
    File source = new File( "src\\Example.java" ); // This File
    JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
    StandardJavaFileManager fileManager = compiler.getStandardFileManager( null, null, null );
    Iterable<? extends JavaFileObject> units;
    units = fileManager.getJavaFileObjectsFromFiles( Arrays.asList( source ) );
    compiler.getTask( null, fileManager, null, null, null, units ).call();
    fileManager.close();
}

}

推荐答案

起初，我认为这肯定是内存泄漏；但是，它与 SoftReference 的工作方式直接相关。

At first, I thought this was a definite memory leak; however, it is directly related to how SoftReferences work.

Oracle的JVM仅会尝试收集似乎无法强制以编程方式收集软引用。

Oracle's JVM will only attempt to collect soft references when the heap is completely used up. It appears that it is not possible to force soft references to be collected programatically.

要确定问题所在，我在无限堆上使用了三个转储：

To identify the problem, I used three dumps on an 'unlimited' heap:

1. 启动应用程序并获取Compiler和ScriptFileManager，然后进行GC最终处理-GC。进行转储。


2. 加载500个脚本。


3. 执行GC-finalize-GC。写统计信息。进行转储。


4. 加载500个脚本。


5. 执行GC-finalize-GC。写统计信息。

1. Start up the app and get the Compiler and ScriptFileManager, then do a GC-finalize-GC. Make a dump.
2. Load 500 'scripts'.
3. Do a GC-finalize-GC. Write stats. Make a dump.
4. Load 500 'scripts'.
5. Do a GC-finalize-GC. Write stats. Make a dump.

明显的（双）实例计数增加：名称 （500-> 1k）， SharedNameTable （500-> 1k）， SharedNameTable $ NameImpl （成千上万）和 [LSharedNameTable $ NameImpl （500-> 1k）。

The obvious (double) instance count increase: Names (500 -> 1k), SharedNameTable (500->1k), SharedNameTable$NameImpl (in the hundreds of thousands) and [LSharedNameTable$NameImpl (500->1k).

使用EMA分析后，很明显 SharedNameTable 对 com.sun.tools.javac.util.List 的静态引用显然是 SoftReference 曾经创建的每个 SharedNameTable （因此在运行时编译的每个源文件都有一个）。所有 $ NameImpl 都是您的源文件所分割的标记。而且显然所有令牌都从不从堆中释放出来并积累到无尽……还是？

After analyzing with EMA, it becomes apparent that SharedNameTable has a static reference to a com.sun.tools.javac.util.List which apparently SoftReferences every single SharedNameTable ever created (so one for each source file you have compiled during runtime). All the $NameImpls are the tokens your source file(s) were split to. And apparently all the tokens are never freed from the heap and accumulate to no end... Or do they?

我决定测试是否实际上是这样。知道软引用与弱引用的区别之后，我决定使用一个小堆（-Xms32m -Xmx32m）。这样，JVM被迫释放 SharedNameTable 或因 OutOfMemoryError 而失败。结果说明一切：

I decided to test if this was actually the case. Knowing the difference of soft vs weak references, I decided to use a small heap (-Xms32m -Xmx32m). This way JVM is forced to either free SharedNameTables or fail with OutOfMemoryError. The results speak for themselves:

-Xmx512m -Xms512m

-Xmx512m -Xms512m

Total memory: 477233152
Free memory: 331507232
Used memory: 138.97506713867188 MB
Loaded scripts: 500

Total memory: 489816064
Free memory: 203307408
Used memory: 273.23594665527344 MB
Loaded scripts: 1000

The classloader/component "java.net.FactoryURLClassLoader @ 0x8a8a748" occupies 279.709.192 (98,37%) bytes.

-Xmx32m -Xms32m

-Xmx32m -Xms32m

Total memory: 29687808
Free memory: 25017112
Used memory: 4.454322814941406 MB
Loaded scripts: 500

Total memory: 29884416
Free memory: 24702728
Used memory: 4.941642761230469 MB
Loaded scripts: 1000

One instance of "com.sun.tools.javac.file.ZipFileIndex" loaded by "java.net.FactoryURLClassLoader @ 0x8aa4cc8" occupies 2.230.736 (47,16%) bytes. The instance is referenced by *.*.script.ScriptFileManager @ 0x8ac8230.

（这只是到JDK库的链接。）

(This is merely a link to a JDK library.)

脚本：

public class Avenger
{
    public Avenger()
    {
        JavaClassScriptCache.doNotCollect(this);
    }

    public static void main(String[] args)
    {
        // this method is called after compiling
        new Avenger();
    }
}

doNotCollect：

doNotCollect:

private static final int TO_LOAD = 1000;
private static final List<Object> _active = new ArrayList<Object>(TO_LOAD);

public static void doNotCollect(Object o)
{
    _active.add(o);
}

System.out.println("Loaded scripts: " + _active.size());

这篇关于在运行时使用JDK编译器时发生内存泄漏的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！