APOC

APOC

关注
发信
关注(28)粉丝(399)

Neo4j OutOfMemory问题

本文介绍了Neo4j OutOfMemory问题的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

这是我的Main.java源代码。它是从neo4j-apoc-1.0示例中获取的。修改以存储2个节点和1个关系的1M记录的目标：

This is my source code of Main.java. It was grabbed from neo4j-apoc-1.0 examples. The goal of modification to store 1M records of 2 nodes and 1 relation:

package javaapplication2;

import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.graphdb.Node;
import org.neo4j.graphdb.RelationshipType;
import org.neo4j.graphdb.Transaction;
import org.neo4j.kernel.EmbeddedGraphDatabase;


public class Main
{
    private static final String DB_PATH = "neo4j-store-1M";
    private static final String NAME_KEY = "name";

    private static enum ExampleRelationshipTypes implements RelationshipType
    {
        EXAMPLE
    }

    public static void main(String[] args)
    {
        GraphDatabaseService graphDb = null;

        try
        {
            System.out.println( "Init database..." );


            graphDb = new EmbeddedGraphDatabase( DB_PATH );

            registerShutdownHook( graphDb );


            System.out.println( "Start of creating database..." );


            int valIndex = 0;

            for(int i=0; i<1000; ++i)
            {
                for(int j=0; j<1000; ++j)
                {
                    Transaction tx = graphDb.beginTx();

                    try
                    {
                        Node firstNode = graphDb.createNode();
            firstNode.setProperty( NAME_KEY, "Hello" + valIndex );

                        Node secondNode = graphDb.createNode();
            secondNode.setProperty( NAME_KEY, "World" + valIndex );

                        firstNode.createRelationshipTo(
                           secondNode, ExampleRelationshipTypes.EXAMPLE );

                        tx.success();

                        ++valIndex;
                    }
                    finally
                    {
                        tx.finish();
                    }
                }
            }

            System.out.println("Ok, client processing finished!");
        }
        finally
        {
            System.out.println( "Shutting down database ..." );

            graphDb.shutdown();
        }
    }

    private static void registerShutdownHook( final GraphDatabaseService graphDb )
    {
        // Registers a shutdown hook for the Neo4j instance so that it
        // shuts down nicely when the VM exits (even if you "Ctrl-C" the
        // running example before it's completed)
        Runtime.getRuntime().addShutdownHook( new Thread()
        {
            @Override
            public void run()
            {
                graphDb.shutdown();
            }
        } );
    }
}

经过几次迭代（约150K）后出现错误消息：

After a few iterations (around 150K) I got error message:

java.lang.OutOfMemoryError：Java堆空间
at java.nio.HeapByteBuffer。（HeapByteBuffer.java:39）
at java.nio.ByteBuffer.allocate（ByteBuffer.java:312）
at org.neo4j.kernel.impl.nioneo.store.PlainPersistenceWindow。（PlainPersistenceWindow.java:30）
at org.neo4j.kernel .impl.nioneo.store.PersistenceWindowPool.allocateNewWindow（PersistenceWindowPool.java:534）
at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.refreshBricks（PersistenceWindowPool.java:430）
at org .neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire（PersistenceWindowPool.java:122）
at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow（CommonAbstractStore.java:459)
at org.neo4j.kernel.impl.nioneo.store.AbstractDynamicStore.updateRecord（AbstractDynamicStore.java:240）
at org.neo4j.kernel.impl.nioneo.store.PropertyStore.updateRecord（PropertyStore.java:209）
at org.neo4j.kernel.impl.nioneo.xa.Command $ PropertyCommand.execute（Command.java ：513）
at org.neo4j.kernel.impl.nioneo.xa.NeoTransaction.doCommit（NeoTransaction.java:443）
at org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit （XaTransaction.java:316）
at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit（XaResourceManager.java:399）
at org.neo4j.kernel.impl.transaction.xaframework .XaResourceHelpImpl.commit（XaResourceHelpImpl.java:64）
at org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit（TransactionImpl.java:514）
at org.neo4j.kernel.impl.transaction .TxManager.commit（TxManager.java:571）
at org.neo4j.kernel.impl.transaction.TxManager.commit（TxManager.java:543）
at org.neo4j.kernel.impl.transaction .TransactionImpl.commit（TransactionImpl.java ：102）
at org.neo4j.kernel.EmbeddedGraphDbImpl $ TransactionImpl.finish（EmbeddedGraphDbImpl.java:329）
at javaapplication2.Main.main（Main.java:62）
28.05.2010 9:52:14 org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool logWarn
警告：[neo4j-store-1M\\\
eostore.propertystore.db.strings]无法分配直接缓冲区

"java.lang.OutOfMemoryError: Java heap space at java.nio.HeapByteBuffer.(HeapByteBuffer.java:39) at java.nio.ByteBuffer.allocate(ByteBuffer.java:312) at org.neo4j.kernel.impl.nioneo.store.PlainPersistenceWindow.(PlainPersistenceWindow.java:30) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.allocateNewWindow(PersistenceWindowPool.java:534) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.refreshBricks(PersistenceWindowPool.java:430) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire(PersistenceWindowPool.java:122) at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow(CommonAbstractStore.java:459) at org.neo4j.kernel.impl.nioneo.store.AbstractDynamicStore.updateRecord(AbstractDynamicStore.java:240) at org.neo4j.kernel.impl.nioneo.store.PropertyStore.updateRecord(PropertyStore.java:209) at org.neo4j.kernel.impl.nioneo.xa.Command$PropertyCommand.execute(Command.java:513) at org.neo4j.kernel.impl.nioneo.xa.NeoTransaction.doCommit(NeoTransaction.java:443) at org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:316) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:399) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) at org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:514) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:571) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:543) at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:102) at org.neo4j.kernel.EmbeddedGraphDbImpl$TransactionImpl.finish(EmbeddedGraphDbImpl.java:329) at javaapplication2.Main.main(Main.java:62)28.05.2010 9:52:14 org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool logWarnWARNING: [neo4j-store-1M\neostore.propertystore.db.strings] Unable to allocate direct buffer"

伙计们！帮助我plzzz，我做错了什么，我怎么修复它？在Windows XP 32bit SP3平台上测试。也许解决方案在创建自定义配置？

Guys! Help me plzzz, what I did wrong, how can I repair it? Tested on platform Windows XP 32bit SP3. Maybe solution within creation custom configuration?

thnx 4每个建议！

thnx 4 every advice!

推荐答案

this is a configuration issue on Windows, where Neo4j cannot use memory mapped buffers. Instead, a Java Buffer on the heap is created. In 1.0 this buffer was 470MB per default, which is more than the default heap for the Windows JVM. You have two options:


  1. 在你的pom.xml中切换到APOC 1.1-SNAPSHOT而不是1.0，它有一个自动配置，分配最多50％的可用JVM堆到Neo4j

  1. Switch to APOC 1.1-SNAPSHOT instead of 1.0 in your pom.xml which has an autoconfiguration, assigning max 50% of the available JVM heap to Neo4j

通过使用

java -Xmx512m ....

java -Xmx512m ....

您甚至可以在Eclipse中的运行配置中的JVM参数下插入

You can even insert that under JVM arguments in the Run Configurations in Eclipse

如果这有帮助，请告诉我们！

Let us know if this helps!

此外，为每个人做一个完整的交易节点对需要很长时间。尝试在第一个循环中打开一个事务，并且每1000个节点对只提交一次吗？

Also, doing a full transaction for every node pair is going to take a long time. Try opening a transaction in the first loop and do commits only every 1000 node pairs?

/ peter

这篇关于Neo4j OutOfMemory问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！

09-02 06:32
Powered by LMLPHP ©2024 APOC 0.044174