我们在 CEP 上看到内存不足错误。线程转储显示大约有 32000 个线程在监视器上休眠。此外,即使 CEP JVM 选项指定在内存不足时生成 HeapDump,我们也看不到任何生成的堆转储..
请指教。 (CEP JVM -Xms256m -Xmx1536m)

1) Cassandra 在这个 CEP 上被禁用
2) CEP 版本为 2.1.0
3) CEP 以 WSO2 ESB 为前端(使用 BAM 调解器)。
4) 除了向 CEP 发送实际的有效载荷数据外,ESB 还会向 CEP 发送一个周期性的心跳(每 15 ec)。
5) 我们还在 ESB 上配置了 JMX 代理,它每 15 分钟监控一次 CEP(cpu/memorythreads)
6) 即使指定了 -XX:HeapDumpPath= 参数,也未找到堆转储

  • 在这次OOM之前,CEP连续运行了7天。重新启动后,我们观察到线程数以每天大约 4000-5000 个线程的速度稳步增加

  • CEP 日志..
    [2013-06-10 05:31:49,040] ERROR -  Thread Thread[ActiveMQ InactivityMonitor  WriteCheckTimer,5,main] died {org.apache.zookeeper.server.NIOServerCnxn}
    java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Thread.java:640)
    at  java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
    at org.apache.activemq.transport.AbstractInactivityMonitor.writeCheck(AbstractInactivityMonitor.java:153)
    at org.apache.activemq.transport.AbstractInactivityMonitor$2.run(AbstractInactivityMonitor.java:117)
    at org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
    at java.util.TimerThread.mainLoop(Timer.java:512)
    at java.util.TimerThread.run(Timer.java:462)
    [2013-06-10 05:31:49,040] ERROR -  Thread Thread[ActiveMQ InactivityMonitor WriteCheckTimer,5,main] died {org.apache.zookeeper.server.NIOServerCnxn}
    java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Thread.java:640)
    at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
    at org.apache.activemq.transport.AbstractInactivityMonitor.writeCheck(AbstractInactivityMonitor.java:153)
    at org.apache.activemq.transport.AbstractInactivityMonitor$2.run(AbstractInactivityMonitor.java:117)
    at org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
    at java.util.TimerThread.mainLoop(Timer.java:512)
    at java.util.TimerThread.run(Timer.java:462)
    

    CEP中配置的一些查询
    <cep:query name="xxxBuildUpQuery">
    <cep:expression><![CDATA[from  xxxCEPIntgDataStream[interfaceInformationInterfaceName=='xxx-xxx' or interfaceInformationInterfaceName=='xxx-xxx'or
                                   interfaceInformationInterfaceName=='xxx-xxx' or interfaceInformationInterfaceName=='xxx-xxx' or
                                   interfaceInformationInterfaceName=='xxx-xxx' or interfaceInformationInterfaceName=='xxxx-xxx' or
                                   interfaceInformationInterfaceName=='xxx-xxx' or
                                   interfaceInformationInterfaceName=='xxx-xxx' ]#window.time(60000)
    insert into buildUpStream interfaceInformationInterfaceName, count(interfaceInformationxxxId) as noOfInflowMsgs group by interfaceInformationInterfaceName]]></cep:expression>
    <cep:output brokerName="activemqJmsBroker" topic="xxxBuildUpInfoTopic">
      <cep:xmlMapping>
        <xxxAnalytics>
          <buildUpInfo>
            <interfaceName>{interfaceInformationInterfaceName}</interfaceName>
            <buildUpPerMin>{noOfInflowMsgs}</buildUpPerMin>
          </buildUpInfo>
        </xxxAnalytics>
      </cep:xmlMapping>
    </cep:output>
    </cep:query>
    <cep:query name="xxxQueueDepthQuery">
    <cep:expression><![CDATA[from xxxIntgrQueueDepthData_v1
    insert into xxxIntgrQueueDepthStream flowName,appName, queueDepth]]>  </cep:expression>
    <cep:output brokerName="activemqJmsBroker" topic="xxxIntgrQueueDepthTopic">
      <cep:xmlMapping>
        <xxxAnalytics>
          <queueDepthInfo>
            <flowName>{flowName}</flowName>
            <appName>{appName}</appName>
            <depth>{queueDepth}</depth>
          </queueDepthInfo>
        </xxxAnalytics>
      </cep:xmlMapping>
    </cep:output>
    </cep:query>
    <cep:query name="xxxClockDataQuery">
      <cep:expression><![CDATA[from testStream
    insert into testOutClockDataStream AEDateTime]]></cep:expression>
      <cep:output brokerName="activemqJmsBroker" topic="xxxClockDataTopic">
        <cep:xmlMapping>
          <xxxClockFeed>
            <data>
              <XXDateTime>{XXDateTime}</XXDateTime>
            </data>
          </xxxClockFeed>
        </cep:xmlMapping>
      </cep:output>
     </cep:query>
     <cep:query name="xxxSimltrPaymntAvgQuery_1">
      <cep:expression><![CDATA[from xxxCEPIntgDataStream#window.time(15000)
        insert into xxxSimltrPymntAvgData avg(amount) as avgAmount, currency group by currency]]></cep:expression>
      <cep:output brokerName="activemqJmsBroker" topic="xxxAvgPaymntDetails">
        <cep:xmlMapping>
          <xxxAnalytics>
            <avgPaymentData>
              <avgAmount>{avgAmount}</avgAmount>
              <currency>{currency}</currency>
            </avgPaymentData>
          </xxxAnalytics>
        </cep:xmlMapping>
      </cep:output>
    

    谢谢
    拉吉夫·帕蒂尔

    最佳答案

    我发现 Siddhi Manager 启动了一个以 Integer.MAX_VALUE 作为核心池大小的调度线程池。这意味着每个请求都会创建一个新线程,没有超时策略。 (引用: ThreadPoolExecutor )

    在 WSO2 解决此问题之前,您可以更改此线程池的大小。 P.e. 在 org.wso2.siddhi.core.SiddhiManager 类中更改行:



    (SiddhiManager 版本 1.1.0-wso2v1 中的第 77 行)

    对这个:



    此更改将创建大小为 100 的核心池、最大池大小为 Integer.MAX_VALUE,并且空闲线程(超过核心池大小)将在完成后立即删除。

    10-08 15:59