问题描述
试图在64位linux(CentOS)计算机上诊断一些奇怪的Tomcat( 7.0.21 )和/或JVM错误。
我负载测试我们的服务器应用程序,并尝试用100K消息击中它。发起 jvisualvm ,并始终关注堆。一切看起来很棒*(见下文),直到我得到大约93K处理的消息,然后Tomcat刚刚死了。
在Tomcat的PID号码上输入 ps ,以确认它已死。
- 负载测试已运行约90分钟;
- CPU在45%左右保持不变
- 已使用的堆大约为2GB减去一堆后的GCs),但堆大小从4GB增加到 MAX_HEAP 大约30分钟后
- 类加载/卸载正常循环
- 线程转储是正常的
服务器代码中没有任何对 System.exit() - 所以我们可以立即规则化(是的,我已经仔细检查了!!!)。
我不知道这是Tomcat崩溃还是JVM(我如何告诉?)。即使我知道,我似乎找不到任何错误的指示:
- 所有的服务器应用程序的日志停止且没有任何ERROR消息(即使我们的日志记录已普遍设置为DEBUG或更高)
- Tomcat catalina.out c $ c> localhost_access _ * 文件只是停止没有任何信息
我听说有可能有Tomcat日志的核心转储,但它不确定如何做,在线示例没有帮助太多。
如何诊断这个?
提前感谢!
对不起,我不得不删除从@erickson的绿色支票。我终于知道是什么杀死了Tomcat。
看起来像一个profiler插件没有正确配置VisualVM,试图运行一个配置文件在Tomcat进程杀了它。 / p>
立即调查原因,一旦我了解更多就会更新此答案。
Trying to diagnose some bizarre Tomcat (7.0.21) and/or JVM errors on a 64-bit linux (CentOS) machine.
I'm load testing our server application and tried hitting it with 100K messages. Launched jvisualvm and kept my eye on the heap the whole time. Everything was looking great* (see below) until I got to about 93K processed messages and then Tomcat just died. Ran a ps on Tomcat's PID number to confirm it was dead.
Up until this crash:
- Load test had been running for about 90 minutes; should have finished shortly thereafter since we were at 93K/100K)
- CPU was holding strong around 45%
- Used heap was around 2GB (plus or minus a bunch after GCs) but heap size grew from 4GB to MAX_HEAP after about 30 minutes
- Class loading/unloading was cycling normally
- Thread dumps were normal
Nowhere in the server code are any calls to System.exit() - so we can rule that right out (and yes I've double-checked!!!).
I'm not sure if this is Tomcat crashing or the JVM (how do I tell?). And even if I did know, I can't seem to find any indication of what went wrong:
- All of the server app's logs just stop without any ERROR messages (even though we have logging universally set to DEBUG and higher)
- Tomcat's catalina.out and respect localhost_access_* files just stop without any info
I've heard it is possible to have Tomcat log a coredump when it does but not sure how to do that and online examples aren't helping much.
How would SO go about diagnosing this? What steps should I take to start ruling out all of the possible factors?
Thanks in advance!
Sorry I had to remove the green check from @erickson. I finally figured out what was killing Tomcat.
It looks like a profiler plugin is not configured correctly with VisualVM and attempting to run a profile on the Tomcat process killed it.
Investigating why right now, and will update this answer once I know more.
这篇关于汤姆突然死了的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!