第四章:虚拟机性能监控与故障处理工具

Java 与 C++ 之间有一堵由内存动态分配和垃圾收集技术所围成的高强,墙外面的人想进去,墙里面的人想出来.

4.1 概述

理论总是作为指导实践的工具,能把这些知识应用到实际工作中才是我们的最终目的.

给一个系统定位问题的时候,知识/经验是关键基础, 数据是依据, 工具是运用知识处理数据的手段.
数据包括: 运行日志, 异常堆栈, GC日志, 线程快照, 堆转储快照等.
经常使用适当的虚拟机监控和分析的工具可以加快我们分析数据,定位解决问题的速度.

4.2 JDK的命令行工具

JDK的bin目录

表:Sun JDK 监控和故障处理工具

名称	主要作用
jps	JVM Process Status Tool, 显示指定系统内所有的HotSpot 虚拟机进程
jstat	JVM Statistics Monitoring Tool, 用于收集 HotSpot 虚拟机各方面的运行数据
jinfo	Configuration Info for Java, 显示虚拟机配置信息
jmap	Memory Map for Java, 生成虚拟机的内存转储快照(headdump 文件)
jhat	JVM Heap Dump Browser, 用于分析heapdump 文件,它会建立一个HTTP/HTML 服务器, 让用户可以在浏览器上查看分析结果
jstack	Stack Trace for Java, 显示虚拟机的线程快照

4.2.1 jps: 虚拟机进程状态工具

jps: 列出正在运行的虚拟机进程,并显示虚拟机执行主类(Main Class, main()函数所在的类)名称以及这些进程的本地虚拟机唯一ID(Local Virtual Machine Identifier, LVMID).
jps 命令格式: jps [options] [hostid]

jps 执行样例:

linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jps -l
30096 blog-0.0.1-SNAPSHOT.jar
13242 sun.tools.jps.Jps
linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jps -q
30096
13254
linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jps -m
30096 jar
13266 Jps -m
linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jps -v
30096 jar
13278 Jps -Denv.class.path=.:/usr/java/jdk1.8.0_161/lib:/usr/java/jdk1.8.0_161/jre/lib -Dapplication.home=/usr/java/jdk1.8.0_161 -Xms8m
linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jps -lv
30096 blog-0.0.1-SNAPSHOT.jar
13292 sun.tools.jps.Jps -Denv.class.path=.:/usr/java/jdk1.8.0_161/lib:/usr/java/jdk1.8.0_161/jre/lib -Dapplication.home=/usr/java/jdk1.8.0_161 -Xms8m

表: jps 工具主要选项

选项	作用
-q	只输出 LVMID, 省略主类的名称
-m	输出虚拟机进程启动时传递给主类main() 函数的参数
-l	输出主类的全名, 如果进程执行的是Jar包,输出 Jar路径
-v	输出虚拟机进程启动时JVM参数

4.2.2 jstat: 虚拟机统计信息监视工具

jstat(JVM Statistics Monitoring Tool) 是用于监视虚拟机各种运行状态信息的命令行工具. 它可以显示本地或者远程虚拟机进程中的类装载,内存,垃圾收集,JIT编译等运行数据, 在没有GUI图形界面,只提供了纯文本控制台环境的服务器上,它将是运行期定位虚拟机性能问题的首选工具.

jstat 命令格式: jstat [option vmid [interval [s | ms] [count] ]]
假设需要每250毫秒查询一次进程2764的垃圾收集情况, 一共查询20次:

jstat -gc 2764 250 20

表: jstat 工具主要选项

选项	作用
-class	监视类装载,卸载数量,总空间以及类装载所耗费的时间
-gc	监视Java堆状况,包括Eden区, 两个Survivor区,老年代,永久代等的容量,已用空间,GC时间合计等信息
-gccapacity	监视内容与-gc基本相同,但输出主要专注Java堆各个区域使用到的最大,最小空间
-gcutil	监视内容与 -gc 基本相同, 但输出主要关注已使用空间占总空间的百分比
-gccause	与 -gcutil 功能一样,但是会额外输出导致上一次GC产生的原因
-gcnew	监视新生代GC状况
-gcnewcapacity	监视内容与 -gcnew 基本相同,输出主要关注使用到的最大,最小空间
-gcold	监视老年代GC状况
-gcoldcapacity	监视内容与i -gcold 基本相同,输出主要关注使用到的最大,最小空间
-gcpermcapacity	输出永久代使用到的最大,最小空间
-compiler	输出JIT 编译器编译过的方法,耗时等信息
-printcompilation	输出已经被JIT编译的方法

linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jstat -class 30096
Loaded  Bytes  Unloaded  Bytes     Time   
 11647 21454.1        0     0.0      19.01
linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jstat -gc 30096    
 S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT   
2624.0 2624.0 100.6   0.0   21184.0   6007.8   52748.0    49099.7   65968.0 64808.9 8112.0 7844.3    378    2.038   3      0.311    2.349
linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jstat -gccapacity 30096
 NGCMN    NGCMX     NGC     S0C   S1C       EC      OGCMN      OGCMX       OGC         OC       MCMN     MCMX      MC     CCSMN    CCSMX     CCSC    YGC    FGC 
 10880.0 171328.0  26432.0 2624.0 2624.0  21184.0    21888.0   342720.0    52748.0    52748.0      0.0 1107968.0  65968.0      0.0 1048576.0   8112.0    378     3
linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jstat -gcutil 30096
  S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT     GCT   
  3.84   0.00  28.36  93.08  98.24  96.70    378    2.038     3    0.311    2.349
linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jstat -gccause 30096
  S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT     GCT    LGCC                 GCC                 
  3.84   0.00  28.36  93.08  98.24  96.70    378    2.038     3    0.311    2.349 Allocation Failure   No GC               
linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jstat -gcnew 30096
 S0C    S1C    S0U    S1U   TT MTT  DSS      EC       EU     YGC     YGCT  
2624.0 2624.0  100.6    0.0 15  15 1312.0  21184.0   6007.8    378    2.038
linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jstat -gcold 30096
   MC       MU      CCSC     CCSU       OC          OU       YGC    FGC    FGCT     GCT   
 65968.0  64808.9   8112.0   7844.3     52748.0     49099.7    378     3    0.311    2.349
linyk3@aliyun02:/usr/java/jdk1.8.0_161/bin$ jstat -compiler 30096
Compiled Failed Invalid   Time   FailedType FailedMethod
   11317      1       0    58.19          1 com/mysql/jdbc/AbandonedConnectionCleanupThread run

4.2.3 jinfo: Java 配置信息工具

jinfo(Configuration Info for Java) 的作用是实时的查看和调整虚拟机各项参数.

jinfo 命令格式: jinfo [ option ] pid
执行样例:

linyk3@aliyun02:~$ jinfo -flag CMSInitiatingOccupancyFraction 30096
-XX:CMSInitiatingOccupancyFraction=-1

4.2.4 jmap : Java 内存映像工具

jmap(Memory Map for Java) 命令用于生成堆转储快照(一般称为 heapdump 或 dump 文件).
jmap 的作用不仅仅是为了获取dump文件,它还可以查询finalize执行队列, Java堆和永久代的详细信息,如空间使用率,当前使用的是哪种收集器等.

jmap 命令格式: jmap [ option ] vmid

表: jmap 工具的主要选项

选项	作用
-dump	生成Java堆转储快照. 格式为 -dump:[live, ] format=b, file=<filename>, 其中live子参数说明是否只dump出存活的对象
-finalizerinfo	显示在F-Queue 中等待 Finalizer 线程执行 finalize 方法的对象. 只在 Linux/Solaris 平台下有效
-heap	显示Java堆详细信息, 如使用何种收集器,参数配置,分代状况等.只在Linux/Solaris平台下有效
-histo	显示堆中对象统计信息, 包括类,实列对象,合计容量
-permstat	以ClassLoader 为统计口径显示永久代内存状态. 只在Linux/Solaris平台下有效
-F	当虚拟机进程对 -dump 选项没有响应时, 可使用这个选项强制生成 dump 快照. 只在 Linux/Solaris 平台下有效

linyk3@aliyun02:~$ jps -l
30096 blog-0.0.1-SNAPSHOT.jar
16257 sun.tools.jps.Jps
linyk3@aliyun02:~$ 
linyk3@aliyun02:~$ jmap -dump:format=b,file=blog.bin 30096
Dumping heap to /home/linyk3/blog.bin ...
Heap dump file created
linyk3@aliyun02:~$

4.2.5 jhat: 虚拟机堆转储快照分析工具

Sun JDK 提供 jhat(JVM Heap Analysis Tool) 命令与 jmap 搭配使用,来分析jmap 生成的堆转储快照.

linyk3@aliyun02:~$ jhat blog
blog/     blog.bin  
linyk3@aliyun02:~$ jhat blog.bin 
Reading from blog.bin...
Dump file created Tue Jan 21 00:59:03 CST 2020
Snapshot read, resolving...
Resolving 1125167 objects...
Chasing references, expect 225 dots.................................................................................................................................................................................................................................
Eliminating duplicate references.................................................................................................................................................................................................................................
Snapshot resolved.
Started HTTP server on port 7000
Server is ready.

当提示"Server is ready" 时, 访问服务器http://locolhost:7000/ 就可以看到分析结果了

image.png

分析内存泄漏只要是用到其中的 "heap histogram"

image.png

4.2.6 jstack : Java 堆栈跟踪工具

jstack(Stack Trace for Java)命令用于生成虚拟机挡墙时刻的线程快照(一般称为threaddump或者javacore文件).

线程快照就是当前虚拟机内每一条线程正在执行的方法堆栈的集合,生成线程快照的主要目的就是定位线程出现长时间停顿的原因,如线程间锁死, 死循环, 请求外部资源导致的长时间等待等都是导致线程长时间停顿的常见原因.

线程出现停顿的时候,通过jstack来查看各个线程的调用堆栈,就可以知道没有响应的线程到底在后台做些什么事, 或者等待着什么资源.

jstack 命令格式: jstack [ option ] vmid

表: jstack 工具主要选项

选项	作用
-F	当正常输出的请求不被响应时, 强制输出线程堆栈
-l	除堆栈外,显示关于锁的附加消息
-m	如果调用到本地方法的话, 可以显示C/C++的堆栈

linyk3@aliyun02:~$ jstack -l 30096
2020-01-21 01:20:55
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.161-b12 mixed mode):

"Attach Listener" #34 daemon prio=9 os_prio=0 tid=0x00007f503802b000 nid=0x3eed waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"logback-1" #33 daemon prio=5 os_prio=0 tid=0x00007f5034558000 nid=0x75d2 waiting on condition [0x00007f502bbbc000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000edb37bd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"DestroyJavaVM" #32 prio=5 os_prio=0 tid=0x00007f505c00a800 nid=0x7592 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"http-nio-9001-AsyncTimeout" #30 daemon prio=5 os_prio=0 tid=0x00007f505e36d800 nid=0x75c9 waiting on condition [0x00007f502bebd000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.coyote.AbstractProtocol$AsyncTimeout.run(AbstractProtocol.java:1211)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-Acceptor-0" #29 daemon prio=5 os_prio=0 tid=0x00007f505e8ad000 nid=0x75c8 runnable [0x00007f502bfbe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        - locked <0x00000000ed9fead0> (a java.lang.Object)
        at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:455)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-ClientPoller-0" #28 daemon prio=5 os_prio=0 tid=0x00007f505d103800 nid=0x75c7 runnable [0x00007f502c0bf000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000000eda5c8a0> (a sun.nio.ch.Util$3)
        - locked <0x00000000eda5c8b0> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000000eda5c858> (a sun.nio.ch.EPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:787)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-exec-10" #27 daemon prio=5 os_prio=0 tid=0x00007f505ca4d000 nid=0x75c6 waiting on condition [0x00007f502c1c0000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000ed9feae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-exec-9" #26 daemon prio=5 os_prio=0 tid=0x00007f505e79e800 nid=0x75c5 waiting on condition [0x00007f502c2c1000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000ed9feae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-exec-8" #25 daemon prio=5 os_prio=0 tid=0x00007f505c15a800 nid=0x75c4 waiting on condition [0x00007f502c3c2000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000ed9feae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-exec-7" #24 daemon prio=5 os_prio=0 tid=0x00007f505c9df800 nid=0x75c3 waiting on condition [0x00007f502c4c3000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000ed9feae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-exec-6" #23 daemon prio=5 os_prio=0 tid=0x00007f505d03f800 nid=0x75c2 waiting on condition [0x00007f502c5c4000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000ed9feae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-exec-5" #22 daemon prio=5 os_prio=0 tid=0x00007f505e51e000 nid=0x75c1 waiting on condition [0x00007f502c6c5000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000ed9feae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-exec-4" #21 daemon prio=5 os_prio=0 tid=0x00007f505e5a3000 nid=0x75c0 waiting on condition [0x00007f502c7c6000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000ed9feae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-exec-3" #20 daemon prio=5 os_prio=0 tid=0x00007f505c387000 nid=0x75bf waiting on condition [0x00007f502c8c7000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000ed9feae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-exec-2" #19 daemon prio=5 os_prio=0 tid=0x00007f505c386000 nid=0x75be waiting on condition [0x00007f502c9c8000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000ed9feae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"http-nio-9001-exec-1" #18 daemon prio=5 os_prio=0 tid=0x00007f505c333000 nid=0x75bd waiting on condition [0x00007f504c1fd000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000ed9feae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"NioBlockingSelector.BlockPoller-1" #17 daemon prio=5 os_prio=0 tid=0x00007f505cdb6000 nid=0x75bc runnable [0x00007f502e0cd000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000000ed9fed78> (a sun.nio.ch.Util$3)
        - locked <0x00000000ed9fed88> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000000ed9fed30> (a sun.nio.ch.EPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:339)

   Locked ownable synchronizers:
        - None

"Abandoned connection cleanup thread" #16 daemon prio=5 os_prio=0 tid=0x00007f505cd44800 nid=0x75a4 in Object.wait() [0x00007f502dac9000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
        - locked <0x00000000ec550f60> (a java.lang.ref.ReferenceQueue$Lock)
        at com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - <0x00000000ec550ff0> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"Tomcat JDBC Pool Cleaner[1804094807:1576085386247]" #15 daemon prio=5 os_prio=0 tid=0x00007f505ccfe800 nid=0x75a3 in Object.wait() [0x00007f502dbca000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        - locked <0x00000000ec5511a0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:505)

   Locked ownable synchronizers:
        - None

"container-0" #14 prio=5 os_prio=0 tid=0x00007f505cb90800 nid=0x75a2 waiting on condition [0x00007f502decb000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.catalina.core.StandardServer.await(StandardServer.java:427)
        at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer$1.run(TomcatEmbeddedServletContainer.java:177)

   Locked ownable synchronizers:
        - None

"ContainerBackgroundProcessor[StandardEngine[Tomcat]]" #13 daemon prio=5 os_prio=0 tid=0x00007f505cb92000 nid=0x75a1 waiting on condition [0x00007f502dfcc000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1355)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

"Service Thread" #7 daemon prio=9 os_prio=0 tid=0x00007f505c0dc000 nid=0x7599 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"C1 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007f505c0d8800 nid=0x7598 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007f505c0d6800 nid=0x7597 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007f505c0d2800 nid=0x7596 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f505c0a2000 nid=0x7595 in Object.wait() [0x00007f504cd24000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
        - locked <0x00000000eb17c9c0> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

   Locked ownable synchronizers:
        - None

"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f505c09d800 nid=0x7594 in Object.wait() [0x00007f504ce25000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
        - locked <0x00000000eb17cb78> (a java.lang.ref.Reference$Lock)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

   Locked ownable synchronizers:
        - None

"VM Thread" os_prio=0 tid=0x00007f505c096000 nid=0x7593 runnable 

"VM Periodic Task Thread" os_prio=0 tid=0x00007f505c0e1000 nid=0x759a waiting on condition 

JNI global references: 289

linyk3@aliyun02:~$

4.2.7 HSDIS: JIT 生成代码反汇编

在Java虚拟机规范中, 详细描述了虚拟机指令集中每条指令的执行过程,执行前后对操作数栈,局部变量表的影响等细节.

分析程序如何执行,通过软件调试工具(GDB, Windbg)来断点调试是最常见的手段.但是在调试Java虚拟机会遇到很大困难.

HSDIS 是一个 Sun 官方推荐的 HotSpot 虚拟机 JIT 编译代码的反汇编插件.它的作用是让HotSpot的-XX:+PrintAssembly 指令调用它来把动态生成的本地代码还原为汇编代码输出,同时还生成大量非常有价值的注释,这样我们就可以通过输出的代码来分析问题.

4.3 JDK 可视化工具

JDK中除了提供大量的命令行工具,还有两个功能强大的可视化工具: JConsole 和 VisualVM.
JConsole: JDK 1.5 就以及提供的虚拟机监控工具
VisualVM: JDK1.6 Update7中才首次发布. 现在已经成为主力推动的多合一故障处理工具.

4.3.1 JConsole: Java 监视与管理控制台

启动:

image.png

4.3.2 VisualVM: 多合一故障处理工具

VisualVM(All-In-One Java Troubleshooting Tool)是到目前为止随JDK发布的功能最强大的运行监视和故障处理程序.

VisualVM 基于 NetBeans 平台开发, 所以具备了插件扩展功能:

1. 显示虚拟机进程以及进程的配置,环境信息(jps, jinfo)
2.监视应用程序的CPU, GC, 堆, 方法区以及线程的信息(jstat, jstack)
3.dump 以及分析堆转储快照(jmap, jhat)
4.方法级的程序运行性能分析,找出被调用最多,运行时间最长的方法
5.离线程序快照: 手机程序的运行时配置, 线程dump, 内存 dump等信息建立一个快照, 可以将快照发送开发者进行bug反馈.
6.其他 plugins 的无限可能性

image.png

4.4 本章小结

本章介绍了随JDK发布的6个命令行工具以及两个可视化的故障处理工具,灵活使用这些工具可以给问题处理带来很大的便利.

除了JDK自带的工具外,常用的故障处理工具还有很多.

最后编辑于：2020.01.21 01:45:38

人面猴
序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
沈念sama阅读 203,324评论 5赞 476
死咒
序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
沈念sama阅读 85,303评论 2赞 381
救了他两次的神仙让他今天三更去死
文/潘晓璐我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
开封第一讲书人阅读 150,192评论 0赞 337
道士缉凶录：失踪的卖姜人
文/不坏的土叔我叫张陵，是天一观的道长。经常有香客问我，道长，这世上最难降的妖魔是什么？我笑而不...
开封第一讲书人阅读 54,555评论 1赞 273
港岛之恋（遗憾婚礼）
正文为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
茶点故事阅读 63,569评论 5赞 365
恶毒庶女顶嫁案：这布局不是一般人想出来的
文/花漫我一把揭开白布。她就那样静静地躺着，像睡着了一般。火红的嫁衣衬着肌肤如雪。梳的纹丝不乱的头发上，一...
开封第一讲书人阅读 48,566评论 1赞 281
城市分裂传说
那天，我揣着相机与录音，去河边找鬼。笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。我是一名探鬼主播，决...
沈念sama阅读 37,927评论 3赞 395
双鸳鸯连环套：你想象不到人心有多黑
文/苍兰香墨我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
开封第一讲书人阅读 36,583评论 0赞 257
万荣杀人案实录
序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
沈念sama阅读 40,827评论 1赞 297
护林员之死
正文独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋以下内容为张勋视角年9月15日...
茶点故事阅读 35,590评论 2赞 320
白月光启示录
正文我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
茶点故事阅读 37,669评论 1赞 329
活死人
序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
沈念sama阅读 33,365评论 4赞 318
日本核电站爆炸内幕
正文年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
茶点故事阅读 38,941评论 3赞 307
男人毒药：我在死后第九天来索命
文/蒙蒙一、第九天我趴在偏房一处隐蔽的房顶上张望。院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
开封第一讲书人阅读 29,928评论 0赞 19
一桩弑父案，背后竟有这般阴谋
文/苍兰香墨我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。一阵脚步声响...
开封第一讲书人阅读 31,159评论 1赞 259
情欲美人皮
我被黑心中介骗来泰国打工，没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。一个月前我还...
沈念sama阅读 42,880评论 2赞 349
代替公主和亲
正文我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
茶点故事阅读 42,399评论 2赞 342

第四章:虚拟机性能监控与故障处理工具

4.1 概述

4.2 JDK的命令行工具

4.2.1 jps: 虚拟机进程状态工具

4.2.2 jstat: 虚拟机统计信息监视工具

4.2.3 jinfo: Java 配置信息工具

4.2.4 jmap : Java 内存映像工具

4.2.5 jhat: 虚拟机堆转储快照分析工具

4.2.6 jstack : Java 堆栈跟踪工具

4.2.7 HSDIS: JIT 生成代码反汇编

4.3 JDK 可视化工具

4.3.1 JConsole: Java 监视与管理控制台

4.3.2 VisualVM: 多合一故障处理工具

4.4 本章小结

推荐阅读更多精彩内容