hadoop►zookeeper►zookeeper monitor

zookeeper monitor

zookeeper监控指令

1
2
3
4
5
6
7
8
9
10
11
12
* conf   输出相关服务配置的详细信息
* cons   列出所有连接到服务器的客户端的完全的连接 / 会话的详细信息(包括"接受 / 发送"的包数量,会话id,操作延迟,最后的操作执行等等信息)
* dump   列出未经处理的会话和临时节点
* envi   输出关于服务环境的详细信息(区别于 conf 命令)
* reqs   列出未经处理的请求
* ruok   测试服务是否运行正常.如果确实如此,那么服务返回"imok",否则不做任何相应
* stat   输出关于性能和连接的客户端的列表
* wchs   列出服务器 watch 的详细信息
* wchc   通过 session 列出服务器 watch 的详细信息,它的输出是一个与 watch 相关的会话的列表
* wchp   通过路径列出服务器 watch 的详细信息。它输出一个与 session 相关的路径
* srvr   列出server的简要信息
* srst   重置server的状态,不建议执行！！！

zookeeper 检测是否正常

1
2
[root@10 ~]# echo ruok | nc 10.0.3.41 2181     
imok     //返回imok,说明zookeeper运行正常

用于监控zookeeper健康状态的各种参数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
[root@10 ~]# echo mntr | nc 10.0.3.41 2181     
zk_version      3.4.6-1569965, built on 02/20/2014 09:09 GMT   //版本
zk_avg_latency  1                                              //平均延迟
zk_max_latency  886                                            //最大延迟
zk_min_latency  0                                              //最小延迟
zk_packets_received     4598897                                //接收包
zk_packets_sent 4727418                                        //发送包
zk_num_alive_connections        4                              //连接数
zk_outstanding_requests 0                                      //未完成客户端请求数
zk_server_state leader                                         //zk角色leader/follower
zk_znode_count  483                                            //znode数
zk_watch_count  28                                             //watch数
zk_ephemerals_count     52                                     //临时节点数
zk_approximate_data_size        35923                          //
zk_open_file_descriptor_count   35                             //打开文件描述符数
zk_max_file_descriptor_count    65536                          //最大打开文件描述符数
zk_followers    2                                              //zk follower数
zk_synced_followers     2                                      //zk 同步follower数
zk_pending_syncs        0                                      //zk 等待同步follower数

zookeeper 配置参数查看

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root@10 ~]# echo conf | nc 10.0.3.41 2181
clientPort=2181                          //客户端端口
dataDir=/mnt/data/zookeeper/version-2    //数据存放路径
dataLogDir=/mnt/log/zookeeper/version-2  //日志存放路径
tickTime=2000                            //间隔单位时间
maxClientCnxns=60                        //
minSessionTimeout=4000
maxSessionTimeout=40000
serverId=41
initLimit=5
syncLimit=2
electionAlg=3
electionPort=4181
quorumPort=3181
peerType=0

列出server的简要信息

1
2
3
4
5
6
7
8
9
10
[root@10 ~]# echo srvr | nc 10.0.3.41 2181     
Zookeeper version: 3.4.6-1569965, built on 02/20/2014 09:09 GMT
Latency min/avg/max: 0/1/886
Received: 4597328
Sent: 4725788
Connections: 5
Outstanding: 0
Zxid: 0x1003e6eeb
Mode: leader
Node count: 483

列服务器环境信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[root@10 ~]# echo envi | nc 10.0.3.41 2181      
Environment:
zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
host.name=localhost
java.version=1.8.0_73
java.vendor=Oracle Corporation
java.home=/mnt/app/java/jre
java.class.path=/mnt/app/zookeeper/bin/../build/classes:/mnt/app/zookeeper/bin/../build/lib/*.jar:/mnt/app/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/mnt/app/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/mnt/app/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/mnt/app/zookeeper/bin/../lib/log4j-1.2.16.jar:/mnt/app/zookeeper/bin/../lib/jline-0.9.94.jar:/mnt/app/zookeeper/bin/../zookeeper-3.4.6.jar:/mnt/app/zookeeper/bin/../src/java/lib/*.jar:/mnt/app/zookeeper/bin/../conf:.:/mnt/app/java/lib:/mnt/app/java/jre/lib
java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
java.io.tmpdir=/tmp
java.compiler=<NA>
os.name=Linux
os.arch=amd64
os.version=3.10.0-123.9.3.el7.x86_64
user.name=wisdom
user.home=/home/wisdom
user.dir=/home/wisdom

列出性能和连接的客户端列表

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[root@10 ~]# echo stats | nc 10.0.3.41 2181     
Zookeeper version: 3.4.6-1569965, built on 02/20/2014 09:09 GMT
Clients:
 /1.1.1.1:64115[1](queued=0,recved=0,sent=0)
 /10.0.3.41:41921[1](queued=0,recved=623,sent=657)
 /2.2.2.2:37177[1](queued=0,recved=27282,sent=27282)
 /10.0.3.41:41925[0](queued=0,recved=1,sent=0)

Latency min/avg/max: 0/1/886
Received: 4594796
Sent: 4723167
Connections: 4
Outstanding: 0
Zxid: 0x1003e669c
Mode: leader
Node count: 483

列出所有连接到服务器的客户端的完全的连接

1
2
3
[root@10 ~]# echo cons | nc 10.0.3.41 2181
/1.1.1.1:64115[1](queued=0,recved=0,sent=0) /2.2.2.2:37177[1](queued=0,recved=26679,sent=26679,sid=0x2a5a6f2424c90056,lop=SETD,est=1488507549618,to=6000,lcxid=0x22183,lzxid=0x1003e5e2e,lresp=1488522616395,llat=5,minlat=0,avglat=2,maxlat=126)
 /3.3.3.3:41910[0](queued=0,recved=1,sent=0)

列出未经处理的请求

1
[root@10 ~]# echo reqs | nc 10.0.3.41 2181

列出未经处理的会话和临时节点

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
[root@10 ~]# echo dump | nc 10.0.3.41 2181     
SessionTracker dump:
Session Sets (8):
0 expire at Fri Mar 03 14:32:32 CST 2017:
0 expire at Fri Mar 03 14:32:34 CST 2017:
0 expire at Fri Mar 03 14:32:36 CST 2017:
11 expire at Fri Mar 03 14:32:38 CST 2017:
        0x295a6f2424ce0060
        0x2a5a6f2424c90057
        0x285a6f2425ba0096
        0x295a6f2424ce00a2
        0x295a6f2424ce00a5
        0x2a5a6f2424c90056
        0x285a6f2425ba00c4
        0x295a6f2424ce0001
        0x285a6f2425ba00c3
        0x2a5a6f2424c90000
        0x2a5a6f2424c90055
0 expire at Fri Mar 03 14:32:44 CST 2017:
0 expire at Fri Mar 03 14:32:46 CST 2017:
1 expire at Fri Mar 03 14:32:54 CST 2017:
        0x285a6f2425ba0019
2 expire at Fri Mar 03 14:32:56 CST 2017:
        0x285a6f2425ba0017
        0x285a6f2425ba0018
ephemeral nodes dump:
Sessions with Ephemerals (11):
0x2a5a6f2424c90057:
        /consumers/flume/owners/v5-inbound-request/4
        /consumers/flume/owners/v5-inbound-request/2
        /consumers/flume/ids/flume_gateway2.zdp.ol-1488187633466-719a2fc5
        /consumers/flume/owners/v5-inbound-request/3
        /consumers/flume/owners/v5-inbound-request/0
        /consumers/flume/owners/v5-inbound-request/1
0x285a6f2425ba00c4:
        /consumers/flume/owners/v5-outbound-metric-avail/4
        /consumers/flume/ids/flume_gateway2.zdp.ol-1488187633262-e6b79549
        /consumers/flume/owners/v5-outbound-metric-avail/0
        /consumers/flume/owners/v5-outbound-metric-avail/1
        /consumers/flume/owners/v5-outbound-metric-avail/2
        /consumers/flume/owners/v5-outbound-metric-avail/3
0x2a5a6f2424c90056:
        /consumers/flume/ids/flume_gateway2.zdp.ol-1488187633465-b1247702
        /consumers/flume/owners/v5-inbound-metric/0
        /consumers/flume/owners/v5-inbound-metric/1
        /consumers/flume/owners/v5-inbound-metric/2
        /consumers/flume/owners/v5-inbound-metric/3
        /consumers/flume/owners/v5-inbound-metric/4
0x2a5a6f2424c90055:
        /consumers/flume/ids/flume_gateway2.zdp.ol-1488187633465-7be69da4
        /consumers/flume/owners/v5-inv-request/0
        /consumers/flume/owners/v5-inv-request/1
        /consumers/flume/owners/v5-inv-request/2
        /consumers/flume/owners/v5-inv-request/3
        /consumers/flume/owners/v5-inv-request/4
0x285a6f2425ba00c3:
        /consumers/flume/owners/v5-3rdservice-request/0
        /consumers/flume/ids/flume_gateway2.zdp.ol-1488187633261-613ec6e4
        /consumers/flume/owners/v5-3rdservice-request/3
        /consumers/flume/owners/v5-3rdservice-request/4
        /consumers/flume/owners/v5-3rdservice-request/1
        /consumers/flume/owners/v5-3rdservice-request/2
0x295a6f2424ce0060:
        /consumers/flume/ids/flume_gateway2.zdp.ol-1488187633465-8124b5c4
        /consumers/flume/owners/v5-avail-request/4
        /consumers/flume/owners/v5-avail-request/2
        /consumers/flume/owners/v5-avail-request/3
        /consumers/flume/owners/v5-avail-request/0
        /consumers/flume/owners/v5-avail-request/1
0x295a6f2424ce0001:
        /brokers/ids/42
0x285a6f2425ba0096:
        /consumers/flume/owners/v5-rate-request/1
        /consumers/flume/owners/v5-rate-request/2
        /consumers/flume/owners/v5-rate-request/3
        /consumers/flume/owners/v5-rate-request/4
        /consumers/flume/owners/v5-rate-request/0
        /consumers/flume/ids/flume_gateway2.zdp.ol-1488187633466-1c0d7513
0x295a6f2424ce00a2:
        /consumers/flume/owners/v5-outbound-metric-rate/2
        /consumers/flume/owners/v5-outbound-metric-rate/1
        /consumers/flume/owners/v5-outbound-metric-rate/0
        /consumers/flume/owners/v5-outbound-metric-rate/4
        /consumers/flume/owners/v5-outbound-metric-rate/3
        /consumers/flume/ids/flume_gateway2.zdp.ol-1488187633263-1ba4161c
0x295a6f2424ce00a5:
        /brokers/ids/40
0x2a5a6f2424c90000:
        /brokers/ids/41
        /controller

列出服务器watch详细信息

1
2
3
[root@10 ~]# echo wchs | nc 10.0.3.41 2181
2 connections watching 21 paths
Total watches:22

通过session列出服务器watch详细信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
[root@10 ~]# echo wchc | nc 10.0.3.41 2181  
0x295a6f2424ce00a5
        /controller
        /config/changes
0x2a5a6f2424c90000
        /isr_change_notification
        /admin/preferred_replica_election
        /brokers/ids
        /admin/delete_topics
        /brokers/topics/v5-inv-request
        /brokers/topics/__consumer_offsets
        /brokers/topics
        /controller
        /brokers/topics/v5-inbound-request
        /brokers/topics/v5-inbound-metric
        /admin/reassign_partitions
        /brokers/topics/v5-apiweb-request-selectroom
        /brokers/topics/v5-outbound-metric-avail
        /brokers/topics/v5-rate-metric
        /brokers/topics/v5-rate-request
        /brokers/topics/v5-outbound-metric-rate
        /brokers/topics/v5-inv-metric
        /brokers/topics/v5-avail-request
        /config/changes
        /brokers/topics/v5-3rdservice-request
0x2a5a6f2424c90056
        /brokers/topics/v5-inbound-metric
        /consumers/flume/ids

通过路列出服务器watch的详细信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
[root@10 ~]# echo wchp | nc 10.0.3.41 2181  
/admin/delete_topics
        0x2a5a6f2424c90000
/brokers/topics
        0x2a5a6f2424c90000
/brokers/topics/v5-inbound-metric
        0x2a5a6f2424c90000
        0x2a5a6f2424c90056
/admin/reassign_partitions
        0x2a5a6f2424c90000
/brokers/topics/v5-apiweb-request-selectroom
        0x2a5a6f2424c90000
/brokers/topics/v5-3rdservice-request
        0x2a5a6f2424c90000
/isr_change_notification
        0x2a5a6f2424c90000
/admin/preferred_replica_election
        0x2a5a6f2424c90000
/brokers/ids
        0x2a5a6f2424c90000
/brokers/topics/v5-inv-request
        0x2a5a6f2424c90000
/brokers/topics/__consumer_offsets
        0x2a5a6f2424c90000
/controller
        0x2a5a6f2424c90000
        0x295a6f2424ce00a5
/consumers/flume/ids
        0x2a5a6f2424c90056
/brokers/topics/v5-inbound-request
        0x2a5a6f2424c90000
/brokers/topics/v5-outbound-metric-avail
        0x2a5a6f2424c90000
/brokers/topics/v5-rate-metric
        0x2a5a6f2424c90000
/brokers/topics/v5-rate-request
        0x2a5a6f2424c90000
/brokers/topics/v5-outbound-metric-rate
        0x2a5a6f2424c90000
/brokers/topics/v5-inv-metric
        0x2a5a6f2424c90000
/brokers/topics/v5-avail-request
        0x2a5a6f2424c90000
/config/changes
        0x2a5a6f2424c90000
        0x295a6f2424ce00a5