Linux 性能优化工具包 sysstat 以及 sysstat 服务,性能优化sysstat


1. sysstat service

1.1. /etc/cron.d/sysstat

Linux 下的性能分析,少不了要使用 sysstat 工具包中的命令。同时 sysstat 服务会自动每个10分钟收集一次系统状态信息,并将它们存储到 /var/log/sa目录下。每天一个文件,文件名分别是sa01, sa02,... sa30..

[root@localhost sa]# pwd
/var/log/sa
[root@localhost sa]# ls
sa01  sa03  sa05  sa06  sa07  sa08  sa09  sa10  sa11  sa12  sa13  sa14  sa15  sa25  sa26  sa27  sa28  sa29  sa30  sar25

自动收集系统信息,是通过 /etc/cron.d/sysstat 来实现的:

[root@localhost sa]# cat /etc/cron.d/sysstat
# Run system activity accounting tool every 10 minutes
*/10 * * * * root /usr/lib/sa/sa1 1 1
# 0 * * * * root /usr/lib/sa/sa1 600 6 &
# Generate a daily summary of process accounting at 23:53
53 23 * * * root /usr/lib/sa/sa2 -A

上面标示每个10分钟,运行一次 /usr/lib/sa/sa1 1 1, 该命令 man sa1:

sa1 - Collect and store binary data in the system activity daily data file.

SYNOPSIS:/usr/lib/sa/sa1 [ --boot | interval count ]

sa1 收集到的是 binary data. 所以 sa2 -A 负责将当天的二进制文件转换成文本文件。

sa2 - Write a daily report in the /var/log/sa directory

其中的收集信息的频率(10分钟一次)我们可以根据需要进行修改。

1.2. /etc/sysconfig/sysstat

/var/log/sa/目录下的文件保存时间等相关配置存在于文件 /etc/sysconfig/sysstat中:

[root@localhost ~]# cat /etc/sysconfig/sysstat
# sysstat-9.0.4 configuration file.

# How long to keep log files (in days).
# If value is greater than 28, then log files are kept in
# multiple directories, one for each month.
HISTORY=28

# Compress (using gzip or bzip2) sa and sar files older than (in days):
COMPRESSAFTER=31

# Parameters for the system activity data collector (see sadc manual page)
# which are used for the generation of log files.
SADC_OPTIONS="-S DISK"

# Compression program to use.
ZIP="bzip2"

其中的 HISTORY=28 表示最多保存28天的数据。保存时间,我们可以根据需要进行修改。

1.3. 确保 sysstat 服务启动:

[root@localhost ~]# chkconfig --level 35 sysstat on
[root@localhost ~]# chkconfig |grep sysstat
sysstat         0:off   1:on    2:on    3:on    4:on    5:on    6:off

上面是 sysstat 服务运行的知识。下面是 sysstat 工具包中的命令行如何使用收集到的信息,来了解系统的运行情况。

2. sysstat 工具包中的命令

sysstat工具包中有很多的分析命令,常用的有:sar, iostat, mpstat(multi processor stat), pidstat, vmstat等等。

其中的sar命令尤其强大,它一个命令就能够收集系统CPU, 内存,磁盘,网络,进程切换,swap, 甚至文件inode,等个方面的信息。sar 的应用比较多,而且也比较复杂,数据更为精确。

2.1 sar 命令

1)sar -u 查看CPU信息:

sar -u, sar -u ALL, sar -u 1 5, sar -u ALL 1 5

[root@localhost ~]# sar -u 1 2
Linux 2.6.32-504.el6.i686 (localhost.localdomain)       10/20/2015      _i686_  (1 CPU)

11:02:27 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
11:02:28 AM     all      0.99      0.00      2.97      0.00      0.00     96.04
11:02:29 AM     all      0.00      0.00      3.03      0.00      0.00     96.97
Average:        all      0.50      0.00      3.00      0.00      0.00     96.50

2)sar -r, sar -R 查看内存信息:

sar -r, sar -R, sar -r 1 5, sar -R 1 5

[root@localhost ~]# sar -r  1 2
Linux 2.6.32-504.el6.i686 (localhost.localdomain)       10/20/2015      _i686_  (1 CPU)

11:05:31 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
11:05:32 AM    824392    206156     20.00     71556     74584     82316      3.96
11:05:33 AM    824392    206156     20.00     71556     74584     82316      3.96
Average:       824392    206156     20.00     71556     74584     82316      3.96
[root@localhost ~]# sar -R  1 2
Linux 2.6.32-504.el6.i686 (localhost.localdomain)       10/20/2015      _i686_  (1 CPU)

11:05:56 AM   frmpg/s   bufpg/s   campg/s
11:05:57 AM    -11.11      0.00      0.00
11:05:58 AM      0.00      0.00      0.00
Average:        -5.56      0.00      0.00

3)sar -S, sar -W 查看 swap:

sar -S, sar -S 1 5, sar -W, sar -W 1 5. sar -S 查看的是swap空间的使用情况,sar -W 是每秒发生的swap/page情况

[root@localhost ~]# sar -S Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 09:43:28 AM LINUX RESTART 09:50:02 AM kbswpfree kbswpused %swpused kbswpcad %swpcad 10:00:01 AM 1048572 0 0.00 0 0.00 10:10:02 AM 1048572 0 0.00 0 0.00 10:20:01 AM 1048572 0 0.00 0 0.00 10:30:01 AM 1048572 0 0.00 0 0.00 10:40:02 AM 1048572 0 0.00 0 0.00 10:50:01 AM 1048572 0 0.00 0 0.00 11:00:01 AM 1048572 0 0.00 0 0.00 Average: 1048572 0 0.00 0 0.00 [root@localhost ~]# sar -S 1 5 Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 11:07:29 AM kbswpfree kbswpused %swpused kbswpcad %swpcad 11:07:30 AM 1048572 0 0.00 0 0.00 11:07:31 AM 1048572 0 0.00 0 0.00 11:07:32 AM 1048572 0 0.00 0 0.00 11:07:33 AM 1048572 0 0.00 0 0.00 11:07:34 AM 1048572 0 0.00 0 0.00 Average: 1048572 0 0.00 0 0.00 [root@localhost ~]# sar -W Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 09:43:28 AM LINUX RESTART 09:50:02 AM pswpin/s pswpout/s 10:00:01 AM 0.00 0.00 10:10:02 AM 0.00 0.00 10:20:01 AM 0.00 0.00 10:30:01 AM 0.00 0.00 10:40:02 AM 0.00 0.00 10:50:01 AM 0.00 0.00 11:00:01 AM 0.00 0.00 Average: 0.00 0.00 [root@localhost ~]# sar -W 1 5 Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 11:09:51 AM pswpin/s pswpout/s 11:09:52 AM 0.00 0.00 11:09:53 AM 0.00 0.00 11:09:54 AM 0.00 0.00 11:09:55 AM 0.00 0.00 11:09:56 AM 0.00 0.00 Average: 0.00 0.00 View Code

4)sar -b, sar -B 查看磁盘IO:

sar -b, sar -b 1 5, sar -B, sar -B 1 5. sar -b 查看磁盘tps(iops);sar -B 主要查看每秒磁盘 page/ fault

[root@localhost ~]# sar -b Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 09:43:28 AM LINUX RESTART 09:50:02 AM tps rtps wtps bread/s bwrtn/s 10:00:01 AM 0.01 0.00 0.01 0.00 0.11 10:10:02 AM 0.06 0.03 0.03 1.41 0.43 10:20:01 AM 0.01 0.00 0.01 0.00 0.11 10:30:01 AM 0.01 0.00 0.01 0.00 0.17 10:40:02 AM 0.16 0.10 0.06 4.84 0.89 10:50:01 AM 18.11 17.81 0.31 334.52 26.32 11:00:01 AM 0.11 0.09 0.02 5.12 0.34 11:10:01 AM 0.02 0.00 0.02 0.00 0.17 Average: 2.31 2.25 0.06 43.13 3.56 [root@localhost ~]# sar -b 1 5 Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 11:12:58 AM tps rtps wtps bread/s bwrtn/s 11:12:59 AM 0.00 0.00 0.00 0.00 0.00 11:13:00 AM 0.00 0.00 0.00 0.00 0.00 11:13:01 AM 0.00 0.00 0.00 0.00 0.00 11:13:02 AM 0.00 0.00 0.00 0.00 0.00 11:13:03 AM 0.00 0.00 0.00 0.00 0.00 Average: 0.00 0.00 0.00 0.00 0.00 [root@localhost ~]# sar -B Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 09:43:28 AM LINUX RESTART 09:50:02 AM pgpgin/s pgpgout/s fault/s majflt/s pgfree/s pgscank/s pgscand/s pgsteal/s %vmeff 10:00:01 AM 0.00 0.05 3.29 0.00 0.86 0.00 0.00 0.00 0.00 10:10:02 AM 0.71 0.22 12.02 0.00 3.16 0.00 0.00 0.00 0.00 10:20:01 AM 0.00 0.05 3.43 0.00 0.85 0.00 0.00 0.00 0.00 10:30:01 AM 0.00 0.09 3.28 0.00 0.83 0.00 0.00 0.00 0.00 10:40:02 AM 2.42 0.44 59.48 0.02 14.86 0.00 0.00 0.00 0.00 10:50:01 AM 167.26 13.16 1580.24 0.09 336.45 0.00 0.00 0.00 0.00 11:00:01 AM 2.56 0.17 15.00 0.02 3.95 0.00 0.00 0.00 0.00 11:10:01 AM 0.00 0.09 25.18 0.00 9.43 0.00 0.00 0.00 0.00 Average: 21.57 1.78 212.26 0.02 46.20 0.00 0.00 0.00 0.00 [root@localhost ~]# sar -B 1 5 Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 11:13:09 AM pgpgin/s pgpgout/s fault/s majflt/s pgfree/s pgscank/s pgscand/s pgsteal/s %vmeff 11:13:10 AM 0.00 0.00 45.45 0.00 59.60 0.00 0.00 0.00 0.00 11:13:11 AM 0.00 0.00 38.38 0.00 59.60 0.00 0.00 0.00 0.00 11:13:12 AM 0.00 0.00 30.30 0.00 59.60 0.00 0.00 0.00 0.00 11:13:13 AM 0.00 0.00 30.30 0.00 59.60 0.00 0.00 0.00 0.00 11:13:14 AM 0.00 0.00 30.00 0.00 59.00 0.00 0.00 0.00 0.00 Average: 0.00 0.00 34.88 0.00 59.48 0.00 0.00 0.00 0.00 View Code

5)sar -d 查看磁盘IO活动:

[root@localhost ~]# sar -d Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 09:43:28 AM LINUX RESTART 09:50:02 AM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util 10:00:01 AM dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 10:00:01 AM dev8-0 0.01 0.00 0.11 12.80 0.00 2.00 1.60 0.00 10:10:02 AM dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 10:10:02 AM dev8-0 0.06 1.41 0.43 29.62 0.00 5.86 5.30 0.03 10:20:01 AM dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 10:20:01 AM dev8-0 0.01 0.00 0.11 12.80 0.00 2.80 2.60 0.00 10:30:01 AM dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 10:30:01 AM dev8-0 0.01 0.00 0.17 14.86 0.00 3.29 2.14 0.00 10:40:02 AM dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 10:40:02 AM dev8-0 0.16 4.84 0.89 35.13 0.00 7.26 6.59 0.11 10:50:01 AM dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 10:50:01 AM dev8-0 18.11 334.52 26.32 19.92 0.05 3.15 2.98 5.40 11:00:01 AM dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:00:01 AM dev8-0 0.11 5.12 0.34 50.62 0.00 20.66 13.55 0.15 11:10:01 AM dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:10:01 AM dev8-0 0.02 0.00 0.17 11.56 0.00 1.67 1.56 0.00 Average: dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: dev8-0 2.31 43.13 3.56 20.25 0.01 3.29 3.08 0.71 [root@localhost ~]# sar -d 1 3 Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 11:16:29 AM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util 11:16:30 AM dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:16:30 AM dev8-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:16:30 AM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util 11:16:31 AM dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:16:31 AM dev8-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:16:31 AM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util 11:16:32 AM dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:16:32 AM dev8-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util Average: dev11-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: dev8-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 View Code

6)sar -n { keyword [,...] | ALL } 查看网络:

Possible keywords are DEV, EDEV, NFS, NFSD, SOCK, IP, EIP, ICMP, EICMP, TCP,  ETCP,  UDP,  SOCK6,  IP6,  EIP6, ICMP6, EICMP6 and UDP6.

你想查看网络哪个方面的信息,就将 keyword 换成上面的那个选项。

sar -n DEV 查看网络设备上的数据包收发情况,sar -n EDEV 查看网络设备上发生的数据包收发错误的情况,sar -n DEV 1 5, sar -n EDEV 1 5.

[root@localhost ~]# sar -n DEV 1 2 Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 11:20:05 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s 11:20:06 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:20:06 AM eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:20:06 AM eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:20:06 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s 11:20:07 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:20:07 AM eth0 2.02 3.03 0.12 0.70 0.00 0.00 0.00 11:20:07 AM eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth0 1.01 1.52 0.06 0.35 0.00 0.00 0.00 Average: eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 [root@localhost ~]# sar -n EDEV 1 2 Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 11:20:16 AM IFACE rxerr/s txerr/s coll/s rxdrop/s txdrop/s txcarr/s rxfram/s rxfifo/s txfifo/s 11:20:17 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:20:17 AM eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:20:17 AM eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:20:17 AM IFACE rxerr/s txerr/s coll/s rxdrop/s txdrop/s txcarr/s rxfram/s rxfifo/s txfifo/s 11:20:18 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:20:18 AM eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 11:20:18 AM eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: IFACE rxerr/s txerr/s coll/s rxdrop/s txdrop/s txcarr/s rxfram/s rxfifo/s txfifo/s Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 View Code

sar -n TCP, sar -n TCP 1 5 查看TCP;sar -n ETCP 查看tcp错误; sar -n SOCK, sar -n SOCK 1 5 查看socket;

7)sar -q 查看系统load负载:

[root@localhost ~]# sar -q
Linux 2.6.32-504.el6.i686 (localhost.localdomain)       10/20/2015      _i686_  (1 CPU)

09:43:28 AM       LINUX RESTART

09:50:02 AM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15
10:00:01 AM         0        66      0.00      0.02      0.09
10:10:02 AM         0        67      0.00      0.00      0.03
10:20:01 AM         0        67      0.00      0.00      0.00
10:30:01 AM         0        67      0.00      0.00      0.00
10:40:02 AM         0        78      0.00      0.00      0.00
10:50:01 AM         0        76      0.00      0.16      0.14
11:00:01 AM         0        80      0.00      0.02      0.06
11:10:01 AM         0        80      0.00      0.00      0.00
11:20:01 AM         0        81      0.00      0.00      0.00
Average:            0        74      0.00      0.02      0.04
[root@localhost ~]# sar -q 1 3
Linux 2.6.32-504.el6.i686 (localhost.localdomain)       10/20/2015      _i686_  (1 CPU)

11:25:44 AM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15
11:25:45 AM         0        80      0.00      0.00      0.00
11:25:46 AM         0        80      0.00      0.00      0.00
11:25:47 AM         0        80      0.00      0.00      0.00
Average:            0        80      0.00      0.00      0.00

runq-sz 表示在CPU上排队等待执行的任务;plist-sz系统所有任务。

8)sar -w 每秒进程创建 和 进程切换:

[root@localhost ~]# sar -w Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 09:43:28 AM LINUX RESTART 09:50:02 AM proc/s cswch/s 10:00:01 AM 0.01 25.14 10:10:02 AM 0.04 25.58 10:20:01 AM 0.01 25.01 10:30:01 AM 0.01 25.40 10:40:02 AM 0.19 27.64 10:50:01 AM 4.02 120.10 11:00:01 AM 0.04 41.77 11:10:01 AM 0.09 38.55 11:20:01 AM 0.23 50.63 Average: 0.51 42.18 [root@localhost ~]# sar -w 1 4 Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 11:28:07 AM proc/s cswch/s 11:28:08 AM 0.00 80.39 11:28:09 AM 0.00 90.82 11:28:10 AM 0.00 45.45 11:28:11 AM 0.00 60.00 Average: 0.00 69.17 View Code

9)sar -W 每秒swap/page:

[root@localhost ~]# sar -W Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 09:43:28 AM LINUX RESTART 09:50:02 AM pswpin/s pswpout/s 10:00:01 AM 0.00 0.00 10:10:02 AM 0.00 0.00 10:20:01 AM 0.00 0.00 10:30:01 AM 0.00 0.00 10:40:02 AM 0.00 0.00 10:50:01 AM 0.00 0.00 11:00:01 AM 0.00 0.00 11:10:01 AM 0.00 0.00 11:20:01 AM 0.00 0.00 Average: 0.00 0.00 [root@localhost ~]# sar -W 1 4 Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 11:28:50 AM pswpin/s pswpout/s 11:28:51 AM 0.00 0.00 11:28:52 AM 0.00 0.00 11:28:53 AM 0.00 0.00 11:28:54 AM 0.00 0.00 Average: 0.00 0.00 View Code

10)sar -v 查看文件inode使用,缓存情况:

[root@localhost ~]# sar -v Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 09:43:28 AM LINUX RESTART 09:50:02 AM dentunusd file-nr inode-nr pty-nr 10:00:01 AM 1963 160 7250 0 10:10:02 AM 1988 160 7266 0 10:20:01 AM 1988 160 7266 0 10:30:01 AM 1988 160 7266 0 10:40:02 AM 2126 480 7418 5 10:50:01 AM 40677 448 43933 5 11:00:01 AM 40772 480 43952 5 11:10:01 AM 40894 480 43952 5 11:20:01 AM 41281 480 43953 5 Average: 19297 334 23584 3 [root@localhost ~]# sar -v 1 3 Linux 2.6.32-504.el6.i686 (localhost.localdomain) 10/20/2015 _i686_ (1 CPU) 11:29:49 AM dentunusd file-nr inode-nr pty-nr 11:29:50 AM 41488 480 43945 5 11:29:51 AM 41488 480 43945 5 11:29:52 AM 41488 480 43945 5 Average: 41488 480 43945 5 View Code

2.2 iostat 命令

注意查看IO和CPU负载情况

参见:http://www.cnblogs.com/digdeep/p/4863502.html 和 man 手册

2.3 mpstat 命令

注意查看多核CPU情况

参见:http://www.cnblogs.com/digdeep/p/4847484.html 和 man 手册

2.4 pidstat命令

注意查看进程情况

参见:http://www.cnblogs.com/digdeep/p/4847484.html 和 man 手册

 

相关内容