Shell应用:统计指定格式超过n秒的数据
Shell应用:统计指定格式超过n秒的数据
该脚本的应用场景是数据分析,一般作线上日志分析的人员需要观察线上数据的变化,如:
1.访问数据库的响应大于1000毫秒的sql
2.apache cookielog响应大于多少毫秒的数据
数据格式:
- 2010-12-14 00:01:26,427 FATAL FUNCTION_TIME - wmmad.alloffer.get 15511ms
- 2010-12-14 00:01:33,164 FATAL FUNCTION_TIME - wmmad.alloffer.get 14213ms
- 2010-12-14 00:02:31,021 FATAL FUNCTION_TIME - wmmad.alloffer.get 14126ms
- 2010-12-14 00:05:08,160 FATAL FUNCTION_TIME - wmmad.alloffer.get 15295ms
- 2010-12-14 00:24:00,372 ERROR FUNCTION_TIME - wmmad.offer.repost 406ms
脚本:
[plain]- #/bin/bash
- # author: madding.lip
- # date 2010.12.14
- # 统计超n毫秒的数据量
- ERROR_USAGE=1
- if [ $# != 2 ]; then
- echo "Usage: $0 file times";
- exit $ERROR_USAGE
- fi
- myfile=$1
- mytime=$2
- all=0
- count=0;
- data=`cat $myfile | awk '{print $7}' | sed 's/ms//g'`;
- for i in $data ;
- do
- all=$(( $all + 1 ));
- if test $(( $mytime < $i )) -eq 1 ;then
- # echo $i; sleep 1;
- count=$(( $count + 1 ));
- fi
- done;
- echo "response time over ${mytime}ms: "$count" times"
- echo "all count: "$all" times"
具体根据数据格式作调整即可。
评论暂时关闭