在服务器存储的测试中,经常需要看performance的性能曲线,这样最能直接观察HDD或者SSD的性能曲线。
如下这是一个针对HDD跑Fio读写的iostat监控log,下面介绍一下分别用shell 和Python3 写画iostat图的方法
1 shell脚本
环境:linux OS gnuplot工具
第一步 :解析iostat log。提取要作图的一行
bash
#!/bin/bash
hdd=$1
cat ${hdd}_iostat.log |grep -i Device |head -1 >${hdd}_iostat.txt
cat ${hdd}_iostat.log |grep -i ${hdd} >>${hdd}_iostat.txt
第二步:把画图的列(监控读写的IOPS)求出来
bash
#!/bin/bash
hdd=$1
cat ${hdd}_iostat.log |grep -i Device |head -1 >${hdd}_iostat.txt
cat ${hdd}_iostat.log |grep -i ${hdd} >>${hdd}_iostat.txt
num_read_iops=`head -1 ${hdd}_iostat.txt | awk '{for (i=1;i<=NF;i++) {if ($i=="r/s") {print i}}}'`
num_write_iops=`head -1 ${hdd}_iostat.txt | awk '{for (i=1;i<=NF;i++) {if ($i=="w/s") {print i}}}'`
第三步:直接调用gnuplot工具作图
bash
#!/bin/bash
hdd=$1
cat ${hdd}_iostat.log |grep -i Device |head -1 >${hdd}_iostat.txt
cat ${hdd}_iostat.log |grep -i ${hdd} >>${hdd}_iostat.txt
num_read_iops=`head -1 ${hdd}_iostat.txt | awk '{for (i=1;i<=NF;i++) {if ($i=="r/s") {print i}}}'`
num_write_iops=`head -1 ${hdd}_iostat.txt | awk '{for (i=1;i<=NF;i++) {if ($i=="w/s") {print i}}}'`
echo "
set terminal png
set title 'Random_${hdd}_IOPS'
set output 'IOPS_${hdd}.png'
set xlabel 'count:1s'
set ylabel 'IOPS'
set key right top
plot '${hdd}_iostat.txt' using :$num_write_iops title 'w/s' with lines lw 1,\
'${hdd}_iostat.txt' using :$num_read_iops title 'r/s' with lines lw 1
" | gnuplot
运行 :bash gnuplot.sh sde
2 python 脚本
环境:python3
python库: sys(引用参数), re(正则匹配), pandas(做表格), matplotlib(画图工具)
第一步:解析iostat log
python
import sys
import re
import pandas as pd
from matplotlib import pyplot as plt
hdd = sys.argv[1]
rs = []
ws = []
logfile = open(f"{hdd}_iostat.log","r")
for log in logfile:
if hdd in log:
data = re.sub(' +', ' ', log)
rs.append(data.split(" ")[3])
ws.append(data.split(" ")[4])
第二步:将需要作图的列(这次监控带宽)导入CSV
python
import sys
import re
import pandas as pd
from matplotlib import pyplot as plt
hdd = sys.argv[1]
rs = []
ws = []
logfile = open(f"{hdd}_iostat.log","r")
for log in logfile:
if hdd in log:
data = re.sub(' +', ' ', log)
rs.append(data.split(" ")[3])
ws.append(data.split(" ")[4])
dic = {"read": rs, "write": ws}
df = pd.DataFrame(dic)
df.to_csv(f"{hdd}_iostat.csv")
data = pd.read_csv(f"{hdd}_iostat.csv")
第三步:用matplotlib库直接作图
python
import sys
import re
import pandas as pd
from matplotlib import pyplot as plt
hdd = sys.argv[1]
rs = []
ws = []
logfile = open(f"{hdd}_iostat.log","r")
for log in logfile:
if hdd in log:
data = re.sub(' +', ' ', log)
rs.append(data.split(" ")[3])
ws.append(data.split(" ")[4])
dic = {"read": rs, "write": ws}
df = pd.DataFrame(dic)
df.to_csv(f"{hdd}_iostat.csv")
data = pd.read_csv(f"{hdd}_iostat.csv")
fig = plt.figure(figsize=(10, 6), dpi=300)
x = data.iloc[:, 0]
y1 = data.iloc[:, 1]
y2 = data.iloc[:, 2]
plt.xlabel(u'time (s)', size=10)
plt.ylabel("BW(kB/s)")
plt.plot(x, y1, label='rKB/s', color='orange', linestyle=':')
plt.plot(x, y2, label='wKB/s', color='cyan', linestyle='-.')
plt.legend(loc="upper right")
plt.title(f"{hdd}_BW")
plt.grid(alpha=0.4)
plt.savefig(str(hdd) + ".png")
运行 :python3 plot.py sde