什么是Netperf
- Netperf是由惠普公司开发的一种网络性能测量工具，主要针对基于TCP或UDP的传输。
- Netperf根据应用的不同，可以进行不同模式的网络性能测试，即批量数据传输（bulk data transfer）模式和请求/应答（request/reponse）模式。
- Netperf测试结果所反映的是一个系统能够以多快的速度向另外一个系统发送数据，以及另
外一个系统能够以多块的速度接收数据 - https://github.com/HewlettPackard/netperf
- https://manpages.org/netperf
- netserver(1) - Linux man page
基础用法
-
两台机器，一台服务端 运行 netserver，另一端 客户端，运行 netperf
-
Netperf 缺省情况下进行 TCP 批量传输，即 -t TCP_STREAM
-
关闭防火墙
[root@localhost ~]# netserver
Starting netserver with host 'IN(6)ADDR_ANY' port '12865' and family AF_UNSPEC[root@localhost netperf]# netperf -H 127.0.0.1 -l 1
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec131072 16384 16384 1.00 24765.25
不指定端口的话 12865 为 netserver 默认端口
显示结果
- Protocol : 传输协议类型 TCP、UDP
- Elapsed : 测试的时间
- Throughput ： 吞吐量
- Throughput Units ： 吞吐量的单位
- Minimum/Maximum/Mean/50th/90th/99th Percentile Latency : 各种时延信息
- 其他更多结果参考 9.3.1 Omni Output Selectors
详细使用方法
命令格式：
netserver [-4] [-6] [-d] [-h] [-L name,family] [-p portnum] [-v verbosity] [-V]
netperf <global> -- <test-specific>
global 是全局设置选项，test-specific 是针对于这个测试的特定配置
一些规律：很多小写的命令表示发送/本地，大写的命令表示远程/接收
help
[root@localhost netperf]# netserver -h
Usage: netserver [options]
Options:
-h Display this text
-D Do not daemonize
-d Increase debugging output
-f Do not spawn chilren for each test, run serially
-L name,family Use name to pick listen address and family for family
-N No debugging output, even if netperf asks
-p portnum Listen for connect requests on portnum.
-4 Do IPv4
-6 Do IPv6
-v verbosity Specify the verbosity level
-V Display version information and exit
-Z passphrase Expect passphrase as the first thing received
[root@localhost ~]# netperf -h
Usage: netperf [global options] -- [test options]
Global options:
-a send,recv Set the local send,recv buffer alignment
-A send,recv Set the remote send,recv buffer alignment
-B brandstr Specify a string to be emitted with brief output
-c [cpu_rate] Report local CPU usage
-C [cpu_rate] Report remote CPU usage
-d Increase debugging output
-D time,[units] * Display interim results at least every time interval
using units as the initial guess for units per second
A negative value for time will make heavy use of the
system's timestamping functionality
-f G|M|K|g|m|k Set the output units
-F lfill[,rfill]* Pre-fill buffers with data from specified file
-h Display this text
-H name|ip,fam * Specify the target machine and/or local ip and family
-i max,min Specify the max and min number of iterations (15,1)
-I lvl[,intvl] Specify confidence level (95 or 99) (99)
and confidence interval in percentage (10)
-j Keep additional timing statistics
-l testlen Specify test duration (>0 secs) (<0 bytes|trans)
-L name|ip,fam * Specify the local ip|name and address family
-o send,recv Set the local send,recv buffer offsets
-O send,recv Set the remote send,recv buffer offset
-n numcpu Set the number of processors for CPU util
-N Establish no control connection, do 'send' side only
-p port,lport* Specify netserver port number and/or local port
-P 0|1 Don't/Do display test headers
-r Allow confidence to be hit on result only
-s seconds Wait seconds between test setup and test start
-S Set SO_KEEPALIVE on the data connection
-t testname Specify test to perform
-T lcpu,rcpu Request netperf/netserver be bound to local/remote cpu
-v verbosity Specify the verbosity level
-W send,recv Set the number of send,recv buffers
-v level Set the verbosity level (default 1, min 0)
-V Display the netperf version and exit
-y local,remote Set the socket priority
-Y local,remote Set the IP_TOS. Use hexadecimal.
-Z passphrase Set and pass to netserver a passphrase
[root@localhost netperf]# netperf -t TCP_RR -- -h
Usage: netperf [global options] -- [test options]
OMNI and Migrated BSD Sockets Test Options:
-b number Send number requests at start of _RR tests
-c Explicitly declare this a connection test such as
TCP_CRR or TCP_CC
-C Set TCP_CORK when available
-d direction Explicitly set test direction based on bitwise OR
of 0x2 for transmit and 0x4 for receive. Default:
based on test type
-D [L][,R] Set TCP_NODELAY locally and/or remotely (TCP_*)
-h Display this text
-H name[/mask],fam Use name (or IP) and family as target of data connection
A mask value will cause randomization of the IP used
-k [file] Generate keyval output optionally based on file
Use filename of '?' to get the list of choices
-K loc[,rem] Set the local and/or remote congestion control
algorithm to use on those platforms where it can
be set.
-L name[/mask],fam Use name (or IP) and family as source of data connection
A mask value will cause randomization of the IP used
-m local,remote Set the send size for _STREAM/_MAERTS tests
-M local,remote Set the recv size for _STREAM/_MAERTS tests
-n Use the connected socket for UDP locally
-N Use the connected socket for UDP remotely
-o [file] Generate CSV output optionally based on file
Use filename of '?' to get the list of choices
-O [file] Generate classic-style output based on file
Use filename of '?' to get the list of choices
-p min[,max] Set the min/max port numbers for TCP_CRR, TCP_TRR
-P local[,remote] Set the local/remote port for the data socket
-r req,[rsp] Set request/response sizes (TCP_RR, UDP_RR)
-R 0/1 Allow routing of traffic on data connection.
Default: 0 (off) for UDP_STREAM, 1 (on) otherwise
-s send[,recv] Set local socket send/recv buffer sizes
-S send[,recv] Set remote socket send/recv buffer sizes
-t type Explicitly set socket type. Default is implicit
based on other settings
-T protocol Explicitly set data connection protocol. Default is
implicit based on other settings
-u uuid Use the supplied string as the UUID for this test.
-4 Use AF_INET (eg IPv4) on both ends of the data conn
-6 Use AF_INET6 (eg IPv6) on both ends of the data conn
For those options taking two parms, at least one must be specified;
specifying one value without a comma will set both parms to that
value, specifying a value with a leading comma will set just the second
parm, a value with a trailing comma will set just the first. To set
each parm to unique values, specify both and separate them with a
comma.
global options
H
指定server的ip
-t testname
- TCP_STREAM, TCP_MAERTS, TCP_SENDFILE, TCP_RR, TCP_CRR, TCP_CC
- UDP_STREAM, UDP_RR
- XTI_TCP_STREAM, XTI_TCP_RR, XTI_TCP_CRR, XTI_TCP_CC
- XTI_UDP_STREAM, XTI_UDP_RR
- SCTP_STREAM, SCTP_RR
- DLCO_STREAM, DLCO_RR, DLCL_STREAM, DLCL_RR
- LOC_CPU, REM_CPU
- OMNI
可分为两类：
一类是单项的传输性能测试， STEAM 后缀的，一类是 往返测试 RR后缀的
常用的：
UDP_STREAM
UDP_RR
-s
在建立链接和传输数据之间sleep一段时间，用于多次运行netperf，不希望之前的netperf对后面的netperf 有影响的场景。
-p
-p remote_port,local_port
注意，指定的端口，不是实际发包用的端口，而是客户端服务端传输基础配置等信息的端口,netstat 时该端口是 ESTABLISHED 状态
The first value of the optionspec passed-in with this option tells netperf the port number at which it should expect the remote netserver to be listening for control connections. The second value of the optionspec will request netperf to bind to that local port number before establishing the control connection
-P
-P 0 # 关闭横幅，默认1
cC
-c # local cpu utilization
-C # remote cpu utilization
当"oOk"中包含cpu参数 时，应该加此命令，否则cpu利用率显示为0.
-T
This option controls the CPU, and probably by extension memory, affinity of netperf and/or netserver.
设置服务端和客户端的cpu亲和性
test options
oOk
-o : 横向显示结果，逗号分隔
-O ： 横向显示结果
-k ： 纵向显示结果
r && mM
r主要用于RR场景
-r request size,response size
request size : Remote Recv Size / Request Size Bytes
response size : Remote Send Size / Response Size Bytes
mM主要用于 STREAM场景
-m 32768/32k # 传给send的buf大小
-M 32768/32k # 传给 recv 的大小
k/m/g/K/M/G : 10^3/10^6/10^9/2^10/2^20/2^30 # 单位
m : Local Send Size
M : Remote Recv Size
RR测试是本地发射和远端的接收 或 本地接收和远端发射相等吗？为什么只有两个参数，如果相等，为什么在RR实测中 Local Send Size,Local Recv Size 的值和r配置无关？
## 这4个size用于 non-request/response，也就是STREAM
LOCAL_SEND_SIZE
This will display the size of the buffers netperf passed in any "send" calls it made on the data connection for a non-request/response test. Units: Bytes.
LOCAL_RECV_SIZE
This will display the size of the buffers netperf passed in any "receive" calls it made on the data connection for a non-request/response test. Units: Bytes.
REMOTE_SEND_SIZE
This will display the size of the buffers netserver passed in any "send" calls it made on the data connection for a non-request/response test. Units: Bytes.
REMOTE_RECV_SIZE
This will display the size of the buffers netserver passed in any "receive" calls it made on the data connection for a non-request/response test. Units: Bytes.
## 这2个size用于 request/response，也就是RR
REQUEST_SIZE
This will display the size of the requests netperf sent in a request-response test. Units: Bytes.
RESPONSE_SIZE
This will display the size of the responses netserver sent in a request-response test. Units: Bytes.
L
指定本地IP地址测试
-P
--- -P remote_port,local_port
#Set the local and/or remote port numbers for the data connection.
我理解-p是用于控制连接的端口，而-P是真正数据传输的测试的端口
eg：
-p 12866,11111 -- -P 12869,11112
[root@localhost netperf]# netstat -antup | grep net
Active Internet connections (servers and established)
tcp 0 0 127.0.0.1:11111 127.0.0.1:12866 ESTABLISHED 2905792/netperf
tcp6 0 0 :::12865 :::* LISTEN 497555/netserver
tcp6 0 0 :::12866 :::* LISTEN 2532430/netserver
tcp6 0 0 127.0.0.1:12866 127.0.0.1:11111 ESTABLISHED 2905794/netserver
udp 0 0 0.0.0.0:12869 0.0.0.0:* 2905792/netperf
udp 4352 0 0.0.0.0:11112 0.0.0.0:* 2905794/netserver
sS
-s : This option sets the local (netperf) send and receive socket buffer sizes for the data connection to the value(s) specified.
-S :This option sets the remote (netserver) send and/or receive socket buffer sizes for the data connection to the value(s) specified.
常用辅助命令
shell
netstat -lnpt
kill -9 pid of netperf
fillwall-cmd --state # 防火墙状态
systemctl stop fillwalld.service # 关闭防火墙其他
https://github.com/HewlettPackard/netperf/blob/master/doc/netperf.html 基本讲了所有使用方法，还有一些测试例子https://github.com/HewlettPackard/netperf/blob/master/doc/examples/tcp_rr_script
文档中描述的的一些英文词汇理解：
control connections： 指用于一些控制信息传输的连接
data connection: 指用于测试数据传输的那部分连接