Ardupilot Rpanion iperf网络性能测试

[1. 源由](#1. 源由)
[2. 分析](#2. 分析)
[3. 安装](#3. 安装)
[4. 测试](#4. 测试)
- [4.1 第一次测试](#4.1 第一次测试)
- - [4.1.1 iperf测试参数A](#4.1.1 iperf测试参数A)
  - - [4.1.1.1 测试链路1](#4.1.1.1 测试链路1)
    - [4.1.1.2 测试链路2](#4.1.1.2 测试链路2)
    - [4.1.1.3 测试链路3](#4.1.1.3 测试链路3)
  - [4.1.2 iperf测试参数B - 测试链路3](#4.1.2 iperf测试参数B - 测试链路3)
  - - [4.1.2.1 测试数据](#4.1.2.1 测试数据)
    - [4.1.2.2 数据简单分析](#4.1.2.2 数据简单分析)
    - [4.1.2.3 数据深入分析](#4.1.2.3 数据深入分析)
    - [4.1.2.4 模拟测试网络](#4.1.2.4 模拟测试网络)
- [4.2 第二次测试](#4.2 第二次测试)
- - [4.2.1 测试数据](#4.2.1 测试数据)
  - [4.2.2 数据分析](#4.2.2 数据分析)
  - [4.2.3 网络延迟分析](#4.2.3 网络延迟分析)
  - [4.2.4 视频效果](#4.2.4 视频效果)
- [4.3 第三次测试](#4.3 第三次测试)
- - [4.3.1 测试数据](#4.3.1 测试数据)
  - [4.3.2 数据分析](#4.3.2 数据分析)
  - [4.3.3 网络延迟分析](#4.3.3 网络延迟分析)
  - [4.3.4 视频效果](#4.3.4 视频效果)
[5. 总结](#5. 总结)
[6. 参考资料](#6. 参考资料)
[7. ZeroTier优化办法 - Moon节点](#7. ZeroTier优化办法 - Moon节点)
- [7.1 服务端配置](#7.1 服务端配置)
- [7.2 客户端配置](#7.2 客户端配置)
[8. DJI RC系统性能](#8. DJI RC系统性能)

1. 源由

网络性能测试主要目的是：

评估网络健康状况：网络性能测试可以帮助评估网络的健康状况，包括带宽、延迟、丢包率等指标。通过了解网络的实际性能，可以及时发现并解决网络瓶颈和问题，确保网络的正常运行。
优化网络性能：通过网络性能测试，可以确定网络中存在的瓶颈和性能问题，并采取相应的优化措施。这包括调整网络设备配置、优化网络拓扑、升级网络硬件等，以提高网络的吞吐量、稳定性和可靠性。
规划网络扩展和升级：网络性能测试可以帮助企业规划网络的扩展和升级，以满足不断增长的业务需求。通过了解当前网络的性能状况和未来的需求，可以更好地规划网络架构、设备采购和部署策略。
保障用户体验：良好的网络性能是保障用户体验的关键。网络性能测试可以帮助确保网络对用户提供良好的服务质量，包括快速的响应时间、稳定的连接和高质量的数据传输。
提高网络安全：网络性能测试还可以帮助发现和解决网络安全漏洞和风险。通过测试网络的性能和安全性，可以识别潜在的安全威胁，并采取相应的防护措施，提高网络的安全性和可靠性。

最近要做的是针对视频、电传链路的评估，其在VPN/4G-LTE下性能到底如何？

2. 分析

在进行网络性能测试时，其主要步骤如下：

清晰的测试目标：确定您要测试的网络性能指标，例如带宽、延迟、丢包率等。确保您的测试目标清晰明确，并且测试过程中能够量化和记录这些指标。
选择适当的测试工具：根据您的测试需求选择合适的工具。一些常用的网络性能测试工具包括iperf、Wireshark、PingPlotter等。确保您选择的工具能够提供您需要的功能和数据输出。
测试环境的准备：在进行测试之前，确保测试环境的稳定性和一致性。避免在网络负载较高的时段进行测试，以免影响测试结果。另外，确保测试设备和网络连接的稳定性，以避免测试过程中出现干扰或故障。
数据采集和分析：在测试过程中，及时记录测试数据，并进行分析和比较。这些数据可以帮助您评估网络性能，并识别潜在的问题和瓶颈。确保您使用的工具能够提供详细的测试结果和分析功能。
安全性和合规性：在进行网络性能测试时，确保您的测试活动不会对网络安全造成威胁，并遵守相关的法律法规和政策要求。在测试之前，与相关部门或团队沟通，确保您的测试活动符合组织的安全和合规标准。
测试结果的验证和确认：在完成测试之后，验证和确认测试结果的准确性和可靠性。如果可能，进行多次测试以确保结果的一致性，并与预期的性能指标进行比较。

综上所述，网络性能测试需要仔细的规划和准备工作，以确保测试结果的准确性和可信度。通过遵循上述注意事项，您可以更好地评估和优化网络性能，提高网络的稳定性和效率。

3. 安装

Jetson Orin Nano

$ sudo apt-get install iperf
$ iperf --version
iperf version 2.1.5 (3 December 2021) pthreads
Raspberrry Pi 3B+

$ sudo apt-get install iperf
$ iperf --version
iperf version 2.0.14a (2 October 2020) pthreads

$ iperf --version
iperf version 2.1.5 (3 December 2021) pthreads
daniel@daniel-nvidia:~$ iperf -h
Usage: iperf [-s|-c host] [options]
iperf [-h|--help] [-v|--version]

Client/Server:
-b, --bandwidth #[kmgKMG | pps] bandwidth to read/send at in bits/sec or packets/sec
-e, --enhanced use enhanced reporting giving more tcp/udp and traffic information
-f, --format [kmgKMG] format to report: Kbits, Mbits, KBytes, MBytes
--hide-ips hide ip addresses and host names within outputs
-i, --interval # seconds between periodic bandwidth reports
-l, --len #[kmKM] length of buffer in bytes to read or write (Defaults: TCP=128K, v4 UDP=1470, v6 UDP=1450)
-m, --print_mss print TCP maximum segment size (MTU - TCP/IP header)
-o, --output <filename> output the report or error message to this specified file
-p, --port # client/server port to listen/send on and to connect
--permit-key permit key to be used to verify client and server (TCP only)
--sum-only output sum only reports
-u, --udp use UDP rather than TCP
-w, --window #[KM] TCP window size (socket buffer size)
-z, --realtime request realtime scheduler
-B, --bind <host>[:<port>][%<dev>] bind to <host>, ip addr (including multicast address) and optional port and device
-C, --compatibility for use with older versions does not sent extra msgs
-M, --mss # set TCP maximum segment size (MTU - 40 bytes)
-N, --nodelay set TCP no delay, disabling Nagle's Algorithm
-S, --tos # set the socket's IP_TOS (byte) field
-Z, --tcp-congestion <algo> set TCP congestion control algorithm (Linux only)

Server specific:
-p, --port #[-#] server port(s) to listen on/connect to
-s, --server run in server mode
-1, --singleclient run one server at a time
--histograms enable latency histograms
--permit-key-timeout set the timeout for a permit key in seconds
--tcp-rx-window-clamp set the TCP receive window clamp size in bytes
--tap-dev #[<dev>] use TAP device to receive at L2 layer
-t, --time # time in seconds to listen for new connections as well as to receive traffic (default not set)
--udp-histogram #,# enable UDP latency histogram(s) with bin width and count, e.g. 1,1000=1(ms),1000(bins)
-B, --bind <ip>[%<dev>] bind to multicast address and optional device
-U, --single_udp run in single threaded UDP mode
--sum-dstip sum traffic threads based upon destination ip address (default is src ip)
-D, --daemon run the server as a daemon
-V, --ipv6_domain Enable IPv6 reception by setting the domain and socket to AF_INET6 (Can receive on both IPv4 and IPv6)

Client specific:
-c, --client <host> run in client mode, connecting to <host>
--connect-only run a connect only test
--connect-retries # number of times to retry tcp connect
-d, --dualtest Do a bidirectional test simultaneously (multiple sockets)
--fq-rate #[kmgKMG] bandwidth to socket pacing
--full-duplex run full duplex test using same socket
--ipg set the the interpacket gap (milliseconds) for packets within an isochronous frame
--isochronous <frames-per-second>:<mean>,<stddev> send traffic in bursts (frames - emulate video traffic)
--incr-dstip Increment the destination ip with parallel (-P) traffic threads
--incr-dstport Increment the destination port with parallel (-P) traffic threads
--incr-srcip Increment the source ip with parallel (-P) traffic threads
--incr-srcport Increment the source port with parallel (-P) traffic threads
--local-only Set don't route on socket
--near-congestion=[w] Use a weighted write delay per the sampled TCP RTT (experimental)
--no-connect-sync No sychronization after connect when -P or parallel traffic threads
--no-udp-fin No final server to client stats at end of UDP test
-n, --num #[kmgKMG] number of bytes to transmit (instead of -t)
-r, --tradeoff Do a fullduplexectional test individually
--tcp-write-prefetch set the socket's TCP_NOTSENT_LOWAT value in bytes and use event based writes
-t, --time # time in seconds to transmit for (default 10 secs)
--trip-times enable end to end measurements (requires client and server clock sync)
--txdelay-time time in seconds to hold back after connect and before first write
--txstart-time unix epoch time to schedule first write and start traffic
-B, --bind [<ip> | ip:port] bind ip (and optional port) from which to source traffic
-F, --fileinput <name> input the data to be transmitted from a file
-H, --ssm-host <ip> set the SSM source, use with -B for (S,G)
-I, --stdin input the data to be transmitted from stdin
-L, --listenport # port to receive fullduplexectional tests back on
-P, --parallel # number of parallel client threads to run
-R, --reverse reverse the test (client receives, server sends)
-S, --tos IP DSCP or tos settings
-T, --ttl # time-to-live, for multicast (default 1)
-V, --ipv6_domain Set the domain to IPv6 (send packets over IPv6)
-X, --peer-detect perform server version detection and version exchange

Miscellaneous:
-x, --reportexclude [CDMSV] exclude C(connection) D(data) M(multicast) S(settings) V(server) reports
-y, --reportstyle C report as a Comma-Separated Values
-h, --help print this message and quit
-v, --version print version information and quit

[kmgKMG] Indicates options that support a k,m,g,K,M or G suffix
Lowercase format characters are 10^3 based and uppercase are 2^n based
(e.g. 1k = 1000, 1K = 1024, 1m = 1,000,000 and 1M = 1,048,576)

The TCP window size option can be set by the environment variable
TCP_WINDOW_SIZE. Most other options can be set by an environment variable
IPERF_<long option name>, such as IPERF_BANDWIDTH.

Source at http://sourceforge.net/projects/iperf2/
Report bugs to iperf-users@lists.sourceforge.net

4. 测试

测试指标：带宽、延迟、丢包率、稳定性。其他的可靠性、安全性等暂不考虑。
测试工具：iperf

4.1 第一次测试

目前，本地已有的测试环境的【网络拓扑A】，如下所示：

4.1.1 iperf测试参数A

服务端：iperf -u -s
客户端：iperf -u -T 10 -b [1M] -c [server address]

4.1.1.1 测试链路1

局域网测试：Pi --> Hub --> Jetson
服务端IP地址：192.168.1.19 (Jetson)
客户端IP地址：192.168.1.201 (Pi)

$ iperf -u -T 10 -b 1M -c 192.168.1.19

Client connecting to 192.168.1.19, UDP port 5001
UDP buffer size: 208 KByte (default)

[ 3] local 192.168.1.201 port 49022 connected with 192.168.1.19 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0000-10.0153 sec 1.25 MBytes 1.05 Mbits/sec
[ 3] Sent 896 datagrams
[ 3] Server Report:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0000-10.0116 sec 1.25 MBytes 1.05 Mbits/sec 3.065 ms 0/ 895 (0%)

$ iperf -u -T 10 -b 2M -c 192.168.1.19

Client connecting to 192.168.1.19, UDP port 5001
UDP buffer size: 208 KByte (default)

[ 3] local 192.168.1.201 port 46959 connected with 192.168.1.19 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0000-10.0098 sec 2.50 MBytes 2.10 Mbits/sec
[ 3] Sent 1788 datagrams
[ 3] Server Report:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0000-9.9998 sec 2.50 MBytes 2.10 Mbits/sec 2.513 ms 0/ 1787 (0%)

$ iperf -u -T 10 -b 3M -c 192.168.1.19

Client connecting to 192.168.1.19, UDP port 5001
UDP buffer size: 208 KByte (default)

[ 3] local 192.168.1.201 port 44713 connected with 192.168.1.19 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0000-10.0079 sec 3.75 MBytes 3.15 Mbits/sec
[ 3] Sent 2680 datagrams
[ 3] Server Report:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0000-10.0206 sec 3.75 MBytes 3.14 Mbits/sec 2.352 ms 1/ 2679 (0.037%)
1M带宽测试：

带宽：1.05 Mbits/sec

丢包率：0%

Jitter：3.065 ms
2M带宽测试：

带宽：2.10 Mbits/sec

丢包率：0%

Jitter：2.513 ms
3M带宽测试：

带宽：3.14 Mbits/sec

丢包率：0.037%

Jitter：2.352 ms

4.1.1.2 测试链路2

预设条件：4G LTE SIM卡移除
VPN测试：Pi --> Hub --> Modem --> Server --> Modem --> Hub --> AP --> Jetson
服务端IP地址：172.24.42.46 (Jetson)
客户端IP地址：172.24.161.33 (Pi)

$ iperf -u -T 10 -b 1M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)

[ 3] local 172.24.161.33 port 43642 connected with 172.24.42.46 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0000-10.0153 sec 1.25 MBytes 1.05 Mbits/sec
[ 3] Sent 896 datagrams
[ 3] Server Report:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0000-9.9983 sec 1.25 MBytes 1.05 Mbits/sec 3.425 ms 0/ 895 (0%)

$ iperf -u -T 10 -b 2M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)

[ 3] local 172.24.161.33 port 43610 connected with 172.24.42.46 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0000-10.0098 sec 2.50 MBytes 2.10 Mbits/sec
[ 3] Sent 1788 datagrams
[ 3] Server Report:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0000-10.0097 sec 2.50 MBytes 2.10 Mbits/sec 3.465 ms 0/ 1787 (0%)

$ iperf -u -T 10 -b 3M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)

[ 3] local 172.24.161.33 port 37048 connected with 172.24.42.46 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0000-10.0041 sec 3.75 MBytes 3.15 Mbits/sec
[ 3] Sent 2679 datagrams
[ 3] Server Report:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0000-10.0336 sec 3.75 MBytes 3.14 Mbits/sec 4.807 ms 0/ 2678 (0%)
1M带宽测试：

带宽：1.05 Mbits/sec

丢包率：0%

Jitter：3.425 ms
2M带宽测试：

带宽：2.10 Mbits/sec

丢包率：0%

Jitter：3.465 ms
3M带宽测试：

带宽：3.14 Mbits/sec

丢包率：0%

Jitter：4.807 ms

4.1.1.3 测试链路3

预设条件：有线IP网络物理断开
VPN测试：Pi --> 4G LTE --> BTS --> Server --> Modem --> Hub --> AP --> Jetson
服务端IP地址：172.24.42.46 (Jetson)
客户端IP地址：172.24.161.33 (Pi)

$ iperf -u -T 10 -b 1M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)

[ 3] local 172.24.161.33 port 34869 connected with 172.24.42.46 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0000-10.0154 sec 1.25 MBytes 1.05 Mbits/sec
[ 3] Sent 896 datagrams
[ 3] Server Report:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0000-9.9907 sec 1.25 MBytes 1.05 Mbits/sec 8.210 ms 0/ 895 (0%)

$ iperf -u -T 10 -b 2M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)

[ 3] local 172.24.161.33 port 35665 connected with 172.24.42.46 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0000-10.0098 sec 2.50 MBytes 2.10 Mbits/sec
[ 3] Sent 1788 datagrams
[ 3] Server Report:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0000-9.9933 sec 2.50 MBytes 2.10 Mbits/sec 7.350 ms 0/ 1787 (0%)

$ iperf -u -T 10 -b 3M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)

[ 3] local 172.24.161.33 port 43883 connected with 172.24.42.46 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0000-10.0079 sec 3.75 MBytes 3.15 Mbits/sec
[ 3] Sent 2680 datagrams
[ 3] Server Report:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0000-10.0150 sec 3.75 MBytes 3.14 Mbits/sec 5.268 ms 0/ 2679 (0%)
1M带宽测试：

带宽：1.05 Mbits/sec

丢包率：0%

Jitter：8.210 ms
2M带宽测试：

带宽：2.10 Mbits/sec

丢包率：0%

Jitter：7.350 ms
3M带宽测试：

带宽：3.14 Mbits/sec

丢包率：0%

Jitter：5.268 ms

4.1.2 iperf测试参数B - 测试链路3

为什么要引入测试参数B？主要原因是：

4G LTE视频发现花屏+卡顿现象

测试参数A给出的1/2/3链路状况都是非常理想的
上述两个现象与实际结果发生了严重的不一致问题，太困惑了。

4G LTE networks results in significant screen freezing and stuttering #225

怀疑：是否是iperf测试参数问题导致测试结果不准确。因此，调整测试参数如下：

服务端：iperf -u -s
客户端：iperf -u -T 10 -b [1M] -l 20 -e -i 1 -t 60 -c [server address]

预设条件：有线IP网络物理断开

VPN测试：Pi --> 4G LTE --> BTS --> Server --> Modem --> Hub --> AP --> Jetson

服务端IP地址：172.24.42.46 (Jetson)

客户端IP地址：172.24.161.33 (Pi)

4.1.2.1 测试数据

$ iperf -u -T 10 -b 2M -l 20 -e -i 1 -t 60 -c 172.24.42.46
------------------------------------------------------------
Client connecting to 172.24.42.46, UDP port 5001 with pid 950 (1 flows)
Write buffer size: 20.0 Byte
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 172.24.161.33%zt44xhvvrc port 57316 connected with 172.24.42.46 port 5001
[ ID] Interval            Transfer     Bandwidth      Write/Err  PPS
[  3] 0.0000-1.0000 sec   256 KBytes  2.10 Mbits/sec  13104/0    13105 pps
[  3] 1.0000-2.0000 sec   256 KBytes  2.10 Mbits/sec  13111/0    13112 pps
[  3] 2.0000-3.0000 sec   256 KBytes  2.10 Mbits/sec  13108/0    13106 pps
[  3] 3.0000-4.0000 sec   256 KBytes  2.10 Mbits/sec  13106/0    13108 pps
[  3] 4.0000-5.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13108 pps
[  3] 5.0000-6.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13105 pps
[  3] 6.0000-7.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13107 pps
[  3] 7.0000-8.0000 sec   256 KBytes  2.10 Mbits/sec  13109/0    13108 pps
[  3] 8.0000-9.0000 sec   256 KBytes  2.10 Mbits/sec  13103/0    13103 pps
[  3] 9.0000-10.0000 sec   256 KBytes  2.10 Mbits/sec  13108/0    13112 pps
[  3] 10.0000-11.0000 sec   256 KBytes  2.10 Mbits/sec  13111/0    13106 pps
[  3] 11.0000-12.0000 sec   256 KBytes  2.10 Mbits/sec  13106/0    13108 pps
[  3] 12.0000-13.0000 sec   256 KBytes  2.10 Mbits/sec  13108/0    13107 pps
[  3] 13.0000-14.0000 sec   256 KBytes  2.10 Mbits/sec  13105/0    13107 pps
[  3] 14.0000-15.0000 sec   256 KBytes  2.10 Mbits/sec  13105/0    13102 pps
[  3] 15.0000-16.0000 sec   256 KBytes  2.10 Mbits/sec  13112/0    13113 pps
[  3] 16.0000-17.0000 sec   256 KBytes  2.10 Mbits/sec  13104/0    13104 pps
[  3] 17.0000-18.0000 sec   256 KBytes  2.10 Mbits/sec  13109/0    13111 pps
[  3] 18.0000-19.0000 sec   256 KBytes  2.10 Mbits/sec  13109/0    13107 pps
[  3] 19.0000-20.0000 sec   256 KBytes  2.10 Mbits/sec  13105/0    13107 pps
[  3] 20.0000-21.0000 sec   256 KBytes  2.10 Mbits/sec  13108/0    13105 pps
[  3] 21.0000-22.0000 sec   256 KBytes  2.10 Mbits/sec  13105/0    13109 pps
[  3] 22.0000-23.0000 sec   256 KBytes  2.10 Mbits/sec  13097/0    13107 pps
[  3] 23.0000-24.0000 sec   256 KBytes  2.10 Mbits/sec  13120/0    13107 pps
[  3] 24.0000-25.0000 sec   256 KBytes  2.10 Mbits/sec  13106/0    13105 pps
[  3] 25.0000-26.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13110 pps
[  3] 26.0000-27.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13105 pps
[  3] 27.0000-28.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13109 pps
[  3] 28.0000-29.0000 sec   256 KBytes  2.10 Mbits/sec  13108/0    13105 pps
[  3] 29.0000-30.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13107 pps
[  3] 30.0000-31.0000 sec   256 KBytes  2.10 Mbits/sec  13106/0    13106 pps
[  3] 31.0000-32.0000 sec   256 KBytes  2.10 Mbits/sec  13109/0    13110 pps
[  3] 32.0000-33.0000 sec   256 KBytes  2.10 Mbits/sec  13108/0    13108 pps
[  3] 33.0000-34.0000 sec   256 KBytes  2.10 Mbits/sec  13104/0    13107 pps
[  3] 34.0000-35.0000 sec   256 KBytes  2.10 Mbits/sec  13109/0    13105 pps
[  3] 35.0000-36.0000 sec   256 KBytes  2.10 Mbits/sec  13109/0    13109 pps
[  3] 36.0000-37.0000 sec   256 KBytes  2.10 Mbits/sec  13106/0    13108 pps
[  3] 37.0000-38.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13105 pps
[  3] 38.0000-39.0000 sec   256 KBytes  2.10 Mbits/sec  13108/0    13109 pps
[  3] 39.0000-40.0000 sec   256 KBytes  2.10 Mbits/sec  13106/0    13107 pps
[  3] 40.0000-41.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13107 pps
[  3] 41.0000-42.0000 sec   256 KBytes  2.10 Mbits/sec  13109/0    13107 pps
[  3] 42.0000-43.0000 sec   256 KBytes  2.10 Mbits/sec  13105/0    13107 pps
[  3] 43.0000-44.0000 sec   256 KBytes  2.10 Mbits/sec  13108/0    13107 pps
[  3] 44.0000-45.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13107 pps
[  3] 45.0000-46.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13104 pps
[  3] 46.0000-47.0000 sec   256 KBytes  2.10 Mbits/sec  13109/0    13109 pps
[  3] 47.0000-48.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13108 pps
[  3] 48.0000-49.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13106 pps
[  3] 49.0000-50.0000 sec   256 KBytes  2.10 Mbits/sec  13105/0    13105 pps
[  3] 50.0000-51.0000 sec   256 KBytes  2.10 Mbits/sec  13110/0    13110 pps
[  3] 51.0000-52.0000 sec   256 KBytes  2.10 Mbits/sec  13104/0    13108 pps
[  3] 52.0000-53.0000 sec   256 KBytes  2.10 Mbits/sec  13110/0    13106 pps
[  3] 53.0000-54.0000 sec   256 KBytes  2.10 Mbits/sec  13108/0    13108 pps
[  3] 54.0000-55.0000 sec   256 KBytes  2.10 Mbits/sec  13105/0    13108 pps
[  3] 55.0000-56.0000 sec   256 KBytes  2.10 Mbits/sec  13108/0    13107 pps
[  3] 56.0000-57.0000 sec   256 KBytes  2.10 Mbits/sec  13108/0    13107 pps
[  3] 57.0000-58.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13106 pps
[  3] 58.0000-59.0000 sec   256 KBytes  2.10 Mbits/sec  13106/0    13106 pps
[  3] 59.0000-60.0000 sec   256 KBytes  2.10 Mbits/sec  13107/0    13108 pps
[  3] 0.0000-60.0007 sec  15.0 MBytes  2.10 Mbits/sec  786443/0    13107 pps
[  3] Sent 786443 datagrams
[  3] Server Report:
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3] 0.0000-60.1233 sec  7.88 MBytes  1.10 Mbits/sec   0.145 ms 373400/786443 (47%)
[  3] 0.0000-60.1233 sec  32 datagrams received out-of-order

4.1.2.2 数据简单分析

测试设置为使用UDP协议，目标带宽为2Mbps，发送数据包大小为20字节，持续时间为60秒。
客户端连接到IP地址为172.24.42.46的服务器。
在整个测试期间，每秒发送的数据包数量保持在大约13100个左右，除非有数据包丢失，否则保持稳定。
服务器报告显示，在60秒的测试期间，共发送了786443个数据包，其中约47%的数据包丢失，且有32个数据包接收时顺序错乱。
带宽的实际测量结果约为1.10 Mbits/sec，明显低于设定的2Mbps目标带宽。

综上所述，这些数据表明在给定的网络环境下存在丢包和顺序错乱的问题，导致带宽表现不佳。可能的原因包括网络拥塞、传输延迟、或网络设备配置问题。为解决这些问题，可以考虑优化网络配置、增加带宽、或改善网络连接质量。

4.1.2.3 数据深入分析

本次测试已经发现一些线索，至少确实可能存在链路状况不佳的情况。

分析对比测试线A v/s B的差异：

测试参数A：默认v4 UDP=1470, v6 UDP=1450
测试参数B：数据包大小20字节，每秒报文数量13105

在维持带宽情况下，小包数量增加，会引起网络拥塞，进而导致丢包率明显增加。此时，出现花屏和卡顿就不足为奇。

从这里可以认识到，之前的测试过于简单。

因为视频流并非只是发送一种长度的报文，可能是变化的；另外，在编码完成后，会有发送报文需求；在图像编码时，视频数据尚没有准备好，此时，并无数据发送需求，网络带宽是空闲的。

引入以下思考：视频传输H264编码P/I帧的数据发送是如何分布的（时空概念）？

报文大小分布情况如何？
报文发送速率如何？

Bit Rate Throttling Algorithm on Video over RTP

举了一个例子：

高峰时刻I帧，5ms 15个数据包，而I帧又是完整的帧图像，如果此时出现丢包，那可不就是花屏、卡顿。
每间隔2秒，传输报文数量出现一个2倍峰值
15FPS(66.7ms)下，H264每隔62ms出现一个峰值大包传输集中点(时刻)

4.1.2.4 模拟测试网络

模拟高峰时刻带宽测试：5ms 15包数据(每秒15x1000/5=3000pps)，1400字节一包。

实际测试网络，还存在其他视频链路，应用程序等等网络开销
模拟始终高压传输，导致带宽更加紧张，容易导致UDP丢包

最终结果：81%丢包率，压根没法看视频了（妥妥的2-8定律）

$ iperf -b 3000pps -u -e -i 1 -l 1400 -c 172.24.42.46
------------------------------------------------------------
Client connecting to 172.24.42.46, UDP port 5001 with pid 1137 (1 flows)
Write buffer size: 1.37 KByte
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 172.24.161.33%zt44xhvvrc port 49273 connected with 172.24.42.46 port 5001
[ ID] Interval            Transfer     Bandwidth      Write/Err  PPS
[  3] 0.0000-1.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3002 pps
[  3] 1.0000-2.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3000 pps
[  3] 2.0000-3.0000 sec  4.00 MBytes  33.6 Mbits/sec  2996/0     3000 pps
[  3] 3.0000-4.0000 sec  4.01 MBytes  33.6 Mbits/sec  3003/0     2999 pps
[  3] 4.0000-5.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3001 pps
[  3] 5.0000-6.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3000 pps
[  3] 6.0000-7.0000 sec  4.00 MBytes  33.6 Mbits/sec  2999/0     3000 pps
[  3] 7.0000-8.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 8.0000-9.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 9.0000-10.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 0.0000-10.0009 sec  40.1 MBytes  33.6 Mbits/sec  30005/0     3000 pps
[  3] Sent 30005 datagrams
[  3] Server Report:
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3] 0.0000-10.3893 sec  7.64 MBytes  6.17 Mbits/sec   2.070 ms 24283/30007 (81%)

4.2 第二次测试

鉴于第一次测试的网络拓扑问题，现在只用一个4G LTE链路：因此链路独享，功能单一，不会有太多的竞争关系。【网络拓扑B】调整如下：

4.2.1 测试数据

$ iperf -b 3000pps -u -e -i 1 -l 1400 -c 172.24.42.46
------------------------------------------------------------
Client connecting to 172.24.42.46, UDP port 5001 with pid 1426 (1 flows)
Write buffer size: 1.37 KByte
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 172.24.161.33%zt44xhvvrc port 51044 connected with 172.24.42.46 port 5001
[ ID] Interval            Transfer     Bandwidth      Write/Err  PPS
[  3] 0.0000-1.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3002 pps
[  3] 1.0000-2.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 2.0000-3.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 3.0000-4.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3001 pps
[  3] 4.0000-5.0000 sec  4.00 MBytes  33.6 Mbits/sec  2999/0     2999 pps
[  3] 5.0000-6.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 6.0000-7.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 7.0000-8.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 8.0000-9.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 9.0000-10.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 0.0000-10.0009 sec  40.1 MBytes  33.6 Mbits/sec  30005/0     3000 pps
[  3] Sent 30005 datagrams
[  3] Server Report:
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3] 0.0000-10.2656 sec  37.5 MBytes  30.7 Mbits/sec   0.268 ms 1890/30004 (6.3%)

注：更多测试数据供参考。

daniel@rpanion:~ $ iperf -b 3000pps -u -e -i 1 -l 1400 -c 172.24.42.46
------------------------------------------------------------
Client connecting to 172.24.42.46, UDP port 5001 with pid 1510 (1 flows)
Write buffer size: 1.37 KByte
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 172.24.161.33%zt44xhvvrc port 43161 connected with 172.24.42.46 port 5001
[ ID] Interval            Transfer     Bandwidth      Write/Err  PPS
[  3] 0.0000-1.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3002 pps
[  3] 1.0000-2.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 2.0000-3.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3001 pps
[  3] 3.0000-4.0000 sec  4.00 MBytes  33.6 Mbits/sec  2999/0     2999 pps
[  3] 4.0000-5.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 5.0000-6.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 6.0000-7.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 7.0000-8.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 8.0000-9.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 9.0000-10.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 0.0000-10.0009 sec  40.1 MBytes  33.6 Mbits/sec  30005/0     3000 pps
[  3] Sent 30005 datagrams
[  3] Server Report:
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3] 0.0000-10.0739 sec  39.7 MBytes  33.0 Mbits/sec   0.191 ms  278/30004 (0.93%)
daniel@rpanion:~ $ iperf -b 3000pps -u -e -i 1 -l 1400 -c 172.24.42.46
------------------------------------------------------------
Client connecting to 172.24.42.46, UDP port 5001 with pid 1516 (1 flows)
Write buffer size: 1.37 KByte
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 172.24.161.33%zt44xhvvrc port 59910 connected with 172.24.42.46 port 5001
[ ID] Interval            Transfer     Bandwidth      Write/Err  PPS
[  3] 0.0000-1.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3002 pps
[  3] 1.0000-2.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 2.0000-3.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 3.0000-4.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 4.0000-5.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 5.0000-6.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 6.0000-7.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 7.0000-8.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 8.0000-9.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 9.0000-10.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 0.0000-10.0008 sec  40.1 MBytes  33.6 Mbits/sec  30005/0     3000 pps
[  3] Sent 30005 datagrams
[  3] Server Report:
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3] 0.0000-10.2943 sec  36.7 MBytes  29.9 Mbits/sec   0.462 ms 2551/30004 (8.5%)
daniel@rpanion:~ $ iperf -b 3000pps -u -e -i 1 -l 1400 -c 172.24.42.46
------------------------------------------------------------
Client connecting to 172.24.42.46, UDP port 5001 with pid 1520 (1 flows)
Write buffer size: 1.37 KByte
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 172.24.161.33%zt44xhvvrc port 40583 connected with 172.24.42.46 port 5001
[ ID] Interval            Transfer     Bandwidth      Write/Err  PPS
[  3] 0.0000-1.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3001 pps
[  3] 1.0000-2.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 2.0000-3.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 3.0000-4.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 4.0000-5.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 5.0000-6.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 6.0000-7.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 7.0000-8.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 8.0000-9.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 9.0000-10.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 0.0000-10.0022 sec  40.1 MBytes  33.6 Mbits/sec  30009/0     3000 pps
[  3] Sent 30009 datagrams
[  3] Server Report:
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3] 0.0000-10.0383 sec  38.3 MBytes  32.0 Mbits/sec   0.262 ms 1311/30008 (4.4%)
daniel@rpanion:~ $ iperf -b 3000pps -u -e -i 1 -l 1400 -c 172.24.42.46
------------------------------------------------------------
Client connecting to 172.24.42.46, UDP port 5001 with pid 1524 (1 flows)
Write buffer size: 1.37 KByte
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 172.24.161.33%zt44xhvvrc port 47580 connected with 172.24.42.46 port 5001
[ ID] Interval            Transfer     Bandwidth      Write/Err  PPS
[  3] 0.0000-1.0000 sec  3.96 MBytes  33.2 Mbits/sec  2967/0     3002 pps
[  3] 1.0000-2.0000 sec  4.00 MBytes  33.6 Mbits/sec  2997/0     3000 pps
[  3] 2.0000-3.0000 sec  4.03 MBytes  33.8 Mbits/sec  3015/0     2979 pps
[  3] 3.0000-4.0000 sec  4.03 MBytes  33.8 Mbits/sec  3016/0     3014 pps
[  3] 4.0000-5.0000 sec  3.98 MBytes  33.4 Mbits/sec  2982/0     3007 pps
[  3] 5.0000-6.0000 sec  4.04 MBytes  33.9 Mbits/sec  3024/0     3000 pps
[  3] 6.0000-7.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 7.0000-8.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 8.0000-9.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 9.0000-10.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 0.0000-10.0009 sec  40.1 MBytes  33.6 Mbits/sec  30005/0     3000 pps
[  3] Sent 30005 datagrams
[  3] Server Report:
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3] 0.0000-10.0151 sec  39.7 MBytes  33.3 Mbits/sec   0.302 ms  235/30004 (0.78%)
daniel@rpanion:~ $ iperf -b 3000pps -u -e -i 1 -l 1400 -c 172.24.42.46
------------------------------------------------------------
Client connecting to 172.24.42.46, UDP port 5001 with pid 1527 (1 flows)
Write buffer size: 1.37 KByte
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 172.24.161.33%zt44xhvvrc port 44434 connected with 172.24.42.46 port 5001
[ ID] Interval            Transfer     Bandwidth      Write/Err  PPS
[  3] 0.0000-1.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3001 pps
[  3] 1.0000-2.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 2.0000-3.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 3.0000-4.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 4.0000-5.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 5.0000-6.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 6.0000-7.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 7.0000-8.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 8.0000-9.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 9.0000-10.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 0.0000-10.0009 sec  40.1 MBytes  33.6 Mbits/sec  30005/0     3000 pps
[  3] Sent 30005 datagrams
[  3] Server Report:
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3] 0.0000-10.0227 sec  40.1 MBytes  33.5 Mbits/sec   0.474 ms    0/30004 (0%)

4.2.2 数据分析

根据提供的iperf数据，我们可以得到以下分析：

带宽：

平均带宽为33.6 Mbits/sec，这是UDP流量在网络上的平均传输速率。
丢包率：

总共发送了30005个数据包，其中有1890个数据包丢失。

丢包率为6.3%。这意味着有6.3%的数据包未能成功到达目的地。这可能是由于网络拥塞、丢包或其他网络问题引起的。
Jitter：

Jitter指的是数据包传输之间的时延变化。在本例中，Jitter为0.268毫秒。Jitter越小，表示数据包传输的稳定性越高。
PPS（每秒数据包数）：

平均每秒传输了3000个数据包。这是一个可用于评估网络负载的指标。

综合来看，虽然仍存在6.3%的丢包率，实际情况不可能长时间高压力，视频码流也有FEC等纠错机制。相对于第一次测试情况，相对来说该网络链路环境更为理想，也更加符合实际使用情况。

4.2.3 网络延迟分析

通过ping命令，iperf(Linux端)和QGC(Windows端)，分析网络延时：

iperf(Linux端)：平均3.35ms

64 bytes from 172.24.161.33: icmp_seq=1 ttl=64 time=3.07 ms
64 bytes from 172.24.161.33: icmp_seq=2 ttl=64 time=3.14 ms
64 bytes from 172.24.161.33: icmp_seq=3 ttl=64 time=3.83 ms
64 bytes from 172.24.161.33: icmp_seq=4 ttl=64 time=3.35 ms
QGC(Windows端)：平均7.75ms

Reply from 172.24.161.33: bytes=32 time=7ms TTL=64
Reply from 172.24.161.33: bytes=32 time=3ms TTL=64
Reply from 172.24.161.33: bytes=32 time=12ms TTL=64
Reply from 172.24.161.33: bytes=32 time=9ms TTL=64

4.2.4 视频效果

实测效果： ~ 245 ms 时延；？？？貌似VPN没啥时延啊？呵呵，后面说，卖个关子。

Rpanion OV5647 ZeroTier 网络拓扑B 性能实测

4.3 第三次测试

使用更加贴近实际场景，两个4G LTE独享带宽【网络拓扑C】。

4.3.1 测试数据

$ iperf -b 3000pps -u -e -i 1 -l 1400 -c 172.24.42.46
------------------------------------------------------------
Client connecting to 172.24.42.46, UDP port 5001 with pid 860 (1 flows)
Write buffer size: 1.37 KByte
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 172.24.161.33%zt44xhvvrc port 36516 connected with 172.24.42.46 port 5001
[ ID] Interval            Transfer     Bandwidth      Write/Err  PPS
[  3] 0.0000-1.0000 sec  4.01 MBytes  33.6 Mbits/sec  3002/0     3002 pps
[  3] 1.0000-2.0000 sec  4.00 MBytes  33.6 Mbits/sec  2999/0     3000 pps
[  3] 2.0000-3.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 3.0000-4.0000 sec  4.00 MBytes  33.6 Mbits/sec  2999/0     3000 pps
[  3] 4.0000-5.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3000 pps
[  3] 5.0000-6.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 6.0000-7.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     2999 pps
[  3] 7.0000-8.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3001 pps
[  3] 8.0000-9.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 9.0000-10.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3001 pps
[  3] 0.0000-10.0015 sec  40.1 MBytes  33.6 Mbits/sec  30007/0     3000 pps
[  3] Sent 30007 datagrams
[  3] Server Report:
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3] 0.0000-10.3614 sec  37.5 MBytes  30.4 Mbits/sec   0.469 ms 1882/30006 (6.3%)
[  3] 0.0000-10.3614 sec  2921 datagrams received out-of-order

注：更多测试数据供参考。

$ iperf -b 3000pps -u -e -i 1 -l 1400 -c 172.24.42.46
------------------------------------------------------------
Client connecting to 172.24.42.46, UDP port 5001 with pid 1484 (1 flows)
Write buffer size: 1.37 KByte
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 172.24.161.33%zt44xhvvrc port 39651 connected with 172.24.42.46 port 5001
[ ID] Interval            Transfer     Bandwidth      Write/Err  PPS
[  3] 0.0000-1.0000 sec  4.01 MBytes  33.6 Mbits/sec  3001/0     3002 pps
[  3] 1.0000-2.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 2.0000-3.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 3.0000-4.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 4.0000-5.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 5.0000-6.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 6.0000-7.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 7.0000-8.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 8.0000-9.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 9.0000-10.0000 sec  4.01 MBytes  33.6 Mbits/sec  3000/0     3000 pps
[  3] 0.0000-10.0005 sec  40.1 MBytes  33.6 Mbits/sec  30004/0     3000 pps
[  3] Sent 30004 datagrams
[  3] Server Report:
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3] 0.0000-10.1356 sec  36.8 MBytes  30.4 Mbits/sec   0.174 ms 2446/30003 (8.2%)

4.3.2 数据分析

测试参数：

使用了以下参数进行测试：带宽设置为3000pps、使用UDP协议、启用错误报告、设置报告间隔为1秒、每个数据包大小为1400字节，连接到IP地址为172.24.42.46的服务器。
客户端信息：

客户端IP地址为172.24.161.33，端口36516，连接到服务器IP地址为172.24.42.46，端口5001。
传输数据：

在每个1秒的时间间隔内，传输了大约4.01兆字节的数据。

带宽在每秒都维持在约33.6兆位/秒。

平均每秒发送的数据包数（PPS）为3000左右。
UDP错误：

未发现UDP错误。
服务器报告：

在10.36秒的测试期间，总共传输了37.5兆字节的数据。

平均带宽为30.4兆位/秒。

报告的抖动（jitter）为0.469毫秒。

6.3%的数据包丢失率，共丢失1882个数据包。

有2921个数据包收到时是乱序的。

综上所述，该iperf测试表明UDP连接在设定的条件下运行良好，但存在一定程度的数据包丢失和乱序。

4.3.3 网络延迟分析

通过ping命令，iperf(Linux端)和QGC(Windows端)，分析网络延时：

iperf(Linux端)：平均607.5ms

64 bytes from 172.24.161.33: icmp_seq=1 ttl=64 time=668 ms
64 bytes from 172.24.161.33: icmp_seq=2 ttl=64 time=583 ms
64 bytes from 172.24.161.33: icmp_seq=4 ttl=64 time=565 ms
64 bytes from 172.24.161.33: icmp_seq=5 ttl=64 time=614 ms
QGC(Windows端)：平均822.25ms

Reply from 172.24.161.33: bytes=32 time=825ms TTL=64
Reply from 172.24.161.33: bytes=32 time=854ms TTL=64
Reply from 172.24.161.33: bytes=32 time=801ms TTL=64
Reply from 172.24.161.33: bytes=32 time=809ms TTL=64

4.3.4 视频效果

实测效果： ~ 569 ms 时延； Linux端607.5/2 = 303ms, 569 - 303 = 266 ms左右。

Rpanion OV5647 ZeroTier 网络拓扑C 性能实测

5. 总结

综上所述：

【网络拓扑A】~ 200 ms, 参考：《ArduPilot开源飞控之lida2003套机+伴机电脑首飞》
【网络拓扑B】~ 245 ms
【网络拓扑C】~ 569 ms
局域网点对点性能最好，直接就是Rpanion+Rpi3B+ 200ms时延。
统一WLAN出口，VPN仍然能够很好的优化，与局域网性能相当。
实际不同发射/接受链路网络，若VPN不做好优化，时延会很显著。

实际使用场景，需要看链路情况，若要进行商业部署，必须做好优化，或者自建网络(VPN/P2P等等)来控制好链路时延问题。

Ardupilot + Rpi3B+ + OV5647 + 4G LTE Copter

6. 参考资料

【1】zerotier - Securely connect any device, anywhere.

7. ZeroTier优化办法 - Moon节点

7.1 服务端配置

Step1：安装zerotier-one

curl -s https://install.zerotier.com/ | sudo bash
Step2：加入自己的网络

zerotier-cli join <Network ID>
Step3：生成moon.json文件

cd /var/lib/zerotier-one

zerotier-idtool initmoon identity.public > moon.json
Step4：修改moon.json文件 //服务器，打开安全组，开放9993端口，采用UDP协议

vi moon.json //字段中修改："stableEndpoints": ["服务器的ip/9993"]
Step5：生成签名文件 //生成一个000000xxxxxxxxx.moon的文件

zerotier-idtool genmoon moon.json
Step6：新建自己的moon节点配置

mkdir moons.d

mv 000000xxxxxxxxx.moon moons.d

service zerotier-one restart //之后重启服务器端B的Zerotier

7.2 客户端配置

Step1：下载配置文件

scp usrname@ServerIP:/var/lib/zerotier-one/moons.d/000000xxxxxxxxx.moon .
Step2：前往zerotier安装目录 //window一般在 C:\ProgramData\ZeroTier\One
Step3：新建ZeroTier\One\moons.d目录，并复制00000xxxx.moon
Step4：重启windows服务
- Step5：查看moon服务器 //确认服务器(IP)和节点已经在列表中，表示成功

zerotier-cli listpeers
Step6：查看ping延迟 //网络跳转越少，路线越短，性能越好。国内典型值在20ms左右。

8. DJI RC系统性能

DJI RC Pro features O3+ video transmission technology, capable of sending a 1080p/60fps live feed from up to 15 km away and latency as low as 120 ms.

Ardupilot Rpanion iperf网络性能测试

Ardupilot Rpanion iperf网络性能测试

1. 源由

2. 分析

3. 安装

4. 测试

4.1 第一次测试

4.1.1 iperf测试参数A

4.1.1.1 测试链路1

$ iperf -u -T 10 -b 1M -c 192.168.1.19

Client connecting to 192.168.1.19, UDP port 5001 UDP buffer size: 208 KByte (default)

$ iperf -u -T 10 -b 2M -c 192.168.1.19

Client connecting to 192.168.1.19, UDP port 5001 UDP buffer size: 208 KByte (default)

$ iperf -u -T 10 -b 3M -c 192.168.1.19

Client connecting to 192.168.1.19, UDP port 5001 UDP buffer size: 208 KByte (default)

4.1.1.2 测试链路2

$ iperf -u -T 10 -b 1M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001 UDP buffer size: 208 KByte (default)

$ iperf -u -T 10 -b 2M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001 UDP buffer size: 208 KByte (default)

$ iperf -u -T 10 -b 3M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001 UDP buffer size: 208 KByte (default)

4.1.1.3 测试链路3

$ iperf -u -T 10 -b 1M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001 UDP buffer size: 208 KByte (default)

$ iperf -u -T 10 -b 2M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001 UDP buffer size: 208 KByte (default)

$ iperf -u -T 10 -b 3M -c 172.24.42.46

Client connecting to 172.24.42.46, UDP port 5001 UDP buffer size: 208 KByte (default)

4.1.2 iperf测试参数B - 测试链路3

4.1.2.1 测试数据

4.1.2.2 数据简单分析

4.1.2.3 数据深入分析

4.1.2.4 模拟测试网络

4.2 第二次测试

4.2.1 测试数据

4.2.2 数据分析

4.2.3 网络延迟分析

4.2.4 视频效果

4.3 第三次测试

4.3.1 测试数据

4.3.2 数据分析

4.3.3 网络延迟分析

4.3.4 视频效果

5. 总结

6. 参考资料

7. ZeroTier优化办法 - Moon节点

7.1 服务端配置

curl -s https://install.zerotier.com/ | sudo bash

zerotier-cli join <Network ID>

cd /var/lib/zerotier-one

zerotier-idtool initmoon identity.public > moon.json

vi moon.json //字段中修改："stableEndpoints": ["服务器的ip/9993"]

zerotier-idtool genmoon moon.json

mkdir moons.d

mv 000000xxxxxxxxx.moon moons.d

service zerotier-one restart //之后重启服务器端B的Zerotier

7.2 客户端配置

scp usrname@ServerIP:/var/lib/zerotier-one/moons.d/000000xxxxxxxxx.moon .

zerotier-cli listpeers

8. DJI RC系统性能

Client connecting to 192.168.1.19, UDP port 5001
UDP buffer size: 208 KByte (default)

Client connecting to 192.168.1.19, UDP port 5001
UDP buffer size: 208 KByte (default)

Client connecting to 192.168.1.19, UDP port 5001
UDP buffer size: 208 KByte (default)

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)

Client connecting to 172.24.42.46, UDP port 5001
UDP buffer size: 208 KByte (default)