Linux bridge开启hairpin模拟测试macvlan vepa模式(续)-联通外部网络

石小千2023-12-27 13:02

之前写了《Linux bridge开启hairpin模拟测试macvlan vepa模式》，记录了Linux bridge开启hairpin，模拟测试macvlan vepa模式下，同一父接口下两个子接口的网络通讯情况，文中缺少了子接口与外部网络通讯的部分，本文补上，详见下：

参考

1.Linux bridge开启hairpin模拟测试macvlan vepa模式

2.Linux虚拟网络设备---之使用Veth pair连接linux网桥bridge

3.Linux 网络设备 - Bridge 详解

4.brctl快速入门与基础

环境

与《Linux bridge开启hairpin模拟测试macvlan vepa模式》环境相同。

测试

1. 测试流程

《Linux bridge开启hairpin模拟测试macvlan vepa模式》只验证了macvlan vepa模式下，同一父网卡下的两个子接口（子网卡）通过开启hairpin的外部交换转发，实现网络通讯。

本文在上述测试基础上，补充Linux bridge开启hairpin模拟测试macvlan vepa时，子接口（子网卡）与外部网络（宿主机以外网络）的通讯情况，测试步骤如下：

  • 详见 《Linux bridge开启hairpin模拟测试macvlan vepa模式》中 1.测试流程
  • 补充测试流程 ：
    • 将宿主机网卡enp0s5 加入Linux bridge br0
    • 删掉网卡enp0s5 绑定的IP（10.211.55.18 ），并将该IP 绑定到br0 上
    • 修改宿主机默认路由，默认路由设备由enp0s5 改为br0
    • 增加ns101 下默认路由，默认路由设备使用veth0_1.101
    • 增加ns102 下默认路由，默认路由设备使用veth0_1.102
    • 测试macvlan vepa网络子接口与宿主机以外网络的通讯情况

详见下图：

  • 网络连接图
    • 物理网卡enp0s5连接外部网络
    • br0 连接物理网卡enp0s5 和虚拟网卡veth0
    • 虚拟网卡veth0 和veth0_1是一对veth pair
    • veth0_1.101 和veth0_1.102 是veth0的子网卡
  • 网络通讯图
    • veth0_1.101 和veth0_1.102 ，通过br0 （veth0接口开启hairpin）转发通讯。
    • veth0_1.101 和veth0_1.102 ，通过br0 的接口（enp0s5）与宿主机以外网络通讯，例如：网关10.211.55.1

2. 将宿主机网卡enp0s5加入Linux bridge br0

请务必确保有其他方式可以登录宿主机，网卡enp0s5加入br0后，当前网络连接会断开。

  • enp0s5 加入br0
javascript 复制代码 
// enp0s5 加入 br0
[root@centos7-18 ~]# brctl addif br0 enp0s5
// 查看bridge
[root@centos7-18 ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.001c426087b2       no              enp0s5
                                                        veth0
virbr0          8000.5254009f1377       yes             virbr0-nic
[root@centos7-18 ~]#

3. 删掉网卡enp0s5 IP（10.211.55.18），将该IP绑定到br0 上

  • 删掉宿主机网卡enp0s5 绑定的IP 10.211.55.18
javascript 复制代码 
// 删掉宿主机网卡enp0s5的IP
[root@centos7-18 ~]# ip addr del 10.211.55.18/24 dev enp0s5
  • 增加br0 绑定IP 10.211.55.18
javascript 复制代码 
// br0绑定IP 10.211.55.18
[root@centos7-18 ~]# ip addr add 10.211.55.18/24 dev br0
  • 查看当前IP
    • enp0s5已经没有IP了
    • br0 的IP为10.211.55.18
javascript 复制代码 
// 查看当前IP
[root@centos7-18 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000
    link/ether 00:1c:42:60:87:b2 brd ff:ff:ff:ff:ff:ff
    inet6 fdb2:2c26:f4e4:0:21c:42ff:fe60:87b2/64 scope global mngtmpaddr dynamic 
       valid_lft 2581396sec preferred_lft 594196sec
    inet6 fe80::21c:42ff:fe60:87b2/64 scope link 
       valid_lft forever preferred_lft forever
6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:1c:42:60:87:b2 brd ff:ff:ff:ff:ff:ff
    inet 10.211.55.18/24 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:0:21c:42ff:fe60:87b2/64 scope global mngtmpaddr dynamic 
       valid_lft 2591815sec preferred_lft 604615sec
    inet6 fe80::8413:97ff:fe70:a2e2/64 scope link 
       valid_lft forever preferred_lft forever
7: veth0_1@veth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 7a:87:ef:c6:77:9b brd ff:ff:ff:ff:ff:ff
    inet6 fdb2:2c26:f4e4:0:7887:efff:fec6:779b/64 scope global mngtmpaddr dynamic 
       valid_lft 2591815sec preferred_lft 604615sec
    inet6 fe80::7887:efff:fec6:779b/64 scope link 
       valid_lft forever preferred_lft forever
8: veth0@veth0_1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether 86:08:8e:91:09:fe brd ff:ff:ff:ff:ff:ff
    inet6 fe80::8408:8eff:fe91:9fe/64 scope link 
       valid_lft forever preferred_lft forever
[root@centos7-18 ~]#

4. 修改宿主机默认路由，设备由enp0s5改为br0

  • 删掉宿主机enp0s5的默认路由
javascript 复制代码 
// 删掉宿主机网卡enp0s5的IP
[root@centos7-18 ~]# ip route del default via 10.211.55.1 dev enp0s5
  • 增加宿主机br0的默认路由
javascript 复制代码 
// 增加宿主机**br0**的默认路由
[root@centos7-18 ~]# ip route add default via 10.211.55.1 dev br0
  • 查看当前路由
javascript 复制代码 
// 查看当前路由
[root@centos7-18 ~]# ip route
default via 10.211.55.1 dev br0 
10.211.55.0/24 dev br0 proto kernel scope link src 10.211.55.18 
10.211.55.0/24 dev enp0s6 proto kernel scope link src 10.211.55.21 metric 100 
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1

5. 增加namespace ns101和ns102的默认路由

  • 增加ns101 下默认路由，设备使用veth0_1.101
javascript 复制代码 
// 增加ns101下默认路由，默认路由设备使用veth0_1.101
[root@centos7-18 ~]# ip netns exec ns101 ip route add default via 10.211.55.1 dev veth0_1.101
// 查看ns101路由表 
[root@centos7-18 ~]# ip netns exec ns101 ip route 
default via 10.211.55.1 dev veth0_1.101 
10.211.55.0/24 dev veth0_1.101 proto kernel scope link src 10.211.55.101 
[root@centos7-18 ~]#
  • 增加ns102 下默认路由，设备使用veth0_1.102
javascript 复制代码 
// 增加ns102下默认路由，默认路由设备使用veth0_1.102
[root@centos7-18 ~]# ip netns exec ns102 ip route add default via 10.211.55.1 dev veth0_1.102 
// 查看ns102路由表 
[root@centos7-18 ~]# ip netns exec ns102 ip route 
default via 10.211.55.1 dev veth0_1.102 
10.211.55.0/24 dev veth0_1.102 proto kernel scope link src 10.211.55.102 
[root@centos7-18 ~]#

6. 测试macvlan vepa网络子接口与宿主机以外网络通讯

  • 检查测试环境-enp0s5 、br0 、veth0 和veth0_1已启用
javascript 复制代码 
// 显示已启用的设备：enp0s5、br0、veth0、veth0_1
[root@centos7-18 ~]# ip address show up
2: enp0s5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000
    link/ether 00:1c:42:60:87:b2 brd ff:ff:ff:ff:ff:ff
    inet6 fdb2:2c26:f4e4:0:21c:42ff:fe60:87b2/64 scope global mngtmpaddr dynamic 
       valid_lft 2569303sec preferred_lft 582103sec
    inet6 fe80::21c:42ff:fe60:87b2/64 scope link 
       valid_lft forever preferred_lft forever
6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:1c:42:60:87:b2 brd ff:ff:ff:ff:ff:ff
    inet 10.211.55.18/24 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:0:21c:42ff:fe60:87b2/64 scope global mngtmpaddr dynamic 
       valid_lft 2591897sec preferred_lft 604697sec
    inet6 fe80::8413:97ff:fe70:a2e2/64 scope link 
       valid_lft forever preferred_lft forever
7: veth0_1@veth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 7a:87:ef:c6:77:9b brd ff:ff:ff:ff:ff:ff
    inet6 fdb2:2c26:f4e4:0:7887:efff:fec6:779b/64 scope global mngtmpaddr dynamic 
       valid_lft 2591897sec preferred_lft 604697sec
    inet6 fe80::7887:efff:fec6:779b/64 scope link 
       valid_lft forever preferred_lft forever
8: veth0@veth0_1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether 86:08:8e:91:09:fe brd ff:ff:ff:ff:ff:ff
    inet6 fe80::8408:8eff:fe91:9fe/64 scope link 
       valid_lft forever preferred_lft forever
  • 检查测试环境-br0 的veth0接口已开启hairpin
javascript 复制代码 
// 开启hairpin
[root@centos7-18 ~]# brctl hairpin br0 veth0 on
// 查看br0的veth0开启hairpin结果
[root@centos7-18 ~]# bridge -d link  | grep -A5 veth0
8: veth0 state UP @veth0_1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 2 
    hairpin on guard off root_block off fastleave off learning on flood on mcast_flood on 
[root@centos7-18 ~]#
  • 检查测试环境-veth0_1.101 和veth0_1.102已启用
javascript 复制代码 
// 显示ns101已启用的设备：veth0_1.101
[root@centos7-18 ~]# ip netns exec ns101 ip a show up
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
9: veth0_1.101@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether b2:3e:6e:ae:74:57 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.211.55.101/24 scope global veth0_1.101
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:0:b03e:6eff:feae:7457/64 scope global mngtmpaddr dynamic 
       valid_lft 2591507sec preferred_lft 604307sec
    inet6 fe80::b03e:6eff:feae:7457/64 scope link 
       valid_lft forever preferred_lft forever
// 显示ns102已启用的设备：veth0_1.102
[root@centos7-18 ~]# ip netns exec ns102 ip a show up
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
10: veth0_1.102@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 22:f8:d5:8b:c1:63 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.211.55.102/24 scope global veth0_1.102
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:0:20f8:d5ff:fe8b:c163/64 scope global mngtmpaddr dynamic 
       valid_lft 2591491sec preferred_lft 604291sec
    inet6 fe80::20f8:d5ff:fe8b:c163/64 scope link 
       valid_lft forever preferred_lft forever
  • 测试namespace ns101 访问宿主机以外网络
    • Ping 网关IP 10.211.55.1 ，通
    • Ping 网络内其它主机IP 10.211.55.10 ，通
javascript 复制代码 
// 查看ns101的IP
[root@centos7-18 ~]# ip netns exec ns101 ip a | grep -A5 veth
9: veth0_1.101@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether b2:3e:6e:ae:74:57 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.211.55.101/24 scope global veth0_1.101
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:0:b03e:6eff:feae:7457/64 scope global mngtmpaddr dynamic 
       valid_lft 2591841sec preferred_lft 604641sec
    inet6 fe80::b03e:6eff:feae:7457/64 scope link 
       valid_lft forever preferred_lft forever
javascript 复制代码 
// Ping 网关IP 10.211.55.1，通
[root@centos7-18 ~]# ip netns exec ns101 ping -c2 10.211.55.1
PING 10.211.55.1 (10.211.55.1) 56(84) bytes of data.
64 bytes from 10.211.55.1: icmp_seq=1 ttl=128 time=0.164 ms
64 bytes from 10.211.55.1: icmp_seq=2 ttl=128 time=0.323 ms

--- 10.211.55.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.164/0.243/0.323/0.081 ms
[root@centos7-18 ~]# 
// Ping 网络内其它主机IP 10.211.55.10，通
[root@centos7-18 ~]# ip netns exec ns101 ping -c2 10.211.55.10
PING 10.211.55.10 (10.211.55.10) 56(84) bytes of data.
64 bytes from 10.211.55.10: icmp_seq=1 ttl=64 time=0.288 ms
64 bytes from 10.211.55.10: icmp_seq=2 ttl=64 time=0.526 ms

--- 10.211.55.10 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.288/0.407/0.526/0.119 ms
[root@centos7-18 ~]#
  • 测试namespace ns101 和ns102 可以互相访问
javascript 复制代码 
// ns101 ping ns102的IP 10.211.55.102，通
[root@centos7-18 ~]# ip netns exec ns101 ping -c2 10.211.55.102
PING 10.211.55.102 (10.211.55.102) 56(84) bytes of data.
64 bytes from 10.211.55.102: icmp_seq=1 ttl=64 time=0.085 ms
64 bytes from 10.211.55.102: icmp_seq=2 ttl=64 time=0.083 ms

--- 10.211.55.102 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1005ms
rtt min/avg/max/mdev = 0.083/0.084/0.085/0.001 ms
// ns102 ping ns101的IP 10.211.55.101，通
[root@centos7-18 ~]# ip netns exec ns102 ping -c2 10.211.55.101
PING 10.211.55.101 (10.211.55.101) 56(84) bytes of data.
64 bytes from 10.211.55.101: icmp_seq=1 ttl=64 time=0.057 ms
64 bytes from 10.211.55.101: icmp_seq=2 ttl=64 time=0.087 ms

--- 10.211.55.101 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.057/0.072/0.087/0.015 ms
[root@centos7-18 ~]#

结果

bridge工作在二层，打通了namespace与宿主机以外的网络，查看br0的mac table可以看到mac结果

1. 查看br0的mac table，包含网关（外部）的mac

  • 先通过宿主机Ping网关IP 10.211.55.1 ，获得网关对应的mac：00:1c:42:00:00:18
javascript 复制代码 
// 宿主机Ping网关IP 10.211.55.1后，查看网关mac
[root@centos7-18 ~]# ping -c2 10.211.55.1
PING 10.211.55.1 (10.211.55.1) 56(84) bytes of data.
64 bytes from 10.211.55.1: icmp_seq=1 ttl=128 time=0.170 ms
64 bytes from 10.211.55.1: icmp_seq=2 ttl=128 time=0.264 ms

--- 10.211.55.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.170/0.217/0.264/0.047 ms
// 网关mac为 00:1c:42:00:00:18
[root@centos7-18 ~]# arp 
Address                  HWtype  HWaddress           Flags Mask            Iface
10.211.55.2              ether   00:1c:42:00:00:08   C                     br0
10.211.55.1              ether   00:1c:42:00:00:18   C                     br0
10.211.55.10                     (incomplete)                              br0
10.211.55.1              ether   00:1c:42:00:00:18   C                     enp0s6
10.211.55.102            ether   22:f8:d5:8b:c1:63   C                     br0
10.211.55.101            ether   b2:3e:6e:ae:74:57   C                     br0
  • 查看br0 的mac table，已包含网关的mac(00:1c:42:00:00:18)
javascript 复制代码 
// 查看br0 mac table
[root@centos7-18 ~]# brctl showmacs br0
port no mac addr                is local?       ageing timer
  2     00:1c:42:00:00:08       no                 0.00
  2     00:1c:42:00:00:18       no                10.43
  2     00:1c:42:60:87:b2       yes                0.00
  2     00:1c:42:60:87:b2       yes                0.00
  2     00:1c:42:d1:70:62       no               193.31
  1     86:08:8e:91:09:fe       yes                0.00
  1     86:08:8e:91:09:fe       yes                0.00

2. 查看br0的mac table，包含macvlan vepa子网卡（内部）的mac

  • 查看ns101 网卡veth0_1.101 的mac为b2:3e:6e:ae:74:57
javascript 复制代码 
// ns101网卡veth0_1.101的mac为b2:3e:6e:ae:74:57
[root@centos7-18 ~]# ip netns exec ns101 ip a | grep -A2 veth
9: veth0_1.101@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether b2:3e:6e:ae:74:57 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.211.55.101/24 scope global veth0_1.101
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:0:b03e:6eff:feae:7457/64 scope global mngtmpaddr dynamic
  • 在ns101 中 ping 网关IP 10.211.55.1
javascript 复制代码 
// ns101 ping 网关IP 10.211.55.1
[root@centos7-18 ~]# ip netns exec ns101 ping -c2 10.211.55.1
PING 10.211.55.1 (10.211.55.1) 56(84) bytes of data.
64 bytes from 10.211.55.1: icmp_seq=1 ttl=128 time=0.169 ms
64 bytes from 10.211.55.1: icmp_seq=2 ttl=128 time=0.225 ms

--- 10.211.55.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.169/0.197/0.225/0.028 ms
  • 查看br0 的mac table，新增加了veth0_1.101 的mac（b2:3e:6e:ae:74:57）
javascript 复制代码 
// 查看br0 mac table，增加了veth0_1.101的mac b2:3e:6e:ae:74:57
[root@centos7-18 ~]# brctl showmacs br0
port no mac addr                is local?       ageing timer
  2     00:1c:42:00:00:08       no                 0.00
  2     00:1c:42:00:00:18       no                 0.93
  2     00:1c:42:60:87:b2       yes                0.00
  2     00:1c:42:60:87:b2       yes                0.00
  2     00:1c:42:d1:70:62       no               177.70
  1     86:08:8e:91:09:fe       yes                0.00
  1     86:08:8e:91:09:fe       yes                0.00
  1     b2:3e:6e:ae:74:57       no                 0.94

总结

通过Linux bridge 开启接口hairpin的方式，模拟macvlan vepa在外部交换支持802.1q的情况下，同一父网卡下的多个子网卡之间是可以通讯的。(详见 Linux bridge开启hairpin模拟测试macvlan vepa模式 )

通过将宿主机物理网卡加入到Linux bridge中，使用bridge桥接内外网络，可以实现内部macvlan vepa子网卡访问外部网络。

相关推荐
Spring_java_gg7 分钟前
如何抵御 Linux 服务器黑客威胁和攻击
linux·服务器·网络·安全·web安全
✿ ༺ ོIT技术༻7 分钟前
Linux：认识文件系统
linux·运维·服务器
会掉头发35 分钟前
Linux进程通信之共享内存
linux·运维·共享内存·进程通信
我言秋日胜春朝★38 分钟前
【Linux】冯诺依曼体系、再谈操作系统
linux·运维·服务器
饮啦冰美式1 小时前
22.04Ubuntu---ROS2使用rclcpp编写节点
linux·运维·ubuntu
wowocpp1 小时前
ubuntu 22.04 server 安装 和 初始化 LTS
linux·运维·ubuntu
Huaqiwill1 小时前
Ubuntun搭建并行计算环境
linux·云计算
wclass-zhengge1 小时前
Netty篇（入门编程）
java·linux·服务器
Lign173141 小时前
ubuntu unrar解压 中文文件名异常问题解决
linux·运维·ubuntu
热门推荐
01【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总02(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明03组基轨迹建模 GBTM的介绍与实现（Stata 或 R）04【AIGC】重塑未来的科技巨轮05全面解析：构建基于深度学习的安全帽检测系统（UI界面+YOLO代码+数据集）06【经验分享】Ubuntu22.04安装微信(linux官方版)07基于YOLOv10深度学习的CT扫描图像肾结石智能检测系统【python源码+Pyqt5界面+数据集+训练代码】深度学习实战、目标检测08Ubuntu 20.04使用Livox mid 360 测试 FAST_LIO09RAG 实践- Ollama+RagFlow 部署本地知识库10【TC3xx芯片】TC3xx芯片电源管理系统PMS详解