搭建高可用OpenStack（Queen版）集群（九）之部署nova计算节点

happy_king_zi2024-08-16 11:24

一、搭建高可用OpenStack（Queen版）集群之部署计算节点

一、部署nova

1、安装nova-compute

在全部计算节点安装nova-compute服务

复制代码 
yum install python-openstackclient openstack-utils openstack-selinux -y
yum install openstack-nova-compute -y

若yum安装时报错如下，解决办法：找到对应包进行单独安装

复制代码 
Failed:
  python-backports-ssl_match_hostname.noarch 0:3.5.0.1-1.el7                                                                                       python2-urllib3.noarch 0:1.21.1-1.el7

解决

python2-urllib3.noarch 0:1.21.1-1.el7下载地址：python-urllib3-1.21.1-1.el7 | Build Info | CentOS Community Build Service 具体包的地址：http://cbs.centos.org/kojifiles/packages/python-urllib3/1.21.1/1.el7/noarch/python2-urllib3-1.21.1-1.el7.noarch.rpm

python-backports-ssl_match_hostname

2、配置nova.conf

在全部计算节点操作

**　　注意：**

  1. "my_ip"参数，根据节点修改
  2. nova.conf文件的权限：root:nova
复制代码 
cp -rp /etc/nova/nova.conf{,.bak}
egrep -v "^$|^#" /etc/nova/nova.conf
复制代码 
[DEFAULT]
my_ip=10.20.9.46
use_neutron=true
firewall_driver=nova.virt.firewall.NoopFirewallDriver
enabled_apis=osapi_compute,metadata
# 前端采用haproxy时，服务连接rabbitmq会出现连接超时重连的情况，可通过各服务与rabbitmq的日志查看；
# transport_url=rabbit://openstack:openstack@controller:5673
# rabbitmq本身具备集群机制，官方文档建议直接连接rabbitmq集群；但采用此方式时服务启动有时会报错，原因不明；如果没有此现象，强烈建议连接rabbitmq直接对接集群而非通过前端haproxy
transport_url=rabbit://openstack:openstack@controller01:5672,openstack:openstack@controller02:5672,openstack:openstack@controller03:5672
[api]
auth_strategy=keystone
[api_database]
[barbican]
[cache]
[cells]
[cinder]
[compute]
[conductor]
[console]
[consoleauth]
[cors]
[crypto]
[database]
[devices]
[ephemeral_storage_encryption]
[filter_scheduler]
[glance]
api_servers=http://controller:9292
[guestfs]
[healthcheck]
[hyperv]
[ironic]
[key_manager]
[keystone]
[keystone_authtoken]
auth_uri = http://controller:5000
auth_url = http://controller:35357
memcached_servers = controller01:11211,controller02:11211,controller03:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = nova
password = nova_pass
[libvirt]
# 通过“egrep -c '(vmx|svm)' /proc/cpuinfo”命令查看主机是否支持硬件加速，返回1或者更大的值表示支持，返回0表示不支持；
# 支持硬件加速使用”kvm”类型，不支持则使用”qemu”类型；
# 一般虚拟机不支持硬件加速
virt_type=qemu
[matchmaker_redis]
[metrics]
[mks]
[neutron]
[notifications]
[osapi_v21]
[oslo_concurrency]
lock_path=/var/lib/nova/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[pci]
[placement]
os_region_name=RegionTest
auth_type=password
auth_url=http://controller:35357/v3
project_name=service
project_domain_name=Default
username=placement
user_domain_name=Default
password=placement_pass
[quota]
[rdp]
[remote_debug]
[scheduler]
[serial_console]
[service_user]
[spice]
[upgrade_levels]
[vault]
[vendordata_dynamic_auth]
[vmware]
[vnc]
enabled=true
vncserver_listen=0.0.0.0
vncserver_proxyclient_address=$my_ip
# 因某些未做主机绑定的客户端不能访问”controller”名字，改为使用具体的ip地址
novncproxy_base_url=http://10.20.9.47:6080/vnc_auto.html
[workarounds]
[wsgi]
[xenserver]
[xvp]
3、启动服务，并设置为开机自启，然后验证服务是否正常

在全部计算节点操作

复制代码 
systemctl enable libvirtd.service openstack-nova-compute.service
systemctl restart libvirtd.service
systemctl restart openstack-nova-compute.service

systemctl status libvirtd.service openstack-nova-compute.service
4、向cell数据库添加计算节点

在任意控制节点操作

确认数据库中含有主机

复制代码 
[root@controller01 ml2]# openstack compute service list --service nova-compute
+-----+--------------+-----------+------+---------+-------+----------------------------+
|  ID | Binary       | Host      | Zone | Status  | State | Updated At                 |
+-----+--------------+-----------+------+---------+-------+----------------------------+
| 123 | nova-compute | compute01 | nova | enabled | up    | 2018-09-15T13:06:40.000000 |
| 126 | nova-compute | compute02 | nova | enabled | up    | 2018-09-15T13:06:46.000000 |
+-----+--------------+-----------+------+---------+-------+----------------------------+
1、手动发现计算节点

手工发现计算节点主机，即添加到cell数据库

复制代码 
su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova
2、自动发现计算节点

在全部控制节点操作

  1. 为避免新加入计算节点时，手动执行注册操作"nova-manage cell_v2 discover_hosts"，可设置控制节点定时自动发现主机；
  2. 涉及控制节点nova.conf文件的[scheduler]字段；
  3. 如下设置自动发现时间为5min，可根据实际环境调节

设置自动发现时间为5min

复制代码 
# vim /etc/nova/nova.conf
[scheduler]
discover_hosts_in_cells_interval=300

重启nova服务，配置生效

复制代码 
systemctl restart openstack-nova-api.service
systemctl status openstack-nova-api.service
5、验证

登陆dashboard，管理员-->计算-->虚拟机管理器

如果已注册成功，在"虚拟机管理器"标签下可发现计算节点，并能展示出各计算节点的资源；如果未注册或注册失败，则"虚拟机管理器"标签下无主机。

http://10.20.9.47/

账号：admin 密码：admin_pass

1、查看计算节点情况

2、查看openstack集群中所有节点情况

二、部署Neutron

1、安装neutron-linuxbridge

在全部计算节点安装neutro-linuxbridge服务

复制代码 
yum install openstack-neutron-linuxbridge ebtables ipset -y
2、配置neutron.conf

在全部计算节点操作

**　　注意：**

  1. "bind_host"参数，根据节点修改
  2. neutron.conf文件的权限：root:neutron
复制代码 
cp -rp /etc/neutron/neutron.conf{,.bak}
egrep -v "^$|^#" /etc/neutron/neutron.conf
复制代码 
[DEFAULT]
state_path = /var/lib/neutron
bind_host = 10.20.9.46
auth_strategy = keystone
# 前端采用haproxy时，服务连接rabbitmq会出现连接超时重连的情况，可通过各服务与rabbitmq的日志查看；
# transport_url = rabbit://openstack:openstack@controller:5673
# rabbitmq本身具备集群机制，官方文档建议直接连接rabbitmq集群；
但采用此方式时服务启动有时会报错，原因不明；如果没有此现象，强烈建议连接rabbitmq直接对接集群而非通过前端haproxy
transport_url=rabbit://openstack:openstack@controller01:5672,openstack:openstack@controller02:5672,openstack:openstack@controller03:5672
[agent]
[cors]
[database]
[keystone_authtoken]
www_authenticate_uri = http://controller:5000
auth_url = http://controller:35357
memcached_servers = controller01:11211,controller02:11211,controller03:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = neutron_pass
[matchmaker_redis]
[nova]
[oslo_concurrency]
lock_path = $state_path/lock
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[quotas]
[ssl]
3、配置linuxbridge_agent.ini
1、配置linuxbridge_agent.ini

在全部计算节点操作

**　　注意：linuxbridge_agent.ini文件的权限：root:neutron**

****　　　　单网卡需要设置：physical_interface_mappings = provider:ens192****

复制代码 
cp /etc/neutron/plugins/ml2/linuxbridge_agent.ini{,.bak}
egrep -v "^$|^#" /etc/neutron/plugins/ml2/linuxbridge_agent.ini
复制代码 
[DEFAULT]
[agent]
[linux_bridge]
# 网络类型名称与物理网卡对应，这里vlan租户网络对应规划的eth3；
# 需要明确的是物理网卡是本地有效，需要根据主机实际使用的网卡名确定；
# 另有”bridge_mappings”参数对应网桥
physical_interface_mappings = vlan:eth3
[network_log]
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
enable_security_group = true
[vxlan]
enable_vxlan = true
# tunnel租户网络（vxlan）vtep端点，这里对应规划的eth2（的地址），根据节点做相应修改
local_ip = 10.0.0.41
l2_population = true
2、配置内核参数
  1. bridge：是否允许桥接；
  2. 如果"sysctl -p"加载不成功，报" No such file or directory"错误，需要加载内核模块"br_netfilter"；
  3. 命令"modinfo br_netfilter"查看内核模块信息；
  4. 命令"modprobe br_netfilter"加载内核模块，解决错误
复制代码 
echo "# bridge" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-ip6tables = 1" >> /etc/sysctl.conf
sysctl -p
4、配置nova.conf

在全部计算节点操作

配置只涉及nova.conf的"[neutron]"字段

复制代码 
# vim /etc/nova/nova.conf
[neutron]
url=http://controller:9696
auth_type=password
auth_url=http://controller:35357
project_name=service
project_domain_name=default
username=neutron
user_domain_name=default
password=neutron_pass
region_name=RegionTest
5、启动服务

在全部计算节点操作

nova.conf文件已变更，首先需要重启全部计算节点的nova服务

复制代码 
systemctl restart openstack-nova-compute.service
systemctl status openstack-nova-compute.service

启动网络服务

复制代码 
systemctl enable neutron-linuxbridge-agent.service
systemctl restart neutron-linuxbridge-agent.service
systemctl status neutron-linuxbridge-agent.service
6、验证

**　　任意控制节点操作**

加载环境变量

复制代码 
. admin-openrc

查看neutron相关的agent

复制代码 
[root@controller01 neutron]# openstack network agent list
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| ID                                   | Agent Type         | Host         | Availability Zone | Alive | State | Binary                    |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| 03637bae-7416-4a23-b478-2fcafe29e11e | Linux bridge agent | controller02 | None              | :-)   | UP    | neutron-linuxbridge-agent |
| 1272cce8-459d-4880-b2ec-ce4f808d4271 | Metadata agent     | controller01 | None              | :-)   | UP    | neutron-metadata-agent    |
| 395b2cbb-87c0-4323-9273-21b9a9e7edaf | DHCP agent         | controller03 | nova              | :-)   | UP    | neutron-dhcp-agent        |
| 70dcfbb4-e60f-44a7-86ed-c07a719591fc | L3 agent           | controller02 | nova              | :-)   | UP    | neutron-l3-agent          |
| 711ef8dc-594d-4e81-8b45-b70944f031b0 | DHCP agent         | controller01 | nova              | :-)   | UP    | neutron-dhcp-agent        |
| 7626f6ec-620e-4eb4-b69b-78081f07cae5 | Linux bridge agent | compute02    | None              | :-)   | UP    | neutron-linuxbridge-agent |
| a1d93fac-d0bf-43d9-b613-3b6c6778e3ea | Linux bridge agent | compute01    | None              | :-)   | UP    | neutron-linuxbridge-agent |
| a5a67133-3218-41df-946e-d6162098b199 | Linux bridge agent | controller01 | None              | :-)   | UP    | neutron-linuxbridge-agent |
| af72caaf-48c3-423d-8526-31f529a1575b | L3 agent           | controller01 | nova              | :-)   | UP    | neutron-l3-agent          |
| b54c108f-a543-43f5-b81f-396b832da9c3 | Linux bridge agent | controller03 | None              | :-)   | UP    | neutron-linuxbridge-agent |
| cde2adf4-6796-4d50-9471-fda0cc060f09 | Metadata agent     | controller03 | None              | :-)   | UP    | neutron-metadata-agent    |
| d1707f54-e626-47fe-ba20-2f5e15abb662 | DHCP agent         | controller02 | nova              | :-)   | UP    | neutron-dhcp-agent        |
| e47a48b8-e7cb-48d1-b10f-895b7a536b70 | L3 agent           | controller03 | nova              | :-)   | UP    | neutron-l3-agent          |
| ec20a89d-2527-4342-9dab-f47ac5d71726 | Metadata agent     | controller02 | None              | :-)   | UP    | neutron-metadata-agent    |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
[root@controller01 neutron]# openstack network agent list --agent-type linux-bridge
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| ID                                   | Agent Type         | Host         | Availability Zone | Alive | State | Binary                    |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| 03637bae-7416-4a23-b478-2fcafe29e11e | Linux bridge agent | controller02 | None              | :-)   | UP    | neutron-linuxbridge-agent |
| 7626f6ec-620e-4eb4-b69b-78081f07cae5 | Linux bridge agent | compute02    | None              | :-)   | UP    | neutron-linuxbridge-agent |
| a1d93fac-d0bf-43d9-b613-3b6c6778e3ea | Linux bridge agent | compute01    | None              | :-)   | UP    | neutron-linuxbridge-agent |
| a5a67133-3218-41df-946e-d6162098b199 | Linux bridge agent | controller01 | None              | :-)   | UP    | neutron-linuxbridge-agent |
| b54c108f-a543-43f5-b81f-396b832da9c3 | Linux bridge agent | controller03 | None              | :-)   | UP    | neutron-linuxbridge-agent |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+

三、部署cinder

https://www.cnblogs.com/netonline/p/9337900.html

在采用ceph或其他商业/非商业后端存储时，建议将cinder-volume服务部署在控制节点，通过pacemaker将服务运行在active/passive模式。

1、安装cinder

在全部计算点安装cinder服务

复制代码 
yum install -y openstack-cinder targetcli python-keystone
2、配置cinder.conf

在全部计算点操作

**　　注意：**

  1. "my_ip"参数，根据节点修改
  2. cinder.conf文件的权限：root:cinder
复制代码 
cp -rp /etc/cinder/cinder.conf{,.bak}
egrep -v "^$|^#" /etc/cinder/cinder.conf
复制代码 
[DEFAULT]
state_path = /var/lib/cinder
my_ip = 10.20.9.46
glance_api_servers = http://controller:9292
auth_strategy = keystone
# 简单的将cinder理解为存储的机头，后端可以采用nfs，ceph等共享存储
enabled_backends = ceph
# 前端采用haproxy时，服务连接rabbitmq会出现连接超时重连的情况，可通过各服务与rabbitmq的日志查看；
# transport_url = rabbit://openstack:openstack@controller:5673
# rabbitmq本身具备集群机制，官方文档建议直接连接rabbitmq集群；但采用此方式时服务启动有时会报错，原因不明；
# 如果没有此现象，强烈建议连接rabbitmq直接对接集群而非通过前端haproxy
transport_url=rabbit://openstack:openstack@controller01:5672,openstack:openstack@controller02:5672,openstack:openstack@controller03:5672
[backend]
[backend_defaults]
[barbican]
[brcd_fabric_example]
[cisco_fabric_example]
[coordination]
[cors]
[database]
connection = mysql+pymysql://cinder:123456@controller/cinder
[fc-zone-manager]
[healthcheck]
[key_manager]
[keystone_authtoken]
www_authenticate_uri = http://controller:5000
auth_url = http://controller:35357
memcached_servers = controller01:11211,controller02:11211,controller03:11211
auth_type = password
project_domain_id = default
user_domain_id = default
project_name = service
username = cinder
password = cinder_pass
[matchmaker_redis]
[nova]
[oslo_concurrency]
lock_path = $state_path/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[oslo_versionedobjects]
[profiler]
[service_user]
[ssl]
[vault]
3、启动服务

在全部计算点操作

复制代码 
systemctl enable openstack-cinder-volume.service target.service
systemctl restart openstack-cinder-volume.service
systemctl restart target.service
systemctl status openstack-cinder-volume.service target.service
4、验证

在任意控制节点操作

加载环境变量

复制代码 
. admin-openrc

查看agent服务或：cinder service-list

此时后端存储服务为ceph，但ceph相关服务尚未启用并集成到cinder-volume，导致cinder-volume服务的状态也是"down"

复制代码 
[root@controller01 ~]# openstack volume service list
+------------------+--------------+------+---------+-------+----------------------------+
| Binary           | Host         | Zone | Status  | State | Updated At                 |
+------------------+--------------+------+---------+-------+----------------------------+
| cinder-scheduler | controller01 | nova | enabled | up    | 2018-09-10T13:09:24.000000 |
| cinder-scheduler | controller03 | nova | enabled | up    | 2018-09-10T13:09:25.000000 |
| cinder-scheduler | controller02 | nova | enabled | up    | 2018-09-10T13:09:23.000000 |
+------------------+--------------+------+---------+-------+----------------------------+
相关推荐
珍珠是蚌的眼泪2 小时前
微服务_1、入门
分布式·微服务·eureka·springcloud·注册中心
问道飞鱼2 小时前
分布式中间件-分布式代理框架Codis和Twemproxy
分布式·中间件
ps酷教程3 小时前
Zookeeper学习
分布式·学习·zookeeper
一见你就笑i4 小时前
关于RabbitMQ重复消费的解决方案
分布式·面试·rabbitmq
weixin_439357201 天前
分布式本地缓存 ehcache 缓存同步复制
java·spring boot·分布式·缓存
qq_485015211 天前
rabbitmq
分布式·rabbitmq
月夜星辉雪1 天前
【RabbitMQ 项目】服务端：数据管理模块之绑定管理
分布式·rabbitmq
kalvin_y_liu1 天前
RabbitMQ
分布式·rabbitmq·ruby
Pdh胖大海1 天前
Redisson实现分布式锁（看门狗机制）
redis·分布式
小阿轩yx1 天前
小阿轩yx-Zabbix企业级分布式监控环境部署
linux·分布式·云计算·zabbix·运维开发·企业级监控部署
热门推荐
01RAG 实践- Ollama+RagFlow 部署本地知识库02组基轨迹建模 GBTM的介绍与实现（Stata 或 R）032024年高教社杯数学建模国赛C题超详细解题思路分析04yolov8实战第五天——yolov8+ffmpg实时视频流检测并进行实时推流——（推流，保姆教学）05Coze扣子平台完整体验和实践（附国内和国际版对比）06【2024数模国赛赛题思路公开】国赛B题思路丨附可运行代码丨无偿自提07【经验分享】Ubuntu22.04安装微信(linux官方版)08苍穹外卖面试总结09【2024高教社杯全国大学生数学建模竞赛】B题 生产过程中的决策问题——解题思路 代码 论文10Ubuntu24.04安装中文输入法