基于华为云ARM架构ECS平台的OpenStack Wallaby All-in-one 完整部署指导

基于华为云ECS平台的OpenStack Wallaby All-in-one 完整部署指导

📦 阶段一：确认离线包（已上传至资源平台）

离线包 export-all.tar.gz 应包含：

export-all/
├── oos-playbooks.tar.gz           # 所有修复过的 playbook 和 role
├── oos-inventory.tar.gz           # inventory 配置
├── oos-workdir.tar.gz             # oos 工作目录
├── oos-keypair.tar.gz             # SSH 密钥对
├── openstack-configs.tar.gz       # OpenStack 配置文件（参考）
├── oos.conf                       # oos 全局配置
├── constants.yaml / openstack_release.yaml / openeuler_repo.yaml
├── openeuler_sig_repo.yaml / package.spec.j2
├── .admin-openrc                  # OpenStack 环境变量
├── rpm-list.txt
├── service-list-snapshot.txt
└── endpoint-list-snapshot.txt

上传到新 ECS 的 /root/ 下备用。

---

🖥️ 阶段二：新 ECS 环境准备

0. 虚拟机规格

项目	推荐配置
CPU	8 核
内存	16 GB 以上（低于 8 GB 会因 MariaDB 连接断开导致部署失败）
系统盘	50 GB
数据盘 1	30 GB（Cinder LVM，vdb/sdb）
数据盘 2	30 GB（Swift，vdc/sdc）
网络	1 张网卡
操作系统	openEuler 22.03 LTS aarch64

1. 系统基础配置

hostnamectl set-hostname openstack-ecs
systemctl stop firewalld && systemctl disable firewalld
setenforce 0
sed -i 's/^SELINUX=enforcing/SELINUX=permissive/' /etc/selinux/config

yum -y install ntpdate
ntpdate cn.pool.ntp.org

pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

2. 安装 OpenStack SIG 工具

pip install openstack-sig-tool==1.0.6

3. 安装 oos 工具及基础依赖

yum -y install openstack-release-wallaby git
yum -y install openblas-0.3.18-3.oe2203.aarch64 python3-XStatic-roboto-fontface-0.5.0.0-1.oe2203.noarch
pip install pyparsing==2.4.2
pip3 uninstall -y pyOpenSSL
pip3 install pyOpenSSL jinja2==3.0.3 werkzeug==2.0.3 cryptography==38.0.3 ipaddress==1.0.23

git clone https://gitee.com/openeuler/openstack.git
cd openstack/tools/oos/
pip3 install -r requirements.txt
python3 setup.py install
cd ~

4. ⭐ Python 库版本降级

库	版本	解决的问题
requests	2.27.1	nova-manage 递归错误
urllib3	1.26.18	配合 requests
numpy	<2.0	Cinder 启动 np.float_ 报错
dnspython	2.2.1	keystone-manage 的 dns.rdtypes 报错

pip3 install requests==2.27.1 urllib3==1.26.18 'numpy<2.0' dnspython==2.2.1

python3 -c "import requests, urllib3, numpy, dns; \
  print(f'requests={requests.__version__}'); \
  print(f'urllib3={urllib3.__version__}'); \
  print(f'numpy={numpy.__version__}'); \
  print(f'dnspython={dns.__version__}')"

期望 ：requests=2.27.1、urllib3=1.26.18、numpy=1.26.x、dnspython=2.2.1。

5. 安装工具

yum install -y python3-libselinux tmux

---

📥 阶段三：导入离线包

1. 解压

cd /root
tar xzf export-all.tar.gz
cd export-all/

2. 覆盖 oos playbooks

mv /usr/local/etc/oos/playbooks /usr/local/etc/oos/playbooks.orig
tar xzf oos-playbooks.tar.gz -C /usr/local/etc/oos/

3. 覆盖 inventory

mv /usr/local/etc/oos/inventory /usr/local/etc/oos/inventory.orig
tar xzf oos-inventory.tar.gz -C /usr/local/etc/oos/

4. 覆盖 oos 全局配置

cp oos.conf /usr/local/etc/oos/oos.conf
cp constants.yaml openstack_release.yaml openeuler_repo.yaml openeuler_sig_repo.yaml package.spec.j2 /usr/local/etc/oos/

修改 oos.conf：

vi /usr/local/etc/oos/oos.conf

neutron_dataplane_interface_name = eth1    # ip addr 查看
cinder_block_device = vdb                  # lsblk 查看
swift_storage_devices = vdc                # lsblk 查看

5. ⭐ 校验 create_identity_service role

grep -n "awk" /usr/local/etc/oos/playbooks/roles/create_identity_service/tasks/main.yaml

期望 ：能看到 awk '$2=="{``{ type }}"' 之类的行。为空则手动改为：

- name: Create {{ service }} identity service
  shell: |
    source ~/.admin-openrc
    if ! openstack service list -f value -c Type | grep -qx "{{ type }}"; then
      openstack service create --name {{ service }} --description "{{ description }}" {{ type }}
    fi
    SERVICE_ID=$(openstack service list -f value -c ID -c Type | awk '$2=="{{ type }}"{print $1}' | head -n1)
    for iface in public internal admin; do
      if ! openstack endpoint list --service "$SERVICE_ID" --interface "$iface" -f value -c ID | grep -q .; then
        case $iface in
          public)   openstack endpoint create --region RegionOne "$SERVICE_ID" public   {{ endpoint }} ;;
          internal) openstack endpoint create --region RegionOne "$SERVICE_ID" internal {{ endpoint }} ;;
          admin)    openstack endpoint create --region RegionOne "$SERVICE_ID" admin    {{ admin_endpoint | default(endpoint) }} ;;
        esac
      fi
    done
  args:
    executable: /bin/bash

6. ⭐ 校验其他 playbook 的幂等性

grep -n "blkid" /usr/local/etc/oos/playbooks/swift.yaml
grep -n "ip link show br-ex" /usr/local/etc/oos/playbooks/neutron.yaml
grep -n "pvs /dev" /usr/local/etc/oos/playbooks/cinder.yaml
grep -n "EVENTLET_NO_GREENDNS" /usr/local/etc/oos/playbooks/nova.yaml

每条都应有输出。

7. 恢复密钥对

tar xzf oos-keypair.tar.gz -C /usr/local/etc/oos/

mkdir -p /root/.ssh
cp /usr/local/etc/oos/key_pair/* /root/.ssh/
chmod 0600 /root/.ssh/id_rsa
chmod 0644 /root/.ssh/id_rsa.pub

mkdir -p /root/oos/key_pair
cp /usr/local/etc/oos/key_pair/* /root/oos/key_pair/
chmod 600 /root/oos/key_pair/id_rsa

8. 恢复 oos 工作目录

mv /root/oos /root/oos.orig 2>/dev/null
tar xzf oos-workdir.tar.gz -C /root/

9. 恢复 admin-openrc 并改 IP

cp .admin-openrc /root/.admin-openrc
chmod 600 /root/.admin-openrc
vi /root/.admin-openrc
# OS_AUTH_URL=http://<新机器内网IP>:5000/v3

---

⚙️ 阶段四：部署准备与运行

1. 注册 oos 环境

oos env manage -n all-in-one -i <新机器内网IP> -r 22.03-lts
ssh-copy-id root@<新机器内网IP>

2. ⭐ 创建三个核心脚本

2.1 fix-endpoints.sh

cat > /root/fix-endpoints.sh <<'SCRIPT_EOF'
#!/bin/bash
# OpenStack endpoint 修复脚本 v3
HOST="192.168.0.193"      # ⚠️ 按新机器 IP 改
REGION="RegionOne"

source /root/.admin-openrc 2>/dev/null

echo "=== [1/5] 检查 Keystone ==="
if ! systemctl is-active --quiet openstack-keystone; then
  echo "❌ Keystone 未运行，请先启动"; exit 1
fi
if ! openstack token issue >/dev/null 2>&1; then
  echo "❌ Keystone 认证失败"; exit 1
fi
echo "✅ Keystone 正常"

echo ""
echo "=== [2/5] 清理空 service ==="
openstack service list -f value -c ID -c Name -c Type 2>/dev/null | while read id name type; do
  if [ -z "$name" ] || [ -z "$type" ]; then
    echo "  删除空 service: id=$id"
    openstack endpoint list --service "$id" -f value -c ID 2>/dev/null | while read eid; do
      openstack endpoint delete "$eid" 2>/dev/null
    done
    openstack service delete "$id" 2>/dev/null
  fi
done

echo ""
echo "=== [3/5] 清理重复 service（保留有 endpoint 的） ==="
for stype in $(openstack service list -f value -c Type 2>/dev/null | sort -u); do
  [ -z "$stype" ] && continue
  ids=($(openstack service list -f value -c ID -c Type 2>/dev/null | awk -v t="$stype" '$2==t{print $1}'))
  [ "${#ids[@]}" -le 1 ] && continue

  keep=""
  for id in "${ids[@]}"; do
    cnt=$(openstack endpoint list --service "$id" -f value -c ID 2>/dev/null | wc -l)
    if [ "$cnt" -ge 1 ] && [ -z "$keep" ]; then
      keep="$id"
    fi
  done
  [ -z "$keep" ] && keep="${ids[0]}"

  for id in "${ids[@]}"; do
    if [ "$id" != "$keep" ]; then
      openstack endpoint list --service "$id" -f value -c ID 2>/dev/null | while read eid; do
        openstack endpoint delete "$eid" 2>/dev/null
      done
      openstack service delete "$id" 2>/dev/null
      echo "  删除重复 service: $id ($stype)"
    fi
  done
done

echo ""
echo "=== [4/5] 补齐 endpoint ==="
declare -A SERVICES=(
  [identity]="5000:/v3/"
  [image]="9292:/"
  [compute]="8774:/v2.1"
  [placement]="8778:/"
  [network]="9696:/"
  [volumev3]="8776:/v3/%(project_id)s"
  [volumev2]="8776:/v2/%(tenant_id)s"
  [object-store]="8080:/v1/AUTH_%(tenant_id)s"
  [orchestration]="8004:/v1/%(tenant_id)s"
  [cloudformation]="8000:/v1"
  [metric]="8041:/"
  [alarming]="8042:/"
  [metering]="8777:/"
  [database]="8779:/v1.0/%(tenant_id)s"
  [accelerator]="8777:/"
)

for stype in "${!SERVICES[@]}"; do
  IFS=':' read -r port path <<< "${SERVICES[$stype]}"
  sid=$(openstack service list -f value -c ID -c Type 2>/dev/null | awk -v t="$stype" '$2==t{print $1}' | head -1)
  [ -z "$sid" ] && continue
  url="http://${HOST}:${port}${path}"
  for iface in public internal admin; do
    if ! openstack endpoint list --service "$sid" --interface "$iface" -f value -c ID 2>/dev/null | grep -q .; then
      if openstack endpoint create --region "$REGION" "$sid" "$iface" "$url" >/dev/null 2>&1; then
        echo "  ✅ $stype $iface"
      else
        echo "  ❌ $stype $iface 创建失败"
      fi
    fi
  done
done

echo ""
echo "=== [5/5] 检查结果 ==="
if ! openstack token issue >/dev/null 2>&1; then
  echo "❌ Keystone 已不可用"; exit 1
fi
for sid in $(openstack service list -f value -c ID 2>/dev/null); do
  sname=$(openstack service show "$sid" -f value -c name 2>/dev/null)
  stype=$(openstack service show "$sid" -f value -c type 2>/dev/null)
  [ -z "$stype" ] && continue
  count=$(openstack endpoint list --service "$sid" -f value -c ID 2>/dev/null | wc -l)
  if [ "$count" -lt 3 ]; then
    echo "  ⚠️  $sname ($stype): only $count"
  else
    echo "  ✅ $sname ($stype): $count"
  fi
done
SCRIPT_EOF

chmod +x /root/fix-endpoints.sh

2.2 clean-dup-services.sh

cat > /root/clean-dup-services.sh <<'SCRIPT_EOF'
#!/bin/bash
source /root/.admin-openrc
if ! openstack token issue >/dev/null 2>&1; then
  echo "❌ Keystone 不可用"; exit 1
fi
echo "=== 清理重复 service（保留有 endpoint 的） ==="
for stype in $(openstack service list -f value -c Type 2>/dev/null | sort -u); do
  [ -z "$stype" ] && continue
  ids=($(openstack service list -f value -c ID -c Type 2>/dev/null | awk -v t="$stype" '$2==t{print $1}'))
  [ "${#ids[@]}" -le 1 ] && continue
  echo ""
  echo "--- $stype 有 ${#ids[@]} 个 ---"
  keep=""
  for id in "${ids[@]}"; do
    cnt=$(openstack endpoint list --service "$id" -f value -c ID 2>/dev/null | wc -l)
    echo "  id=$id endpoints=$cnt"
    if [ "$cnt" -ge 1 ] && [ -z "$keep" ]; then keep="$id"; fi
  done
  [ -z "$keep" ] && keep="${ids[0]}"
  echo "  ➡️  保留: $keep"
  for id in "${ids[@]}"; do
    if [ "$id" != "$keep" ]; then
      openstack endpoint list --service "$id" -f value -c ID 2>/dev/null | while read eid; do
        openstack endpoint delete "$eid" 2>/dev/null
      done
      openstack service delete "$id" 2>/dev/null
      echo "  ❌ 删除: $id"
    fi
  done
done
echo ""
echo "=== 清理后 ==="
openstack service list
SCRIPT_EOF

chmod +x /root/clean-dup-services.sh

2.3 watch-endpoints.sh（部署期间并行监控）

cat > /root/watch-endpoints.sh <<'SCRIPT_EOF'
#!/bin/bash
HOST="192.168.0.193"      # ⚠️ 按新机器 IP 改
REGION="RegionOne"
INTERVAL=30

declare -A SERVICES=(
  [identity]="5000:/v3/" [image]="9292:/" [compute]="8774:/v2.1"
  [placement]="8778:/" [network]="9696:/"
  [volumev3]="8776:/v3/%(project_id)s" [volumev2]="8776:/v2/%(tenant_id)s"
  [object-store]="8080:/v1/AUTH_%(tenant_id)s"
  [orchestration]="8004:/v1/%(tenant_id)s" [cloudformation]="8000:/v1"
  [metric]="8041:/" [alarming]="8042:/" [metering]="8777:/"
  [database]="8779:/v1.0/%(tenant_id)s" [accelerator]="8777:/"
)

source /root/.admin-openrc 2>/dev/null

while true; do
  if ! openstack token issue >/dev/null 2>&1; then
    echo "[$(date +%T)] ⏸  Keystone 不可用，等待..."
    sleep $INTERVAL; continue
  fi
  FIXED=0
  for stype in "${!SERVICES[@]}"; do
    IFS=':' read -r port path <<< "${SERVICES[$stype]}"
    sid=$(openstack service list -f value -c ID -c Type 2>/dev/null | awk -v t="$stype" '$2==t{print $1}' | head -1)
    [ -z "$sid" ] && continue
    url="http://${HOST}:${port}${path}"
    for iface in public internal admin; do
      if ! openstack endpoint list --service "$sid" --interface "$iface" -f value -c ID 2>/dev/null | grep -q .; then
        if openstack endpoint create --region "$REGION" "$sid" "$iface" "$url" >/dev/null 2>&1; then
          echo "[$(date +%T)] ✅ 补齐 $stype $iface"; FIXED=$((FIXED+1))
        fi
      fi
    done
  done
  [ "$FIXED" -eq 0 ] && echo "[$(date +%T)] 所有 endpoint 完整"
  sleep $INTERVAL
done
SCRIPT_EOF

chmod +x /root/watch-endpoints.sh

3. ⭐ 启动三会话模式

# 会话 1：watch
tmux new -s watch
/root/watch-endpoints.sh
# Ctrl+B D

# 会话 2：deploy
tmux new -s deploy
cd /root/oos
oos env setup -r wallaby all-in-one 2>&1 | tee /root/oos/deploy-$(date +%F_%H%M%S).log
# Ctrl+B D

# 会话 3：log
tmux new -s log
tail -f $(ls -t /root/oos/deploy-*.log | head -1)

4. 部署成功标志

PLAY RECAP *********************************************************************
controller                 : ok=XXX  changed=XXX  unreachable=0    failed=0    skipped=X    rescued=0    ignored=X

关键是 failed=0。

---

🔄 阶段五：部署中断后的标准恢复流程

五步法

# 1. 修依赖
pip3 install dnspython==2.2.1                    # dns.rdtypes
pip3 install requests==2.27.1 urllib3==1.26.18   # RecursionError
pip3 install 'numpy<2.0'                         # np.float_

# 2. 清理重复/空 service
/root/clean-dup-services.sh

# 3. 补齐所有 endpoint
systemctl status openstack-keystone
/root/fix-endpoints.sh

# 4. 重启失败的服务
systemctl restart openstack-nova-api openstack-nova-conductor \
  openstack-nova-scheduler openstack-nova-compute

# 5. 重跑 setup
cd /root/oos
oos env setup -r wallaby all-in-one 2>&1 | tee /root/oos/deploy-retry-$(date +%F_%H%M%S).log

报错速查表

报错关键词	原因	命令
AttributeError: module 'dns.rdtypes'	dnspython 版本高	pip3 install dnspython==2.2.1
RecursionError	requests/urllib3 版本高	pip3 install requests==2.27.1 urllib3==1.26.18
AttributeError: np.float_	numpy 2.x	pip3 install 'numpy<2.0'
EndpointNotFound ... placement	placement endpoint 缺失	/root/fix-endpoints.sh
EndpointNotFound ... gnocchi / metric	gnocchi endpoint 缺失	/root/fix-endpoints.sh
EndpointNotFound ... 其他服务	对应服务 endpoint 缺失	同上
Multiple service matches	Keystone service 重复	/root/clean-dup-services.sh
HTTP 500 / Lost connection to MySQL	内存不足	增加 ECS 内存至 16 GB+
Connection refused 到 :5000	Keystone 挂了	systemctl restart openstack-keystone
No package ... available	yum 仓库未启用	检查 EPOL 和 OpenStack_Wallaby

---

✅ 阶段六：部署后验证

1. Endpoint 完整性

source /root/.admin-openrc
openstack endpoint list | wc -l       # 期望 33（11 服务 × 3）
/root/fix-endpoints.sh | tail -n 15   # 全部 ✅

2. 服务命令测试

openstack token issue
openstack image list
openstack network list
openstack server list
openstack volume list
openstack stack list
openstack object store account show
openstack compute service list
openstack network agent list
openstack volume service list

3. 服务进程状态

systemctl list-units --type=service --state=running | \
  grep -iE "openstack|nova|neutron|cinder|glance|heat|swift|trove|cyborg|aodh|gnocchi|ceilometer|placement"

4. ⭐ Horizon session 清理（必做！）

在正式访问 Horizon 之前，先清掉旧的 session 缓存（即使这次是新部署，如果之前有任何 endpoint 缺失的时期访问过 dashboard，也会污染 session）：

systemctl restart memcached
systemctl restart httpd
sleep 2
systemctl status memcached
systemctl status httpd

为什么必须做：Horizon 会把登录用户的服务目录（service catalog）缓存在 session 中。如果用户在 endpoint 缺失时访问过 dashboard，旧 session 会一直用残缺的 catalog，导致后续即使 endpoint 补齐了，页面仍然报"出错啦"。

5. 浏览器访问 Horizon

浏览器访问：http://<ECS 公网 IP>/dashboard

首次访问时：

• 使用无痕窗口 或清除该站点 cookie（避免浏览器缓存的旧 session）
• 用户 admin，密码见 /root/.admin-openrc 中的 OS_PASSWORD

能正常登录并看到面板即部署完整成功。

6. ⭐ Horizon 故障排查

如果 Horizon 报"出错啦"，按以下流程排查。

6.1 日志路径速查

日志	路径
dashboard 错误日志	/var/log/httpd/openstack-dashboard-error_log（不是 dashboard_error.log）
dashboard 访问日志	/var/log/httpd/openstack-dashboard-access_log
httpd 主错误日志	/var/log/httpd/error_log
Horizon 应用日志目录	/var/log/horizon/（通常为空）

关键 ：openEuler 上 dashboard 的日志文件名为 openstack-dashboard-error_log（带 openstack- 前缀），别找错。

6.2 查看真实错误

# 实时跟踪，同时刷新浏览器页面
tail -f /var/log/httpd/openstack-dashboard-error_log

6.3 常见报错与修复

报错 1 ：horizon.exceptions.ServiceCatalogException: 无效的服务目录：network（或其他服务名）

原因：session 缓存了 endpoint 缺失时的 catalog。

修复：

systemctl restart memcached
systemctl restart httpd
# 浏览器退出登录或清 cookie，重新登录

报错 2 ：SESSION_ENGINE 相关错误

检查：

grep -A 10 "^CACHES" /etc/openstack-dashboard/local_settings

期望 LOCATION 指向本机 memcached：

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '192.168.0.193:11211',
    },
}

报错 3：无法访问，浏览器显示连接被拒

检查：

systemctl status httpd
ss -tlnp | grep :80
firewall-cmd --list-all | grep http
# 云控制台 ECS 安全组放通 80 端口

报错 4：能打开登录页，但登录后跳"出错啦"

检查：

# dashboard 关键配置
grep -E "OPENSTACK_HOST|ALLOWED_HOSTS|OPENSTACK_KEYSTONE_URL" \
  /etc/openstack-dashboard/local_settings

# 期望：
# ALLOWED_HOSTS = ['*'] 或 ["localhost", "*"]
# OPENSTACK_HOST = "192.168.0.193"
# OPENSTACK_KEYSTONE_URL = "http://%s:5000/v3" % OPENSTACK_HOST

# identity endpoint
openstack endpoint list | grep identity
# 三条 URL 都应是 http://192.168.0.193:5000/v3/

6.4 Horizon 修复通用流程

# 1. 补齐 endpoint
/root/fix-endpoints.sh

# 2. 清 session
systemctl restart memcached
systemctl restart httpd

# 3. 浏览器清 cookie / 无痕窗口访问
# 4. 若仍报错，看日志
tail -f /var/log/httpd/openstack-dashboard-error_log

7. 停止监控脚本

pkill -f watch-endpoints.sh

---

📋 服务端口 / URL 速查表

服务	类型	端口	URL 路径
keystone	identity	5000	/v3/
glance	image	9292	/
nova	compute	8774	/v2.1
placement	placement	8778	/
neutron	network	9696	/
cinder v3	volumev3	8776	/v3/%(project_id)s
cinder v2	volumev2	8776	/v2/%(tenant_id)s
swift	object-store	8080	/v1/AUTH_%(tenant_id)s
heat	orchestration	8004	/v1/%(tenant_id)s
heat-cfn	cloudformation	8000	/v1
gnocchi	metric	8041	/
aodh	alarming	8042	/
ceilometer	metering	8777	/
trove	database	8779	/v1.0/%(tenant_id)s
cyborg	accelerator	8777	/

URL 含 %(tenant_id)s / %(project_id)s 的，在 bash 中必须用单引号包裹。

---

📌 关键文件与日志路径速查

配置文件

文件/目录	用途
/usr/local/etc/oos/playbooks/	所有部署脚本
/usr/local/etc/oos/playbooks/roles/create_identity_service/tasks/main.yaml	endpoint 创建关键 role
/usr/local/etc/oos/inventory/	inventory 配置
/usr/local/etc/oos/oos.conf	oos 全局配置
/usr/local/etc/oos/key_pair/	SSH 密钥对
/root/oos/	oos 工作目录
/root/oos/key_pair/id_rsa	setup 使用的私钥
/root/.admin-openrc	OpenStack 环境变量
/etc/openstack-dashboard/local_settings	Horizon 配置

脚本

脚本	用途
/root/fix-endpoints.sh	endpoint 修复（v3）
/root/clean-dup-services.sh	重复 service 清理
/root/watch-endpoints.sh	部署期间并行监控

日志

日志	路径
dashboard 错误日志	/var/log/httpd/openstack-dashboard-error_log
dashboard 访问日志	/var/log/httpd/openstack-dashboard-access_log
httpd 主错误日志	/var/log/httpd/error_log
Keystone	/var/log/httpd/keystone.log / /var/log/keystone/keystone.log
Nova（各类）	/var/log/nova/nova-*.log
Neutron	/var/log/neutron/*.log
Cinder	/var/log/cinder/*.log
Glance	/var/log/glance/*.log
Heat	/var/log/heat/*.log
Swift	/var/log/swift/*.log
部署日志	/root/oos/deploy-*.log

---