Play with sunbeam again (by quqi99)

作者：张华 发表于：2026-06-05
版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明(http://blog.csdn.net/quqi99)

问题

之前玩过sunbeam, 都命令都过时了，Using sunbeam to deploy openstack - https://zhhuabj.blog.csdn.net/article/details/133840856

并且这次是将sunbean部署在公司测试机器上，但全是内网环境，需要通过proXX才能访问外网。诀窍是：

• 对于juju controller IP必须设置no_proxy

• no_proxy不仅得在/etc/environment里设置(设置了之后得重新登录ssh让它生效

• no_proxy还得设置在snap中: sudo snap set system proxy.no-proxy=$NO_PROXY && sudo snap get system proxy

• no_proxy更得设置在juju中：

  juju model-config -m localhost-localhost:controller no-proxy=NO_PROXY juju-no-proxy=NO_PROXY apt-no-proxy=NO_PROXY juju model-config -m localhost-localhost:openstack-machines no-proxy=NO_PROXY juju-no-proxy=NO_PROXY apt-no-proxy=NO_PROXY
  juju model-config -m localhost-localhost:controller | grep -i no-proxy
  juju model-config -m localhost-localhost:openstack-machines | grep -i no-proxy

步骤

下面是步骤：

#We don't have permission to create flavor with 'root-disk=100G mem=16G cores=8', so we're using a volume disk instead  (root-disk-source=volume)
#juju add-model sunbeam && juju add-machine --base ubuntu@24.04 --constraints "root-disk=100G mem=16G cores=8"
#ssh-keygen -t rsa -N '' -f ~/.ssh/id_rsa
openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey
openstack server create --image auto-sync/ubuntu-noble-24.04-amd64-server-20260518-disk1.img --flavor shared.xlarge --key-name mykey --network net_stg-reproducer-zhhuabj-ps7-psd-extra --boot-from-volume 100 sunbeam
SUNBEAM_VM_IP=10.159.26.62
ssh -i ~/.ssh/id_rsa ubuntu@$SUNBEAM_VM_IP -v
openstack volume create --size 10 --type Ceph_NVMe ceph1 && openstack server add volume sunbeam ceph1
openstack volume create --size 10 --type Ceph_NVMe ceph2 && openstack server add volume sunbeam ceph2
openstack volume create --size 10 --type Ceph_NVMe ceph3 && openstack server add volume sunbeam ceph3
#sudo parted /dev/sdb mklabel gpt && sudo parted /dev/sdb mkpart primary ext4 0% 100% && sudo mkfs.ext4 /dev/sdb1
#UUID=$(sudo blkid -s UUID -o value /dev/sdb1)
#sudo mkdir -p /data/sunbeam && sudo chown -R $USER:$USER /data/sunbeam
#echo "UUID=$UUID /data/sunbeam ext4 defaults 0 0" | sudo tee -a /etc/fstab
#sudo mount -a && df -h

juju ssh 0
#reset the env if it's necessary
sudo snap remove --purge openstack
sudo snap remove --purge juju
sudo snap remove --purge juju-db
sudo snap remove --purge kubectl
sudo /usr/sbin/remove-juju-services
sudo rm -rf /var/lib/juju
rm -rf ~/.local/share/juju
rm -rf ~/snap/juju/
rm -rf ~/snap/openstack
rm -rf ~/snap/openstack-hypervisor
rm -rf ~/snap/microstack/
rm -rf ~/snap/microk8s/
sudo snap remove --purge vault
sudo snap remove --purge microk8s
sudo snap remove --purge openstack-hypervisor
rm -rf $USER/.local/share/openstack/deployments.yaml
#it's best to restart, otherwise some calico NICs and namespaces may not be able to access
sudo init 6

#https://canonical-openstack.readthedocs-hosted.com/en/latest/tutorial/get-started-with-openstack/
#Configuring passwordless access to the sudo command for all terminal commands for the currently logged in user 
#echo "$USER NOPASSWORD=(ALL) ALL" | sudo tee /etc/sudoers.d/nopasswd && sudo chmod 440 /etc/sudoers.d/nopasswd

#这里设置no_proxy代理异常关键，假设controller ip是10.250.150.28/24, 那no_proxy里一定要添加10.250.150.0/24(设置对了之后使用juju show-user --debug能看到'proxy=direct'字眼
#controller ip只有在运行了（sunbeam prepare-node-script --bootstrap | bash -x && newgrp snap_daemon）之后才会有，需之后再修改no_proxy，但为简便第一次就加了10.250.150.0/24
#NO_PROXY也不应该随便加入别的，如若加入了cloud-images.ubuntu.com, 就会发生'no matching image found'
NO_PROXY="localhost,127.0.0.1,::1,10.149.95.128/25,172.24.0.0/24,172.22.0.0/24,172.20.0.0/24,172.26.0.0/24,172.28.0.0/24,10.159.26.128/25,10.159.25.128/25,10.159.26.0/25,10.250.150.0/24";
export HTTP_PROXY=http://egress.ps7.internal:3128 HTTPS_PROXY=http://egress.ps7.internal:3128 NO_PROXY="$NO_PROXY" http_proxy=http://egress.ps7.internal:3128 https_proxy=http://egress.ps7.internal:3128 no_proxy="$NO_PROXY"
JUJU_DATA=$HOME/.local/share/juju;
juju show-user --debug | sed -n '1,40p'
#ERROR LOG: api dial attempt failed: url=wss://252.46.0.1:17070/api address=wss://252.46.0.1:17070 ip=252.46.0.1:17070 attempt=1 proxy=http://egress.ps7.internal:3128 elapsed=2ms err=Forbidden
#RIGHT LOG: api dial attempt succeeded: url=wss://10.159.26.151:17070/api ip=10.159.26.151:17070 attempt=1 proxy=direct elapsed=6ms

#make sure you don't add 'cloud-images.ubuntu.com' in no_proxy
curl https://cloud-images.ubuntu.com/releases/ -o /dev/null

sudo snap install openstack --channel 2024.1/stable

#先将环境变量写到 /etc/environment
echo 'HTTP_PROXY=http://egress.ps7.internal:3128' |sudo tee -a /etc/environment
echo 'HTTPS_PROXY=http://egress.ps7.internal:3128' |sudo tee -a /etc/environment
echo "NO_PROXY=$NO_PROXY" | sudo tee -a /etc/environment
#退出ssh再重新登录ssh让它生效： env |grep -i proxy

rm -rf ~/.local/share/juju/controllers.yaml && sudo remove-juju-services
sunbeam prepare-node-script --bootstrap | bash -x && newgrp snap_daemon
#lxc exec juju-03be01-0 -- tail -f /var/log/syslog


#但上步创建的juju controller的IP是10.9.136.162，不是我们之前设置的NO_PROXY=10.250.150.0/24, 所以我们得添加10.9.136.0/24, 之后再重新登录ssh让它生效, 此时juju status将不在hang在那
juju status
juju show-user --debug | sed -n '1,40p'

#接着运行bootstrap
newgrp snap_daemon
sunbeam cluster bootstrap --accept-defaults --role control,compute,storage
tail -f ~/snap/openstack/common/logs/sunbeam*
tail -f ~/snap/openstack/common/etc/*/deploy-sunbeam-machine/terraform-apply-*.log
juju machines
ubuntu@sunbeam:~$ juju machines
Machine  State    Address       Inst id        Base          AZ       Message
0        started  10.9.136.162  juju-03be01-0  ubuntu@24.04  sunbeam  Running
ubuntu@sunbeam:~$ juju models
Controller: localhost-localhost
Model               Cloud/Region         Type    Status     Machines  Cores  Units  Access  Last connection
controller*         localhost/localhost  lxd     available         1      -  1      admin   just now
openstack-machines  close-swine/default  manual  available         1      8  -      admin   2 minutes ago

#在设置了/etc/envirnment之后(并重新登录ssh)运行了上步的bootstrap会自动设置juju proxy, 但snap proxy还是差一个no_proxy
ubuntu@sunbeam:~$ juju model-config -m localhost-localhost:controller | grep -i no-proxy
apt-no-proxy                                default     ""
juju-no-proxy                               controller  localhost,127.0.0.1,::1,10.149.95.128/25,172.24.0.0/24,172.22.0.0/24,172.20.0.0/24,172.26.0.0/24,172.28.0.0/24,10.159.26.128/25,10.159.25.128/25,10.159.26.0/25,10.250.150.0/24,10.159.26.62,10.9.136.1/24
no-proxy                                    controller  localhost,127.0.0.1,::1,10.149.95.128/25,172.24.0.0/24,172.22.0.0/24,172.20.0.0/24,172.26.0.0/24,172.28.0.0/24,10.159.26.128/25,10.159.25.128/25,10.159.26.0/25,10.250.150.0/24,10.159.26.62,10.9.136.1/24
ubuntu@sunbeam:~$ juju model-config -m localhost-localhost:openstack-machines | grep -i no-proxy
apt-no-proxy                                default  ""
juju-no-proxy                               model    10.149.95.128/25,10.9.136.0/24,172.22.0.0/24,10.159.26.0/25,10.152.183.0/24,172.26.0.0/24,172.20.0.0/24,10.159.25.128/25,172.24.0.0/24,10.159.26.128/25,172.28.0.0/24,localhost,127.0.0.1,10.1.0.0/16,::1,.svc.cluster.local,.svc
no-proxy                                    default  127.0.0.1,localhost,::1
ubuntu@sunbeam:~$ sudo snap get system proxy
Key          Value
proxy.http   http://egress.ps7.internal:3128
proxy.https  http://egress.ps7.internal:3128
proxy.store 

#也需设置snap no_proxy，不设置会报：ERROR unable to contact api server after 0 attempts: unknown error in bootstrap api connect: unable to connect to API: Forbidden）
sudo snap set system proxy.no-proxy=$NO_PROXY
sudo snap get system proxy
sunbeam cluster bootstrap --accept-defaults --role control,compute,storage
tail -f /home/ubuntu/snap/openstack/common/logs/*
sudo k8s kubectl get pods --all-namespaces
alias kubectl='sudo /snap/k8s/current/bin/kubectl'
source <(kubectl completion bash) && kubectl completion bash |sudo tee /etc/bash_completion.d/kubectl
sudo /snap/k8s/current/bin/kubectl get pods --all-namespaces
sudo /snap/k8s/current/bin/ctr namespaces list
sudo /snap/k8s/current/bin/ctr -n k8s.io images ls
sudo /snap/k8s/current/bin/ctr version

#现在改成直接用NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/16 , 然后报下列错, 再设置juju proxy就问题解决了：
#subprocess.CalledProcessError: Command '['/snap/openstack/1005/juju/bin/juju', 'migrate', 'localhost-localhost:admin/openstack-machines', 'sunbeam-controller']' returned non-zero exit status 1
juju model-config -m localhost-localhost:controller no-proxy=$NO_PROXY juju-no-proxy=$NO_PROXY apt-no-proxy=$NO_PROXY
juju model-config -m localhost-localhost:openstack-machines no-proxy=$NO_PROXY juju-no-proxy=$NO_PROXY apt-no-proxy=$NO_PROXY
juju model-config -m localhost-localhost:controller | grep -i no-proxy
juju model-config -m localhost-localhost:openstack-machines | grep -i no-proxy
sunbeam cluster bootstrap --accept-defaults --role control,compute,storage
sudo /snap/k8s/current/bin/ctr -n k8s.io images ls
sunbeam utils juju-login
sunbeam configure --accept-defaults --openrc demo-openrc
sunbeam launch ubuntu --name test
ssh -i /home/ubuntu/snap/openstack/1005/sunbeam ubuntu@172.16.2.44

sudo microceph.ceph status
for l in a b c; do
  loop_file="$(sudo mktemp -p /mnt XXXX.img)" 
  sudo truncate -s 1G "${loop_file}"
  loop_dev="$(sudo losetup --show -f "${loop_file}")"
  # the block-devices plug doesn't allow accessing /dev/loopX
  # devices so we make those same devices available under alternate
  # names (/dev/sdiY)
  minor="${loop_dev##/dev/loop}"
  sudo mknod -m 0660 "/dev/sdi${l}" b 7 "${minor}"
  sudo microceph disk add --wipe "/dev/sdi${l}"
done 
sudo microceph disk list

国内机器如何部署 sunbeam设想

国内家中机器由于特色跟国外公司机器一样也都属于内网机器，所以上面的办法一样适用。只是注意一点，国内做实验不要直接在物理机上做(因为有ipv6), 弄一个only ipv4的lxd容器做更好。费劲， 略。

curl -4 -k -I --max-time 15 https://cloud-images.ubuntu.com/releases/streams/v1/index.json
curl -6 -k -I --max-time 15 https://cloud-images.ubuntu.com/releases/streams/v1/index.json
curl -4 -k -I -x http://192.168.99.179:3128 --max-time 15 https://cloud-images.ubuntu.com/releases/streams/v1/index.json
curl -6 -k -I -x http://[2409:8a00:7881:20c0:a236:bcff:fe58:2bff]:3128 --max-time 15 https://cloud-images.ubuntu.com/releases/streams/v1/index.json