kubeedge安装并接入摄像头操作说明
一、说明
| 组件 | 版本 | 下载地址 | 说明 |
|---|---|---|---|
| k8s | 1.22.0 | github | 安装步骤可参考我之前的帖子 |
| kubeedge | 1.22.0 | github | 截至发帖最新 |
| 摄像头 | 淘宝(非广告，选类似的) | 支持RTSP、ONVIF协议，使用有线 |
且准备至少2台server（服务器或虚拟机）。
二、安装
1、安装keadm
在已安装的k8s集群中操作，首先获取压缩包：
https://github.com/kubeedge/kubeedge/releases/download/v1.22.0/keadm-v1.22.0-linux-amd64.tar.gz安装keadm：
tar -zxvf keadm-v1.22.0-linux-amd64.tar.gz
cp keadm /usr/bin/2、安装cloudcore
执行：
#配置docker代理,方便拉取相关镜像
keadm init --advertise-address=192.168.135.134 --kubeedge-version=v1.22.0 --kube-config=/root/.kube/config查看cloudcore安装情况：
root@ubuntu:/usr/local/src/github.com/kubeedge/kubeedge# kubectl get all -n kubeedge
NAME READY STATUS RESTARTS AGE
pod/cloud-iptables-manager-9p9lf 1/1 Running 0 87s
pod/cloudcore-55fcd4f859-f2brv 1/1 Running 0 87s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/cloudcore ClusterIP 10.96.3.145 <none> 10000/TCP,10001/UDP,10002/TCP,10003/TCP,10004/TCP 87s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/cloud-iptables-manager 1 1 1 1 1 <none> 87s
daemonset.apps/edge-eclipse-mosquitto 0 0 0 0 0 <none> 87s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/cloudcore 1/1 1 1 87s
NAME DESIRED CURRENT READY AGE
replicaset.apps/cloudcore-55fcd4f859 1 1 1 87s3、安装edgecore
准备一台服务器作为edge节点（还未纳管进k8s集群），且该节点需要安装containerd（docker不兼容）。
1）修改主机名
节点名称需要和cloudcore端不同，否则无法加入k8s集群。以edge-1为例：
hostnamectl set-hostname edge-1
vim /etc/hosts对/etc/hosts的内容进行修改：将127.0.1.1改为新的不在集群内重复的名称。
127.0.0.1 localhost
127.0.1.1 edge-1修改完后重启该节点：
reboot -h now2）安装containerd
（1）更新系统并安装依赖
sudo apt update && sudo apt upgrade -y
sudo apt install -y apt-transport-https ca-certificates curl gnupg lsb-release（2）添加Docker官方GPG密钥和仓库源*
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null（3）安装containerd*
sudo apt update
sudo apt install -y containerd.io（4）启动服务并设置开机自启
sudo systemctl enable --now containerd
sudo systemctl status containerd # 检查状态（应为active）（5）验证安装
sudo ctr version # 显示Client和Server版本即成功3）安装CNI
安装完containerd后，需要安装对应的cni插件。
（1）containerd中添加cni插件配置
sudo vi /etc/containerd/config.toml
# disabled_plugins = ["cri"]
[plugins."io.containerd.grpc.v1.cri".cni]
bin_dir = "/opt/cni/bin" # CNI插件二进制文件路径
conf_dir = "/etc/cni/net.d" # CNI配置文件路径（2）安装CNI插件
安装cni插件：
mkdir -p /opt/cni/bin
wget https://github.com/containernetworking/plugins/releases/download/v1.7.1/cni-plugins-linux-amd64-v1.7.1.tgz
sudo tar -C /opt/cni/bin -xzf cni-plugins-linux-amd64-v1.7.1.tgz配置cni：
/etc/cni/net.d/10-mynet.conflist的内容如下：
{
"cniVersion": "1.0.0",
"name": "mynet",
"plugins": [
{
"type": "bridge",
"bridge": "cni0",
"isGateway": true,
"ipMasq": true,
"ipam": {
"type": "host-local",
"ranges": [
[{"subnet": "10.88.0.0/16"}]
],
"routes": [{"dst": "0.0.0.0/0"}]
}
},
{
"type": "portmap",
"capabilities": {"portMappings": true}
}
]
}重启containerd：
sudo systemctl restart containerd
sudo systemctl status containerd # 确认服务状态正常4）、安装edgecore
配置containerd的代理，下载镜像：
ctr images pull docker.io/kubeedge/installation-package:v1.22.0在cloudcore节点中获取token：
root@ubuntu:/usr/local/src/github.com/kubeedge/kubeedge# keadm gettoken
e1c7098d412bf8df63e4f3f3bb44415147dda285164ef6cac2e19fc3aa73c71a.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3NjI3NjE5NTZ9.dVqpfvh6_vWhgZoQXft1D8hmki4prBv-AIGXwT2tX_A在edgecore节点执行：
keadm join --cloudcore-ipport=192.168.20.50:10000 --token=27a37ef16159f7d3be8fae95d588b79b3adaaf92727b72659eb89758c66ffda2.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1OTAyMTYwNzd9.JBj8LLYWXwbbvHKffJBpPd5CyxqapRQYDIXtFZErgYE --kubeedge-version=v1.17.05）、适配容器环境
修改端侧端口监听地址：
由于相关容器跑在容器环境中，故需要修改监听地址为该edgecore节点的地址：
vim /etc/kubeedge/config/edgecore.yaml将127.0.0.1改为：
...
tailoredKubeletConfig:
address: 192.168.135.135
...在/etc/hosts中加入域名：
192.168.135.135 localhost重启edgecore：
systemctl restart edgecore.service
#查看edgecore是否异常
systemctl status edgecore.service注意：当前情况下，calico的pod出现问题也没有影响。若需要解决calico的问题请安装edgemesh组件。
5）、卸载kubeedge
若之前步骤操作有问题，或者需要重新安装，则需要执行以下步骤：
keadm deprecated reset
sudo rm -rf /etc/kubeedge/
sudo rm -rf /var/lib/kubeedge/三、接入摄像头
1、确认摄像头信息
需要到摄像头官网下载对应操作app，来获取摄像头ip，以及对摄像头进行配置。
该部分摄像头的说明书中有介绍。
2、部署 ONVIF Mapper
1）搭建编译环境
kubeedge没有提供已编译好的镜像（估计是每个厂商的onvif协议理解和处理的不一样）。故需要自行编译。本人本地虚拟机（Ubuntu20.04）编译没成功。故找到官方提供的编译镜像。（估计Ubuntu18.0.4能成）。
下载编译镜像：
docker pull kubeedge/build-tools:1.22.9-ke1搭建编译环境：
#启动容器
docker run -it kubeedge/build-tools:1.22.9-ke1 bash
#找个目录下载onvif的代码
git clone git@github.com:kubeedge/mappers-go
#更新环境
apt-get update &&
apt-get install -y upx-ucl gcc-aarch64-linux-gnu libc6-dev-arm64-cross gcc-arm-linux-gnueabi libc6-dev-armel-cross libva-dev libva-drm2 libx11-dev libvdpau-dev libxext-dev libsdl1.2-dev libxcb1-dev libxau-dev libxdmcp-dev yasm
apt install pkg-config
apt install curl
curl -sLO https://ffmpeg.org/releases/ffmpeg-4.1.6.tar.bz2 &&
tar -jx --strip-components=1 -f ffmpeg-4.1.6.tar.bz2 &&
./configure && make &&
sudo make install2）编译onvif-mapper二进制
#编译onvif代码
cd /usr/local/src/mappers-go/mappers/onvif/cmd
#后续调试会遇到解析onvif协议的问题，需要修改对应的代码。
#主要有两点：1、解析摄像头返回的RTSP协议地方。2、获取RTSPurl的地方。
go build -o onvif .3）制作onvif-mapper镜像
编译好后，将二进制拷贝出来（根据/usr/local/src/mappers-go/mappers/onvif/中DockeFile的说明放入bin目录下）。
docker cp 容器号:/usr/local/src/mappers-go/mappers/onvif/cmd/onvif /usr/local/src/mappers-go/mappers/onvif/cmd/bin/然后拉取制作DockerFile要求的镜像：
docker pull ubuntu:18.04
docker run -it --rm ubuntu:18.04 bash
#在该镜像中安装如下依赖：
apt install libva2 libva-drm2 libva-x11-2 libvdpau1 libvdpau-dev
#再次提交镜像
docker commit 容器号 ubuntu:18.05修改DockerFile文件，将ubuntu:18.04改为ubuntu:18.05
然后制作镜像：
docker build -t onvif-mapper:v1.7-linux-amd64 .4）在端侧导入
docker save onvif-mapper:v1.7-linux-amd64>onvif-mapper.tar.gz
scp onvif-mapper.tar.gz root@端侧ip:/目录
ctr -n k8s.io images import onvif-mapper.tar.gz5）启动摄像头接入pod
修改配置文件
cd /usr/local/src/github.com/kubeedge/mappers-go/mappers/onvif/中部署文件deployment.yaml，本人重命名为：5-onvif-mapper-deployment.yaml，并修改内容为：
apiVersion: apps/v1
kind: Deployment
metadata:
name: onvif-mapper
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: onvif-mapper
template:
metadata:
labels:
app: onvif-mapper
spec:
# 使用节点选择器，确保 Pod 被调度到有摄像头的边缘节点
nodeSelector:
kubernetes.io/hostname: edge-1
containers:
- name: onvif-mapper
image: docker.io/library/onvif-mapper:v1.7-linux-amd64
volumeMounts:
- mountPath: /etc/secret
name: camera-secret
- name: device-profile-volume
mountPath: /opt/kubeedge/
readOnly: true
- name: frames-volume
mountPath: /home/btg/frames
- name: videos-volume
mountPath: /home/btg/videos
volumes:
- name: camera-secret
secret:
secretName: onvif-camera-secret
- name: device-profile-volume
configMap:
name: device-profile-config-01
items:
- key: deviceProfile.json
path: deviceProfile.json
- name: frames-volume
hostPath:
path: /home/btg/frames
type: DirectoryOrCreate
- name: videos-volume
hostPath:
path: /home/btg/videos
type: DirectoryOrCreate其对应的cm文件为：4-onvif-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: device-profile-config-01
namespace: default
data:
deviceProfile.json: |-
{
"deviceInstances": [
{
"id": "my-onvif-camera",
"name": "my-onvif-camera",
"protocol": "customized-protocol-camera01",
"model": "onvif-camera-model",
"twins": [
{
"propertyName": "SaveFrame",
"desired": {
"value": "true",
"metadata": {
"type": "boolean"
}
},
"reported": {
"value": "true",
"metadata": {
"type": "boolean"
}
}
},
{
"propertyName": "SaveVideo",
"desired": {
"value": "true",
"metadata": {
"type": "boolean"
}
},
"reported": {
"value": "true",
"metadata": {
"type": "boolean"
}
}
},
{
"propertyName": "reboot",
"desired": {
"value": "false",
"metadata": {
"timestamp": "1550049403598",
"type": "boolean"
}
},
"reported": {
"value": "false",
"metadata": {
"timestamp": "1550049403598",
"type": "boolean"
}
}
}
],
"propertyVisitors": [
{
"name": "reboot",
"propertyName": "reboot",
"modelName": "onvif-camera-model",
"collectCycle": 2000000000,
"protocol": "customized-protocol-camera01",
"visitorConfig": {
"protocolName": "onvif",
"configData": {
"method": "SystemReboot"
}
}
},
{
"name": "SaveFrame",
"propertyName": "SaveFrame",
"modelName": "onvif-camera-model",
"collectCycle": 10000000000,
"protocol": "customized-protocol-camera01",
"visitorConfig": {
"protocolName": "onvif",
"configData": {
"method": "SaveFrame",
"outputDir": "/home/btg/frames",
"format": "jpg",
"frameCount": 10,
"frameInterval": 20000000000
}
}
},
{
"name": "SaveVideo",
"propertyName": "SaveVideo",
"modelName": "onvif-camera-model",
"collectCycle": 30000000000,
"protocol": "customized-protocol-camera01",
"visitorConfig": {
"protocolName": "onvif",
"configData": {
"method": "SaveVideo",
"outputDir": "/home/btg/videos",
"format": "avi",
"frameCount": 2
}
}
}
]
}
],
"deviceModels": [
{
"name": "onvif-camera-model",
"properties": [
{
"name": "reboot",
"dataType": "boolean",
"description": "Reboot the camera",
"accessMode": "ReadWrite",
"defaultValue": false
},
{
"name": "SaveFrame",
"dataType": "boolean",
"description": "Save video frames",
"accessMode": "ReadWrite",
"defaultValue": true
},
{
"name": "SaveVideo",
"dataType": "boolean",
"description": "Save video stream",
"accessMode": "ReadWrite",
"defaultValue": true
}
]
}
],
"protocols": [
{
"name": "customized-protocol-camera01",
"protocol": "customized-protocol",
"protocolConfig": {
"protocolName": "onvif",
"configData": {
"url": "192.168.1.188:80",
"userName": "admin",
"password": "/etc/secret/password"
}
},
"protocolCommonConfig": null
}
]
}注意：容器现在还不支持mp4加解码。故需要配置为avi。
此外还需要将摄像头的密码也进行配置，放入1-onvif-secret.yaml文件
# onvif-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: onvif-camera-secret
namespace: default
type: Opaque
data:
password: MTIzNDU2Cg==其中password使用base64编码获得。
echo 123456 |base64启动相关pod：
kubectl apply -f 1-onvif-secret.yaml
kubectl apply -f 4-onvif-cm.yaml
kubectl apply -f 5-onvif-mapper-deployment.yaml此时从k8s中看到的相关pod如下：
default onvif-mapper-5f666d8769-2md8c 1/1 Running
kubeedge cloud-iptables-manager-dx55x 1/1 Running
kubeedge cloudcore-55fcd4f859-pffmv 1/1 Running
kubeedge edge-eclipse-mosquitto-bdnz5 1/1 Running6）验证结果
此时到边缘节点中查看相应的目录：
ll /home/btg/frames/
ll /home/btg/videos/注意：/home/btg/videos中保存的是视频帧，每秒数量在摄像头界面进行配置。
四、附录
1、查看日志
edgecore日志：
journalctl -u edgecore -n 50 -f2、测试rtsp是否通
# 使用ffmpeg测试RTSP流,具体rul写法参见摄像头的说明书
ffmpeg -i "rtsp://admin:123456@192.168.1.188:554/cho01.264" -t 10 -f null -
# 使用vlc测试
vlc "rtsp://admin:123456@192.168.1.188:554"3、控制摄像头运动
http://192.168.1.188:80/digest/frmPTZControl
post方式，发送的body为：
摄像头向左转动：
{"Type":1,"Dev":1,"Ch":1,"Data":{"Cmd":23,"IsStop":0,"Speed":5}}
摄像头向右转动：
{"Type":1,"Dev":1,"Ch":1,"Data":{"Cmd":24,"IsStop":1,"Speed":5}}
摄像头向上转动：
{"Type":1,"Dev":1,"Ch":1,"Data":{"Cmd":21,"IsStop":1,"Speed":5}}
摄像头向下转动：
{"Type":1,"Dev":1,"Ch":1,"Data":{"Cmd":22,"IsStop":1,"Speed":5}}
返回的body为：
{"Result":0,"Data":{"StatusCode":"Operation Ok"}}
curl -X POST \
http://192.168.1.188:80/digest/frmPTZControl \
-H "Content-Type: application/json" \
--digest -u admin:123456 \
-d '{"Type":1,"Dev":1,"Ch":1,"Data":{"Cmd":24,"IsStop":0,"Speed":5}}'