K8s 中部署 LNMP 架构 ECShop 电商
项目需求
在 K8s 中部署 LNMP 架构 ECShop 电商。
-
部署 NFS 服务器,提供共享目录 /ecshop
-
部署 NFS 动态卷制备,存储类型 ecshop
-
电商相关资源部署在 ecshop 命名空间
-
为 ecshop 命名空间设置 Quota:
- cpu: "4"
- memory: 8G
- requests.cpu: "4"
- requests.memory: 8G
- requests.storage: 500G
-
为 ecshop 命名空间设置 LimitRange:
- requests.cpu:默认:200m,最小:100m
- requests.memory:默认:256Mi,最小:256Mi
- cpu:默认:500m,最大:1
- memory:默认:512Mi,最大:2048Mi
-
部署 mysql 应用:
- 使用 StatefullSets 部署,基于 mysql:5.7 镜像,副本数1个
- 存储由 NFS 动态卷制备提供
- 使用 secret 存储mysql 用户 root 密码,ecshop应用用户名ecshop、密码Laoma@123、数据库ecshop
- 配置合适的Probe,进行健康监测
- 额外配置一个 svc,用于暴漏 mysql 应用
-
部署 php 应用:
- 使用 deployment 部署,基于 php:7.2-fpm-ecshop 镜像,副本数1个
- 配置合适的Probe,进行健康监测
- 使用 service-clusterIP 暴露
-
部署 nginx 应用:
- 使用 deployment 部署,基于 nginx:1.24 镜像,副本数1个
- 存储由 NFS 动态卷制备提供
- 使用 cm 存储 nginx 配置文件,
- 配置合适的Probe,进行健康监测
- 配置一个 svc,用于暴漏 ecshop 应用
-
部署网络组件:
- 部署 metallab
- 部署 ingress,为ingress-nginx控制器提供自谦名证书和私钥,用于后续的应用的https通信
-
配置 ingress 规则,通过 https://shop.laoma.cloud 访问商城。
-
部署 Metric Server,针对 php 和 nginx 应用,配置 HPA 规则,最小2个pod,最大5个pod。
-
部署 Kubernetes Dashboard 用于监视和管理 Kubernetes 集群,配置 ingress 规则,通过 https://dashboard.laoma.cloud 访问 Kubernetes Dashboard。
站点架构
Dashboard_GROUP
Nginx_GROUP
PHP_GROUP
MySQL_GROUP
right
NFS服务器
10.1.8.30 /ecshop
PV 动态卷制备
PVC-mysql
ConfigMap: nginx-config
PVC-ecshop
Secret: mysql账号密码
Secret: nginx-tls
通用TLS证书
Statefulset: mysql
Service: mysql HeadLess
Service: mysql ClusterIP
Deployment: php
Service: php ClusterIP
Deployment: nginx
Service: nginx ClusterIP
Deployment: kubernetes-dashboard
Service: kubernetes-dashboard ClusterIP
MetalLB 负载均衡
分配外部LB VIP
Ingress-Nginx Controller
集群入口网关
Ingress规则: shop.laoma.cloud
Ingress规则: dashboard.laoma.cloud
步骤 1:部署存储
1.1 部署 NFS 服务
NFS 服务器操作
bash
# 安装 NFS 服务
[root@master30 ~]# apt install -y nfs-kernel-server
# 创建共享目录
[root@master30 ~]# mkdir -p -m 777 /ecshop
# 配置 NFS 共享(地址已改为10.1.8.30,允许所有节点访问)
[root@master30 ~]# echo "/ecshop *(rw,sync,no_root_squash,no_all_squash)" >> /etc/exports
# 启动服务
[root@master30 ~]# systemctl restart nfs-server
NFS 客户端操作
bash
## 安装 NFS 客户端
[root@worker31-32 ~]# apt install -y nfs-common
## 验证共享
[root@worker31-32 ~]# showmount -e master30
Export list for master30:
/ecshop *
1.2 部署 NFS provisioner
NFS没有内置制备器,这里我们自定义外部分配器。
bash
# 部署 NFS provisioner到命名空间:kube-storage
[root@master30 ~]# kubectl create ns kube-storage
创建 nfs-rbac.yaml 文件,赋予 Provisioner 操作 PV/PVC 的权限。
yaml
[root@master30 ~]# cat > nfs-rbac.yaml <<'EOF'
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
namespace: kube-storage
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: nfs-client-provisioner-runner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: run-nfs-client-provisioner
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
namespace: kube-storage
roleRef:
kind: ClusterRole
name: nfs-client-provisioner-runner
apiGroup: rbac.authorization.k8s.io
EOF
bash
[root@master30 ~]# kubectl apply -f nfs-rbac.yaml
部署 NFS Provisioner 应用
yaml
[root@master30 ~]# cat > nfs-provisioner.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-client-provisioner
namespace: kube-storage
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
volumes:
- name: nfs-client-root
nfs:
server: 10.1.8.30 # 同上 NFS 服务器 IP
path: /ecshop # 核心:同步改为 /ecshop
containers:
- name: nfs-client-provisioner
image: registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifs
- name: NFS_SERVER
value: 10.1.8.30 # 替换为你的 NFS 服务器 IP
- name: NFS_PATH
value: /ecshop # 核心:共享路径改为 /ecshop
EOF
bash
# 部署 NFS Provisioner 应用
[root@master30 ~]# kubectl apply -f nfs-provisioner.yaml
# 查看 NFS Provisioner 应用
[root@master30 ~]# kubectl get deployments.apps -n kube-storage
NAME READY UP-TO-DATE AVAILABLE AGE
nfs-client-provisioner 1/1 1 1 1m
创建 NFS StorageClass
yaml
[root@master30 ~]# cat > ecshop-StorageClass.yaml <<'EOF'
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: ecshop
namespace: kube-storage
provisioner: fuseim.pri/ifs # 必须和 Provisioner 名称一致
parameters:
onDelete: retain # 删除 PVC 保留 NFS 数据
archiveOnDelete: "false"
# 按需设置回收策略
#reclaimPolicy: Retain
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: Immediate
EOF
bash
# 创建 StorageClass
[root@master30 ~]# kubectl apply -f ecshop-StorageClass.yaml
[root@master30 ~]# kubectl get sc ecshop
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
ecshop fuseim.pri/ifs Delete Immediate true 11s
步骤 2:配置命名空间
2.1 创建 Namespace
bash
[root@master30 ~]# kubectl create ns ecshop
[root@master30 ~]# kubectl config set-context --current --namespace ecshop
2.2 配置 Quota
bash
[root@master30 ~]# kubectl create quota ecshop \
--hard=cpu=4,memory=8G,requests.cpu=4,requests.memory=8G,requests.storage=500G
[root@master30 ~]# kubectl describe quota ecshop
Name: ecshop
Namespace: ecshop
Resource Used Hard
-------- ---- ----
cpu 0 4
memory 0 8G
requests.cpu 0 4
requests.memory 0 8G
requests.storage 0 500G
2.3 配置 LimitRange
yaml
[root@master30 ~]# cat > ecshop-LimitRange.yaml <<'EOF'
apiVersion: v1
kind: LimitRange
metadata:
name: ecshop
namespace: ecshop
spec:
limits:
- type: Container
max:
memory: 2048Mi
cpu: 1
min:
memory: 256Mi
cpu: 100m
default:
memory: 512Mi
cpu: 500m
defaultRequest:
memory: 256Mi
cpu: 200m
EOF
yaml
[root@master30 ~]# kubectl apply -f ecshop-LimitRange.yaml
[root@master30 ~]# kubectl describe limitranges ecshop
Name: ecshop
Namespace: ecshop
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Container cpu 100m 1 200m 500m -
Container memory 256Mi 2Gi 256Mi 512Mi -
步骤 3:部署 MySQL
3.1 创建 Secret
创建 Secret 存储 MySQL root 密码、ECShop 专用用户及密码、数据库,替代明文配置,提升安全性。
这里定义:
- MySQL root 密码:
Root@123(可按需修改) - ECShop 专用数据库用户:
ecshop - ECShop 专用用户密码:
ECShop@123(可按需修改) - ECShop 专用数据库:
ecshop
执行命令创建 Secret:
bash
# 创建 Secret 存储数据库敏感信息
[root@master30 ~]# kubectl create secret generic mysql \
--namespace ecshop \
--from-literal=mysql-root-password=Root@123 \
--from-literal=ecshop-db-user=ecshop \
--from-literal=ecshop-db-password=ECShop@123
# 验证 Secret
[root@master30 ~]# kubectl get secrets
NAME TYPE DATA AGE
mysql Opaque 3 10s
3.2 创建 Service
创建 mysql-service.yaml:
- mysql-headless,为后续 StatefulSet 管理的 Pod 提供固定名称
- mysql,为后续 StatefulSet 管理的 Pod 提供负载均衡
yaml
[root@master30 ~]# cat > mysql-service.yaml <<'EOF'
apiVersion: v1
kind: Service
metadata:
name: mysql-headless
namespace: ecshop
spec:
type: ClusterIP
selector:
app: mysql ## 匹配 StatefulSet 管理的 Pod 标签
clusterIP: None ## 无头服务,ClusterIP 设为 None
ports:
- port: 3306
targetPort: 3306
name: mysql-port ## 端口名称,便于识别
---
apiVersion: v1
kind: Service
metadata:
name: mysql
namespace: ecshop
spec:
type: ClusterIP
selector:
app: mysql ## 匹配 StatefulSet 管理的 Pod 标签
ports:
- port: 3306
targetPort: 3306
name: mysql-port ## 端口名称,便于识别
EOF
# 创建服务
[root@master30 ~]# kubectl apply -f mysql-service.yaml
# 没有 CLUSTER-IP
[root@master30 ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mysql ClusterIP 10.110.201.224 <none> 3306/TCP 4s
mysql-headless ClusterIP None <none> 3306/TCP 4s
3.3 部署 MySQL
创建 mysql-statefulset.yaml,副本数保持1个,Service为ClusterIP,无其他修改:
yaml
[root@master30 ~]# cat > mysql-statefulset.yaml <<'EOF'
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mysql
namespace: ecshop
spec:
serviceName: mysql-headless
replicas: 1
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:5.7
ports:
- containerPort: 3306
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mysql
key: mysql-root-password
- name: MYSQL_USER
valueFrom:
secretKeyRef:
name: mysql
key: ecshop-db-user
- name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
name: mysql
key: ecshop-db-password
- name: MYSQL_DATABASE
value: "ecshop"
volumeMounts:
- name: mysql-data ## 挂载存储卷,名称与 volumeClaimTemplates 一致
mountPath: /var/lib/mysql ## MySQL 数据存储路径
# 1. 启动探针:等待MySQL完全启动(适配初始化耗时)
startupProbe:
exec:
command:
- sh
- -c
- "mysql -uroot -p${MYSQL_ROOT_PASSWORD} -e 'SELECT 1;'"
initialDelaySeconds: 10 # 启动后10秒开始检测
periodSeconds: 5 # 每5秒检测一次
failureThreshold: 30 # 最多重试30次(总超时150秒)
timeoutSeconds: 3 # 单次检测超时3秒
# 2. 存活探针:检测MySQL是否存活,异常则重启
livenessProbe:
exec:
command:
- sh
- -c
- "mysqladmin ping -uroot -p${MYSQL_ROOT_PASSWORD}"
initialDelaySeconds: 30 # 启动后30秒开始检测(避开初始化)
periodSeconds: 10 # 每10秒检测一次
timeoutSeconds: 5 # 单次检测超时5秒
failureThreshold: 3 # 失败3次则判定异常
# 3. 就绪探针:检测MySQL是否可接收请求
readinessProbe:
exec:
command:
- sh
- -c
- "mysql -uroot -p${MYSQL_ROOT_PASSWORD} -e 'SHOW DATABASES;'"
initialDelaySeconds: 20 # 比存活探针早启动
periodSeconds: 5 # 每5秒检测一次
timeoutSeconds: 2 # 单次检测超时2秒
successThreshold: 1 # 成功1次则判定就绪
failureThreshold: 2 # 失败2次则移除服务端点
volumeClaimTemplates: ## PVC 模板,为每个 Pod 自动创建 PVC
- metadata:
name: mysql-data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "ecshop" ## 与 PV 的 storageClassName 一致
resources:
requests:
storage: 10Gi
EOF
bash
[root@master30 ~]# kubectl apply -f mysql-statefulset.yaml
[root@master30 ~]# kubectl get statefulsets.apps
NAME READY AGE
mysql 1/1 52s
[root@master30 ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-0 1/1 Running 0 78s
[root@master30 ~]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
mysql-data-mysql-0 Bound pvc-c40fcf08-0f68-481f-b266-29b03eca925b 10Gi RWO ecshop <unset> 3m2s
验证数据库
bash
[root@master30 ~]# apt install -y mysql-client
[root@master30 ~]# mysql -uecshop -pECShop@123 -h 10.110.201.224 -e 'show databases;'
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------------------+
| Database |
+--------------------+
| information_schema |
| ecshop |
+--------------------+
步骤 4:部署 PHP
4.1 创建 PVC
PHP 和 Nginx 使用相同存储存储,创建 ecshop-pvc.yaml:
yaml
[root@master30 ~]# cat > ecshop-pvc.yaml <<'EOF'
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ecshop
namespace: ecshop
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 10Gi
storageClassName: ecshop
EOF
执行创建:
bash
[root@master30 ~]# kubectl apply -f ecshop-pvc.yaml
[root@master30 ~]# kubectl get pvc ecshop
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
ecshop Bound pvc-3d01bfc0-2a4e-4a54-8e1e-fa5650e4507e 10Gi RWX ecshop <unset> 4s
4.2 部署 PHP
创建 php-deploy.yaml。
提示:镜像找老师获取。
yaml
[root@master30 ~]# cat > php-deploy.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: php
namespace: ecshop
spec:
replicas: 1
selector:
matchLabels:
app: php
template:
metadata:
labels:
app: php
spec:
containers:
- name: php
image: php:7.2-fpm-ecshop
ports:
- containerPort: 9000
# 1. 存活探针:验证 FPM 进程存活(核心)
livenessProbe:
tcpSocket:
port: 9000
initialDelaySeconds: 10 # 启动后10秒开始检测(避开初始化)
periodSeconds: 10 # 每10秒检测一次
timeoutSeconds: 5 # 单次检测超时5秒
failureThreshold: 3 # 失败3次重启容器
# 2. 就绪探针:验证 FPM 可接收请求(推荐用状态页)
readinessProbe:
tcpSocket:
port: 9000
initialDelaySeconds: 5 # 比存活探针早启动
periodSeconds: 5 # 每5秒检测一次
timeoutSeconds: 3 # 单次检测超时3秒
successThreshold: 1 # 成功1次即就绪
failureThreshold: 2 # 失败2次移除服务端点
volumeMounts:
## PHP 网站根目录
- name: ecshop-data
mountPath: /usr/share/nginx/html
volumes:
## PHP 网站根目录
- name: ecshop-data
persistentVolumeClaim:
claimName: ecshop
EOF
bash
[root@master30 ~]# kubectl apply -f php-deploy.yaml
[root@master30 ~]# kubectl get deploy php
NAME READY UP-TO-DATE AVAILABLE AGE
php 1/1 1 1 6s
4.3 创建 Service
创建 php-service.yaml:
bash
[root@master30 ~]# cat > php-service.yaml <<'EOF'
---
apiVersion: v1
kind: Service
metadata:
name: php
namespace: ecshop
spec:
type: ClusterIP
selector:
app: php
ports:
- port: 9000
targetPort: 9000
EOF
bash
[root@master30 ~]# kubectl apply -f php-service.yaml
[root@master30 ~]# kubectl get svc php
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
php ClusterIP 10.104.30.180 <none> 9000/TCP 21s
步骤 5:部署 Nginx
5.1 部署 ECShop 代码
bash
## 进入 NFS 服务器(IP:10.1.8.30),配置 ECShop 源码
[root@master30 ~]# unzip ECShop_V4.1.20_UTF8.zip
# 查看代码物理位置名称
[root@master30 ~]# kubectl get pvc ecshop
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
ecshop Bound pvc-3d01bfc0-2a4e-4a54-8e1e-fa5650e4507e 10Gi RWX ecshop <unset> 10m
[root@master30 ~]# cp -a ECShop_V4.1.20_UTF8_release20250416/source/ecshop/* /ecshop/ecshop-ecshop-pvc-3d01bfc0-2a4e-4a54-8e1e-fa5650e4507e/
[root@master30 ~]# chmod 777 -R /ecshop/ecshop-ecshop-pvc-3d01bfc0-2a4e-4a54-8e1e-fa5650e4507e/
# 存储服务器准备测试页面
[root@master30 ~]# echo Hello World From Nginx > /ecshop/ecshop-ecshop-pvc-3d01bfc0-2a4e-4a54-8e1e-fa5650e4507e/index.html
[root@master30 ~]# echo "<?php phpinfo(); ?>" > /ecshop/ecshop-ecshop-pvc-3d01bfc0-2a4e-4a54-8e1e-fa5650e4507e/phpinfo.php
[root@master30 ~]# cat > /ecshop/ecshop-ecshop-pvc-3d01bfc0-2a4e-4a54-8e1e-fa5650e4507e/php-mysql.php <<'EOF'
<?php
$link=mysqli_connect('mysql','ecshop','ECShop@123');
if($link)
echo "Connect Mysql Success !\n";
else
echo "Connect Mysql Failed !\n";
$link->close();
?>
EOF
5.2 创建 Nginx 配置
bash
[root@master30 ~]# cat > nginx-configmap.yaml <<'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx
namespace: ecshop
data:
nginx.conf: |
worker_processes auto;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
# ECShop 站点配置
server {
listen 80;
server_name shop.laoma.cloud; # 替换为实际域名
# 网站根目录
root /usr/share/nginx/html;
index index.php index.html;
# PHP 转发配置(指向ecshop命名空间的PHP Service)
location ~ \.php$ {
fastcgi_pass php.ecshop.svc.cluster.local:9000; # 跨命名空间访问,格式:服务名.命名空间.svc.cluster.local
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
fastcgi_param HTTPS on;
}
# 静态资源缓存
location ~* \.(jpg|jpeg|png|gif|css|js)$ {
expires 30d;
add_header Cache-Control "public, max-age=2592000";
}
}
}
EOF
bash
[root@master30 ~]# kubectl apply -f nginx-configmap.yaml
[root@master30 ~]# kubectl get cm nginx
NAME DATA AGE
nginx 1 27s
5.4 部署 Nginx
创建 nginx-deploy.yaml:
bash
[root@master30 ~]# cat > nginx-deploy.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: ecshop
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.24
ports:
- containerPort: 80
- containerPort: 443
# 1. 启动探针(可选):适配首次启动耗时
startupProbe:
tcpSocket:
port: 80
initialDelaySeconds: 3
periodSeconds: 2
failureThreshold: 10 # 总超时20秒,确保启动完成
# 2. 存活探针:验证 Nginx 核心服务存活
livenessProbe:
exec:
# 访问 Nginx 内置状态页(需提前配置),返回 200 则正常
command:
- sh
- -c
- "curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1/index.html | grep '200'"
initialDelaySeconds: 10 # 启动后10秒开始检测
periodSeconds: 10 # 每10秒检测一次
timeoutSeconds: 5 # 单次检测超时5秒
failureThreshold: 3 # 失败3次重启容器
# 3. 就绪探针:验证 Nginx 可接收外部请求
readinessProbe:
httpGet:
path: / # 业务根路径(可替换为你的实际业务路径)
port: 80
scheme: HTTP
initialDelaySeconds: 5 # 比存活探针早启动
periodSeconds: 5 # 每5秒检测一次
timeoutSeconds: 2 # 单次检测超时2秒
successThreshold: 1 # 成功1次即就绪
failureThreshold: 2 # 失败2次移除服务端点
volumeMounts:
- name: nginx
mountPath: /etc/nginx/nginx.conf ## 挂载 Nginx 配置
subPath: nginx.conf
- name: ecshop-data
mountPath: /usr/share/nginx/html ## 挂载 ECShop 代码目录
volumes:
- name: nginx
configMap:
name: nginx
- name: ecshop-data
persistentVolumeClaim:
claimName: ecshop
EOF
bash
[root@master30 ~]# kubectl apply -f nginx-deploy.yaml
[root@master30 ~]# kubectl get deploy nginx
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 21s
5.5 创建 Service
创建 ecshop-service.yaml:
bash
[root@master30 ~]# cat > nginx-service.yaml <<'EOF'
---
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: ecshop
spec:
type: ClusterIP
selector:
app: nginx
ports:
- port: 80
targetPort: 80
name: http
EOF
bash
[root@master30 ~]# kubectl apply -f nginx-service.yaml
[root@master30 ~]# kubectl get svc nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx ClusterIP 10.98.1.27 <none> 80/TCP,443/TCP 50s
5.6 测试 Nginx
bash
[root@master30 ~]# kubectl get svc nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx ClusterIP 10.98.1.27 <none> 80/TCP,443/TCP 50s
# 欢迎页面
[root@master30 ~]# curl http://10.98.1.27/index.html
Hello World From Nginx
# php页面
[root@master30 ~]# curl http://10.98.1.27/phpinfo.php -s|grep -o 'PHP Version 7.2'
PHP Version 7.2
# mysql连接
[root@master30 ~]# curl http://10.98.1.27/php-mysql.php
Connect Mysql Success !
步骤 6:部署 LoadBalancer
bash
[root@master30 ~]# tar -xf metallb-0.14.8.tar.gz
[root@master30 ~]# kubectl apply -f metallb-0.14.8/config/manifests/metallb-native.yaml
# 等待着所有pod正常运行再进行下一步
[root@master30 ~]# kubectl get pods -n metallb-system
NAME READY STATUS RESTARTS AGE
pod/controller-786f9df989-98bjh 1/1 Running 0 85s
pod/speaker-gthhx 1/1 Running 0 85s
pod/speaker-jwj25 1/1 Running 0 85s
pod/speaker-s5zvq 1/1 Running 0 85s
# 配置地址池
[root@master30 ~]# cat << 'EOF' > ippool.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: first-pool
namespace: metallb-system
spec:
addresses:
- 10.1.8.40-10.1.8.80
EOF
[root@master30 ~]# kubectl apply -f ippool.yaml
# 配置 lay2
[root@master30 ~]# cat << 'EOF' > L2.yaml
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: example
namespace: metallb-system
EOF
[root@master30 ~]# kubectl apply -f L2.yaml
步骤 7:部署 Ingress
bash
[root@master30 ~]# tar -xf ingress-nginx-controller-v1.11.2.tar.gz
[root@master30 ~]# kubectl apply -f ingress-nginx-controller-v1.11.2/deploy/static/provider/cloud/deploy.yaml
[root@master30 ~]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller LoadBalancer 10.107.255.83 10.1.8.40 80:30973/TCP,443:32344/TCP 14s
service/ingress-nginx-controller-admission ClusterIP 10.106.37.158 <none> 443/TCP 8m15s
准备证书
bash
## 生成自签名证书(替换域名)
[root@master30 ~]# openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
-keyout tls.key -out tls.crt \
-subj "/CN=*.laoma.cloud"
## 创建 Secret 存储证书
[root@master30 ~]# kubectl create secret tls ingress-nginx-tls \
--namespace ecshop \
--cert=tls.crt \
--key=tls.key
[root@master30 ~]# kubectl get secrets ingress-nginx-tls
NAME TYPE DATA AGE
ingress-nginx-tls kubernetes.io/tls 2 28s
步骤 8:初始化站点
8.1 配置 ingress 规则
bash
[root@master30 ~]# cat > shop-ingress.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-tls-shop
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- shop.laoma.cloud
secretName: ingress-nginx-tls
rules:
- host: shop.laoma.cloud
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx
port:
number: 80
EOF
bash
[root@master30 ~]# kubectl apply -f shop-ingress.yaml
[root@master30 ~]# kubectl describe ingress ingress-tls-shop
Name: ingress-tls-shop
Labels: <none>
Namespace: ecshop
Address:
Ingress Class: nginx
Default backend: <default>
TLS:
ingress-nginx-tls terminates shop.laoma.cloud
Rules:
Host Path Backends
---- ---- --------
shop.laoma.cloud
/ nginx:80 (10.224.113.130:80)
Annotations: nginx.ingress.kubernetes.io/force-ssl-redirect: true
nginx.ingress.kubernetes.io/forwarded-for-header: X-Forwarded-For
nginx.ingress.kubernetes.io/proxy-set-header: X-Forwarded-Proto $scheme
nginx.ingress.kubernetes.io/ssl-redirect: true
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 10s nginx-ingress-controller Scheduled for sync
8.2 访问站点
windows 访问,需提前配置域名解析。
访问 https://shop.laoma.cloud,点击高级继续。

点击继续前往shop.laoma.cloud(不安全)继续。

8.3 初始化站点
进入 ECShop 商城站点欢迎页面,勾选我已仔细阅读,并同意上述条款中的所有内容,点击下一步:配置安装环境继续。

点击下一步:配置系统继续。

根据提示填写相关内容,然后点击立即安装继续。

安装完成后访问主页https://shop.laoma.cloud/。

访问后台 https://shop.laoma.cloud/admin,使用ECShop账号登录。

输入用户名和密码,点击登录。

登录成功。

步骤 9:部署 Metric Server
bash
# 下载 Metrics-Server
[root@master30 ~]# wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.7.1/components.yaml
# 修改 Metrics-Server,不校验tls
[root@master30 ~]# sed -i '/metric-resolution/a\ - --kubelet-insecure-tls' components.yaml
# 下载镜像
[root@master30 ~]# grep image: components.yaml
image: registry.k8s.io/metrics-server/metrics-server:v0.7.1
# 部署 Metrics-Server
[root@master30 ~]# kubectl apply -f components.yaml
# 等30秒再查看使用情况
[root@master30 ~]# kubectl top nodes
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
master30.laoma.cloud 70m 3% 1709Mi 45%
worker31.laoma.cloud 22m 1% 1166Mi 30%
worker32.laoma.cloud 24m 1% 1069Mi 28
步骤 10:配置 HPA
bash
[root@master30 ~]# kubectl autoscale deployment php --min=2 --max=5 --cpu-percent=80
[root@master30 ~]# kubectl get deployments.apps php
NAME READY UP-TO-DATE AVAILABLE AGE
php 2/2 2 2 3h33m
[root@master30 ~]# kubectl autoscale deployment nginx --min=2 --max=5 --cpu-percent=80
[root@master30 ~]# kubectl get deployments.apps nginx
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 2/2 2 2 3h27m
[root@master30 ~]# kubectl describe quota ecshop
Name: ecshop
Namespace: ecshop
Resource Used Hard
-------- ---- ----
cpu 1 4
memory 1342177280 8G
requests.cpu 1 4
requests.memory 1342177280 8G
requests.storage 21474836480 500G
步骤 11:部署 Dashboard
11.1 部署 Dashboard
bash
# 下载资源yaml文件
[root@master30 ~]# wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.1/aio/deploy/recommended.yaml
# 查看images
[root@master30 ~]# grep image: recommended.yaml
image: kubernetesui/dashboard:v2.6.1
image: kubernetesui/metrics-scraper:v1.0.8
# 执行部署
[root@master30 ~]# kubectl apply -f recommended.yaml
11.2 配置 ingress 规则
bash
[root@master30 ~]# cat > dashborad-ingress.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-dashborad
namespace: kubernetes-dashboard
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
ingressClassName: nginx
rules:
- host: dashboard.laoma.cloud
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kubernetes-dashboard
port:
number: 443
EOF
bash
[root@master30 ~]# kubectl apply -f dashborad-ingress.yaml
[root@master30 ~]# kubectl describe ingress ingress-dashborad -n kubernetes-dashboard
Name: ingress-dashborad
Labels: <none>
Namespace: kubernetes-dashboard
Address: 10.1.8.40
Ingress Class: nginx
Default backend: <default>
TLS:
ingress-nginx-tls terminates dashboard.laoma.cloud
Rules:
Host Path Backends
---- ---- --------
dashboard.laoma.cloud
/ kubernetes-dashboard:443 (10.224.19.9:8443)
Annotations: nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/force-ssl-redirect: true
nginx.ingress.kubernetes.io/ssl-redirect: true
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 68s (x2 over 80s) nginx-ingress-controller Scheduled for sync
11.3 访问 dashboard
访问 https://dashboard.laoma.cloud

要访问 https://dashboard.laoma.cloud/,需要创建一个 ClusterRolebinding,将 ClusterRole cluster-admin绑定到 kubernetes-dashboard命名空间中服务账户kubernetes-dashboard。
bash
[root@master30 ~]# kubectl create clusterrolebinding cluster-admin-kubernetes-dashboard --clusterrole cluster-admin --serviceaccount kubernetes-dashboard:kubernetes-dashboard
# 创建 Bearer Token
[root@master30 ~]# kubectl -n kubernetes-dashboard create token kubernetes-dashboard
eyJhbGciOiJSUzI1NiIsImtpZCI6ImhLWEhaTWQtNGZTek5zUFUyUmNobzN1V2ZlOG1iWkwzSU5kZDZLbU1WUTAifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNjc2NDU4ODI3LCJpYXQiOjE2NzY0NTUyMjcsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsInNlcnZpY2VhY2NvdW50Ijp7Im5hbWUiOiJhZG1pbi11c2VyIiwidWlkIjoiYmViNGI3MWMtNmI5MS00ODk4LThiMTMtZWQwZWE3N2JiMmQ5In19LCJuYmYiOjE2NzY0NTUyMjcsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDphZG1pbi11c2VyIn0.kBwU3g3Euqj5jMKD9AYu-OQW2yc9pc2hb7uaDi2PO3AdkOBVMkaIiXnWEUC98hpFyXUWGSiaZWPsylJHXMpWz_X-J03nd7zZegUhPbzVxPh7vHyaOaILmqATlakAHMHbwcEQDygBQPBNbPNd-hD-rzaVNkmr2oARNGVRaX7fMNRkuvcbji0-s6IO5yotKkWaGXkIAtwmMTgtgcGnOpVsB97UYjPgWz6RMBL9ZCZIVXHxE_FIl_If7ewhObyaijZIbWeZ21job5PPvb1H1VAHOZoDXtFK_lZdIRIK6e6MYYU4mfYqTcOyiEG8c5SxCOaD4xchWGdzFcfZfn52VQJ7hQ
提示:Token默认有效期1小时,过期后自动删除。下一次登录需要生成新的Token。
