目录
[2. K8S 1.29版本 部署ingress-nginx](#2. K8S 1.29版本 部署ingress-nginx)
[1.kubectl 如何强制删除 Pod、Namespace 资源](#1.kubectl 如何强制删除 Pod、Namespace 资源)
一、实验
1.环境
（1）主机
表1 主机
|--------|--------------|--------|----------------|------------|
| 主机 | 架构 | 版本 | IP | 备注 |
| master | K8S master节点 | 1.29.0 | 192.168.204.8 | |
| node1 | K8S node节点 | 1.29.0 | 192.168.204.9 | |
| node2 | K8S node节点 | 1.29.0 | 192.168.204.10 | 已部署Kuboard |
（2）master节点查看集群
bash
1）查看node
kubectl get node
2)查看node详细信息
kubectl get node -o wide
（3）查看pod
bash
[root@master ~]# kubectl get pod -A
(4) 访问Kuboard
bash
http://192.168.204.10:30080/kuboard/cluster
查看节点
2. K8S 1.29版本 部署ingress-nginx
（1）查阅
bash
https://github.com/kubernetes/ingress-nginx（2）版本支持图
（3）查看K8S版本
bash
[root@master ~]# kubectl version
（4）选择ingress-nginx版本
版本为1.29.0需使用ingress-nginx v1.10.0版本
下载
bash
https://github.com/kubernetes/ingress-nginx/blob/controller-v1.10.0/deploy/static/provider/cloud/deploy.yaml
（5）查看配置文件
bash
[root@master ~]# vim deploy.yaml 
bash
1 apiVersion: v1
2 kind: Namespace
3 metadata:
4 labels:
5 app.kubernetes.io/instance: ingress-nginx
6 app.kubernetes.io/name: ingress-nginx
7 name: ingress-nginx
8 ---
9 apiVersion: v1
10 automountServiceAccountToken: true
11 kind: ServiceAccount
12 metadata:
13 labels:
14 app.kubernetes.io/component: controller
15 app.kubernetes.io/instance: ingress-nginx
16 app.kubernetes.io/name: ingress-nginx
17 app.kubernetes.io/part-of: ingress-nginx
18 app.kubernetes.io/version: 1.10.0
19 name: ingress-nginx
20 namespace: ingress-nginx
21 ---
22 apiVersion: v1
23 kind: ServiceAccount
24 metadata:
25 labels:
26 app.kubernetes.io/component: admission-webhook
27 app.kubernetes.io/instance: ingress-nginx
28 app.kubernetes.io/name: ingress-nginx
29 app.kubernetes.io/part-of: ingress-nginx
30 app.kubernetes.io/version: 1.10.0
31 name: ingress-nginx-admission
32 namespace: ingress-nginx
33 ---
34 apiVersion: rbac.authorization.k8s.io/v1
35 kind: Role
36 metadata:
37 labels:
38 app.kubernetes.io/component: controller
39 app.kubernetes.io/instance: ingress-nginx
40 app.kubernetes.io/name: ingress-nginx
41 app.kubernetes.io/part-of: ingress-nginx
42 app.kubernetes.io/version: 1.10.0
43 name: ingress-nginx
44 namespace: ingress-nginx
45 rules:
46 - apiGroups:
47 - ""
48 resources:
49 - namespaces
50 verbs:
51 - get
52 - apiGroups:
53 - ""
54 resources:
55 - configmaps
56 - pods
57 - secrets
58 - endpoints
59 verbs:
60 - get
61 - list
62 - watch
63 - apiGroups:
64 - ""
65 resources:
66 - services
67 verbs:
68 - get
69 - list
70 - watch
71 - apiGroups:
72 - networking.k8s.io
73 resources:
74 - ingresses
75 verbs:
76 - get
77 - list
78 - watch
79 - apiGroups:
80 - networking.k8s.io
81 resources:
82 - ingresses/status
83 verbs:
84 - update
85 - apiGroups:
86 - networking.k8s.io
87 resources:
88 - ingressclasses
89 verbs:
90 - get
91 - list
92 - watch
93 - apiGroups:
94 - coordination.k8s.io
95 resourceNames:
96 - ingress-nginx-leader
97 resources:
98 - leases
99 verbs:
100 - get
101 - update
102 - apiGroups:
103 - coordination.k8s.io
104 resources:
105 - leases
106 verbs:
107 - create
108 - apiGroups:
109 - ""
110 resources:
111 - events
112 verbs:
113 - create
114 - patch
115 - apiGroups:
116 - discovery.k8s.io
117 resources:
118 - endpointslices
119 verbs:
120 - list
121 - watch
122 - get
123 ---
124 apiVersion: rbac.authorization.k8s.io/v1
125 kind: Role
126 metadata:
127 labels:
128 app.kubernetes.io/component: admission-webhook
129 app.kubernetes.io/instance: ingress-nginx
130 app.kubernetes.io/name: ingress-nginx
131 app.kubernetes.io/part-of: ingress-nginx
132 app.kubernetes.io/version: 1.10.0
133 name: ingress-nginx-admission
134 namespace: ingress-nginx
135 rules:
136 - apiGroups:
137 - ""
138 resources:
139 - secrets
140 verbs:
141 - get
142 - create
143 ---
144 apiVersion: rbac.authorization.k8s.io/v1
145 kind: ClusterRole
146 metadata:
147 labels:
148 app.kubernetes.io/instance: ingress-nginx
149 app.kubernetes.io/name: ingress-nginx
150 app.kubernetes.io/part-of: ingress-nginx
151 app.kubernetes.io/version: 1.10.0
152 name: ingress-nginx
153 rules:
154 - apiGroups:
155 - ""
156 resources:
157 - configmaps
158 - endpoints
159 - nodes
160 - pods
161 - secrets
162 - namespaces
163 verbs:
164 - list
165 - watch
166 - apiGroups:
167 - coordination.k8s.io
168 resources:
169 - leases
170 verbs:
171 - list
172 - watch
173 - apiGroups:
174 - ""
175 resources:
176 - nodes
177 verbs:
178 - get
179 - apiGroups:
180 - ""
181 resources:
182 - services
183 verbs:
184 - get
185 - list
186 - watch
187 - apiGroups:
188 - networking.k8s.io
189 resources:
190 - ingresses
191 verbs:
192 - get
193 - list
194 - watch
195 - apiGroups:
196 - ""
197 resources:
198 - events
199 verbs:
200 - create
201 - patch
202 - apiGroups:
203 - networking.k8s.io
204 resources:
205 - ingresses/status
206 verbs:
207 - update
208 - apiGroups:
209 - networking.k8s.io
210 resources:
211 - ingressclasses
212 verbs:
213 - get
214 - list
215 - watch
216 - apiGroups:
217 - discovery.k8s.io
218 resources:
219 - endpointslices
220 verbs:
221 - list
222 - watch
223 - get
224 ---
225 apiVersion: rbac.authorization.k8s.io/v1
226 kind: ClusterRole
227 metadata:
228 labels:
229 app.kubernetes.io/component: admission-webhook
230 app.kubernetes.io/instance: ingress-nginx
231 app.kubernetes.io/name: ingress-nginx
232 app.kubernetes.io/part-of: ingress-nginx
233 app.kubernetes.io/version: 1.10.0
234 name: ingress-nginx-admission
235 rules:
236 - apiGroups:
237 - admissionregistration.k8s.io
238 resources:
239 - validatingwebhookconfigurations
240 verbs:
241 - get
242 - update
243 ---
244 apiVersion: rbac.authorization.k8s.io/v1
245 kind: RoleBinding
246 metadata:
247 labels:
248 app.kubernetes.io/component: controller
249 app.kubernetes.io/instance: ingress-nginx
250 app.kubernetes.io/name: ingress-nginx
251 app.kubernetes.io/part-of: ingress-nginx
252 app.kubernetes.io/version: 1.10.0
253 name: ingress-nginx
254 namespace: ingress-nginx
255 roleRef:
256 apiGroup: rbac.authorization.k8s.io
257 kind: Role
258 name: ingress-nginx
259 subjects:
260 - kind: ServiceAccount
261 name: ingress-nginx
262 namespace: ingress-nginx
263 ---
264 apiVersion: rbac.authorization.k8s.io/v1
265 kind: RoleBinding
266 metadata:
267 labels:
268 app.kubernetes.io/component: admission-webhook
269 app.kubernetes.io/instance: ingress-nginx
270 app.kubernetes.io/name: ingress-nginx
271 app.kubernetes.io/part-of: ingress-nginx
272 app.kubernetes.io/version: 1.10.0
273 name: ingress-nginx-admission
274 namespace: ingress-nginx
275 roleRef:
276 apiGroup: rbac.authorization.k8s.io
277 kind: Role
278 name: ingress-nginx-admission
279 subjects:
280 - kind: ServiceAccount
281 name: ingress-nginx-admission
282 namespace: ingress-nginx
283 ---
284 apiVersion: rbac.authorization.k8s.io/v1
285 kind: ClusterRoleBinding
286 metadata:
287 labels:
288 app.kubernetes.io/instance: ingress-nginx
289 app.kubernetes.io/name: ingress-nginx
290 app.kubernetes.io/part-of: ingress-nginx
291 app.kubernetes.io/version: 1.10.0
292 name: ingress-nginx
293 roleRef:
294 apiGroup: rbac.authorization.k8s.io
295 kind: ClusterRole
296 name: ingress-nginx
297 subjects:
298 - kind: ServiceAccount
299 name: ingress-nginx
300 namespace: ingress-nginx
301 ---
302 apiVersion: rbac.authorization.k8s.io/v1
303 kind: ClusterRoleBinding
304 metadata:
305 labels:
306 app.kubernetes.io/component: admission-webhook
307 app.kubernetes.io/instance: ingress-nginx
308 app.kubernetes.io/name: ingress-nginx
309 app.kubernetes.io/part-of: ingress-nginx
310 app.kubernetes.io/version: 1.10.0
311 name: ingress-nginx-admission
312 roleRef:
313 apiGroup: rbac.authorization.k8s.io
314 kind: ClusterRole
315 name: ingress-nginx-admission
316 subjects:
317 - kind: ServiceAccount
318 name: ingress-nginx-admission
319 namespace: ingress-nginx
320 ---
321 apiVersion: v1
322 data:
323 allow-snippet-annotations: "false"
324 kind: ConfigMap
325 metadata:
326 labels:
327 app.kubernetes.io/component: controller
328 app.kubernetes.io/instance: ingress-nginx
329 app.kubernetes.io/name: ingress-nginx
330 app.kubernetes.io/part-of: ingress-nginx
331 app.kubernetes.io/version: 1.10.0
332 name: ingress-nginx-controller
333 namespace: ingress-nginx
334 ---
335 apiVersion: v1
336 kind: Service
337 metadata:
338 labels:
339 app.kubernetes.io/component: controller
340 app.kubernetes.io/instance: ingress-nginx
341 app.kubernetes.io/name: ingress-nginx
342 app.kubernetes.io/part-of: ingress-nginx
343 app.kubernetes.io/version: 1.10.0
344 name: ingress-nginx-controller
345 namespace: ingress-nginx
346 spec:
347 externalTrafficPolicy: Local
348 ipFamilies:
349 - IPv4
350 ipFamilyPolicy: SingleStack
351 ports:
352 - appProtocol: http
353 name: http
354 port: 80
355 protocol: TCP
356 targetPort: http
357 - appProtocol: https
358 name: https
359 port: 443
360 protocol: TCP
361 targetPort: https
362 selector:
363 app.kubernetes.io/component: controller
364 app.kubernetes.io/instance: ingress-nginx
365 app.kubernetes.io/name: ingress-nginx
366 type: LoadBalancer
367 ---
368 apiVersion: v1
369 kind: Service
370 metadata:
371 labels:
372 app.kubernetes.io/component: controller
373 app.kubernetes.io/instance: ingress-nginx
374 app.kubernetes.io/name: ingress-nginx
375 app.kubernetes.io/part-of: ingress-nginx
376 app.kubernetes.io/version: 1.10.0
377 name: ingress-nginx-controller-admission
378 namespace: ingress-nginx
379 spec:
380 ports:
381 - appProtocol: https
382 name: https-webhook
383 port: 443
384 targetPort: webhook
385 selector:
386 app.kubernetes.io/component: controller
387 app.kubernetes.io/instance: ingress-nginx
388 app.kubernetes.io/name: ingress-nginx
389 type: ClusterIP
390 ---
391 apiVersion: apps/v1
392 kind: Deployment
393 metadata:
394 labels:
395 app.kubernetes.io/component: controller
396 app.kubernetes.io/instance: ingress-nginx
397 app.kubernetes.io/name: ingress-nginx
398 app.kubernetes.io/part-of: ingress-nginx
399 app.kubernetes.io/version: 1.10.0
400 name: ingress-nginx-controller
401 namespace: ingress-nginx
402 spec:
403 minReadySeconds: 0
404 revisionHistoryLimit: 10
405 selector:
406 matchLabels:
407 app.kubernetes.io/component: controller
408 app.kubernetes.io/instance: ingress-nginx
409 app.kubernetes.io/name: ingress-nginx
410 strategy:
411 rollingUpdate:
412 maxUnavailable: 1
413 type: RollingUpdate
414 template:
415 metadata:
416 labels:
417 app.kubernetes.io/component: controller
418 app.kubernetes.io/instance: ingress-nginx
419 app.kubernetes.io/name: ingress-nginx
420 app.kubernetes.io/part-of: ingress-nginx
421 app.kubernetes.io/version: 1.10.0
422 spec:
423 containers:
424 - args:
425 - /nginx-ingress-controller
426 - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
427 - --election-id=ingress-nginx-leader
428 - --controller-class=k8s.io/ingress-nginx
429 - --ingress-class=nginx
430 - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
431 - --validating-webhook=:8443
432 - --validating-webhook-certificate=/usr/local/certificates/cert
433 - --validating-webhook-key=/usr/local/certificates/key
434 - --enable-metrics=false
435 env:
436 - name: POD_NAME
437 valueFrom:
438 fieldRef:
439 fieldPath: metadata.name
440 - name: POD_NAMESPACE
441 valueFrom:
442 fieldRef:
443 fieldPath: metadata.namespace
444 - name: LD_PRELOAD
445 value: /usr/local/lib/libmimalloc.so
446 image: registry.k8s.io/ingress-nginx/controller:v1.10.0@sha256:42b3f0e5d0846876b1791cd3afeb5f1cbbe4259d6f35651dcc1b5c980925379c
447 imagePullPolicy: IfNotPresent
448 lifecycle:
449 preStop:
450 exec:
451 command:
452 - /wait-shutdown
453 livenessProbe:
454 failureThreshold: 5
455 httpGet:
456 path: /healthz
457 port: 10254
458 scheme: HTTP
459 initialDelaySeconds: 10
460 periodSeconds: 10
461 successThreshold: 1
462 timeoutSeconds: 1
463 name: controller
464 ports:
465 - containerPort: 80
466 name: http
467 protocol: TCP
468 - containerPort: 443
469 name: https
470 protocol: TCP
471 - containerPort: 8443
472 name: webhook
473 protocol: TCP
474 readinessProbe:
475 failureThreshold: 3
476 httpGet:
477 path: /healthz
478 port: 10254
479 scheme: HTTP
480 initialDelaySeconds: 10
481 periodSeconds: 10
482 successThreshold: 1
483 timeoutSeconds: 1
484 resources:
485 requests:
486 cpu: 100m
487 memory: 90Mi
488 securityContext:
489 allowPrivilegeEscalation: false
490 capabilities:
491 add:
492 - NET_BIND_SERVICE
493 drop:
494 - ALL
495 readOnlyRootFilesystem: false
496 runAsNonRoot: true
497 runAsUser: 101
498 seccompProfile:
499 type: RuntimeDefault
500 volumeMounts:
501 - mountPath: /usr/local/certificates/
502 name: webhook-cert
503 readOnly: true
504 dnsPolicy: ClusterFirst
505 nodeSelector:
506 kubernetes.io/os: linux
507 serviceAccountName: ingress-nginx
508 terminationGracePeriodSeconds: 300
509 volumes:
510 - name: webhook-cert
511 secret:
512 secretName: ingress-nginx-admission
513 ---
514 apiVersion: batch/v1
515 kind: Job
516 metadata:
517 labels:
518 app.kubernetes.io/component: admission-webhook
519 app.kubernetes.io/instance: ingress-nginx
520 app.kubernetes.io/name: ingress-nginx
521 app.kubernetes.io/part-of: ingress-nginx
522 app.kubernetes.io/version: 1.10.0
523 name: ingress-nginx-admission-create
524 namespace: ingress-nginx
525 spec:
526 template:
527 metadata:
528 labels:
529 app.kubernetes.io/component: admission-webhook
530 app.kubernetes.io/instance: ingress-nginx
531 app.kubernetes.io/name: ingress-nginx
532 app.kubernetes.io/part-of: ingress-nginx
533 app.kubernetes.io/version: 1.10.0
534 name: ingress-nginx-admission-create
535 spec:
536 containers:
537 - args:
538 - create
539 - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
540 - --namespace=$(POD_NAMESPACE)
541 - --secret-name=ingress-nginx-admission
542 env:
543 - name: POD_NAMESPACE
544 valueFrom:
545 fieldRef:
546 fieldPath: metadata.namespace
547 image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0@sha256:44d1d0e9f19c63f58b380c5fddaca7cf22c7cee564adeff365225a5df5ef3334
548 imagePullPolicy: IfNotPresent
549 name: create
550 securityContext:
551 allowPrivilegeEscalation: false
552 capabilities:
553 drop:
554 - ALL
555 readOnlyRootFilesystem: true
556 runAsNonRoot: true
557 runAsUser: 65532
558 seccompProfile:
559 type: RuntimeDefault
560 nodeSelector:
561 kubernetes.io/os: linux
562 restartPolicy: OnFailure
563 serviceAccountName: ingress-nginx-admission
564 ---
565 apiVersion: batch/v1
566 kind: Job
567 metadata:
568 labels:
569 app.kubernetes.io/component: admission-webhook
570 app.kubernetes.io/instance: ingress-nginx
571 app.kubernetes.io/name: ingress-nginx
572 app.kubernetes.io/part-of: ingress-nginx
573 app.kubernetes.io/version: 1.10.0
574 name: ingress-nginx-admission-patch
575 namespace: ingress-nginx
576 spec:
577 template:
578 metadata:
579 labels:
580 app.kubernetes.io/component: admission-webhook
581 app.kubernetes.io/instance: ingress-nginx
582 app.kubernetes.io/name: ingress-nginx
583 app.kubernetes.io/part-of: ingress-nginx
584 app.kubernetes.io/version: 1.10.0
585 name: ingress-nginx-admission-patch
586 spec:
587 containers:
588 - args:
589 - patch
590 - --webhook-name=ingress-nginx-admission
591 - --namespace=$(POD_NAMESPACE)
592 - --patch-mutating=false
593 - --secret-name=ingress-nginx-admission
594 - --patch-failure-policy=Fail
595 env:
596 - name: POD_NAMESPACE
597 valueFrom:
598 fieldRef:
599 fieldPath: metadata.namespace
600 image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0@sha256:44d1d0e9f19c63f58b380c5fddaca7cf22c7cee564adeff365225a5df5ef3334
601 imagePullPolicy: IfNotPresent
602 name: patch
603 securityContext:
604 allowPrivilegeEscalation: false
605 capabilities:
606 drop:
607 - ALL
608 readOnlyRootFilesystem: true
609 runAsNonRoot: true
610 runAsUser: 65532
611 seccompProfile:
612 type: RuntimeDefault
613 nodeSelector:
614 kubernetes.io/os: linux
615 restartPolicy: OnFailure
616 serviceAccountName: ingress-nginx-admission
617 ---
618 apiVersion: networking.k8s.io/v1
619 kind: IngressClass
620 metadata:
621 labels:
622 app.kubernetes.io/component: controller
623 app.kubernetes.io/instance: ingress-nginx
624 app.kubernetes.io/name: ingress-nginx
625 app.kubernetes.io/part-of: ingress-nginx
626 app.kubernetes.io/version: 1.10.0
627 name: nginx
628 spec:
629 controller: k8s.io/ingress-nginx
630 ---
631 apiVersion: admissionregistration.k8s.io/v1
632 kind: ValidatingWebhookConfiguration
633 metadata:
634 labels:
635 app.kubernetes.io/component: admission-webhook
636 app.kubernetes.io/instance: ingress-nginx
637 app.kubernetes.io/name: ingress-nginx
638 app.kubernetes.io/part-of: ingress-nginx
639 app.kubernetes.io/version: 1.10.0
640 name: ingress-nginx-admission
641 webhooks:
642 - admissionReviewVersions:
643 - v1
644 clientConfig:
645 service:
646 name: ingress-nginx-controller-admission
647 namespace: ingress-nginx
648 path: /networking/v1/ingresses
649 failurePolicy: Fail
650 matchPolicy: Equivalent
651 name: validate.nginx.ingress.kubernetes.io
652 rules:
653 - apiGroups:
654 - networking.k8s.io
655 apiVersions:
656 - v1
657 operations:
658 - CREATE
659 - UPDATE
660 resources:
661 - ingresses
662 sideEffects: None（5）替换镜像源
注意：k8s.io 的image需要修改为阿里云的镜像或其他指定镜像源
参考其他镜像：
bash
https://hub.docker.com/u/anjia0532更换镜像源
bash
docker pull anjia0532/google-containers.ingress-nginx.controller:v1.10.0
bash
docker pull anjia0532/google-containers.ingress-nginx.kube-webhook-certgen:v1.4.0
这里已重新打标签上传个人docker hub
（6） 查看已拉取镜像
bash
[root@node1 ~]# docker images | grep ingress-nginx
（7）生成资源创建ingress-nginx
bash
[root@master ~]# kubectl apply -f deploy.yaml
(8)查看启动的pod和service
bash
[root@master ~]# kubectl get pod,svc -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/ingress-nginx-admission-create-dgzgd 0/1 Completed 0 2m2s 10.244.166.139 node1 <none> <none>
pod/ingress-nginx-admission-patch-c4vgh 0/1 Completed 1 2m2s 10.244.166.138 node1 <none> <none>
pod/ingress-nginx-controller-5dc4b769bd-mmgc6 1/1 Running 0 2m2s 10.244.166.140 node1 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/ingress-nginx-controller LoadBalancer 10.101.23.182 <pending> 80:31820/TCP,443:32442/TCP 2m2s app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
service/ingress-nginx-controller-admission ClusterIP 10.103.254.63 <none> 443/TCP 2m2s app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
bash
#主要看ingress-nginx-controller是否启动成功，是通过这个控制器把生成的nginx配置写入/etc/nginx.conf文件中。ingress-nginx-admission-create和ingress-nginx-admission-patch不管，看pod状态为Completed，他们有可能是一次性执行任务，已经运行完成了
（9）Kuboard查看
工作负载
容器组
服务
（10）确认nginx版本
K8S版本为1.29.0需使用nginx版本1.25.3
（11）拉取镜像
docker hub查看
node1节点拉取
bash
[root@node1 ~]# docker pull nginx:1.25.3
node2节点拉取
bash
[root@node2 ~]# docker pull nginx:1.25.3
（12）编写测试的yaml
bash
[root@master ~]# vim nginx-test.yaml
bash
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
namespace: test
spec:
replicas: 2
selector:
matchLabels:
app: my-nginx
template:
metadata:
labels:
app: my-nginx
spec:
containers:
- name: my-nginx
image: nginx:1.25.3
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-svc
namespace: test
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: my-nginx
（13）生成资源
bash
[root@master ~]# kubectl create ns test
[root@master ~]# kubectl apply -f nginx-test.yaml
（14）查看pod与service
bash
[root@master ~]# kubectl get pod,svc -n test
（15）Kuboard查看
工作负载
容器组
服务
（16）创建对应的ingress
bash
[root@master ~]# vim ingress-http.yaml
bash
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-http
namespace: test
spec:
ingressClassName: "nginx"
rules:
- host: devops.site
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-svc
port:
number: 80
（17）生成资源
bash
[root@master ~]# kubectl apply -f ingress-http.yaml 
(18) 查看ingress
bash
[root@master ~]# kubectl get ingress -n test
详细查看
bash
[root@master ~]# kubectl describe ingress ingress-http -n test
（19）Kuboard查看
应用路由
（20）node1节点修改hosts
bash
[root@node1 ~]# vim /etc/hosts
ingress-nginx-controller部署到node1节点的IP为10.244.166.140
bash
10.244.166.140 devops.site
（21）node1节点访问
bash
[root@node1 ~]# curl devops.site
（22）查看nginx 的pod
bash
[root@master ~]# kubectl get pod -n test
NAME READY STATUS RESTARTS AGE
my-nginx-7bbcf4d985-2sg9h 1/1 Running 0 48m
my-nginx-7bbcf4d985-ztvrb 1/1 Running 0 48m
修改第一个nginx容器内html文件
bash
[root@master ~]# kubectl exec -it my-nginx-7bbcf4d985-2sg9h -n test /bin/bash
......
# cd /usr/share/nginx/html
# ls
# cat index.html
# echo "my-nginx-7bbcf4d985-2sg9h" > index.html
# cat index.html
# exit
修改第二个nginx容器内html文件
bash
[root@master ~]# kubectl exec -it my-nginx-7bbcf4d985-ztvrb -n test /bin/bash
......
# cd /usr/share/nginx/html
# ls
# echo "my-nginx-7bbcf4d985-ztvrb" > index.html
# cat index.html
# exit
（23）node节点访问
node1节点访问
bash
[root@node1 ~]# curl devops.site
node2节点访问（目前为拒绝连接）
bash
[root@node2 ~]# curl devops.site
（24）ingress扩容
完成：
容器组
（25）查看pod与service
ingress-nginx-controller部署到node2节点的IP为10.244.104.13
bash
[root@master ~]# kubectl get pod,svc -n ingress-nginx -o wide
（26） node2节点修改hosts
bash
[root@node2 ~]# vim /etc/hosts
ingress-nginx-controller部署到node2节点的IP为10.244.104.13
bash
10.244.104.13 devops.site
（27）node2节点访问
bash
[root@node2 ~]# curl devops.site
（28）最后再次查看容器组
kube-system名称空间
ingress-nginx名称空间
test名称空间
二、问题
1.kubectl 如何强制删除 Pod、Namespace 资源
（1）报错
devops名称空间的jenkins-bc7986c64-rhcr5一直为Terminating状态
（2）原因分析
资源未成功删除。
（3）解决方法
可以通过 kubectl delete 命令中的 "--force --grace-period=0" 来强制删除资源。
bash
# 删除 Pod
kubectl delete pod ${podname} --force --grace-period=0
# 删除 Namespace
kubectl delete namespace ${namespace_name} --force --grace-period=0删除pod:
bash
[root@master ~]# kubectl delete pods jenkins-bc7986c64-rhcr5 -n devops --force --grace-period=0
成功：
2.创建pod失败
（1）报错
节点创建Pod会一直卡在ContainerCreating的状态无法顺利创建并且就绪，READY状态一直为0/1
Kuboard显示
（2）原因分析
①查看pod
bash
[root@master ~]# kubectl describe pod ingress-nginx-admission-create-2m2hs -n ingress-nginx
②node1节点继续查看cni的日志
bash
sudo journalctl -xe | grep cni
③CNI的配置文件默认在/etc/cni/net.d/目录，进入目录查看
bash
[root@node1 ~]# cd /etc/cni/net.d/
[root@node1 net.d]# ls
nodename为node1，正确的
bash
[root@node1 net.d]# vim 10-calico.conflist
④ 查看kubelet日志
bash
[root@node1 ~]# journalctl --since="2024-04-21 9:50:00" --until="2024-04-21 10:14:00" -fu kubelet
显示Failed to stop sandbox
bash
4月 21 10:13:53 node1 kubelet[1083]: E0421 10:13:53.733547 1083 kuberuntime_manager.go:1381] "Failed to stop sandbox" podSandboxID={"Type":"docker","ID":"f0c0260d8f529498d31a198543cc021365e87eb03729d9ef11b0e55c69d0c8b6"}
⑤ 查看节点cri-docker 并重启服务
bash
systemctl status cri-docker
systemctl restart cri-docker
⑥ 综上分析
原因是node1节点的cni容器出现了异常无法为pod分配ip导致的卡在ContainerCreating的状态。
（3）解决方法
删除异常节点的calico-node容器，让它拉起重新同步数据即可修复。
① 删除 calico-node-7wqzs
②已重新拉活
3.pod报错ImagePullBackOff
（1）原因
pod状态为ImagePullBackOff
（2）原因分析
官方给出的yaml文件中拉取的镜像不在docker hub中，在k8s.gcr.io中，所以在国内我们拉取就会报错：ErrImagePull
相关问题的issue:（相关镜像没法上传到docker hub）
bash
https://github.com/kubernetes/ingress-nginx/issues/6335（3）解决方法
参考项目：
bash
1）GitHub
https://github.com/anjia0532/gcr.io_mirror
2)docker hub地址
https://hub.docker.com/u/anjia0532更换镜像源
bash
[root@node1 ~]# docker pull anjia0532/google-containers.ingress-nginx.controller:v1.10.0
bash
[root@node1 ~]# docker pull anjia0532/google-containers.ingress-nginx.kube-webhook-certgen:v1.4.0
4.docker如何将镜像上传到官方仓库
（1）ingress-nginx.controller
①添加新的标签，在镜像名称前加上仓库名，jiajianwei为仓库名称
bash
[root@node1 ~]# docker tag anjia0532/google-containers.ingress-nginx.controller:v1.10.0 jiajianwei/google-containers.ingress-nginx.controller:v1.10.0
查看镜像：
bash
[root@node1 ~]# docker images
② 登录公共仓库
bash
docker login #登录公共仓库
Username： #账号
password： #密码
③ 上传镜像
bash
[root@node1 ~]# docker push jiajianwei/google-containers.ingress-nginx.controller:v1.10.0
成功：
④在自己的仓库中可以看到上传的镜像，默认上传到公共仓库中
私有仓库需要付费
⑤登出公共仓库
bash
[root@node1 ~]# docker logout
（2）kube-webhook-certgen
①添加新的标签，在镜像名称前加上仓库名，jiajianwei为仓库名称
bash
[root@node1 ~]# docker tag anjia0532/google-containers.ingress-nginx.kube-webhook-certgen:v1.4.0 jiajianwei/google-containers.ingress-nginx.kube-webhook-certgen:v1.4.0
查看镜像：
bash
[root@node1 ~]# docker images
② 登录公共仓库
bash
docker login #登录公共仓库
Username： #账号
password： #密码
③ 上传镜像
bash
[root@node1 ~]# docker push jiajianwei/google-containers.ingress-nginx.kube-webhook-certgen:v1.4.0
成功：
④在自己的仓库中可以看到上传的镜像，默认上传到公共仓库中
私有仓库需要付费
⑤登出公共仓库
bash
[root@node1 ~]# docker logout（3）删除本地已有镜像
bash
[root@node1 ~]# docker rmi -f anjia0532/google-containers.ingress-nginx.controller:v1.10.0 anjia0532/google-containers.ingress-nginx.kube-webhook-certgen:v1.4.0
bash
[root@node1 ~]# docker rmi -f jiajianwei/google-containers.ingress-nginx.controller:v1.10.0 jiajianwei/google-containers.ingress-nginx.kube-webhook-certgen:v1.4.0
(4) 拉取jiajianwei仓库中的镜像
node1节点
bash
[root@node1 ~]# docker pull jiajianwei/google-containers.ingress-nginx.controller:v1.10.0
bash
[root@node1 ~]# docker pull jiajianwei/google-containers.ingress-nginx.kube-webhook-certgen:v1.4.0
node2节点
bash
[root@node2 ~]# docker pull jiajianwei/google-containers.ingress-nginx.controller:v1.10.0
bash
[root@node2 ~]# docker pull jiajianwei/google-containers.ingress-nginx.kube-webhook-certgen:v1.4.0
5.创建ingress报错
（1）报错
bash
Error from server (InternalError): error when creating "ingress-http.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": failed to call webhook: Post "https://ingress-nginx-controller-admission.ingress-nginx.svc:443/networking/v1/ingresses?timeout=10s": context deadline exceeded
（2）原因分析
ValidatingWebhookConfiguration未删掉。
（3）解决方法
查看
bash
[root@master ~]# kubectl get ValidatingWebhookConfiguration
删除
bash
[root@master ~]# kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission
成功创建ingress: