k8s V1.35.2用cilium gateway访问服务的方式(适用于中小型环境,无LB的情况下)后续会写一个生产环境上一的应用
软件版本:
ubuntu24.04.2,
kubeadm v1.35.2
kubernetes v1.35.2
containerd v2.0.2
cilium version v1.19.1
本环境基于 Cilium 构建 Kubernetes 网络与流量入口体系,实现:
替代 kube-proxy 的 eBPF 数据平面转发能力
替代 Calico 的 CNI 网络功能
基于 Gateway API 替代传统 NGINX Ingress Controller 实现服务七层流量接入
一、目标架构
Client
↓
VIP(L2 ARP 漂移)只是入口调度
↓
K8s Node(承载 VIP 的入口节点)
↓
Cilium Gateway(Envoy)
↓
HTTPRoute
↓
ClusterIP Service
↓
Pod(nginx / nacos等)
二、安装 Gateway API CRD
(必须先做)
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/experimental-install.yaml
查看ReferenceGrant 状态
kubectl api-resources | grep ReferenceGrant
root@ops-test-025:/apps/cilium# kubectl api-resources | grep ReferenceGrant
referencegrants refgrant gateway.networking.k8s.io/v1beta1 true ReferenceGrant
这步是必须的
ReferenceGrant = 允许某个 namespace 的 HTTPRoute 去引用另一个 namespace 的 Service / Secret / Backend
多个业务共用一个gateway的时候会遇到跨namespace的问题
验证
kubectl get crd | grep gateway
root@ops-test-025:/apps/cilium# kubectl get crd | grep gateway
backendlbpolicies.gateway.networking.k8s.io 2026-04-27T05:51:07Z
backendtlspolicies.gateway.networking.k8s.io 2026-04-27T05:51:07Z
ciliumgatewayclassconfigs.cilium.io 2026-02-28T02:57:57Z
gatewayclasses.gateway.networking.k8s.io 2026-04-27T05:51:07Z
gateways.gateway.networking.k8s.io 2026-04-27T05:51:08Z
grpcroutes.gateway.networking.k8s.io 2026-04-27T05:51:08Z
httproutes.gateway.networking.k8s.io 2026-04-27T05:51:08Z
referencegrants.gateway.networking.k8s.io 2026-04-27T05:51:08Z
tcproutes.gateway.networking.k8s.io 2026-04-27T05:51:08Z
tlsroutes.gateway.networking.k8s.io 2026-04-27T05:51:08Z
udproutes.gateway.networking.k8s.io 2026-04-27T05:51:09Z
三、安装 Cilium
---安装执行---
root@ops-test-025:/apps/cilium# cilium install \
--version 1.19.1 \
\
--set kubeProxyReplacement=true \
--set kubeProxyReplacementMode=strict \
\
--set k8sServiceHost=192.168.2.25 \
--set k8sServicePort=6443 \
\
--set routingMode=native \
--set ipam.mode=kubernetes \
--set autoDirectNodeRoutes=true \
--set ipv4NativeRoutingCIDR=10.244.0.0/16 \
\
--set loadBalancer.mode=snat \
\
--set nodePort.enabled=true \
--set externalIPs.enabled=true \
--set hostServices.enabled=true \
\
--set l2announcements.enabled=true \
\
--set gatewayAPI.enabled=true \
\
--set hubble.enabled=true \
--set hubble.relay.enabled=true \
--set hubble.ui.enabled=true \
\
--set prometheus.enabled=true \
--set operator.prometheus.enabled=true
ℹ️ Using Cilium version 1.19.1
🔮 Auto-detected cluster name: kubernetes
🔮 Auto-detected kube-proxy has not been installed
ℹ️ Cilium will fully replace all functionalities of kube-proxy
查看cilium状态
root@ops-test-025:/apps/cilium# cilium status
/¯¯\
/¯¯\__/¯¯\ Cilium: OK
\__/¯¯\__/ Operator: OK
/¯¯\__/¯¯\ Envoy DaemonSet: OK
\__/¯¯\__/ Hubble Relay: OK
\__/ ClusterMesh: disabled
DaemonSet cilium Desired: 6, Ready: 6/6, Available: 6/6
DaemonSet cilium-envoy Desired: 6, Ready: 6/6, Available: 6/6
Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1
Deployment hubble-relay Desired: 1, Ready: 1/1, Available: 1/1
Deployment hubble-ui Desired: 1, Ready: 1/1, Available: 1/1
Containers: cilium Running: 6
cilium-envoy Running: 6
cilium-operator Running: 1
clustermesh-apiserver
hubble-relay Running: 1
hubble-ui Running: 1
Cluster Pods: 9/9 managed by Cilium
Helm chart version: 1.19.1
Image versions cilium quay.io/cilium/cilium:v1.19.1@sha256:41f1f74a0000de8656f1de4088ea00c8f2d49d6edea579034c73c5fd5fe01792: 6
cilium-envoy quay.io/cilium/cilium-envoy:v1.35.9-1770979049-232ed4a26881e4ab4f766f251f258ed424fff663@sha256:8188114a2768b5f49d6ce58e168b20d765e0fbc64eee0d83241aa2b150ccd788: 6
cilium-operator quay.io/cilium/operator-generic:v1.19.1@sha256:e7278d763e448bf6c184b0682cf98cdca078d58a27e1b2f3c906792670aa211a: 1
hubble-relay quay.io/cilium/hubble-relay:v1.19.1@sha256:d8c4e13bc36a56179292bb52bc6255379cb94cb873700d316ea3139b1bdb8165: 1
hubble-ui quay.io/cilium/hubble-ui-backend:v0.13.3@sha256:db1454e45dc39ca41fbf7cad31eec95d99e5b9949c39daaad0fa81ef29d56953: 1
hubble-ui quay.io/cilium/hubble-ui:v0.13.3@sha256:661d5de7050182d495c6497ff0b007a7a1e379648e60830dd68c4d78ae21761d: 1
root@ops-test-025:/apps/cilium# kubectl get gatewayclass
NAME CONTROLLER ACCEPTED AGE
cilium io.cilium/gateway-controller Unknown 3m40s
root@ops-test-025:/apps/cilium# kubectl get crd | grep gateway
backendlbpolicies.gateway.networking.k8s.io 2026-04-27T05:51:07Z
backendtlspolicies.gateway.networking.k8s.io 2026-04-27T05:51:07Z
ciliumgatewayclassconfigs.cilium.io 2026-02-28T02:57:57Z
gatewayclasses.gateway.networking.k8s.io 2026-04-27T05:51:07Z
gateways.gateway.networking.k8s.io 2026-04-27T05:51:08Z
grpcroutes.gateway.networking.k8s.io 2026-04-27T05:51:08Z
httproutes.gateway.networking.k8s.io 2026-04-27T05:51:08Z
referencegrants.gateway.networking.k8s.io 2026-04-27T05:51:08Z
tcproutes.gateway.networking.k8s.io 2026-04-27T05:51:08Z
tlsroutes.gateway.networking.k8s.io 2026-04-27T05:51:08Z
udproutes.gateway.networking.k8s.io 2026-04-27T05:51:09Z
root@ops-test-025:/apps/cilium#
安装后必须确认:
cilium status
验证,必须全部:
Cilium OK
Operator OK
Envoy OK
Gateway API enabled
下发L2 策略
vim l2-polocy.yml
apiVersion: cilium.io/v2alpha1
kind: CiliumL2AnnouncementPolicy
metadata:
name: default-l2
spec:
loadBalancerIPs: true
externalIPs: true
interfaces:
- ens160
配置lb地址池
vim lb-pool.yml
apiVersion: cilium.io/v2
kind: CiliumLoadBalancerIPPool
metadata:
name: lb-pool
spec:
blocks:
- start: 192.168.2.23
stop: 192.168.2.24
创建gateway
vim gateway_input.yml
apiVersion: v1
kind: Namespace
metadata:
name: infra
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: cilium-gw
namespace: infra
spec:
gatewayClassName: cilium
listeners:
- name: http
protocol: HTTP
port: 80
allowedRoutes:
namespaces:
from: All
执行
kubectl apply -f l2-policy.yaml
kubectl apply -f lb-pool.yaml
kubectl apply -f gateway.yaml
查看状态
root@ops-test-025:/apps/cilium# kubectl get gateway -n infra
NAME CLASS ADDRESS PROGRAMMED AGE
cilium-gw cilium 192.168.2.23 True 20s
root@ops-test-025:/apps/cilium#
创建nginx实例,验证
kubectl create deployment nginx --image=docker.1ms.run/library/nginx
kubectl expose deployment nginx --port=80
创建nginx实例httproute
root@ops-test-025:/apps/cilium# cat nginx_httproute.yml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: nginx-route
namespace: default
spec:
parentRefs:
- name: cilium-gw
namespace: infra
hostnames:
- nginx.cctbb.com
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: nginx
port: 80
查看gateway状态,httproute状态
root@ops-test-025:/apps/cilium# kubectl get pod -n default
NAME READY STATUS RESTARTS AGE
nginx-554d6b476b-hwpm7 1/1 Running 0 4d
root@ops-test-025:/apps/cilium# kubectl get gateway -A
NAMESPACE NAME CLASS ADDRESS PROGRAMMED AGE
infra cilium-gw cilium 192.168.2.23 True 2m59s
root@ops-test-025:/apps/cilium# kubectl get gatewayclass
NAME CONTROLLER ACCEPTED AGE
cilium io.cilium/gateway-controller Unknown 8m51s
root@ops-test-025:/apps/cilium# kubectl get httproute -A
NAMESPACE NAME HOSTNAMES AGE
default nginx-route ["nginx.cctbb.com"] 54s
验证nginx状态
root@ops-test-025:/apps/cilium# curl http://nginx.cctbb.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, nginx is successfully installed and working.
Further configuration is required for the web server, reverse proxy,
API gateway, load balancer, content cache, or other features.</p>
<p>For online documentation and support please refer to
<a href="https://nginx.org/">nginx.org</a>.<br/>
To engage with the community please visit
<a href="https://community.nginx.org/">community.nginx.org</a>.<br/>
For enterprise grade support, professional services, additional
security features and capabilities please refer to
<a href="https://f5.com/nginx">f5.com/nginx</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
页面访问
