大家好,我是博哥爱运维,K8s是如何来进行服务配置管理的呢?
对于容器而言,如果我们想修改一个容器镜像里面的配置,可以在Dockerfile这一步,将修改好的配置复制到镜像里面再重新打包,对于不用变动配置的镜像而言,这样做属于硬编码当然也可以,但一旦我们的镜像服务需要修改配置,那么就需要重新重新打包非常麻烦,对于K8s而言,对于配置这么重要的一个环节,自然有它的解决方案,那就是configmap(通常普通配置使用)和secret(对于一些机密配置信息使用),在上面的部分章节里面,有提前涉及到这部分内容,但没有进行仔细的讲解,这里就对它们作下详细的实践。
我这里会准备一个deployment的yaml配置,用busybox来作为服务镜像,通过一个完整的yaml就可以快速带大家理解并能熟练在K8s上使用configmap和secret,如果一下子理解不了,后面可以保存这份yaml来作来生产配置参考也是没问题的,用多了自然就熟了,yaml配置如下:
configmap-secret-example-simple.yaml
yaml
---
# configmap
# kubectl create configmap localconfig-env --from-literal=log_level_test=TEST --from-literal=log_level_produce=PRODUCE
apiVersion: v1
kind: ConfigMap
metadata:
name: localconfig-env
data:
log_level_test: TEST
log_level_produce: PRODUCE
---
# configmap
# kubectl create configmap localconfig-file --from-file=localconfig-test=localconfig-test.conf --from-file=localconfig-produce=localconfig-produce.conf
apiVersion: v1
kind: ConfigMap
metadata:
name: localconfig-file
data:
localconfig-produce: |
TEST_RELEASE = False
PORT = 80
PROCESSES = 0
MESSAGE = Produce
localconfig-test: |
TEST_RELEASE = True
PORT = 8080
PROCESSES = 1
MESSAGE = Test
---
# secret
# kubectl create secret generic mysecret --from-literal=mysql-root-password='BogeMysqlPassword' --from-literal=redis-root-password='BogeRedisPassword' --from-file=my_id_rsa=/root/.ssh/id_rsa --from-file=my_id_rsa_pub=/root/.ssh/id_rsa.pub
apiVersion: v1
kind: Secret
metadata:
name: mysecret
namespace: default
type: Opaque
data:
my_id_rsa: bXlfaWRfcnNhCg==
my_id_rsa_pub: bXlfaWRfcnNhX3B1Ygo=
mysql-root-password: Qm9nZU15c3FsUGFzc3dvcmQ=
redis-root-password: Qm9nZVJlZGlzUGFzc3dvcmQ=
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
run: test-busybox
name: test-busybox
namespace: default
spec:
replicas: 1
selector:
matchLabels:
run: test-busybox
template:
metadata:
labels:
run: test-busybox
spec:
containers:
- name: test-busybox
image: registry.cn-shanghai.aliyuncs.com/acs/busybox:v1.29.2
args:
- /bin/sh
- -c
- >
echo "-------------------------------------------------";
echo "TEST_ENV is:$(TEST_ENV)";
echo "-------------------------------------------------";
echo "PRODUCE_ENV is:$(PRODUCE_ENV)";
echo "-------------------------------------------------";
echo "secret MYSQL_ROOT_PASSWORD is:$(MYSQL_ROOT_PASSWORD)";
echo "-------------------------------------------------";
echo "secret REDIS_ROOT_PASSWORD is:$(REDIS_ROOT_PASSWORD)";
echo "-------------------------------------------------";
echo "/etc/local_config_test.py body is:";
cat /etc/local_config_test.py;
echo "-------------------------------------------------";
echo "/etc/local_config_produce.py body is:";
cat /etc/local_config_produce.py;
echo "-------------------------------------------------";
echo "/etc/id_rsa body is:";
cat /etc/id_rsa;
echo "-------------------------------------------------";
echo "/etc/id_rsa.pub body is:";
cat /etc/id_rsa.pub;
echo "-------------------------------------------------";
ls -ltr /etc;
sleep 30000;
env:
- name: TEST_ENV
valueFrom:
configMapKeyRef:
name: localconfig-env
key: log_level_test
- name: PRODUCE_ENV
valueFrom:
configMapKeyRef:
name: localconfig-env
key: log_level_produce
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mysecret
key: mysql-root-password
- name: REDIS_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mysecret
key: redis-root-password
volumeMounts:
- name: testconfig
mountPath: "/etc/local_config_test.py"
subPath: localconfig-test
- name: testconfig
mountPath: "/etc/local_config_produce.py"
subPath: localconfig-produce
readOnly: true
- name: testsecret
mountPath: "/etc/id_rsa"
subPath: my_id_rsa
readOnly: true
- name: testsecret
mountPath: "/etc/id_rsa.pub"
subPath: my_id_rsa_pub
readOnly: true
volumes:
- name: testconfig
configMap:
name: localconfig-file
defaultMode: 0660
- name: testsecret
secret:
secretName: mysecret
defaultMode: 0600
配置自动更新器 reloader
https://github.com/stakater/Reloader
what is reloader
A Kubernetes controller to watch changes in ConfigMap and Secrets and then restart pods for Deployment, StatefulSet and DaemonSet
How to use Reloader
## kind: Deployment
## metadata:
## annotations:
## #------ all(ConfigMap and/or Secret)
## reloader.stakater.com/auto: "true"
## #------ only configmap for name: "foo-configmap"
## configmap.reloader.stakater.com/reload: "foo-configmap"
## #------ many configmaps
## configmap.reloader.stakater.com/reload: "foo-configmap,bar-configmap,baz-configmap"
## #------ only secret for name: "foo-secret"
## secret.reloader.stakater.com/reload: "foo-secret"
## #------ many secrets
## secret.reloader.stakater.com/reload: "foo-secret,bar-secret,baz-secret"
## spec:
## template:
## metadata:
部署yaml配置
yaml
---
# Source: reloader/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
meta.helm.sh/release-namespace: "default"
meta.helm.sh/release-name: "reloader"
labels:
app: reloader-reloader
chart: "reloader-1.0.51"
release: "reloader"
heritage: "Helm"
app.kubernetes.io/managed-by: "Helm"
name: reloader-reloader
namespace: default
---
# Source: reloader/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
meta.helm.sh/release-namespace: "default"
meta.helm.sh/release-name: "reloader"
labels:
app: reloader-reloader
chart: "reloader-1.0.51"
release: "reloader"
heritage: "Helm"
app.kubernetes.io/managed-by: "Helm"
name: reloader-reloader-role
rules:
- apiGroups:
- ""
resources:
- secrets
- configmaps
verbs:
- list
- get
- watch
- apiGroups:
- "apps"
resources:
- deployments
- daemonsets
- statefulsets
verbs:
- list
- get
- update
- patch
- apiGroups:
- "extensions"
resources:
- deployments
- daemonsets
verbs:
- list
- get
- update
- patch
- apiGroups:
- "batch"
resources:
- cronjobs
verbs:
- list
- get
- apiGroups:
- "batch"
resources:
- jobs
verbs:
- create
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
---
# Source: reloader/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
meta.helm.sh/release-namespace: "default"
meta.helm.sh/release-name: "reloader"
labels:
app: reloader-reloader
chart: "reloader-1.0.51"
release: "reloader"
heritage: "Helm"
app.kubernetes.io/managed-by: "Helm"
name: reloader-reloader-role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: reloader-reloader-role
subjects:
- kind: ServiceAccount
name: reloader-reloader
namespace: default
---
# Source: reloader/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
meta.helm.sh/release-namespace: "default"
meta.helm.sh/release-name: "reloader"
labels:
app: reloader-reloader
chart: "reloader-1.0.51"
release: "reloader"
heritage: "Helm"
app.kubernetes.io/managed-by: "Helm"
group: com.stakater.platform
provider: stakater
version: v1.0.51
name: reloader-reloader
namespace: default
spec:
replicas: 1
revisionHistoryLimit: 2
selector:
matchLabels:
app: reloader-reloader
release: "reloader"
template:
metadata:
labels:
app: reloader-reloader
chart: "reloader-1.0.51"
release: "reloader"
heritage: "Helm"
app.kubernetes.io/managed-by: "Helm"
group: com.stakater.platform
provider: stakater
version: v1.0.51
spec:
containers:
- image: "ghcr.io/stakater/reloader:v1.0.51"
imagePullPolicy: IfNotPresent
name: reloader-reloader
ports:
- name: http
containerPort: 9090
livenessProbe:
httpGet:
path: /live
port: http
timeoutSeconds: 5
failureThreshold: 5
periodSeconds: 10
successThreshold: 1
initialDelaySeconds: 10
readinessProbe:
httpGet:
path: /metrics
port: http
timeoutSeconds: 5
failureThreshold: 5
periodSeconds: 10
successThreshold: 1
initialDelaySeconds: 10
securityContext:
{}
securityContext:
runAsNonRoot: true
runAsUser: 65534
serviceAccountName: reloader-reloader