进程间通信一直是工程实现经常遇到的场景，比如说数据处理的进程将结果发送给日志进程记录；亦或者多个数据处理进程，在处理数据的的时候，会有一部分配置需要共享，这样以来这些配置文件一个node只存一份即可，并且在内存中查询肯定比redis快的；再或者就是处理大型数据集的时候，需要一个控制进程和多个数据线程，控制线程进行数据的分片和整合，数据进程进行无脑计算，这个时候控制线程和数据线程是需要进行通信的。
进程间通信方式也有很多，这次我们介绍一种最快的方式--共享内存，快的原因简单说就是这些进程间数据传递不再涉及到内核，即不再通过执行进入内核的系统调用来传递彼此的数据。
共享内存实现原理
裸金属共享内存通信
arduino
// 1.查看共享内存段
ipcs -m
ipcrm -m shmid
// 2.创建共享内存
/* shmget() returns the identifier of the shared memory segment associated with the value of the argument key. */
/* size equal to the value of size rounded up to a multiple of PAGE_SIZE */
/* PC_CREAT | IPC_EXCL */
int shmget(key_t key, size_t size, int shmflg);
// 3.挂接共享内存
/* shmat() attaches the shared memory segment identified by shmid to the address space of the calling process. */
/* If shmaddr is NULL, the system chooses a suitable (unused) address at which to attach the segment. */
/* If shmaddr isn't NULL and SHM_RND is specified in shmflg, the attach occurs at the address equal
to shmaddr rounded down to the nearest multiple of SHMLBA. Otherwise shmaddr must be a page-aligned
address at which the attach occurs. */
void *shmat(int shmid, const void *shmaddr, int shmflg);
// 4.去关联共享内存
int shmdt(const void *shmaddr);
// 5.以cmd操纵共享内存
/* shmctl() performs the control operation specified by cmd on the shared memory segment whose
identifier is given in shmid. */
/* cmd:
IPC_RMID: Mark the segment to be destroyed.
IPC_SET: Write the values of some members of the shmid_ds structure pointed to by arg.buf
to the kernel data structure associated with this shared memory segment
IPC_STAT: Copy information from the kernel data structure associated with shmid
into the shmid_ds structure pointed to by buf.
*/
int shmctl(int shmid, int cmd, struct shmid_ds *buf);
简单写一个demo，分为两个进程，一个是reader进程，一个writer进程。writer进程负责初始化创建一段共享内存，reader进程启动后不断轮询共享内存是否创建好，如果创建好就可以将数据读取出来。
arduino
//Makefile:
.PHONY:all
all:reader writer
writer:writer.c
gcc -o $@ $^
reader:reader.c
gcc -o $@ $^
.PHONY:clean
clean:
rm -f reader writer
//common.h
#ifndef _COMMON_H__
#define _COMMON_H__
#include<stdio.h>
#include<sys/types.h>
#include<sys/ipc.h>
#include<sys/shm.h>
#define PATHNAME "zhihushm"
#define PROJ_ID 0x6666
#endif
//writer.c
#include "writer.h"
static char *shm_addr = NULL;
int create_shm()
{
key_t key = ftok(PATHNAME, PROJ_ID);
if(key < 0)
{
perror("ftok");
return -1;
}
int shmid = shmget(key, 4096, IPC_CREAT | IPC_EXCL | 0666);
if(shmid < 0)
{
perror("shmget");
return -2;
}
shm_addr = shmat(shmid, NULL, 0);
return 0;
}
void write_something()
{
int i = 0;
while (i < 4096)
{
shm_addr[0] = i % 128;
i++;
sleep(1);
}
}
int main()
{
int ret = create_shm();
if (ret < 0)
{
return -1;
}
write_something();
return 0;
}
//reader.c
#include "writer.h"
static char *shm_addr = NULL;
int init()
{
key_t key = ftok(PATHNAME, PROJ_ID);
if(key < 0)
{
perror("ftok");
return -1;
}
int shmid = shmget(key, 4096, IPC_CREAT);
if(shmid < 0)
{
perror("shmget");
return -2;
}
shm_addr = shmat(shmid, NULL, 0);
return 0;
}
void read_something()
{
int i = 0;
while (i < 4096)
{
printf("we get %c\n", shm_addr[0]);
sleep(1);
}
}
int main()
{
int ret = init();
while (ret < 0)
{
ret = init();
}
read_something();
return 0;
}
k8s中pod间共享内存通信
好啦，现在请出这章的主角---多pod共享内存通信，而实现的关键就在各pod同时挂载/dev/shm即可，详细方案如下图：
这样，我们简单把上面的reader和writer两个进程分别容器化，再由k8s调度起来，并起多个reader pod和一个writer pod，来看看是否多个reader pod可以读到writer写的数据，以达到每一个Node上数据只有一份，多pod共享的目的。
先写两个Dockerfile:
bash
//reader.Dockerfile
FROM centos
ADD . /home/
RUN chmod 777 /home/reader
ENTRYPOINT ["/home/reader"]
//writer.Dockerfile
FROM centos
ADD . /home/
RUN chmod 777 /home/writer
ENTRYPOINT ["/home/writer"] 再build出镜像并运行：
bash
docker build -t reader:v1 . -f reader.Dockerfile
docker build -t writer:v1 -f writer.Dockerfile
docker run -ti --name writer -v /dev/shm:/dev/shm --net=host --ipc=host writer:v1 bash
docker run -ti --name reader -v /dev/shm:/dev/shm --net=host --ipc=host reader:v1 bash
既然起两个docker看来是可以通信的了，那我们再写两个yaml文件并用k8s部署出来：
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: huhuxx
name: huhuxx
spec:
selector:
matchLabels:
app: huhuxx
template:
metadata:
labels:
app: huhuxx
spec:
hostNetwork: true
hostIPC: true
containers:
- name: huhuxx
image: reader:v4
imagePullPolicy: Never
volumeMounts:
- mountPath: /dev/shm
name: shm
volumes:
- name: shm
hostPath:
path: /dev/shm
type: Directory
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: huhuxx
name: huhuxx
spec:
selector:
matchLabels:
app: huhuxx
template:
metadata:
labels:
app: huhuxx
spec:
hostNetwork: true
hostIPC: true
containers:
- name: huhuxx
image: writer:v4
imagePullPolicy: Never
volumeMounts:
- mountPath: /dev/shm
name: shm
volumes:
- name: shm
hostPath:
path: /dev/shm
type: Directory查看同一node上的，writer pod和其中两个reader pod:
成啦，虽然实现比较粗糙，毕竟demo嘛，证明这条路是可以走通哒。
Reference
Kubernetes中Pod间共享内存方案 - 腾讯云开发者社区-腾讯云 (tencent.com)
进程间通信------共享内存（Shared Memory）_victory_523的博客-CSDN博客_共享内存
shmget() - Unix, Linux System Call (tutorialspoint.com)
docker警告：WARNING: IPv4 forwarding is disabled. Networking will not work._杰哥的技术杂货铺的博客-CSDN博客