ETCD底层源码设计与实现1

相关链接：ETCD底层源码设计与实现2 https://blog.csdn.net/qq_41822345/article/details/163730617

文章目录

[01 - etcd前置理论与环境准备](#01 - etcd前置理论与环境准备)

[1.1 etcd简介](#1.1 etcd简介)

[1.1.1 etcd定位](#1.1.1 etcd定位)

[1.1.2 核心特性](#1.1.2 核心特性)

[1.1.3 应用场景](#1.1.3 应用场景)

[1.2 Raft共识算法理论](#1.2 Raft共识算法理论)

[1.2.1 Raft概述](#1.2.1 Raft概述)

[1.2.2 Raft节点状态](#1.2.2 Raft节点状态)

[1.2.3 Leader选举](#1.2.3 Leader选举)

选举流程

源码对应

[1.2.4 日志复制](#1.2.4 日志复制)

复制流程

日志结构

[1.2.5 安全性保证](#1.2.5 安全性保证)

[1.2.6 快照机制](#1.2.6 快照机制)

[1.3 MVCC与WAL原理](#1.3 MVCC与WAL原理)

[1.3.1 MVCC概述](#1.3.1 MVCC概述)

[1.3.2 etcd中的MVCC实现](#1.3.2 etcd中的MVCC实现)

版本管理

版本压缩

[1.3.3 WAL（Write-Ahead Log）预写日志](#1.3.3 WAL（Write-Ahead Log）预写日志)

WAL作用

WAL特性

WAL文件结构

[1.4 BoltDB存储基础](#1.4 BoltDB存储基础)

[1.4.1 BoltDB概述](#1.4.1 BoltDB概述)

[1.4.2 核心特性](#1.4.2 核心特性)

[1.4.3 数据结构](#1.4.3 数据结构)

Bucket结构

源码位置

[1.5 etcd v2与v3架构差异](#1.5 etcd v2与v3架构差异)

[1.5.1 核心差异对比](#1.5.1 核心差异对比)

[1.5.2 v3架构改进](#1.5.2 v3架构改进)

[1.6 源码编译与环境配置](#1.6 源码编译与环境配置)

[1.6.1 环境要求](#1.6.1 环境要求)

[1.6.2 编译步骤](#1.6.2 编译步骤)

[1.6.3 单节点启动](#1.6.3 单节点启动)

[1.6.4 集群启动](#1.6.4 集群启动)

[1.6.5 测试命令](#1.6.5 测试命令)

[1.7 项目目录结构](#1.7 项目目录结构)

[1.7.1 根目录结构](#1.7.1 根目录结构)

[1.7.2 server目录详解（核心）](#1.7.2 server目录详解（核心）)

[1.7.3 pkg目录详解](#1.7.3 pkg目录详解)

[1.7.4 核心入口文件](#1.7.4 核心入口文件)

[1.7.5 源码阅读路径建议](#1.7.5 源码阅读路径建议)

参考文献

[02 - etcd整体分层架构](#02 - etcd整体分层架构)

[2.1 分层架构总览](#2.1 分层架构总览)

[2.1.1 各层职责](#2.1.1 各层职责)

[2.2 全局请求流转](#2.2 全局请求流转)

[2.2.1 写请求完整流程](#2.2.1 写请求完整流程)

[2.2.2 读请求优化流程](#2.2.2 读请求优化流程)

线性读流程

[2.3 核心组件依赖关系](#2.3 核心组件依赖关系)

[2.3.1 EtcdServer核心结构体](#2.3.1 EtcdServer核心结构体)

源码位置

核心字段示意

[2.3.2 raftNode结构体](#2.3.2 raftNode结构体)

源码位置

[2.3.3 组件依赖图](#2.3.3 组件依赖图)

[2.4 模块间交互边界设计](#2.4 模块间交互边界设计)

[2.4.1 接口定义](#2.4.1 接口定义)

RaftKV接口

Lessor接口

Authenticator接口

[2.4.2 关键交互流程](#2.4.2 关键交互流程)

[1. 提案提交与应用流程](#1. 提案提交与应用流程)

[2. Ready处理循环](#2. Ready处理循环)

[3. toApply结构体](#3. toApply结构体)

[2.4.3 模块边界总结](#2.4.3 模块边界总结)

参考文献

[03 - 客户端v3 gRPC模块源码解析](#03 - 客户端v3 gRPC模块源码解析)

[3.1 客户端设计概览](#3.1 客户端设计概览)

[3.1.1 核心接口](#3.1.1 核心接口)

源码位置

核心接口图

[3.1.2 Client初始化](#3.1.2 Client初始化)

[3.2 KV与Range查询](#3.2 KV与Range查询)

[3.2.1 KV接口](#3.2.1 KV接口)

[3.2.2 Range查询选项](#3.2.2 Range查询选项)

[3.2.3 使用示例](#3.2.3 使用示例)

[3.3 Txn事务处理](#3.3 Txn事务处理)

[3.3.1 Txn接口](#3.3.1 Txn接口)

[3.3.2 Txn实现](#3.3.2 Txn实现)

[3.3.3 Txn使用示例](#3.3.3 Txn使用示例)

[3.3.4 Compare类型](#3.3.4 Compare类型)

[3.4 Lease租约机制](#3.4 Lease租约机制)

[3.4.1 Lease接口](#3.4.1 Lease接口)

[3.4.2 租约工作流程](#3.4.2 租约工作流程)

[3.4.3 租约使用示例](#3.4.3 租约使用示例)

[3.4.4 关键数据结构](#3.4.4 关键数据结构)

[3.5 Watch监听机制](#3.5 Watch监听机制)

[3.5.1 Watcher接口](#3.5.1 Watcher接口)

[3.5.2 Watch工作原理](#3.5.2 Watch工作原理)

[3.5.3 Watch使用示例](#3.5.3 Watch使用示例)

[3.5.4 Event事件](#3.5.4 Event事件)

[3.6 使用坑点总结](#3.6 使用坑点总结)

[3.6.1 常见问题](#3.6.1 常见问题)

[3.6.2 最佳实践](#3.6.2 最佳实践)

参考文献

[04 - API网关服务层](#04 - API网关服务层)

[4.1 gRPC服务注册](#4.1 gRPC服务注册)

[4.1.1 v3rpc包结构](#4.1.1 v3rpc包结构)

[4.1.2 gRPC服务注册](#4.1.2 gRPC服务注册)

[4.2 请求拦截器](#4.2 请求拦截器)

[4.2.1 拦截器链](#4.2.1 拦截器链)

[4.2.2 拦截器实现](#4.2.2 拦截器实现)

[4.3 认证与鉴权](#4.3 认证与鉴权)

[4.3.1 Auth服务](#4.3.1 Auth服务)

[4.3.2 鉴权流程](#4.3.2 鉴权流程)

[4.4 参数校验](#4.4 参数校验)

[4.5 配额与限流](#4.5 配额与限流)

参考文献

[05 - Raft集群共识核心（重点）](#05 - Raft集群共识核心（重点）)

[5.1 Raft论文核心回顾](#5.1 Raft论文核心回顾)

[5.1.1 Raft三个子问题](#5.1.1 Raft三个子问题)

[5.1.2 Raft状态转换](#5.1.2 Raft状态转换)

[5.2 raftNode与状态机](#5.2 raftNode与状态机)

[5.2.1 raftNode结构体](#5.2.1 raftNode结构体)

[5.2.2 Raft Ready处理循环](#5.2.2 Raft Ready处理循环)

[5.3 Leader选举源码](#5.3 Leader选举源码)

[5.3.1 选举流程](#5.3.1 选举流程)

[5.3.2 选举相关参数](#5.3.2 选举相关参数)

[5.4 日志复制源码](#5.4 日志复制源码)

[5.4.1 日志复制流程](#5.4.1 日志复制流程)

[5.4.2 日志结构](#5.4.2 日志结构)

[5.5 快照同步源码](#5.5 快照同步源码)

[5.5.1 快照触发时机](#5.5.1 快照触发时机)

[5.5.2 快照处理](#5.5.2 快照处理)

[5.6 脑裂与一致性保证](#5.6 脑裂与一致性保证)

[5.6.1 脑裂场景](#5.6.1 脑裂场景)

[5.6.2 一致性保证](#5.6.2 一致性保证)

参考文献

[06 - WAL预写日志持久化](#06 - WAL预写日志持久化)

[6.1 WAL理论基础](#6.1 WAL理论基础)

[6.1.1 WAL作用](#6.1.1 WAL作用)

[6.1.2 WAL特点](#6.1.2 WAL特点)

[6.2 WAL文件结构](#6.2 WAL文件结构)

[6.2.1 目录结构](#6.2.1 目录结构)

[6.2.2 WAL格式](#6.2.2 WAL格式)

[6.3 日志写入流程](#6.3 日志写入流程)

[6.3.1 WAL结构体](#6.3.1 WAL结构体)

[6.3.2 写入流程](#6.3.2 写入流程)

[6.4 崩溃恢复](#6.4 崩溃恢复)

[6.4.1 读取WAL](#6.4.1 读取WAL)

[6.4.2 恢复流程](#6.4.2 恢复流程)

[6.5 日志截断](#6.5 日志截断)

[6.5.1 截断时机](#6.5.1 截断时机)

参考文献

[07 - MVCC+BoltDB存储层](#07 - MVCC+BoltDB存储层)

[7.1 BoltDB基础](#7.1 BoltDB基础)

[7.1.1 BoltDB特点](#7.1.1 BoltDB特点)

[7.1.2 etcd对BoltDB的封装](#7.1.2 etcd对BoltDB的封装)

[7.2 MVCC多版本控制](#7.2 MVCC多版本控制)

[7.2.1 Revision结构](#7.2.1 Revision结构)

[7.2.2 版本存储示例](#7.2.2 版本存储示例)

[7.3 key索引](#7.3 key索引)

[7.3.1 keyIndex结构](#7.3.1 keyIndex结构)

[7.3.2 索引查询流程](#7.3.2 索引查询流程)

[7.4 事务处理](#7.4 事务处理)

[7.4.1 KV事务](#7.4.1 KV事务)

[7.4.2 事务提交流程](#7.4.2 事务提交流程)

[7.5 版本压缩](#7.5 版本压缩)

[7.5.1 压缩触发](#7.5.1 压缩触发)

[7.5.2 压缩后查询](#7.5.2 压缩后查询)

参考文献

01 - etcd前置理论与环境准备

1.1 etcd简介

1.1.1 etcd定位

etcd是一个分布式可靠键值存储系统，专注于以下特性：

Simple：清晰的用户API（gRPC）
Secure：自动TLS，支持客户端证书认证
Fast：基准测试显示10,000写/秒
Reliable：使用Raft算法实现分布式一致性

1.1.2 核心特性

特性	说明
线性一致性	提供强一致性保证
分布式共识	基于Raft算法实现
键值存储	支持多级目录结构
Lease机制	支持TTL过期和续租
Watch机制	监听键变更并推送通知
事务支持	多操作原子执行
MVCC	多版本并发控制

1.1.3 应用场景

Kubernetes配置存储与服务发现
分布式协调与锁
元数据存储
服务发现

1.2 Raft共识算法理论

1.2.1 Raft概述

Raft是一种用于管理复制日志的共识算法，比Paxos更容易理解和实现。Raft将问题分解为三个独立子问题：

Leader选举
日志复制
安全性保证

1.2.2 Raft节点状态

每个节点处于三种状态之一：
#mermaid-svg-aQd2TsXOcH6skJ2j{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-aQd2TsXOcH6skJ2j .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-aQd2TsXOcH6skJ2j .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-aQd2TsXOcH6skJ2j .error-icon{fill:#552222;}#mermaid-svg-aQd2TsXOcH6skJ2j .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-aQd2TsXOcH6skJ2j .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-aQd2TsXOcH6skJ2j .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-aQd2TsXOcH6skJ2j .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-aQd2TsXOcH6skJ2j .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-aQd2TsXOcH6skJ2j .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-aQd2TsXOcH6skJ2j .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-aQd2TsXOcH6skJ2j .marker{fill:#333333;stroke:#333333;}#mermaid-svg-aQd2TsXOcH6skJ2j .marker.cross{stroke:#333333;}#mermaid-svg-aQd2TsXOcH6skJ2j svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-aQd2TsXOcH6skJ2j p{margin:0;}#mermaid-svg-aQd2TsXOcH6skJ2j defs #statediagram-barbEnd{fill:#333333;stroke:#333333;}#mermaid-svg-aQd2TsXOcH6skJ2j g.stateGroup text{fill:#9370DB;stroke:none;font-size:10px;}#mermaid-svg-aQd2TsXOcH6skJ2j g.stateGroup text{fill:#333;stroke:none;font-size:10px;}#mermaid-svg-aQd2TsXOcH6skJ2j g.stateGroup .state-title{font-weight:bolder;fill:#131300;}#mermaid-svg-aQd2TsXOcH6skJ2j g.stateGroup rect{fill:#ECECFF;stroke:#9370DB;}#mermaid-svg-aQd2TsXOcH6skJ2j g.stateGroup line{stroke:#333333;stroke-width:1;}#mermaid-svg-aQd2TsXOcH6skJ2j .transition{stroke:#333333;stroke-width:1;fill:none;}#mermaid-svg-aQd2TsXOcH6skJ2j .stateGroup .composit{fill:white;border-bottom:1px;}#mermaid-svg-aQd2TsXOcH6skJ2j .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px;}#mermaid-svg-aQd2TsXOcH6skJ2j .state-note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-aQd2TsXOcH6skJ2j .state-note text{fill:black;stroke:none;font-size:10px;}#mermaid-svg-aQd2TsXOcH6skJ2j .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5;}#mermaid-svg-aQd2TsXOcH6skJ2j .edgeLabel .label rect{fill:#ECECFF;opacity:0.5;}#mermaid-svg-aQd2TsXOcH6skJ2j .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-aQd2TsXOcH6skJ2j .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-aQd2TsXOcH6skJ2j .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-aQd2TsXOcH6skJ2j .edgeLabel .label text{fill:#333;}#mermaid-svg-aQd2TsXOcH6skJ2j .label div .edgeLabel{color:#333;}#mermaid-svg-aQd2TsXOcH6skJ2j .stateLabel text{fill:#131300;font-size:10px;font-weight:bold;}#mermaid-svg-aQd2TsXOcH6skJ2j .node circle.state-start{fill:#333333;stroke:#333333;}#mermaid-svg-aQd2TsXOcH6skJ2j .node .fork-join{fill:#333333;stroke:#333333;}#mermaid-svg-aQd2TsXOcH6skJ2j .node circle.state-end{fill:#9370DB;stroke:white;stroke-width:1.5;}#mermaid-svg-aQd2TsXOcH6skJ2j .end-state-inner{fill:white;stroke-width:1.5;}#mermaid-svg-aQd2TsXOcH6skJ2j .node rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-aQd2TsXOcH6skJ2j .node polygon{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-aQd2TsXOcH6skJ2j #statediagram-barbEnd{fill:#333333;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-cluster rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-aQd2TsXOcH6skJ2j .cluster-label,#mermaid-svg-aQd2TsXOcH6skJ2j .nodeLabel{color:#131300;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-cluster rect.outer{rx:5px;ry:5px;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-state .divider{stroke:#9370DB;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-state .title-state{rx:5px;ry:5px;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-cluster.statediagram-cluster .inner{fill:white;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-cluster.statediagram-cluster-alt .inner{fill:#f0f0f0;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-cluster .inner{rx:0;ry:0;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-state rect.basic{rx:5px;ry:5px;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#f0f0f0;}#mermaid-svg-aQd2TsXOcH6skJ2j .note-edge{stroke-dasharray:5;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-note text{fill:black;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram-note .nodeLabel{color:black;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagram .edgeLabel{color:red;}#mermaid-svg-aQd2TsXOcH6skJ2j #dependencyStart,#mermaid-svg-aQd2TsXOcH6skJ2j #dependencyEnd{fill:#333333;stroke:#333333;stroke-width:1;}#mermaid-svg-aQd2TsXOcH6skJ2j .statediagramTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-aQd2TsXOcH6skJ2j :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 超时未收到心跳
获得多数投票
收到更高term的RPC
选举超时
收到更高term的RPC
Follower
Candidate
Leader

1.2.3 Leader选举

选举流程

Follower等待来自Leader的心跳
如果超时（150ms~300ms随机），转换为Candidate
Candidate增加当前term，投自己一票，并行向其他节点发送RequestVote RPC
三种可能结果：
- 获得多数票 → 成为Leader
- 收到其他合法Leader的AppendEntries → 退回Follower
- 一段时间无结果 → 新的选举

源码对应

Raft核心模块：go.etcd.io/raft/v3
选举逻辑在raft模块中实现

1.2.4 日志复制

复制流程

Followers Leader Client Followers Leader Client #mermaid-svg-bJHo2stZyknVtQ8o{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-bJHo2stZyknVtQ8o .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-bJHo2stZyknVtQ8o .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-bJHo2stZyknVtQ8o .error-icon{fill:#552222;}#mermaid-svg-bJHo2stZyknVtQ8o .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-bJHo2stZyknVtQ8o .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-bJHo2stZyknVtQ8o .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-bJHo2stZyknVtQ8o .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-bJHo2stZyknVtQ8o .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-bJHo2stZyknVtQ8o .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-bJHo2stZyknVtQ8o .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-bJHo2stZyknVtQ8o .marker{fill:#333333;stroke:#333333;}#mermaid-svg-bJHo2stZyknVtQ8o .marker.cross{stroke:#333333;}#mermaid-svg-bJHo2stZyknVtQ8o svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-bJHo2stZyknVtQ8o p{margin:0;}#mermaid-svg-bJHo2stZyknVtQ8o .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-bJHo2stZyknVtQ8o text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-bJHo2stZyknVtQ8o .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-bJHo2stZyknVtQ8o .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-bJHo2stZyknVtQ8o .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-bJHo2stZyknVtQ8o .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-bJHo2stZyknVtQ8o #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-bJHo2stZyknVtQ8o .sequenceNumber{fill:white;}#mermaid-svg-bJHo2stZyknVtQ8o #sequencenumber{fill:#333;}#mermaid-svg-bJHo2stZyknVtQ8o #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-bJHo2stZyknVtQ8o .messageText{fill:#333;stroke:none;}#mermaid-svg-bJHo2stZyknVtQ8o .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-bJHo2stZyknVtQ8o .labelText,#mermaid-svg-bJHo2stZyknVtQ8o .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-bJHo2stZyknVtQ8o .loopText,#mermaid-svg-bJHo2stZyknVtQ8o .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-bJHo2stZyknVtQ8o .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-bJHo2stZyknVtQ8o .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-bJHo2stZyknVtQ8o .noteText,#mermaid-svg-bJHo2stZyknVtQ8o .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-bJHo2stZyknVtQ8o .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-bJHo2stZyknVtQ8o .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-bJHo2stZyknVtQ8o .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-bJHo2stZyknVtQ8o .actorPopupMenu{position:absolute;}#mermaid-svg-bJHo2stZyknVtQ8o .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-bJHo2stZyknVtQ8o .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-bJHo2stZyknVtQ8o .actor-man circle,#mermaid-svg-bJHo2stZyknVtQ8o line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-bJHo2stZyknVtQ8o :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 发送请求追加日志到本地发送AppendEntries RPC 确认接收提交日志返回结果

日志结构

复制代码

每个日志条目包含：
- term：创建时的任期号
- index：日志在日志中的位置
- command：状态机命令

1.2.5 安全性保证

保证	说明
选举安全性	一个term内最多一个Leader
Leader完整性	Leader拥有所有已提交的日志
日志匹配	相同index和term的日志相同
Leader追加	Leader只追加日志，不覆盖
状态机安全	只有包含所有已提交条目的节点才能成为Leader

1.2.6 快照机制

当日志过大时，Raft使用快照机制：

记录当前状态机状态
丢弃快照之前的日志
帮助落后节点快速追赶

1.3 MVCC与WAL原理

1.3.1 MVCC概述

MVCC（Multi-Version Concurrency Control，多版本并发控制）允许读写操作并发执行而不阻塞。

1.3.2 etcd中的MVCC实现

版本管理

复制代码

每个键的每个修改都创建新版本：
key1: {version: 1, value: "v1"}  → Put
key1: {version: 2, value: "v2"}  → Put
key1: {version: 3, value: "v3"}  → Put

版本压缩

保留最近N个版本
压缩旧版本以节省空间

1.3.3 WAL（Write-Ahead Log）预写日志

WAL作用

持久化：保证操作不丢失
崩溃恢复：重启时恢复状态
同步复制：确保集群一致性

WAL特性

特性	说明
顺序写入	只追加，性能好
分段存储	按大小切分文件
CRC校验	验证完整性
同步刷盘	fsync保证持久化

WAL文件结构

复制代码

wal/
├── 0000000000000000-0000000000000000.wal  # 第1段
├── 0000000000000001-0000000000001000.wal  # 第2段
└── ...

1.4 BoltDB存储基础

1.4.1 BoltDB概述

BoltDB是一个嵌入式键值存储，提供ACID事务和B+树索引。

1.4.2 核心特性

特性	说明
纯Go实现	无CGO依赖
嵌入式	单文件存储
B+树索引	支持范围查询
事务支持	ACID保证
无锁读	MVCC实现

1.4.3 数据结构

Bucket结构

复制代码

DB
├── key (bucket)
│   ├── subkey1: value1
│   └── subkey2: value2
└── another_key (bucket)
    └── ...

源码位置

复制代码

go.etcd.io/bbolt (外部依赖)
server/storage/backend/  (etcd封装层)

1.5 etcd v2与v3架构差异

1.5.1 核心差异对比

维度	v2	v3
API	HTTP/JSON	gRPC/Protobuf
存储	内存树 + 快照	BoltDB + MVCC
Watch机制	HTTP长轮询	gRPC流式
租约	TTL独立管理	Lease集成
事务	无	支持多操作事务
性能	较低	更高

1.5.2 v3架构改进

#mermaid-svg-4mMQxMWJqOInzHku{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-4mMQxMWJqOInzHku .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-4mMQxMWJqOInzHku .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-4mMQxMWJqOInzHku .error-icon{fill:#552222;}#mermaid-svg-4mMQxMWJqOInzHku .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-4mMQxMWJqOInzHku .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-4mMQxMWJqOInzHku .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-4mMQxMWJqOInzHku .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-4mMQxMWJqOInzHku .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-4mMQxMWJqOInzHku .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-4mMQxMWJqOInzHku .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-4mMQxMWJqOInzHku .marker{fill:#333333;stroke:#333333;}#mermaid-svg-4mMQxMWJqOInzHku .marker.cross{stroke:#333333;}#mermaid-svg-4mMQxMWJqOInzHku svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-4mMQxMWJqOInzHku p{margin:0;}#mermaid-svg-4mMQxMWJqOInzHku .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-4mMQxMWJqOInzHku .cluster-label text{fill:#333;}#mermaid-svg-4mMQxMWJqOInzHku .cluster-label span{color:#333;}#mermaid-svg-4mMQxMWJqOInzHku .cluster-label span p{background-color:transparent;}#mermaid-svg-4mMQxMWJqOInzHku .label text,#mermaid-svg-4mMQxMWJqOInzHku span{fill:#333;color:#333;}#mermaid-svg-4mMQxMWJqOInzHku .node rect,#mermaid-svg-4mMQxMWJqOInzHku .node circle,#mermaid-svg-4mMQxMWJqOInzHku .node ellipse,#mermaid-svg-4mMQxMWJqOInzHku .node polygon,#mermaid-svg-4mMQxMWJqOInzHku .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-4mMQxMWJqOInzHku .rough-node .label text,#mermaid-svg-4mMQxMWJqOInzHku .node .label text,#mermaid-svg-4mMQxMWJqOInzHku .image-shape .label,#mermaid-svg-4mMQxMWJqOInzHku .icon-shape .label{text-anchor:middle;}#mermaid-svg-4mMQxMWJqOInzHku .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-4mMQxMWJqOInzHku .rough-node .label,#mermaid-svg-4mMQxMWJqOInzHku .node .label,#mermaid-svg-4mMQxMWJqOInzHku .image-shape .label,#mermaid-svg-4mMQxMWJqOInzHku .icon-shape .label{text-align:center;}#mermaid-svg-4mMQxMWJqOInzHku .node.clickable{cursor:pointer;}#mermaid-svg-4mMQxMWJqOInzHku .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-4mMQxMWJqOInzHku .arrowheadPath{fill:#333333;}#mermaid-svg-4mMQxMWJqOInzHku .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-4mMQxMWJqOInzHku .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-4mMQxMWJqOInzHku .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-4mMQxMWJqOInzHku .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-4mMQxMWJqOInzHku .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-4mMQxMWJqOInzHku .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-4mMQxMWJqOInzHku .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-4mMQxMWJqOInzHku .cluster text{fill:#333;}#mermaid-svg-4mMQxMWJqOInzHku .cluster span{color:#333;}#mermaid-svg-4mMQxMWJqOInzHku div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-4mMQxMWJqOInzHku .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-4mMQxMWJqOInzHku rect.text{fill:none;stroke-width:0;}#mermaid-svg-4mMQxMWJqOInzHku .icon-shape,#mermaid-svg-4mMQxMWJqOInzHku .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-4mMQxMWJqOInzHku .icon-shape p,#mermaid-svg-4mMQxMWJqOInzHku .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-4mMQxMWJqOInzHku .icon-shape .label rect,#mermaid-svg-4mMQxMWJqOInzHku .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-4mMQxMWJqOInzHku .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-4mMQxMWJqOInzHku .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-4mMQxMWJqOInzHku :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 客户端v3
gRPC层
API网关
Raft层
认证鉴权
WAL日志
状态机应用
MVCC存储
BoltDB

1.6 源码编译与环境配置

1.6.1 环境要求

bash 复制代码

# Go版本要求
Go 1.26+ (根据go.mod)

# 系统要求
Linux/MacOS/Windows

1.6.2 编译步骤

bash 复制代码

# 1. 克隆代码
cd D:/dev/code/github/etcd

# 2. 编译etcd
go build -o bin/etcd ./server

# 3. 编译etcdctl
go build -o bin/etcdctl ./etcdctl

# 4. 使用Makefile
make build

1.6.3 单节点启动

bash 复制代码

# 方式1：直接启动
./bin/etcd

# 方式2：指定数据目录
./bin/etcd --data-dir /tmp/etcd-data

# 方式3：指定监听地址
./bin/etcd --listen-client-urls http://localhost:2379 \
           --advertise-client-urls http://localhost:2379

1.6.4 集群启动

bash 复制代码

# 使用Procfile启动3节点集群
# 需要先安装goreman
go install github.com/mattn/goreman@latest

# 启动集群
goreman start

1.6.5 测试命令

bash 复制代码

# 运行单元测试
go test ./server/...

# 运行e2e测试
cd tests
./e2e-test.sh

# 运行性能测试
go test -bench=. ./server/etcdserver

1.7 项目目录结构

1.7.1 根目录结构

复制代码

etcd/
├── api/             # API定义 (Protobuf/gRPC)
├── cache/           # 缓存组件
├── client/          # 客户端库
│   ├── pkg/         # 客户端公共包
│   └── v3/          # v3客户端实现
├── contrib/         # 贡献代码和示例
├── Documentation/   # 官方文档
├── etcdctl/         # 命令行工具
├── etcdutl/         # etcd工具集
├── hack/            # 开发脚本
├── logos/           # Logo资源
├── pkg/             # 公共工具包
├── scripts/         # 构建脚本
├── security/        # 安全相关
├── server/          # 服务端实现 (核心)
├── tests/           # 测试代码
└── tools/           # 工具集合

1.7.2 server目录详解（核心）

复制代码

server/
├── main.go                 # 入口点（简单包装）
├── auth/                   # 认证授权
├── config/                 # 配置管理
├── embed/                  # 嵌入式etcd
├── etcdmain/               # 主程序逻辑
├── etcdserver/             # etcd服务核心
│   ├── api/                # API层
│   ├── apply/              # 状态机应用
│   ├── cindex/             # 一致性索引
│   ├── errors/             # 错误定义
│   ├── read/               # 只读请求处理
│   ├── txn/                # 事务处理
│   ├── version/            # 版本管理
│   ├── server.go           # 核心服务
│   └── v3_server.go        # v3 API服务
├── features/               # 特性开关
├── lease/                  # 租约管理
├── proxy/                  # 代理
├── storage/                # 存储层
│   ├── backend/            # BoltDB后端封装
│   ├── mvcc/               # MVCC实现
│   ├── schema/             # 数据模式
│   └── wal/                # WAL日志
└── verify/                 # 验证工具

1.7.3 pkg目录详解

复制代码

pkg/
├── adt/            # 抽象数据类型
├── contention/     # 竞争检测
├── cpuutil/        # CPU工具
├── crc/            # 校验码
├── debugutil/      # 调试工具
├── expect/         # 测试断言
├── featuregate/    # 特性门控
├── flags/          # 命令行标志
├── grpctesting/    # gRPC测试工具
├── httputil/       # HTTP工具
├── idutil/         # ID工具
├── ioutil/         # IO工具
├── netutil/        # 网络工具
├── notify/         # 通知机制
├── osutil/         # 系统工具
├── pbutil/         # Protobuf工具
├── report/         # 报告工具
├── runtime/        # 运行时工具
├── schedule/       # 调度器
├── stringutil/     # 字符串工具
├── traceutil/      # 追踪工具
└── wait/           # 等待机制

1.7.4 核心入口文件

文件	作用
`server/main.go`	主入口（简单包装）
`server/etcdmain/`	实际主程序逻辑
`server/etcdserver/server.go`	etcd核心服务
`server/etcdserver/v3_server.go`	v3 API实现

1.7.5 源码阅读路径建议

复制代码

第一阶段：理解入口
server/main.go → server/etcdmain/ → server/etcdserver/server.go

第二阶段：理解请求流程
client/v3/ → server/etcdserver/v3_server.go → server/etcdserver/apply/

第三阶段：理解Raft
(外部模块) go.etcd.io/raft/v3/

第四阶段：理解存储
server/storage/wal/ → server/storage/mvcc/ → server/storage/backend/

参考文献

02 - etcd整体分层架构

2.1 分层架构总览

etcd采用清晰的分层设计，从客户端到底层存储共分为5层：
#mermaid-svg-63TkHj5tO3jJ6dRF{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-63TkHj5tO3jJ6dRF .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-63TkHj5tO3jJ6dRF .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-63TkHj5tO3jJ6dRF .error-icon{fill:#552222;}#mermaid-svg-63TkHj5tO3jJ6dRF .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-63TkHj5tO3jJ6dRF .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-63TkHj5tO3jJ6dRF .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-63TkHj5tO3jJ6dRF .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-63TkHj5tO3jJ6dRF .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-63TkHj5tO3jJ6dRF .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-63TkHj5tO3jJ6dRF .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-63TkHj5tO3jJ6dRF .marker{fill:#333333;stroke:#333333;}#mermaid-svg-63TkHj5tO3jJ6dRF .marker.cross{stroke:#333333;}#mermaid-svg-63TkHj5tO3jJ6dRF svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-63TkHj5tO3jJ6dRF p{margin:0;}#mermaid-svg-63TkHj5tO3jJ6dRF .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-63TkHj5tO3jJ6dRF .cluster-label text{fill:#333;}#mermaid-svg-63TkHj5tO3jJ6dRF .cluster-label span{color:#333;}#mermaid-svg-63TkHj5tO3jJ6dRF .cluster-label span p{background-color:transparent;}#mermaid-svg-63TkHj5tO3jJ6dRF .label text,#mermaid-svg-63TkHj5tO3jJ6dRF span{fill:#333;color:#333;}#mermaid-svg-63TkHj5tO3jJ6dRF .node rect,#mermaid-svg-63TkHj5tO3jJ6dRF .node circle,#mermaid-svg-63TkHj5tO3jJ6dRF .node ellipse,#mermaid-svg-63TkHj5tO3jJ6dRF .node polygon,#mermaid-svg-63TkHj5tO3jJ6dRF .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-63TkHj5tO3jJ6dRF .rough-node .label text,#mermaid-svg-63TkHj5tO3jJ6dRF .node .label text,#mermaid-svg-63TkHj5tO3jJ6dRF .image-shape .label,#mermaid-svg-63TkHj5tO3jJ6dRF .icon-shape .label{text-anchor:middle;}#mermaid-svg-63TkHj5tO3jJ6dRF .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-63TkHj5tO3jJ6dRF .rough-node .label,#mermaid-svg-63TkHj5tO3jJ6dRF .node .label,#mermaid-svg-63TkHj5tO3jJ6dRF .image-shape .label,#mermaid-svg-63TkHj5tO3jJ6dRF .icon-shape .label{text-align:center;}#mermaid-svg-63TkHj5tO3jJ6dRF .node.clickable{cursor:pointer;}#mermaid-svg-63TkHj5tO3jJ6dRF .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-63TkHj5tO3jJ6dRF .arrowheadPath{fill:#333333;}#mermaid-svg-63TkHj5tO3jJ6dRF .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-63TkHj5tO3jJ6dRF .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-63TkHj5tO3jJ6dRF .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-63TkHj5tO3jJ6dRF .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-63TkHj5tO3jJ6dRF .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-63TkHj5tO3jJ6dRF .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-63TkHj5tO3jJ6dRF .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-63TkHj5tO3jJ6dRF .cluster text{fill:#333;}#mermaid-svg-63TkHj5tO3jJ6dRF .cluster span{color:#333;}#mermaid-svg-63TkHj5tO3jJ6dRF div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-63TkHj5tO3jJ6dRF .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-63TkHj5tO3jJ6dRF rect.text{fill:none;stroke-width:0;}#mermaid-svg-63TkHj5tO3jJ6dRF .icon-shape,#mermaid-svg-63TkHj5tO3jJ6dRF .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-63TkHj5tO3jJ6dRF .icon-shape p,#mermaid-svg-63TkHj5tO3jJ6dRF .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-63TkHj5tO3jJ6dRF .icon-shape .label rect,#mermaid-svg-63TkHj5tO3jJ6dRF .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-63TkHj5tO3jJ6dRF .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-63TkHj5tO3jJ6dRF .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-63TkHj5tO3jJ6dRF :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 存储持久层
Raft共识层
etcdserver服务层
API网关层
客户端层
client/v3

客户端库
etcdctl

命令行工具
v3rpc

gRPC服务
auth

认证鉴权
api

HTTP API
EtcdServer

核心服务
txn

事务处理
apply

状态机应用
lease

租约管理
raftNode

Raft节点封装
rafthttp

网络传输
go.etcd.io/raft/v3

Raft算法库
wal

预写日志
snap

快照管理
mvcc

多版本存储
backend

BoltDB封装
BoltDB

底层存储

2.1.1 各层职责

层级	职责	关键文件
客户端层	提供用户友好的API	`client/v3/`
API网关层	gRPC/HTTP服务、鉴权	`server/etcdserver/api/v3rpc/`
etcdserver服务层	业务逻辑、事务、租约	`server/etcdserver/`
Raft共识层	分布式一致性	`server/etcdserver/api/rafthttp/`
存储持久层	WAL、MVCC、BoltDB	`server/storage/`

2.2 全局请求流转

2.2.1 写请求完整流程

MVCC存储 apply应用 Raft状态机 WAL日志 raftNode EtcdServer v3rpc层客户端 MVCC存储 apply应用 Raft状态机 WAL日志 raftNode EtcdServer v3rpc层客户端 #mermaid-svg-3cufupYJPmsShsw7{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-3cufupYJPmsShsw7 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-3cufupYJPmsShsw7 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-3cufupYJPmsShsw7 .error-icon{fill:#552222;}#mermaid-svg-3cufupYJPmsShsw7 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-3cufupYJPmsShsw7 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-3cufupYJPmsShsw7 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-3cufupYJPmsShsw7 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-3cufupYJPmsShsw7 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-3cufupYJPmsShsw7 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-3cufupYJPmsShsw7 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-3cufupYJPmsShsw7 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-3cufupYJPmsShsw7 .marker.cross{stroke:#333333;}#mermaid-svg-3cufupYJPmsShsw7 svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-3cufupYJPmsShsw7 p{margin:0;}#mermaid-svg-3cufupYJPmsShsw7 .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-3cufupYJPmsShsw7 text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-3cufupYJPmsShsw7 .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-3cufupYJPmsShsw7 .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-3cufupYJPmsShsw7 .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-3cufupYJPmsShsw7 .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-3cufupYJPmsShsw7 #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-3cufupYJPmsShsw7 .sequenceNumber{fill:white;}#mermaid-svg-3cufupYJPmsShsw7 #sequencenumber{fill:#333;}#mermaid-svg-3cufupYJPmsShsw7 #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-3cufupYJPmsShsw7 .messageText{fill:#333;stroke:none;}#mermaid-svg-3cufupYJPmsShsw7 .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-3cufupYJPmsShsw7 .labelText,#mermaid-svg-3cufupYJPmsShsw7 .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-3cufupYJPmsShsw7 .loopText,#mermaid-svg-3cufupYJPmsShsw7 .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-3cufupYJPmsShsw7 .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-3cufupYJPmsShsw7 .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-3cufupYJPmsShsw7 .noteText,#mermaid-svg-3cufupYJPmsShsw7 .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-3cufupYJPmsShsw7 .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-3cufupYJPmsShsw7 .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-3cufupYJPmsShsw7 .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-3cufupYJPmsShsw7 .actorPopupMenu{position:absolute;}#mermaid-svg-3cufupYJPmsShsw7 .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-3cufupYJPmsShsw7 .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-3cufupYJPmsShsw7 .actor-man circle,#mermaid-svg-3cufupYJPmsShsw7 line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-3cufupYJPmsShsw7 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 认证鉴权检查 Leader复制到Followers Put请求 Range/Put/Delete/Txn 序列化请求提交提案 (Propose) Step追加日志持久化日志持久化确认等待提交已提交通知应用到状态机更新键值确认应用完成返回响应响应结果

2.2.2 读请求优化流程

etcd支持两种读模式：

模式	说明	一致性	性能
Linearizable	线性读	强一致	稍慢
Serializable	串行化读	会话一致	快

线性读流程

go 复制代码

// server/etcdserver/v3_server.go
func (s *EtcdServer) Range(ctx context.Context, r *pb.RangeRequest) (*pb.RangeResponse, error) {
    if !r.Serializable {
        // 线性化读：需与Leader确认
        err = s.read.LinearizableReadNotify(ctx)
    }
    // ...
}

2.3 核心组件依赖关系

2.3.1 EtcdServer核心结构体

源码位置

server/etcdserver/server.go

核心字段示意

go 复制代码

type EtcdServer struct {
    // Raft层
    r *raftNode
    raftReadyHandler *raftReadyHandler

    // 存储层
    kv mvcc.ConsistentWatchableKV
    lessor lease.Lessor

    // 应用层
    applyV3 applierV3
    authStore auth.AuthStore

    // 集群成员
    cluster *membership.RaftCluster
}

2.3.2 raftNode结构体

源码位置

server/etcdserver/raft.go

go 复制代码

type raftNode struct {
    lg *zap.Logger

    tickMu *sync.RWMutex
    latestTickTs time.Time
    raftNodeConfig

    msgSnapC chan *raftpb.Message
    applyc chan toApply
    readStateC chan raft.ReadState

    ticker *time.Ticker
    td *contention.TimeoutDetector

    stopped chan struct{}
    done    chan struct{}
}

2.3.3 组件依赖图

#mermaid-svg-P1fj18nY414iNYhT{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-P1fj18nY414iNYhT .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-P1fj18nY414iNYhT .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-P1fj18nY414iNYhT .error-icon{fill:#552222;}#mermaid-svg-P1fj18nY414iNYhT .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-P1fj18nY414iNYhT .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-P1fj18nY414iNYhT .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-P1fj18nY414iNYhT .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-P1fj18nY414iNYhT .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-P1fj18nY414iNYhT .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-P1fj18nY414iNYhT .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-P1fj18nY414iNYhT .marker{fill:#333333;stroke:#333333;}#mermaid-svg-P1fj18nY414iNYhT .marker.cross{stroke:#333333;}#mermaid-svg-P1fj18nY414iNYhT svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-P1fj18nY414iNYhT p{margin:0;}#mermaid-svg-P1fj18nY414iNYhT .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-P1fj18nY414iNYhT .cluster-label text{fill:#333;}#mermaid-svg-P1fj18nY414iNYhT .cluster-label span{color:#333;}#mermaid-svg-P1fj18nY414iNYhT .cluster-label span p{background-color:transparent;}#mermaid-svg-P1fj18nY414iNYhT .label text,#mermaid-svg-P1fj18nY414iNYhT span{fill:#333;color:#333;}#mermaid-svg-P1fj18nY414iNYhT .node rect,#mermaid-svg-P1fj18nY414iNYhT .node circle,#mermaid-svg-P1fj18nY414iNYhT .node ellipse,#mermaid-svg-P1fj18nY414iNYhT .node polygon,#mermaid-svg-P1fj18nY414iNYhT .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-P1fj18nY414iNYhT .rough-node .label text,#mermaid-svg-P1fj18nY414iNYhT .node .label text,#mermaid-svg-P1fj18nY414iNYhT .image-shape .label,#mermaid-svg-P1fj18nY414iNYhT .icon-shape .label{text-anchor:middle;}#mermaid-svg-P1fj18nY414iNYhT .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-P1fj18nY414iNYhT .rough-node .label,#mermaid-svg-P1fj18nY414iNYhT .node .label,#mermaid-svg-P1fj18nY414iNYhT .image-shape .label,#mermaid-svg-P1fj18nY414iNYhT .icon-shape .label{text-align:center;}#mermaid-svg-P1fj18nY414iNYhT .node.clickable{cursor:pointer;}#mermaid-svg-P1fj18nY414iNYhT .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-P1fj18nY414iNYhT .arrowheadPath{fill:#333333;}#mermaid-svg-P1fj18nY414iNYhT .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-P1fj18nY414iNYhT .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-P1fj18nY414iNYhT .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-P1fj18nY414iNYhT .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-P1fj18nY414iNYhT .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-P1fj18nY414iNYhT .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-P1fj18nY414iNYhT .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-P1fj18nY414iNYhT .cluster text{fill:#333;}#mermaid-svg-P1fj18nY414iNYhT .cluster span{color:#333;}#mermaid-svg-P1fj18nY414iNYhT div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-P1fj18nY414iNYhT .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-P1fj18nY414iNYhT rect.text{fill:none;stroke-width:0;}#mermaid-svg-P1fj18nY414iNYhT .icon-shape,#mermaid-svg-P1fj18nY414iNYhT .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-P1fj18nY414iNYhT .icon-shape p,#mermaid-svg-P1fj18nY414iNYhT .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-P1fj18nY414iNYhT .icon-shape .label rect,#mermaid-svg-P1fj18nY414iNYhT .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-P1fj18nY414iNYhT .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-P1fj18nY414iNYhT .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-P1fj18nY414iNYhT :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} EtcdServer
raftNode
applierV3
lease.Lessor
auth.AuthStore
mvcc.KV
raft.Node
WAL存储
rafthttp.Transport
backend.Backend
BoltDB

2.4 模块间交互边界设计

2.4.1 接口定义

RaftKV接口

源码位置 : server/etcdserver/v3_server.go

go 复制代码

type RaftKV interface {
    Range(ctx context.Context, r *pb.RangeRequest) (*pb.RangeResponse, error)
    RangeStream(r *pb.RangeRequest, rs pb.KV_RangeStreamServer) error
    Put(ctx context.Context, r *pb.PutRequest) (*pb.PutResponse, error)
    DeleteRange(ctx context.Context, r *pb.DeleteRangeRequest) (*pb.DeleteRangeResponse, error)
    Txn(ctx context.Context, r *pb.TxnRequest) (*pb.TxnResponse, error)
    Compact(ctx context.Context, r *pb.CompactionRequest) (*pb.CompactionResponse, error)
}

Lessor接口

go 复制代码

type Lessor interface {
    LeaseGrant(ctx context.Context, r *pb.LeaseGrantRequest) (*pb.LeaseGrantResponse, error)
    LeaseRevoke(ctx context.Context, r *pb.LeaseRevokeRequest) (*pb.LeaseRevokeResponse, error)
    LeaseRenew(ctx context.Context, id lease.LeaseID) (int64, error)
    LeaseTimeToLive(ctx context.Context, r *pb.LeaseTimeToLiveRequest) (*pb.LeaseTimeToLiveResponse, error)
    LeaseLeases(ctx context.Context, r *pb.LeaseLeasesRequest) (*pb.LeaseLeasesResponse, error)
}

Authenticator接口

go 复制代码

type Authenticator interface {
    AuthEnable(ctx context.Context, r *pb.AuthEnableRequest) (*pb.AuthEnableResponse, error)
    AuthDisable(ctx context.Context, r *pb.AuthDisableRequest) (*pb.AuthDisableResponse, error)
    // ... 其他认证方法
}

2.4.2 关键交互流程

1. 提案提交与应用流程

复制代码

EtcdServer.Propose()
    ↓
raftNode.Propose()
    ↓
raft.Node.Step()
    ↓
WAL持久化
    ↓
(复制到Followers)
    ↓
raftNode.applyc
    ↓
EtcdServer.applyAll()
    ↓
applierV3.Apply()
    ↓
MVCC更新

2. Ready处理循环

源码位置 : server/etcdserver/raft.go - raftNode.start()

go 复制代码

func (r *raftNode) start(rh *raftReadyHandler) {
    go func() {
        for {
            select {
            case <-r.ticker.C:
                r.tick()
            case rd := <-r.Ready():
                // 处理Raft Ready
                // 1. 更新Leader状态
                // 2. 保存快照和WAL
                // 3. 发送待应用条目
                // 4. 发送消息给其他节点
                r.Advance()
            }
        }
    }()
}

3. toApply结构体

源码位置 : server/etcdserver/raft.go

go 复制代码

// toApply包含待应用的entries和snapshot
type toApply struct {
    entries  []*raftpb.Entry
    snapshot *raftpb.Snapshot
    notifyc  chan struct{}
    raftAdvancedC <-chan struct{}
}

2.4.3 模块边界总结

边界	接口	作用
客户端 ↔ 服务端	gRPC/Protobuf	网络通信
API层 ↔ etcdserver	RaftKV/Lessor	业务抽象
etcdserver ↔ Raft	Propose/Ready	共识提交
etcdserver ↔ 存储	applierV3	状态应用
Raft ↔ 存储	Storage接口	日志持久化

参考文献

etcd架构设计文档

03 - 客户端v3 gRPC模块源码解析

3.1 客户端设计概览

3.1.1 核心接口

源码位置

client/v3/kv.go、client/v3/txn.go、client/v3/lease.go、client/v3/watch.go

核心接口图

#mermaid-svg-SC1iyiCERV8ZQU0B{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-SC1iyiCERV8ZQU0B .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-SC1iyiCERV8ZQU0B .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-SC1iyiCERV8ZQU0B .error-icon{fill:#552222;}#mermaid-svg-SC1iyiCERV8ZQU0B .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-SC1iyiCERV8ZQU0B .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-SC1iyiCERV8ZQU0B .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-SC1iyiCERV8ZQU0B .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-SC1iyiCERV8ZQU0B .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-SC1iyiCERV8ZQU0B .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-SC1iyiCERV8ZQU0B .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-SC1iyiCERV8ZQU0B .marker{fill:#333333;stroke:#333333;}#mermaid-svg-SC1iyiCERV8ZQU0B .marker.cross{stroke:#333333;}#mermaid-svg-SC1iyiCERV8ZQU0B svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-SC1iyiCERV8ZQU0B p{margin:0;}#mermaid-svg-SC1iyiCERV8ZQU0B .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-SC1iyiCERV8ZQU0B .cluster-label text{fill:#333;}#mermaid-svg-SC1iyiCERV8ZQU0B .cluster-label span{color:#333;}#mermaid-svg-SC1iyiCERV8ZQU0B .cluster-label span p{background-color:transparent;}#mermaid-svg-SC1iyiCERV8ZQU0B .label text,#mermaid-svg-SC1iyiCERV8ZQU0B span{fill:#333;color:#333;}#mermaid-svg-SC1iyiCERV8ZQU0B .node rect,#mermaid-svg-SC1iyiCERV8ZQU0B .node circle,#mermaid-svg-SC1iyiCERV8ZQU0B .node ellipse,#mermaid-svg-SC1iyiCERV8ZQU0B .node polygon,#mermaid-svg-SC1iyiCERV8ZQU0B .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-SC1iyiCERV8ZQU0B .rough-node .label text,#mermaid-svg-SC1iyiCERV8ZQU0B .node .label text,#mermaid-svg-SC1iyiCERV8ZQU0B .image-shape .label,#mermaid-svg-SC1iyiCERV8ZQU0B .icon-shape .label{text-anchor:middle;}#mermaid-svg-SC1iyiCERV8ZQU0B .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-SC1iyiCERV8ZQU0B .rough-node .label,#mermaid-svg-SC1iyiCERV8ZQU0B .node .label,#mermaid-svg-SC1iyiCERV8ZQU0B .image-shape .label,#mermaid-svg-SC1iyiCERV8ZQU0B .icon-shape .label{text-align:center;}#mermaid-svg-SC1iyiCERV8ZQU0B .node.clickable{cursor:pointer;}#mermaid-svg-SC1iyiCERV8ZQU0B .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-SC1iyiCERV8ZQU0B .arrowheadPath{fill:#333333;}#mermaid-svg-SC1iyiCERV8ZQU0B .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-SC1iyiCERV8ZQU0B .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-SC1iyiCERV8ZQU0B .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-SC1iyiCERV8ZQU0B .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-SC1iyiCERV8ZQU0B .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-SC1iyiCERV8ZQU0B .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-SC1iyiCERV8ZQU0B .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-SC1iyiCERV8ZQU0B .cluster text{fill:#333;}#mermaid-svg-SC1iyiCERV8ZQU0B .cluster span{color:#333;}#mermaid-svg-SC1iyiCERV8ZQU0B div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-SC1iyiCERV8ZQU0B .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-SC1iyiCERV8ZQU0B rect.text{fill:none;stroke-width:0;}#mermaid-svg-SC1iyiCERV8ZQU0B .icon-shape,#mermaid-svg-SC1iyiCERV8ZQU0B .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-SC1iyiCERV8ZQU0B .icon-shape p,#mermaid-svg-SC1iyiCERV8ZQU0B .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-SC1iyiCERV8ZQU0B .icon-shape .label rect,#mermaid-svg-SC1iyiCERV8ZQU0B .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-SC1iyiCERV8ZQU0B .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-SC1iyiCERV8ZQU0B .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-SC1iyiCERV8ZQU0B :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} If
Then
Else
Commit
Client
KV
Txn创建
Lease
Watcher
Auth
Maintenance
Cluster
Put
Get
Delete
Do
Txn
Compare
Op
Op
TxnResponse

3.1.2 Client初始化

go 复制代码

type Client struct {
    // 连接配置
    cfg Config

    // gRPC连接
    conn *grpc.ClientConn

    // 各功能子客户端
    KV
    Lease
    Watcher
    Cluster
    Maintenance
    Auth
}

// 创建客户端示例
cli, err := clientv3.New(clientv3.Config{
    Endpoints:   []string{"localhost:2379"},
    DialTimeout: 5 * time.Second,
})

3.2 KV与Range查询

3.2.1 KV接口

源码位置 : client/v3/kv.go

go 复制代码

type KV interface {
    Put(ctx context.Context, key, val string, opts ...OpOption) (*PutResponse, error)
    Get(ctx context.Context, key string, opts ...OpOption) (*GetResponse, error)
    GetStream(ctx context.Context, key string, opts ...OpOption) (GetStreamChan, error)
    Delete(ctx context.Context, key string, opts ...OpOption) (*DeleteResponse, error)
    Compact(ctx context.Context, rev int64, opts ...CompactOption) (*CompactResponse, error)
    Do(ctx context.Context, op Op) (OpResponse, error)
    Txn(ctx context.Context) Txn
}

3.2.2 Range查询选项

选项	说明
WithRange(end)	查询[key, end)范围
WithFromKey()	查询>=key的所有键
WithPrefix()	查询前缀为key的键
WithRev(rev)	查询指定历史版本
WithLimit(limit)	限制返回数量
WithSort()	排序结果
WithCountOnly()	只返回数量
WithKeysOnly()	只返回键

3.2.3 使用示例

go 复制代码

// 单键查询
resp, err := cli.Get(ctx, "foo")

// 范围查询
resp, err := cli.Get(ctx, "a", clientv3.WithRange("c"))

// 前缀查询
resp, err := cli.Get(ctx, "foo/", clientv3.WithPrefix())

// 历史版本查询
resp, err := cli.Get(ctx, "foo", clientv3.WithRev(10))

3.3 Txn事务处理

3.3.1 Txn接口

源码位置 : client/v3/txn.go

go 复制代码

type Txn interface {
    If(cs ...Cmp) Txn
    Then(ops ...Op) Txn
    Else(ops ...Op) Txn
    Commit() (*TxnResponse, error)
}

3.3.2 Txn实现

go 复制代码

type txn struct {
    kv  *kv
    ctx context.Context

    mu    sync.Mutex
    cif   bool
    cthen bool
    celse bool

    isWrite bool

    cmps []*pb.Compare
    sus  []*pb.RequestOp
    fas  []*pb.RequestOp
    callOpts []grpc.CallOption
}

func (txn *txn) If(cs ...Cmp) Txn {
    txn.mu.Lock()
    defer txn.mu.Unlock()

    if txn.cif { panic("cannot call If twice!") }
    txn.cif = true

    for i := range cs {
        cmp := cs[i].Clone()
        txn.cmps = append(txn.cmps, cmp.GetCompare())
    }
    return txn
}

func (txn *txn) Commit() (*TxnResponse, error) {
    r := &pb.TxnRequest{
        Compare: txn.cmps,
        Success: txn.sus,
        Failure: txn.fas,
    }
    resp, err := txn.kv.remote.Txn(txn.ctx, r, txn.callOpts...)
    return (*TxnResponse)(resp), err
}

3.3.3 Txn使用示例

#mermaid-svg-w71EIDK6QV3paIR2{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-w71EIDK6QV3paIR2 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-w71EIDK6QV3paIR2 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-w71EIDK6QV3paIR2 .error-icon{fill:#552222;}#mermaid-svg-w71EIDK6QV3paIR2 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-w71EIDK6QV3paIR2 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-w71EIDK6QV3paIR2 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-w71EIDK6QV3paIR2 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-w71EIDK6QV3paIR2 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-w71EIDK6QV3paIR2 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-w71EIDK6QV3paIR2 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-w71EIDK6QV3paIR2 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-w71EIDK6QV3paIR2 .marker.cross{stroke:#333333;}#mermaid-svg-w71EIDK6QV3paIR2 svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-w71EIDK6QV3paIR2 p{margin:0;}#mermaid-svg-w71EIDK6QV3paIR2 .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-w71EIDK6QV3paIR2 .cluster-label text{fill:#333;}#mermaid-svg-w71EIDK6QV3paIR2 .cluster-label span{color:#333;}#mermaid-svg-w71EIDK6QV3paIR2 .cluster-label span p{background-color:transparent;}#mermaid-svg-w71EIDK6QV3paIR2 .label text,#mermaid-svg-w71EIDK6QV3paIR2 span{fill:#333;color:#333;}#mermaid-svg-w71EIDK6QV3paIR2 .node rect,#mermaid-svg-w71EIDK6QV3paIR2 .node circle,#mermaid-svg-w71EIDK6QV3paIR2 .node ellipse,#mermaid-svg-w71EIDK6QV3paIR2 .node polygon,#mermaid-svg-w71EIDK6QV3paIR2 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-w71EIDK6QV3paIR2 .rough-node .label text,#mermaid-svg-w71EIDK6QV3paIR2 .node .label text,#mermaid-svg-w71EIDK6QV3paIR2 .image-shape .label,#mermaid-svg-w71EIDK6QV3paIR2 .icon-shape .label{text-anchor:middle;}#mermaid-svg-w71EIDK6QV3paIR2 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-w71EIDK6QV3paIR2 .rough-node .label,#mermaid-svg-w71EIDK6QV3paIR2 .node .label,#mermaid-svg-w71EIDK6QV3paIR2 .image-shape .label,#mermaid-svg-w71EIDK6QV3paIR2 .icon-shape .label{text-align:center;}#mermaid-svg-w71EIDK6QV3paIR2 .node.clickable{cursor:pointer;}#mermaid-svg-w71EIDK6QV3paIR2 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-w71EIDK6QV3paIR2 .arrowheadPath{fill:#333333;}#mermaid-svg-w71EIDK6QV3paIR2 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-w71EIDK6QV3paIR2 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-w71EIDK6QV3paIR2 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-w71EIDK6QV3paIR2 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-w71EIDK6QV3paIR2 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-w71EIDK6QV3paIR2 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-w71EIDK6QV3paIR2 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-w71EIDK6QV3paIR2 .cluster text{fill:#333;}#mermaid-svg-w71EIDK6QV3paIR2 .cluster span{color:#333;}#mermaid-svg-w71EIDK6QV3paIR2 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-w71EIDK6QV3paIR2 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-w71EIDK6QV3paIR2 rect.text{fill:none;stroke-width:0;}#mermaid-svg-w71EIDK6QV3paIR2 .icon-shape,#mermaid-svg-w71EIDK6QV3paIR2 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-w71EIDK6QV3paIR2 .icon-shape p,#mermaid-svg-w71EIDK6QV3paIR2 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-w71EIDK6QV3paIR2 .icon-shape .label rect,#mermaid-svg-w71EIDK6QV3paIR2 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-w71EIDK6QV3paIR2 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-w71EIDK6QV3paIR2 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-w71EIDK6QV3paIR2 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 是
否
开始
If条件判断
条件是否成立?
Then操作
Else操作
Commit事务
返回结果

go 复制代码

txn := cli.Txn(ctx)

// If: 检查条件
txn = txn.If(
    clientv3.Compare(clientv3.Value("key"), "=", "old"),
    clientv3.Compare(clientv3.Version("key"), "=", 1),
)

// Then: 条件成立时执行
txn = txn.Then(
    clientv3.OpPut("key", "new"),
    clientv3.OpPut("status", "updated"),
)

// Else: 条件失败时执行
txn = txn.Else(
    clientv3.OpPut("status", "conflict"),
)

// 提交
resp, err := txn.Commit()
if err != nil {
    // 处理错误
}

// 检查事务是否成功
if resp.Succeeded {
    fmt.Println("事务成功")
} else {
    fmt.Println("事务失败")
}

3.3.4 Compare类型

源码位置 : client/v3/compare.go

比较对象	说明
Value	值比较
Version	版本号比较
CreateRevision	创建版本比较
ModRevision	修改版本比较
Lease	租约ID比较

3.4 Lease租约机制

3.4.1 Lease接口

源码位置 : client/v3/lease.go

go 复制代码

type Lease interface {
    Grant(ctx context.Context, ttl int64) (*LeaseGrantResponse, error)
    Revoke(ctx context.Context, id LeaseID) (*LeaseRevokeResponse, error)
    TimeToLive(ctx context.Context, id LeaseID, opts ...LeaseOption) (*LeaseTimeToLiveResponse, error)
    Leases(ctx context.Context) (*LeaseLeasesResponse, error)
    KeepAlive(ctx context.Context, id LeaseID) (<-chan *LeaseKeepAliveResponse, error)
    KeepAliveOnce(ctx context.Context, id LeaseID) (*LeaseKeepAliveResponse, error)
    Close() error
}

3.4.2 租约工作流程

Server Client Server Client #mermaid-svg-zDbtziwWKO6kFC2R{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-zDbtziwWKO6kFC2R .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-zDbtziwWKO6kFC2R .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-zDbtziwWKO6kFC2R .error-icon{fill:#552222;}#mermaid-svg-zDbtziwWKO6kFC2R .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-zDbtziwWKO6kFC2R .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-zDbtziwWKO6kFC2R .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-zDbtziwWKO6kFC2R .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-zDbtziwWKO6kFC2R .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-zDbtziwWKO6kFC2R .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-zDbtziwWKO6kFC2R .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-zDbtziwWKO6kFC2R .marker{fill:#333333;stroke:#333333;}#mermaid-svg-zDbtziwWKO6kFC2R .marker.cross{stroke:#333333;}#mermaid-svg-zDbtziwWKO6kFC2R svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-zDbtziwWKO6kFC2R p{margin:0;}#mermaid-svg-zDbtziwWKO6kFC2R .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-zDbtziwWKO6kFC2R text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-zDbtziwWKO6kFC2R .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-zDbtziwWKO6kFC2R .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-zDbtziwWKO6kFC2R .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-zDbtziwWKO6kFC2R .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-zDbtziwWKO6kFC2R #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-zDbtziwWKO6kFC2R .sequenceNumber{fill:white;}#mermaid-svg-zDbtziwWKO6kFC2R #sequencenumber{fill:#333;}#mermaid-svg-zDbtziwWKO6kFC2R #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-zDbtziwWKO6kFC2R .messageText{fill:#333;stroke:none;}#mermaid-svg-zDbtziwWKO6kFC2R .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-zDbtziwWKO6kFC2R .labelText,#mermaid-svg-zDbtziwWKO6kFC2R .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-zDbtziwWKO6kFC2R .loopText,#mermaid-svg-zDbtziwWKO6kFC2R .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-zDbtziwWKO6kFC2R .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-zDbtziwWKO6kFC2R .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-zDbtziwWKO6kFC2R .noteText,#mermaid-svg-zDbtziwWKO6kFC2R .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-zDbtziwWKO6kFC2R .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-zDbtziwWKO6kFC2R .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-zDbtziwWKO6kFC2R .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-zDbtziwWKO6kFC2R .actorPopupMenu{position:absolute;}#mermaid-svg-zDbtziwWKO6kFC2R .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-zDbtziwWKO6kFC2R .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-zDbtziwWKO6kFC2R .actor-man circle,#mermaid-svg-zDbtziwWKO6kFC2R line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-zDbtziwWKO6kFC2R :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} loop $每5秒续约$ alt $主动撤销$ $超时$ Grant(ttl=10) LeaseID + TTL Put(key, value, WithLease(id)) KeepAlive(id) KeepAliveResponse Revoke(id) OK (key删除) 过期检测删除关联key

3.4.3 租约使用示例

go 复制代码

// 1. 创建租约
leaseResp, err := cli.Grant(ctx, 10) // TTL=10秒
if err != nil { /* handle */ }
leaseID := leaseResp.ID

// 2. 关联键值
_, err = cli.Put(ctx, "key", "value", clientv3.WithLease(leaseID))

// 3. 自动续约 (KeepAlive会自动维持)
kaChan, err := cli.KeepAlive(ctx, leaseID)
if err != nil { /* handle */ }

// 监听续约响应
go func() {
    for ka := range kaChan {
        if ka == nil {
            fmt.Println("续约通道关闭")
            return
        }
        fmt.Printf("续约成功, TTL=%d\n", ka.TTL)
    }
}()

// 或者使用KeepAliveOnce手动续约
kaResp, err := cli.KeepAliveOnce(ctx, leaseID)

// 查看租约信息
ttlResp, err := cli.TimeToLive(ctx, leaseID, clientv3.WithAttachedKeys())
fmt.Printf("剩余TTL: %d, 关联键: %v\n", ttlResp.TTL, ttlResp.Keys)

// 撤销租约 (会同时删除关联键)
_, err = cli.Revoke(ctx, leaseID)

3.4.4 关键数据结构

go 复制代码

type LeaseGrantResponse struct {
    *pb.ResponseHeader
    ID    LeaseID
    TTL   int64
    Error string
}

type LeaseKeepAliveResponse struct {
    *pb.ResponseHeader
    ID  LeaseID
    TTL int64
}

type LeaseTimeToLiveResponse struct {
    *pb.ResponseHeader
    ID         LeaseID
    TTL        int64
    GrantedTTL int64
    Keys       [][]byte
}

3.5 Watch监听机制

3.5.1 Watcher接口

源码位置 : client/v3/watch.go

go 复制代码

type Watcher interface {
    Watch(ctx context.Context, key string, opts ...OpOption) WatchChan
    Close() error
    RequestProgress(ctx context.Context) error
}

type WatchChan <-chan WatchResponse

type WatchResponse struct {
    Header pb.ResponseHeader
    Events []*Event

    Canceled bool
    Created bool
    CompactRevision int64
}

3.5.2 Watch工作原理

#mermaid-svg-KvmndskiITxOiLes{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-KvmndskiITxOiLes .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-KvmndskiITxOiLes .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-KvmndskiITxOiLes .error-icon{fill:#552222;}#mermaid-svg-KvmndskiITxOiLes .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-KvmndskiITxOiLes .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-KvmndskiITxOiLes .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-KvmndskiITxOiLes .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-KvmndskiITxOiLes .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-KvmndskiITxOiLes .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-KvmndskiITxOiLes .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-KvmndskiITxOiLes .marker{fill:#333333;stroke:#333333;}#mermaid-svg-KvmndskiITxOiLes .marker.cross{stroke:#333333;}#mermaid-svg-KvmndskiITxOiLes svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-KvmndskiITxOiLes p{margin:0;}#mermaid-svg-KvmndskiITxOiLes .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-KvmndskiITxOiLes .cluster-label text{fill:#333;}#mermaid-svg-KvmndskiITxOiLes .cluster-label span{color:#333;}#mermaid-svg-KvmndskiITxOiLes .cluster-label span p{background-color:transparent;}#mermaid-svg-KvmndskiITxOiLes .label text,#mermaid-svg-KvmndskiITxOiLes span{fill:#333;color:#333;}#mermaid-svg-KvmndskiITxOiLes .node rect,#mermaid-svg-KvmndskiITxOiLes .node circle,#mermaid-svg-KvmndskiITxOiLes .node ellipse,#mermaid-svg-KvmndskiITxOiLes .node polygon,#mermaid-svg-KvmndskiITxOiLes .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-KvmndskiITxOiLes .rough-node .label text,#mermaid-svg-KvmndskiITxOiLes .node .label text,#mermaid-svg-KvmndskiITxOiLes .image-shape .label,#mermaid-svg-KvmndskiITxOiLes .icon-shape .label{text-anchor:middle;}#mermaid-svg-KvmndskiITxOiLes .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-KvmndskiITxOiLes .rough-node .label,#mermaid-svg-KvmndskiITxOiLes .node .label,#mermaid-svg-KvmndskiITxOiLes .image-shape .label,#mermaid-svg-KvmndskiITxOiLes .icon-shape .label{text-align:center;}#mermaid-svg-KvmndskiITxOiLes .node.clickable{cursor:pointer;}#mermaid-svg-KvmndskiITxOiLes .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-KvmndskiITxOiLes .arrowheadPath{fill:#333333;}#mermaid-svg-KvmndskiITxOiLes .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-KvmndskiITxOiLes .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-KvmndskiITxOiLes .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-KvmndskiITxOiLes .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-KvmndskiITxOiLes .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-KvmndskiITxOiLes .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-KvmndskiITxOiLes .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-KvmndskiITxOiLes .cluster text{fill:#333;}#mermaid-svg-KvmndskiITxOiLes .cluster span{color:#333;}#mermaid-svg-KvmndskiITxOiLes div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-KvmndskiITxOiLes .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-KvmndskiITxOiLes rect.text{fill:none;stroke-width:0;}#mermaid-svg-KvmndskiITxOiLes .icon-shape,#mermaid-svg-KvmndskiITxOiLes .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-KvmndskiITxOiLes .icon-shape p,#mermaid-svg-KvmndskiITxOiLes .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-KvmndskiITxOiLes .icon-shape .label rect,#mermaid-svg-KvmndskiITxOiLes .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-KvmndskiITxOiLes .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-KvmndskiITxOiLes .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-KvmndskiITxOiLes :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Watch请求
创建流
事件推送
重连机制
自动重试
从上次版本恢复
连接断开
重新连接
同步历史
客户端
gRPC流
服务端
事件通道

3.5.3 Watch使用示例

go 复制代码

// 1. 基础监听
rch := cli.Watch(ctx, "key")
for wresp := range rch {
    for _, ev := range wresp.Events {
        fmt.Printf("%s %q : %q\n", ev.Type, ev.Kv.Key, ev.Kv.Value)
    }
}

// 2. 前缀监听
rch := cli.Watch(ctx, "foo/", clientv3.WithPrefix())

// 3. 范围监听
rch := cli.Watch(ctx, "a", clientv3.WithRange("c"))

// 4. 从历史版本开始监听
// 会先回放从rev=100开始的所有事件
rch := cli.Watch(ctx, "key", clientv3.WithRev(100))

// 5. 带过滤的监听
rch := cli.Watch(ctx, "key", clientv3.WithPrevKV()) // 包含旧值

// 6. 处理监听中断
ctx, cancel := context.WithCancel(context.Background())
defer cancel()

rch := cli.Watch(ctx, "key")
for wresp := range rch {
    if wresp.Canceled {
        fmt.Println("监听被取消:", wresp.Err())
        break
    }
    if wresp.CompactRevision != 0 {
        fmt.Printf("历史被压缩，需要从revision=%d重新监听\n", wresp.CompactRevision)
    }
    // 处理事件
}

3.5.4 Event事件

go 复制代码

type Event struct {
    Type EventType
    Kv   *mvccpb.KeyValue
    PrevKv *mvccpb.KeyValue
}

const (
    PUT EventType = 0
    DELETE EventType = 1
)

3.6 使用坑点总结

3.6.1 常见问题

问题	说明	解决方案
Txn中只能有一次If/Then/Else	Txn是线性API设计	按顺序调用，不要重复
Watch断开后不自动重连	需要自己处理重连	检测Canceled，重新Watch
KeepAlive通道关闭	网络问题或租约过期	检查ka是否为nil
WithRev查询被压缩	历史版本可能已清理	捕获ErrCompacted，使用最新版本
租约撤销后键立即删除	设计行为	确认后再撤销

3.6.2 最佳实践

go 复制代码

// 1. 始终使用context控制超时
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()

// 2. Watch重连模式
watchKey := "key"
var watchRev int64 = 0

for {
    opts := []clientv3.OpOption{}
    if watchRev > 0 {
        opts = append(opts, clientv3.WithRev(watchRev))
    }

    rch := cli.Watch(ctx, watchKey, opts...)

    for wresp := range rch {
        if wresp.Canceled {
            fmt.Println("监听断开，重连中...")
            break
        }
        if wresp.CompactRevision > 0 {
            watchRev = wresp.CompactRevision
            continue
        }
        // 处理事件
        for _, ev := range wresp.Events {
            watchRev = ev.Kv.ModRevision + 1
        }
    }
}

// 3. KeepAlive保活模式
func keepAliveForever(cli *clientv3.Client, leaseID clientv3.LeaseID) {
    for {
        kaChan, err := cli.KeepAlive(ctx, leaseID)
        if err != nil {
            time.Sleep(time.Second)
            continue
        }
        // 消费keepalive响应
        for ka := range kaChan {
            if ka == nil {
                break
            }
        }
    }
}

参考文献

etcd客户端API文档

04 - API网关服务层

4.1 gRPC服务注册

4.1.1 v3rpc包结构

源码位置 : server/etcdserver/api/v3rpc/

复制代码

v3rpc/
├── grpc.go          # 服务注册
├── interceptor.go   # 拦截器
├── key.go           # KV服务
├── lease.go         # Lease服务
├── watch.go         # Watch服务
├── auth.go          # Auth服务
├── maintenance.go   # 维护服务
├── member.go        # 成员管理
└── ...

4.1.2 gRPC服务注册

源码位置 : server/etcdserver/api/v3rpc/grpc.go

go 复制代码

// Register registers etcd server services.
func Register(s *grpc.Server, etcdserver *etcdserver.EtcdServer) {
    pb.RegisterKVServer(s, NewKVServer(etcdserver))
    pb.RegisterWatchServer(s, NewWatchServer(etcdserver))
    pb.RegisterLeaseServer(s, NewLeaseServer(etcdserver))
    pb.RegisterClusterServer(s, NewClusterServer(etcdserver))
    pb.RegisterMaintenanceServer(s, NewMaintenanceServer(etcdserver))
    pb.RegisterAuthServer(s, NewAuthServer(etcdserver))
}

4.2 请求拦截器

4.2.1 拦截器链

源码位置 : server/etcdserver/api/v3rpc/interceptor.go
#mermaid-svg-Z8GooFZeML7LRSoW{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-Z8GooFZeML7LRSoW .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-Z8GooFZeML7LRSoW .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-Z8GooFZeML7LRSoW .error-icon{fill:#552222;}#mermaid-svg-Z8GooFZeML7LRSoW .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-Z8GooFZeML7LRSoW .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-Z8GooFZeML7LRSoW .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-Z8GooFZeML7LRSoW .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-Z8GooFZeML7LRSoW .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-Z8GooFZeML7LRSoW .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-Z8GooFZeML7LRSoW .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-Z8GooFZeML7LRSoW .marker{fill:#333333;stroke:#333333;}#mermaid-svg-Z8GooFZeML7LRSoW .marker.cross{stroke:#333333;}#mermaid-svg-Z8GooFZeML7LRSoW svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-Z8GooFZeML7LRSoW p{margin:0;}#mermaid-svg-Z8GooFZeML7LRSoW .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-Z8GooFZeML7LRSoW .cluster-label text{fill:#333;}#mermaid-svg-Z8GooFZeML7LRSoW .cluster-label span{color:#333;}#mermaid-svg-Z8GooFZeML7LRSoW .cluster-label span p{background-color:transparent;}#mermaid-svg-Z8GooFZeML7LRSoW .label text,#mermaid-svg-Z8GooFZeML7LRSoW span{fill:#333;color:#333;}#mermaid-svg-Z8GooFZeML7LRSoW .node rect,#mermaid-svg-Z8GooFZeML7LRSoW .node circle,#mermaid-svg-Z8GooFZeML7LRSoW .node ellipse,#mermaid-svg-Z8GooFZeML7LRSoW .node polygon,#mermaid-svg-Z8GooFZeML7LRSoW .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-Z8GooFZeML7LRSoW .rough-node .label text,#mermaid-svg-Z8GooFZeML7LRSoW .node .label text,#mermaid-svg-Z8GooFZeML7LRSoW .image-shape .label,#mermaid-svg-Z8GooFZeML7LRSoW .icon-shape .label{text-anchor:middle;}#mermaid-svg-Z8GooFZeML7LRSoW .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-Z8GooFZeML7LRSoW .rough-node .label,#mermaid-svg-Z8GooFZeML7LRSoW .node .label,#mermaid-svg-Z8GooFZeML7LRSoW .image-shape .label,#mermaid-svg-Z8GooFZeML7LRSoW .icon-shape .label{text-align:center;}#mermaid-svg-Z8GooFZeML7LRSoW .node.clickable{cursor:pointer;}#mermaid-svg-Z8GooFZeML7LRSoW .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-Z8GooFZeML7LRSoW .arrowheadPath{fill:#333333;}#mermaid-svg-Z8GooFZeML7LRSoW .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-Z8GooFZeML7LRSoW .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-Z8GooFZeML7LRSoW .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Z8GooFZeML7LRSoW .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-Z8GooFZeML7LRSoW .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Z8GooFZeML7LRSoW .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-Z8GooFZeML7LRSoW .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-Z8GooFZeML7LRSoW .cluster text{fill:#333;}#mermaid-svg-Z8GooFZeML7LRSoW .cluster span{color:#333;}#mermaid-svg-Z8GooFZeML7LRSoW div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-Z8GooFZeML7LRSoW .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-Z8GooFZeML7LRSoW rect.text{fill:none;stroke-width:0;}#mermaid-svg-Z8GooFZeML7LRSoW .icon-shape,#mermaid-svg-Z8GooFZeML7LRSoW .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Z8GooFZeML7LRSoW .icon-shape p,#mermaid-svg-Z8GooFZeML7LRSoW .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-Z8GooFZeML7LRSoW .icon-shape .label rect,#mermaid-svg-Z8GooFZeML7LRSoW .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Z8GooFZeML7LRSoW .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-Z8GooFZeML7LRSoW .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-Z8GooFZeML7LRSoW :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 客户端请求
日志拦截器
认证拦截器
指标拦截器
Panic恢复拦截器
实际处理

4.2.2 拦截器实现

go 复制代码

func newLogUnaryInterceptor(lg *zap.Logger) grpc.UnaryServerInterceptor {
    return func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
        start := time.Now()
        resp, err := handler(ctx, req)
        lg.Debug(
            "unary request",
            zap.String("method", info.FullMethod),
            zap.Duration("time", time.Since(start)),
            zap.Error(err),
        )
        return resp, err
    }
}

4.3 认证与鉴权

4.3.1 Auth服务

源码位置 : server/etcdserver/api/v3rpc/auth.go

go 复制代码

type authServer struct {
    as auth.AuthStore
}

func (as *authServer) AuthEnable(ctx context.Context, r *pb.AuthEnableRequest) (*pb.AuthEnableResponse, error) {
    resp, err := as.as.AuthEnable(ctx, r)
    return resp, togRPCError(err)
}

func (as *authServer) Authenticate(ctx context.Context, r *pb.AuthenticateRequest) (*pb.AuthenticateResponse, error) {
    resp, err := as.as.Authenticate(ctx, r)
    return resp, togRPCError(err)
}

4.3.2 鉴权流程

EtcdServer AuthStore Server Client EtcdServer AuthStore Server Client #mermaid-svg-VjyOawYL5HMsyDDT{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-VjyOawYL5HMsyDDT .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-VjyOawYL5HMsyDDT .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-VjyOawYL5HMsyDDT .error-icon{fill:#552222;}#mermaid-svg-VjyOawYL5HMsyDDT .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-VjyOawYL5HMsyDDT .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-VjyOawYL5HMsyDDT .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-VjyOawYL5HMsyDDT .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-VjyOawYL5HMsyDDT .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-VjyOawYL5HMsyDDT .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-VjyOawYL5HMsyDDT .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-VjyOawYL5HMsyDDT .marker{fill:#333333;stroke:#333333;}#mermaid-svg-VjyOawYL5HMsyDDT .marker.cross{stroke:#333333;}#mermaid-svg-VjyOawYL5HMsyDDT svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-VjyOawYL5HMsyDDT p{margin:0;}#mermaid-svg-VjyOawYL5HMsyDDT .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-VjyOawYL5HMsyDDT text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-VjyOawYL5HMsyDDT .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-VjyOawYL5HMsyDDT .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-VjyOawYL5HMsyDDT .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-VjyOawYL5HMsyDDT .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-VjyOawYL5HMsyDDT #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-VjyOawYL5HMsyDDT .sequenceNumber{fill:white;}#mermaid-svg-VjyOawYL5HMsyDDT #sequencenumber{fill:#333;}#mermaid-svg-VjyOawYL5HMsyDDT #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-VjyOawYL5HMsyDDT .messageText{fill:#333;stroke:none;}#mermaid-svg-VjyOawYL5HMsyDDT .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-VjyOawYL5HMsyDDT .labelText,#mermaid-svg-VjyOawYL5HMsyDDT .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-VjyOawYL5HMsyDDT .loopText,#mermaid-svg-VjyOawYL5HMsyDDT .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-VjyOawYL5HMsyDDT .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-VjyOawYL5HMsyDDT .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-VjyOawYL5HMsyDDT .noteText,#mermaid-svg-VjyOawYL5HMsyDDT .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-VjyOawYL5HMsyDDT .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-VjyOawYL5HMsyDDT .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-VjyOawYL5HMsyDDT .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-VjyOawYL5HMsyDDT .actorPopupMenu{position:absolute;}#mermaid-svg-VjyOawYL5HMsyDDT .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-VjyOawYL5HMsyDDT .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-VjyOawYL5HMsyDDT .actor-man circle,#mermaid-svg-VjyOawYL5HMsyDDT line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-VjyOawYL5HMsyDDT :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 请求 + Token 验证Token 用户信息检查权限权限确认执行业务返回结果

4.4 参数校验

源码位置 : server/etcdserver/api/v3rpc/key.go

go 复制代码

func (kv *kvServer) Range(ctx context.Context, r *pb.RangeRequest) (*pb.RangeResponse, error) {
    if err := checkRangeRequest(r); err != nil {
        return nil, err
    }
    resp, err := kv.kv.Range(ctx, r)
    return resp, togRPCError(err)
}

func checkRangeRequest(r *pb.RangeRequest) error {
    if len(r.Key) == 0 {
        return ErrEmptyKey
    }
    return nil
}

4.5 配额与限流

源码位置 : server/etcdserver/api/v3rpc/quota.go

go 复制代码

func checkRequestSize(r interface{}) error {
    size := 0
    // 计算请求大小
    if size > maxRequestBytes {
        return rpctypes.ErrRequestTooLarge
    }
    return nil
}

参考文献

etcd gRPC API文档

05 - Raft集群共识核心（重点）

5.1 Raft论文核心回顾

5.1.1 Raft三个子问题

问题	解决
Leader选举	Term + 超时 + 投票
日志复制	AppendEntries + 多数确认
安全性	日志匹配 + Leader完整性

5.1.2 Raft状态转换

#mermaid-svg-yBhhpr2TTKi6GzBG{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-yBhhpr2TTKi6GzBG .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-yBhhpr2TTKi6GzBG .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-yBhhpr2TTKi6GzBG .error-icon{fill:#552222;}#mermaid-svg-yBhhpr2TTKi6GzBG .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-yBhhpr2TTKi6GzBG .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-yBhhpr2TTKi6GzBG .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-yBhhpr2TTKi6GzBG .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-yBhhpr2TTKi6GzBG .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-yBhhpr2TTKi6GzBG .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-yBhhpr2TTKi6GzBG .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-yBhhpr2TTKi6GzBG .marker{fill:#333333;stroke:#333333;}#mermaid-svg-yBhhpr2TTKi6GzBG .marker.cross{stroke:#333333;}#mermaid-svg-yBhhpr2TTKi6GzBG svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-yBhhpr2TTKi6GzBG p{margin:0;}#mermaid-svg-yBhhpr2TTKi6GzBG defs #statediagram-barbEnd{fill:#333333;stroke:#333333;}#mermaid-svg-yBhhpr2TTKi6GzBG g.stateGroup text{fill:#9370DB;stroke:none;font-size:10px;}#mermaid-svg-yBhhpr2TTKi6GzBG g.stateGroup text{fill:#333;stroke:none;font-size:10px;}#mermaid-svg-yBhhpr2TTKi6GzBG g.stateGroup .state-title{font-weight:bolder;fill:#131300;}#mermaid-svg-yBhhpr2TTKi6GzBG g.stateGroup rect{fill:#ECECFF;stroke:#9370DB;}#mermaid-svg-yBhhpr2TTKi6GzBG g.stateGroup line{stroke:#333333;stroke-width:1;}#mermaid-svg-yBhhpr2TTKi6GzBG .transition{stroke:#333333;stroke-width:1;fill:none;}#mermaid-svg-yBhhpr2TTKi6GzBG .stateGroup .composit{fill:white;border-bottom:1px;}#mermaid-svg-yBhhpr2TTKi6GzBG .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px;}#mermaid-svg-yBhhpr2TTKi6GzBG .state-note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-yBhhpr2TTKi6GzBG .state-note text{fill:black;stroke:none;font-size:10px;}#mermaid-svg-yBhhpr2TTKi6GzBG .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5;}#mermaid-svg-yBhhpr2TTKi6GzBG .edgeLabel .label rect{fill:#ECECFF;opacity:0.5;}#mermaid-svg-yBhhpr2TTKi6GzBG .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-yBhhpr2TTKi6GzBG .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-yBhhpr2TTKi6GzBG .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-yBhhpr2TTKi6GzBG .edgeLabel .label text{fill:#333;}#mermaid-svg-yBhhpr2TTKi6GzBG .label div .edgeLabel{color:#333;}#mermaid-svg-yBhhpr2TTKi6GzBG .stateLabel text{fill:#131300;font-size:10px;font-weight:bold;}#mermaid-svg-yBhhpr2TTKi6GzBG .node circle.state-start{fill:#333333;stroke:#333333;}#mermaid-svg-yBhhpr2TTKi6GzBG .node .fork-join{fill:#333333;stroke:#333333;}#mermaid-svg-yBhhpr2TTKi6GzBG .node circle.state-end{fill:#9370DB;stroke:white;stroke-width:1.5;}#mermaid-svg-yBhhpr2TTKi6GzBG .end-state-inner{fill:white;stroke-width:1.5;}#mermaid-svg-yBhhpr2TTKi6GzBG .node rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-yBhhpr2TTKi6GzBG .node polygon{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-yBhhpr2TTKi6GzBG #statediagram-barbEnd{fill:#333333;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-cluster rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-yBhhpr2TTKi6GzBG .cluster-label,#mermaid-svg-yBhhpr2TTKi6GzBG .nodeLabel{color:#131300;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-cluster rect.outer{rx:5px;ry:5px;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-state .divider{stroke:#9370DB;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-state .title-state{rx:5px;ry:5px;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-cluster.statediagram-cluster .inner{fill:white;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-cluster.statediagram-cluster-alt .inner{fill:#f0f0f0;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-cluster .inner{rx:0;ry:0;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-state rect.basic{rx:5px;ry:5px;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#f0f0f0;}#mermaid-svg-yBhhpr2TTKi6GzBG .note-edge{stroke-dasharray:5;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-note text{fill:black;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram-note .nodeLabel{color:black;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagram .edgeLabel{color:red;}#mermaid-svg-yBhhpr2TTKi6GzBG #dependencyStart,#mermaid-svg-yBhhpr2TTKi6GzBG #dependencyEnd{fill:#333333;stroke:#333333;stroke-width:1;}#mermaid-svg-yBhhpr2TTKi6GzBG .statediagramTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-yBhhpr2TTKi6GzBG :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 选举超时
获得多数票
收到更高Term
选举超时
收到更高Term
Follower
Candidate
Leader

5.2 raftNode与状态机

5.2.1 raftNode结构体

源码位置 : server/etcdserver/raft.go

go 复制代码

type raftNode struct {
    lg *zap.Logger

    tickMu *sync.RWMutex
    latestTickTs time.Time
    raftNodeConfig

    msgSnapC chan *raftpb.Message
    applyc chan toApply
    readStateC chan raft.ReadState

    ticker *time.Ticker
    td *contention.TimeoutDetector

    stopped chan struct{}
    done chan struct{}
}

type toApply struct {
    entries  []*raftpb.Entry
    snapshot *raftpb.Snapshot
    notifyc  chan struct{}
    raftAdvancedC <-chan struct{}
}

5.2.2 Raft Ready处理循环

源码位置 : server/etcdserver/raft.go - raftNode.start()

go 复制代码

func (r *raftNode) start(rh *raftReadyHandler) {
    go func() {
        for {
            select {
            case <-r.ticker.C:
                r.tick()
            case rd := <-r.Ready():
                if rd.SoftState != nil {
                    newLeader := rd.SoftState.Lead != raft.None && rh.getLead() != rd.SoftState.Lead
                    rh.updateLead(rd.SoftState.Lead)
                    islead = rd.RaftState == raft.StateLeader
                }

                if len(rd.ReadStates) != 0 {
                    select {
                    case r.readStateC <- rd.ReadStates[len(rd.ReadStates)-1]:
                    case <-time.After(internalTimeout):
                    }
                }

                committedEntries := rd.CommittedEntries
                notifyc := make(chan struct{}, 1)
                ap := toApply{
                    entries:  committedEntries,
                    snapshot: proto.Clone(rd.Snapshot).(*raftpb.Snapshot),
                    notifyc: notifyc,
                }
                updateCommittedIndex(&ap, rh)

                select {
                case r.applyc <- ap:
                }

                if islead {
                    r.transport.Send(r.processMessages(rd.Messages))
                }

                if !raft.IsEmptySnap(raftSnap) {
                    if err := r.storage.SaveSnap(raftSnap); err != nil {
                        r.lg.Fatal("failed to save Raft snapshot", zap.Error(err))
                    }
                }

                if err := r.storage.Save(rd.HardState, rd.Entries); err != nil {
                    r.lg.Fatal("failed to save Raft hard state and entries", zap.Error(err))
                }

                if !raft.IsEmptySnap(raftSnap) {
                    if err := r.storage.Sync(); err != nil {
                        r.lg.Fatal("failed to sync Raft snapshot", zap.Error(err))
                    }
                    notifyc <- struct{}{}
                    r.raftStorage.ApplySnapshot(raftSnap)
                    if err := r.storage.Release(raftSnap); err != nil {
                        r.lg.Fatal("failed to release Raft wal", zap.Error(err))
                    }
                }

                r.raftStorage.Append(rd.Entries)

                if !islead {
                    r.transport.Send(r.processMessages(rd.Messages))
                    notifyc <- struct{}{}
                } else {
                    notifyc <- struct{}{}
                }

                r.Advance()
            case <-r.stopped:
                return
            }
        }
    }()
}

5.3 Leader选举源码

5.3.1 选举流程

Others Leader Candidate Follower Others Leader Candidate Follower #mermaid-svg-mC4FlX0b6C2PTJXJ{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-mC4FlX0b6C2PTJXJ .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-mC4FlX0b6C2PTJXJ .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-mC4FlX0b6C2PTJXJ .error-icon{fill:#552222;}#mermaid-svg-mC4FlX0b6C2PTJXJ .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-mC4FlX0b6C2PTJXJ .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-mC4FlX0b6C2PTJXJ .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-mC4FlX0b6C2PTJXJ .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-mC4FlX0b6C2PTJXJ .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-mC4FlX0b6C2PTJXJ .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-mC4FlX0b6C2PTJXJ .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-mC4FlX0b6C2PTJXJ .marker{fill:#333333;stroke:#333333;}#mermaid-svg-mC4FlX0b6C2PTJXJ .marker.cross{stroke:#333333;}#mermaid-svg-mC4FlX0b6C2PTJXJ svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-mC4FlX0b6C2PTJXJ p{margin:0;}#mermaid-svg-mC4FlX0b6C2PTJXJ .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-mC4FlX0b6C2PTJXJ text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-mC4FlX0b6C2PTJXJ .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-mC4FlX0b6C2PTJXJ .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-mC4FlX0b6C2PTJXJ .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-mC4FlX0b6C2PTJXJ .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-mC4FlX0b6C2PTJXJ #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-mC4FlX0b6C2PTJXJ .sequenceNumber{fill:white;}#mermaid-svg-mC4FlX0b6C2PTJXJ #sequencenumber{fill:#333;}#mermaid-svg-mC4FlX0b6C2PTJXJ #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-mC4FlX0b6C2PTJXJ .messageText{fill:#333;stroke:none;}#mermaid-svg-mC4FlX0b6C2PTJXJ .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-mC4FlX0b6C2PTJXJ .labelText,#mermaid-svg-mC4FlX0b6C2PTJXJ .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-mC4FlX0b6C2PTJXJ .loopText,#mermaid-svg-mC4FlX0b6C2PTJXJ .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-mC4FlX0b6C2PTJXJ .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-mC4FlX0b6C2PTJXJ .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-mC4FlX0b6C2PTJXJ .noteText,#mermaid-svg-mC4FlX0b6C2PTJXJ .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-mC4FlX0b6C2PTJXJ .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-mC4FlX0b6C2PTJXJ .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-mC4FlX0b6C2PTJXJ .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-mC4FlX0b6C2PTJXJ .actorPopupMenu{position:absolute;}#mermaid-svg-mC4FlX0b6C2PTJXJ .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-mC4FlX0b6C2PTJXJ .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-mC4FlX0b6C2PTJXJ .actor-man circle,#mermaid-svg-mC4FlX0b6C2PTJXJ line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-mC4FlX0b6C2PTJXJ :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 等待心跳超时，变为Candidate 增加Term 投自己一票 RequestVote RPC 投票响应获得多数票，变为Leader 发送心跳

5.3.2 选举相关参数

go 复制代码

const (
    DefaultSnapshotCount = 10000
    DefaultSnapshotCatchUpEntries uint64 = 5000
)

5.4 日志复制源码

5.4.1 日志复制流程

StateMachine Followers Leader Client StateMachine Followers Leader Client #mermaid-svg-OByn4ZXwq9RwXhBr{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-OByn4ZXwq9RwXhBr .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-OByn4ZXwq9RwXhBr .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-OByn4ZXwq9RwXhBr .error-icon{fill:#552222;}#mermaid-svg-OByn4ZXwq9RwXhBr .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-OByn4ZXwq9RwXhBr .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-OByn4ZXwq9RwXhBr .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-OByn4ZXwq9RwXhBr .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-OByn4ZXwq9RwXhBr .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-OByn4ZXwq9RwXhBr .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-OByn4ZXwq9RwXhBr .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-OByn4ZXwq9RwXhBr .marker{fill:#333333;stroke:#333333;}#mermaid-svg-OByn4ZXwq9RwXhBr .marker.cross{stroke:#333333;}#mermaid-svg-OByn4ZXwq9RwXhBr svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-OByn4ZXwq9RwXhBr p{margin:0;}#mermaid-svg-OByn4ZXwq9RwXhBr .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-OByn4ZXwq9RwXhBr text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-OByn4ZXwq9RwXhBr .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-OByn4ZXwq9RwXhBr .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-OByn4ZXwq9RwXhBr .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-OByn4ZXwq9RwXhBr .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-OByn4ZXwq9RwXhBr #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-OByn4ZXwq9RwXhBr .sequenceNumber{fill:white;}#mermaid-svg-OByn4ZXwq9RwXhBr #sequencenumber{fill:#333;}#mermaid-svg-OByn4ZXwq9RwXhBr #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-OByn4ZXwq9RwXhBr .messageText{fill:#333;stroke:none;}#mermaid-svg-OByn4ZXwq9RwXhBr .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-OByn4ZXwq9RwXhBr .labelText,#mermaid-svg-OByn4ZXwq9RwXhBr .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-OByn4ZXwq9RwXhBr .loopText,#mermaid-svg-OByn4ZXwq9RwXhBr .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-OByn4ZXwq9RwXhBr .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-OByn4ZXwq9RwXhBr .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-OByn4ZXwq9RwXhBr .noteText,#mermaid-svg-OByn4ZXwq9RwXhBr .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-OByn4ZXwq9RwXhBr .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-OByn4ZXwq9RwXhBr .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-OByn4ZXwq9RwXhBr .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-OByn4ZXwq9RwXhBr .actorPopupMenu{position:absolute;}#mermaid-svg-OByn4ZXwq9RwXhBr .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-OByn4ZXwq9RwXhBr .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-OByn4ZXwq9RwXhBr .actor-man circle,#mermaid-svg-OByn4ZXwq9RwXhBr line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-OByn4ZXwq9RwXhBr :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Put请求追加到本地日志 AppendEntries RPC 确认提交日志应用到状态机返回结果

5.4.2 日志结构

复制代码

每个日志条目：
- Term: 日志创建时的任期
- Index: 日志在日志流中的位置
- Type: 配置变更或普通
- Data: 实际数据

5.5 快照同步源码

5.5.1 快照触发时机

日志数量达到 SnapshotCount
Follower落后太多，需要快照同步

5.5.2 快照处理

go 复制代码

if !raft.IsEmptySnap(raftSnap) {
    if err := r.storage.SaveSnap(raftSnap); err != nil {
        r.lg.Fatal("failed to save Raft snapshot", zap.Error(err))
    }
    if err := r.storage.Sync(); err != nil {
        r.lg.Fatal("failed to sync Raft snapshot", zap.Error(err))
    }
    notifyc <- struct{}{}
    r.raftStorage.ApplySnapshot(raftSnap)
    if err := r.storage.Release(raftSnap); err != nil {
        r.lg.Fatal("failed to release Raft wal", zap.Error(err))
    }
}

5.6 脑裂与一致性保证

5.6.1 脑裂场景

复制代码

场景：网络分区导致两个Leader
  - Term高的会获胜
  - Term低的会发现更高Term并退位

5.6.2 一致性保证

选举安全性：一个Term只有一个Leader
Leader完整性：Leader包含所有已提交日志
日志匹配：相同Index+Term的日志相同

参考文献

06 - WAL预写日志持久化

6.1 WAL理论基础

6.1.1 WAL作用

功能	说明
持久化	确保操作不丢失
崩溃恢复	重启时恢复状态
同步复制	确保集群一致性

6.1.2 WAL特点

顺序写入：性能高
分段存储：按大小切分
CRC校验：验证完整性
同步刷盘：fsync保证

6.2 WAL文件结构

6.2.1 目录结构

复制代码

member/
└── wal/
    ├── 0000000000000000-0000000000000000.wal
    ├── 0000000000000001-0000000000001000.wal
    ├── 0000000000001001-0000000000002000.wal
    └── ...

6.2.2 WAL格式

复制代码

每个Segment:
┌───────────────────────────────┐
│ Record 0 (Header + Data)       │
├───────────────────────────────┤
│ Record 1 (Header + Data)     │
├───────────────────────────────┤
│ ...                           │
└───────────────────────────────┘

6.3 日志写入流程

6.3.1 WAL结构体

源码位置 : server/storage/wal/wal.go

go 复制代码

type WAL struct {
    dir string

    mu sync.Mutex

    metadata []byte

    state raftpb.HardState

    entries []raftpb.Entry

    start walpb.Snapshot

    decoder *decoder

    encoder *encoder

    locks []*fileutil.LockedFile

    fp *filePipeline

    valid bool
}

6.3.2 写入流程

File Encoder WAL EtcdServer File Encoder WAL EtcdServer #mermaid-svg-lTViQyq9CB5b0kpi{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-lTViQyq9CB5b0kpi .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-lTViQyq9CB5b0kpi .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-lTViQyq9CB5b0kpi .error-icon{fill:#552222;}#mermaid-svg-lTViQyq9CB5b0kpi .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-lTViQyq9CB5b0kpi .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-lTViQyq9CB5b0kpi .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-lTViQyq9CB5b0kpi .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-lTViQyq9CB5b0kpi .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-lTViQyq9CB5b0kpi .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-lTViQyq9CB5b0kpi .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-lTViQyq9CB5b0kpi .marker{fill:#333333;stroke:#333333;}#mermaid-svg-lTViQyq9CB5b0kpi .marker.cross{stroke:#333333;}#mermaid-svg-lTViQyq9CB5b0kpi svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-lTViQyq9CB5b0kpi p{margin:0;}#mermaid-svg-lTViQyq9CB5b0kpi .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-lTViQyq9CB5b0kpi text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-lTViQyq9CB5b0kpi .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-lTViQyq9CB5b0kpi .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-lTViQyq9CB5b0kpi .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-lTViQyq9CB5b0kpi .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-lTViQyq9CB5b0kpi #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-lTViQyq9CB5b0kpi .sequenceNumber{fill:white;}#mermaid-svg-lTViQyq9CB5b0kpi #sequencenumber{fill:#333;}#mermaid-svg-lTViQyq9CB5b0kpi #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-lTViQyq9CB5b0kpi .messageText{fill:#333;stroke:none;}#mermaid-svg-lTViQyq9CB5b0kpi .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-lTViQyq9CB5b0kpi .labelText,#mermaid-svg-lTViQyq9CB5b0kpi .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-lTViQyq9CB5b0kpi .loopText,#mermaid-svg-lTViQyq9CB5b0kpi .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-lTViQyq9CB5b0kpi .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-lTViQyq9CB5b0kpi .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-lTViQyq9CB5b0kpi .noteText,#mermaid-svg-lTViQyq9CB5b0kpi .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-lTViQyq9CB5b0kpi .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-lTViQyq9CB5b0kpi .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-lTViQyq9CB5b0kpi .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-lTViQyq9CB5b0kpi .actorPopupMenu{position:absolute;}#mermaid-svg-lTViQyq9CB5b0kpi .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-lTViQyq9CB5b0kpi .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-lTViQyq9CB5b0kpi .actor-man circle,#mermaid-svg-lTViQyq9CB5b0kpi line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-lTViQyq9CB5b0kpi :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Save(entries) Encode Write fsync同步 OK

6.4 崩溃恢复

6.4.1 读取WAL

go 复制代码

func Open(lg *zap.Logger, dirpath string, snap walpb.Snapshot) (*WAL, error) {
    // 读取所有WAL文件
    // 验证完整性
    // 返回WAL实例
}

6.4.2 恢复流程

复制代码

1. 找到最新快照
2. 从快照开始读取WAL
3. 回放日志到内存
4. 重建状态

6.5 日志截断

6.5.1 截断时机

快照创建后
释放旧的WAL文件

go 复制代码

func (w *WAL) Release(snap walpb.Snapshot) error {
    // 删除小于snap.index的WAL文件
}

参考文献

WAL设计文档

07 - MVCC+BoltDB存储层

7.1 BoltDB基础

7.1.1 BoltDB特点

特点	说明
嵌入式	单文件存储
B+树索引	支持范围查询
ACID事务	保证一致性
MVCC读	无锁读

7.1.2 etcd对BoltDB的封装

源码位置 : server/storage/backend/

go 复制代码

type Backend interface {
    ReadTx() ReadTx
    BatchTx() BatchTx
    ConcurrentReadTx() ReadTx
    Lock()
    Unlock()
    Close() error
}

7.2 MVCC多版本控制

7.2.1 Revision结构

go 复制代码

type Revision struct {
    Main int64
    Sub int64
}

7.2.2 版本存储示例

复制代码

Key="foo":
  Rev:1 => Value="v1"
  Rev:2 => Value="v2"
  Rev:3 => Value="v3"
  ...

7.3 key索引

7.3.1 keyIndex结构

源码位置 : server/storage/mvcc/key_index.go

go 复制代码

type keyIndex struct {
    key []byte
    modified revision
    generations []generation
}

type generation struct {
    rev revisions
    created revision
}

7.3.2 索引查询流程

#mermaid-svg-EESsaBV82Q4tdQIx{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-EESsaBV82Q4tdQIx .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-EESsaBV82Q4tdQIx .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-EESsaBV82Q4tdQIx .error-icon{fill:#552222;}#mermaid-svg-EESsaBV82Q4tdQIx .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-EESsaBV82Q4tdQIx .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-EESsaBV82Q4tdQIx .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-EESsaBV82Q4tdQIx .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-EESsaBV82Q4tdQIx .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-EESsaBV82Q4tdQIx .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-EESsaBV82Q4tdQIx .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-EESsaBV82Q4tdQIx .marker{fill:#333333;stroke:#333333;}#mermaid-svg-EESsaBV82Q4tdQIx .marker.cross{stroke:#333333;}#mermaid-svg-EESsaBV82Q4tdQIx svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-EESsaBV82Q4tdQIx p{margin:0;}#mermaid-svg-EESsaBV82Q4tdQIx .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-EESsaBV82Q4tdQIx .cluster-label text{fill:#333;}#mermaid-svg-EESsaBV82Q4tdQIx .cluster-label span{color:#333;}#mermaid-svg-EESsaBV82Q4tdQIx .cluster-label span p{background-color:transparent;}#mermaid-svg-EESsaBV82Q4tdQIx .label text,#mermaid-svg-EESsaBV82Q4tdQIx span{fill:#333;color:#333;}#mermaid-svg-EESsaBV82Q4tdQIx .node rect,#mermaid-svg-EESsaBV82Q4tdQIx .node circle,#mermaid-svg-EESsaBV82Q4tdQIx .node ellipse,#mermaid-svg-EESsaBV82Q4tdQIx .node polygon,#mermaid-svg-EESsaBV82Q4tdQIx .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-EESsaBV82Q4tdQIx .rough-node .label text,#mermaid-svg-EESsaBV82Q4tdQIx .node .label text,#mermaid-svg-EESsaBV82Q4tdQIx .image-shape .label,#mermaid-svg-EESsaBV82Q4tdQIx .icon-shape .label{text-anchor:middle;}#mermaid-svg-EESsaBV82Q4tdQIx .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-EESsaBV82Q4tdQIx .rough-node .label,#mermaid-svg-EESsaBV82Q4tdQIx .node .label,#mermaid-svg-EESsaBV82Q4tdQIx .image-shape .label,#mermaid-svg-EESsaBV82Q4tdQIx .icon-shape .label{text-align:center;}#mermaid-svg-EESsaBV82Q4tdQIx .node.clickable{cursor:pointer;}#mermaid-svg-EESsaBV82Q4tdQIx .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-EESsaBV82Q4tdQIx .arrowheadPath{fill:#333333;}#mermaid-svg-EESsaBV82Q4tdQIx .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-EESsaBV82Q4tdQIx .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-EESsaBV82Q4tdQIx .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-EESsaBV82Q4tdQIx .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-EESsaBV82Q4tdQIx .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-EESsaBV82Q4tdQIx .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-EESsaBV82Q4tdQIx .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-EESsaBV82Q4tdQIx .cluster text{fill:#333;}#mermaid-svg-EESsaBV82Q4tdQIx .cluster span{color:#333;}#mermaid-svg-EESsaBV82Q4tdQIx div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-EESsaBV82Q4tdQIx .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-EESsaBV82Q4tdQIx rect.text{fill:none;stroke-width:0;}#mermaid-svg-EESsaBV82Q4tdQIx .icon-shape,#mermaid-svg-EESsaBV82Q4tdQIx .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-EESsaBV82Q4tdQIx .icon-shape p,#mermaid-svg-EESsaBV82Q4tdQIx .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-EESsaBV82Q4tdQIx .icon-shape .label rect,#mermaid-svg-EESsaBV82Q4tdQIx .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-EESsaBV82Q4tdQIx .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-EESsaBV82Q4tdQIx .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-EESsaBV82Q4tdQIx :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 是
否
Get(key, rev)
rev=0?
最新版本
查找<=rev的版本
返回K/V

7.4 事务处理

7.4.1 KV事务

源码位置 : server/storage/mvcc/kvstore_txn.go

go 复制代码

type TxnWrite interface {
    TxnRead
    Put(key, value []byte, lease lease.LeaseID) (rev int64)
    DeleteRange(key, end []byte) (n, rev int64)
    End()
}

7.4.2 事务提交流程

复制代码

1. 修改内存索引
2. 写BoltDB
3. 更新revision
4. 触发watch事件

7.5 版本压缩

7.5.1 压缩触发

go 复制代码

func (s *store) Compact(rev int64) (<-chan struct{}, error) {
    // 删除比rev小的版本
}

7.5.2 压缩后查询

复制代码

如果查询的rev已压缩，返回ErrCompacted

参考文献

BoltDB GitHub