功能点 11-12：客户端、RPC 通信、监控与安全

功能点 11-12：客户端、RPC 通信、监控与安全 ------ 源码阅读笔记

对应源码阅读计划功能点 11-12：FlussClient、Netty RPC、LogScanner、MetricRegistry、PrometheusReporter、FlussAuthorizer。

---

笔记 11.1：FlussClient ------ 客户端入口

文件：FlussClient.java、FlussConnection.java

路径 ：fluss-client/src/main/java/org/apache/fluss/client/

核心结构

public class FlussClient implements AutoCloseable {
    private final FlussConnection connection;
    private final Map<TabletKey, LogScanner> scanners;
    private final Map<TabletKey, LogWriter> writers;
    
    // ===== 流式读取 =====
    public LogScanner newLogScanner(
            TablePath tablePath, 
            int bucketId, 
            long startOffset,
            int[] projectedColumns) {
        
        // 1. 查找 Bucket 的 Leader TabletServer
        TabletServerInfo leader = findLeader(tablePath, bucketId);
        
        // 2. 连接到 Leader TabletServer
        return new LogScanner(
            connection.connect(leader),
            tablePath, bucketId, startOffset,
            projectedColumns  // ★ 传递给服务端用于列裁剪
        );
    }
    
    // ===== PK 查询 =====
    public byte[] pointLookup(TablePath tablePath, byte[] key) {
        // 1. 计算分桶
        int bucketId = bucketingFunction.getBucket(key, numBuckets);
        
        // 2. 查找 Leader
        TabletServerInfo leader = findLeader(tablePath, bucketId);
        
        // 3. 发送 PK Lookup RPC
        KvLookupRequest request = KvLookupRequest.builder()
            .tablePath(tablePath)
            .bucketId(bucketId)
            .key(key)
            .build();
        
        KvLookupResponse response = connection.send(leader, request);
        return response.getValue();  // 亚毫秒级
    }
    
    // ===== 流式写入 =====
    public LogWriter newLogWriter(
            TablePath tablePath, 
            int bucketId, 
            WriteMode mode) {
        
        // 1. 查找 Leader
        TabletServerInfo leader = findLeader(tablePath, bucketId);
        
        // 2. 创建 Writer
        return new LogWriter(
            connection.connect(leader),
            tablePath, bucketId,
            mode,  // APPEND / UPSERT / DELETE
            producerId, producerEpoch  // ★ 幂等写入参数
        );
    }
    
    // ===== 管理操作 =====
    public void createTable(TablePath tablePath, TableDescriptor descriptor) {
        // 发送到 CoordinatorServer
        CoordinatorServerInfo coordinator = findCoordinatorLeader();
        CreateTableRequest request = CreateTableRequest.builder()
            .tablePath(tablePath)
            .descriptor(descriptor)
            .build();
        connection.send(coordinator, request);
    }
}

---

笔记 11.2：LogScanner ------ 日志扫描器

文件：LogScanner.java

public class LogScanner {
    
    private long currentOffset;
    private final int[] projectedColumns;
    
    /**
     * 拉取下一批数据
     */
    public ArrowRecordBatch nextBatch() {
        // 1. 构造 Fetch RPC 请求
        FetchRequest request = FetchRequest.builder()
            .tabletId(tabletId)
            .startOffset(currentOffset)
            .maxBytes(MAX_FETCH_SIZE)      // 默认 1MB
            .isolation(FetchIsolation.READ_COMMITTED)  // 只读已提交数据
            .projectedColumns(projectedColumns)  // ★ 列裁剪
            .build();
        
        // 2. 发送到 TabletServer
        FetchResponse response = rpcClient.fetch(tabletServer, request);
        
        // 3. 解析 Arrow IPC 格式的响应
        if (response.getRecordBatch() != null) {
            ArrowRecordBatch batch = ArrowRecordBatch.fromIpc(
                response.getRecordBatch()
            );
            currentOffset = response.getNextOffset();
            return batch;
        }
        
        return null;  // 没有新数据
    }
}

---

笔记 11.3：Netty RPC 通信层

文件：NettyClient.java、NettyServer.java、RpcGateway.java

路径 ：fluss-rpc/src/main/java/org/apache/fluss/rpc/netty/

RPC 协议栈

应用层:
  FlussClient / FlussServer
      ↓
RPC 层:
  RpcGateway (接口定义)
      ↓
  RpcRequest / RpcResponse (封装)
      ↓
序列化:
  ProtobufSerializer (Protobuf 3.25.5)
      ↓
传输层:
  Netty 4.x (NIO, EventLoop)
      ↓
网络:
  TCP (直接连接)

Netty Server 实现

public class NettyServer {
    private final EventLoopGroup bossGroup;    // 接受连接
    private final EventLoopGroup workerGroup;  // 处理 I/O
    
    public void start() {
        ServerBootstrap bootstrap = new ServerBootstrap()
            .group(bossGroup, workerGroup)
            .channel(NioServerSocketChannel.class)
            .childHandler(new ChannelInitializer<>() {
                @Override
                protected void initChannel(SocketChannel ch) {
                    ch.pipeline()
                        .addLast("frameDecoder", 
                            new LengthFieldBasedFrameDecoder(
                                MAX_FRAME_SIZE, 0, 4, 0, 4
                            ))  // ★ 处理 TCP 粘包/拆包
                        .addLast("protobufDecoder",
                            new ProtobufDecoder(RpcRequest.getDefaultInstance()))
                        .addLast("protobufEncoder",
                            new ProtobufEncoder())
                        .addLast("rpcHandler",
                            new RpcServerHandler(rpcService));
                }
            })
            .option(ChannelOption.SO_BACKLOG, 128)
            .childOption(ChannelOption.TCP_NODELAY, true)  // ★ 关闭 Nagle，降低延迟
            .childOption(ChannelOption.SO_KEEPALIVE, true);
        
        ChannelFuture future = bootstrap.bind(port).sync();
    }
}

关键优化：

• TCP_NODELAY：关闭 Nagle 算法，避免小数据包延迟（延迟敏感场景）
• LengthFieldBasedFrameDecoder：处理 TCP 流式传输的边界问题
• ProtobufEncoder/Decoder：高效二进制序列化

---

笔记 12.1：MetricRegistry ------ 监控指标系统

文件：MetricRegistry.java、PrometheusReporter.java

指标注册与上报

public class MetricRegistry {
    
    // 指标类型
    private final Map<String, Counter> counters;      // 计数器（累积）
    private final Map<String, Gauge> gauges;          // 瞬时值
    private final Map<String, Histogram> histograms;  // 分布
    
    /**
     * TabletServer 关键指标注册
     */
    public void registerTabletMetrics() {
        // 日志读写速率
        registerCounter("fluss.tablet.log.write.rate", 
            () -> logWriteCount.getAndReset());  // 每秒写入条数
        
        registerCounter("fluss.tablet.log.read.rate",
            () -> logReadCount.getAndReset());   // 每秒读取条数
        
        // KV 读写延迟
        registerHistogram("fluss.tablet.kv.get.latency",
            new long[]{1, 5, 10, 50, 100, 500});  // P50/P90/P99
        
        registerHistogram("fluss.tablet.kv.put.latency", ...);
        
        // 磁盘使用
        registerGauge("fluss.tablet.disk.usage.bytes",
            () -> getDiskUsage());
        
        // Tiering 延迟
        registerGauge("fluss.tablet.tiering.offset.lag",
            () -> LEO - lastTieredOffset);
    }
}

Prometheus 输出格式

public class PrometheusReporter implements MetricReporter {
    
    @Override
    public String report() {
        StringBuilder sb = new StringBuilder();
        
        // Prometheus 文本格式
        // fluss_tablet_log_write_rate{job="fluss",instance="ts-1"} 52340
        // fluss_tablet_kv_get_latency_p99{job="fluss",instance="ts-1"} 3.2
        
        for (var entry : counters.entrySet()) {
            sb.append(formatPrometheusMetric(
                entry.getKey(), 
                labels, 
                entry.getValue().get()
            )).append("\n");
        }
        
        return sb.toString();
    }
}

---

笔记 12.2：FlussAuthorizer ------ 安全授权

文件：FlussAuthorizer.java

路径 ：fluss-server/src/main/java/org/apache/fluss/server/authorizer/FlussAuthorizer.java

授权模型

public interface Authorizer {
    
    /**
     * 检查用户是否有权限执行操作
     * 
     * @param user      请求用户
     * @param operation 操作类型（READ/WRITE/CREATE/ALTER/DROP）
     * @param resource  操作资源（TablePath/DatabaseName）
     */
    boolean authorize(User user, Operation operation, Resource resource);
    
    enum Operation {
        READ,       // SELECT / 流式读取
        WRITE,      // INSERT / UPDATE / DELETE
        CREATE,     // CREATE TABLE / DATABASE
        ALTER,      // ALTER TABLE
        DROP,       // DROP TABLE / DATABASE
        DESCRIBE,   // SHOW / DESCRIBE
    }
    
    /**
     * SASL/PLAIN 认证配置
     * 
     * fluss-conf.yaml:
     *   security.enabled: true
     *   sasl.mechanism: PLAIN
     *   sasl.users: "user1:password1,user2:password2"
     */
    void authenticate(String username, String password);
}

---

阅读小结（全系列）

功能点	核心文件数	关键收获
1. 启动流程	3	Standby→Leader 两阶段、ZK Fence 防脑裂
2. 元数据	3	ZK 元数据树、createTable 6步链路、Schema Evolution
3. 数据分布	3	MurmurHash3、轮询+容量感知分配、渐进式 Rebalance
4. LogStore	4	Segment 布局、稀疏索引二分查找、LEO/HW、幂等写入
5. KvStore	3	RocksDB 配置、三种 Merge Engine、WAL 恢复
6. Arrow	4	列式写入、零拷贝列裁剪、类型映射、Direct Memory
7. 副本/ISR	4	Leader/Follower 状态机、HW 推进、ZK 选举
8. Tiering	3	Arrow→Parquet、Union Read、3种 Lake Format
9. Flink Connector	4	Catalog/Source/Sink/Lookup、Two-Phase Commit
10. Delta Join	3	无状态算子、JoinStateStore、秒级恢复
11-12. 客户端/监控	4	Netty RPC、Prometheus、SASL 认证

---

全系列 12 个功能点源码阅读笔记完成。基于 Apache Fluss main 分支 (2026年8月)