第 10 篇：「Fluss 实战案例集」------ 完整解决方案与最佳实践
阅读本文你将了解： 5 个完整的生产级端到端案例------电商实时大屏、实时特征存储、CDC 数据管道、实时风控系统、客户 360。每个案例包含架构设计、表结构 DDL、Flink 作业代码和运行指南。最后附 Kafka 迁移指南和生产就绪评估清单。
10.1 案例一：电商实时大屏
10.1.1 业务需求
构建一个实时数据大屏，展示电商平台的核心指标：
- 实时 GMV（成交额）
- 订单量、支付量、发货量
- 各品类 Top 10 排行
- 1 分钟/5 分钟/1 小时粒度聚合
10.1.2 架构设计
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ 订单系统 │ │ 支付系统 │ │ 物流系统 │
│ (Kafka/API) │ │ (Kafka/API) │ │ (Kafka/API) │
└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
│ │ │
└───────────┬───────┴───────────┬───────┘
│ │
┌──────▼───────────────────▼──────┐
│ Apache Fluss │
│ ┌─────────┐ ┌───────────────┐ │
│ │orders │ │ order_wide │ │
│ │payments │ │ (宽表) │ │
│ │shipments│ │ │ │
│ └─────────┘ └───────────────┘ │
└──────────────┬──────────────────┘
│
┌──────────────▼──────────────────┐
│ Flink 流计算 │
│ ┌────────────────────────────┐ │
│ │ 多时间窗口聚合 │ │
│ │ Lookup Join 富化 │ │
│ │ 实时 TOP N 排行 │ │
│ └────────────┬───────────────┘ │
└───────────────┼──────────────────┘
│
┌───────────────▼──────────────────┐
│ 实时大屏 │
│ (WebSocket → Dashboard) │
└──────────────────────────────────┘10.1.3 DDL
sql
CREATE CATALOG fluss_catalog WITH (
'type' = 'fluss',
'bootstrap.servers' = 'coord-1:9123,coord-2:9123'
);
USE CATALOG fluss_catalog;
CREATE DATABASE ecommerce_realtime;
USE ecommerce_realtime;
-- 订单事实表
CREATE TABLE orders (
order_id BIGINT,
user_id BIGINT,
product_id BIGINT,
category_id INT,
amount DECIMAL(10, 2),
quantity INT,
order_time TIMESTAMP(3),
dt STRING,
PRIMARY KEY (order_id, dt) NOT ENFORCED
) PARTITIONED BY (dt)
WITH (
'bucket.num' = '32',
'table.merge-engine' = 'deduplicate'
);
-- 支付事实表
CREATE TABLE payments (
payment_id BIGINT,
order_id BIGINT,
amount DECIMAL(10, 2),
status STRING, -- 'success', 'failed', 'refund'
pay_time TIMESTAMP(3),
dt STRING,
PRIMARY KEY (payment_id, dt) NOT ENFORCED
) PARTITIONED BY (dt)
WITH (
'bucket.num' = '16',
'table.merge-engine' = 'deduplicate'
);
-- 商品维表
CREATE TABLE product_dim (
product_id BIGINT,
product_name STRING,
category_id INT,
category_name STRING,
price DECIMAL(10, 2),
PRIMARY KEY (product_id) NOT ENFORCED
) WITH ('bucket.num' = '8');
-- 实时大屏指标表
CREATE TABLE dashboard_metrics (
metric_key STRING, -- 'gmv', 'order_count', 'payment_count'
window_start TIMESTAMP(3),
window_size STRING, -- '1min', '5min', '1hour'
metric_value DECIMAL(14, 2),
update_time TIMESTAMP(3),
PRIMARY KEY (metric_key, window_start, window_size) NOT ENFORCED
) WITH (
'bucket.num' = '4',
'table.merge-engine' = 'deduplicate'
);
-- 品类排行表
CREATE TABLE category_ranking (
window_start TIMESTAMP(3),
window_size STRING,
category_id INT,
category_name STRING,
total_amount DECIMAL(14, 2),
order_count BIGINT,
ranking INT,
PRIMARY KEY (window_start, window_size, category_id) NOT ENFORCED
) WITH ('bucket.num' = '4');10.1.4 Flink SQL 作业
sql
SET 'execution.runtime-mode' = 'streaming';
SET 'pipeline.name' = 'ecommerce-realtime-dashboard';
-- 作业 1：1 分钟窗口聚合
INSERT INTO dashboard_metrics
SELECT
'gmv' AS metric_key,
TUMBLE_START(order_time, INTERVAL '1' MINUTE) AS window_start,
'1min' AS window_size,
SUM(amount * quantity) AS metric_value,
NOW() AS update_time
FROM orders
GROUP BY TUMBLE(order_time, INTERVAL '1' MINUTE)
UNION ALL
SELECT
'order_count' AS metric_key,
TUMBLE_START(order_time, INTERVAL '1' MINUTE) AS window_start,
'1min' AS window_size,
CAST(COUNT(*) AS DECIMAL(14, 2)) AS metric_value,
NOW() AS update_time
FROM orders
GROUP BY TUMBLE(order_time, INTERVAL '1' MINUTE);
-- 作业 2：品类实时排行（1 分钟）
INSERT INTO category_ranking
SELECT
window_start,
window_size,
category_id,
category_name,
total_amount,
order_count,
ROW_NUMBER() OVER (
PARTITION BY window_start, window_size
ORDER BY total_amount DESC
) AS ranking
FROM (
SELECT
TUMBLE_START(o.order_time, INTERVAL '1' MINUTE) AS window_start,
'1min' AS window_size,
p.category_id,
p.category_name,
SUM(o.amount * o.quantity) AS total_amount,
COUNT(*) AS order_count
FROM orders AS o
LEFT JOIN product_dim FOR SYSTEM_TIME AS OF o.order_time AS p
ON o.product_id = p.product_id
GROUP BY
TUMBLE(o.order_time, INTERVAL '1' MINUTE),
p.category_id, p.category_name
);10.2 案例二：实时特征存储（ML Feature Store）
10.2.1 业务需求
为推荐和风控模型提供实时特征服务：
- 用户过去 7/30 天的统计特征（购买次数、金额、活跃度）
- 用户实时行为特征（最近 10 次浏览/购买的商品）
- 亚毫秒级查询延迟（模型推理场景）
10.2.2 表设计
sql
-- 用户统计特征表（Aggregation Merge Engine）
CREATE TABLE user_stats_features (
user_id BIGINT,
-- 7 天特征
purchase_count_7d INT,
total_spent_7d DECIMAL(12, 2),
active_days_7d INT,
-- 30 天特征
purchase_count_30d INT,
total_spent_30d DECIMAL(12, 2),
active_days_30d INT,
-- 全量特征
lifetime_purchases INT,
avg_order_value DECIMAL(10, 2),
last_purchase_time TIMESTAMP(3),
PRIMARY KEY (user_id) NOT ENFORCED
) WITH (
'bucket.num' = '32',
'table.merge-engine' = 'aggregation',
'fields.purchase_count_7d.aggregate-function' = 'sum',
'fields.total_spent_7d.aggregate-function' = 'sum',
'fields.active_days_7d.aggregate-function' = 'max',
'fields.purchase_count_30d.aggregate-function' = 'sum',
'fields.total_spent_30d.aggregate-function' = 'sum',
'fields.active_days_30d.aggregate-function' = 'max',
'fields.lifetime_purchases.aggregate-function' = 'sum',
'fields.last_purchase_time.aggregate-function' = 'last_value'
);
-- 用户行为序列特征表（Partial Update）
CREATE TABLE user_behavior_features (
user_id BIGINT,
last_10_views ARRAY<BIGINT>, -- 最近浏览的 10 个商品
last_10_purchases ARRAY<BIGINT>, -- 最近购买的 10 个商品
favorite_categories ARRAY<INT>, -- 偏好品类
update_time TIMESTAMP(3),
PRIMARY KEY (user_id) NOT ENFORCED
) WITH (
'bucket.num' = '16',
'table.merge-engine' = 'partial-update'
);
-- 特征查询接口（PYTHON 示例）
-- 模型推理时通过 Fluss Rust/Python Client 进行亚毫秒级查询10.2.3 特征查询示例
python
"""
Python ML 推理中使用 Fluss 特征存储
使用 Fluss Rust Client (PyO3 绑定) 或 Arrow Flight
"""
import pyarrow as pa
from fluss_client import FlussClient
class FeatureService:
def __init__(self):
self.client = FlussClient(bootstrap_servers="coord-1:9123")
def get_user_features(self, user_id: int) -> dict:
"""获取用户特征用于模型推理"""
# 亚毫秒级 PK 查询
stats = self.client.point_lookup(
table="user_stats_features",
key={"user_id": user_id}
)
behavior = self.client.point_lookup(
table="user_behavior_features",
key={"user_id": user_id}
)
return {
"purchase_count_7d": stats["purchase_count_7d"],
"total_spent_7d": stats["total_spent_7d"],
"last_10_purchases": behavior["last_10_purchases"],
"favorite_categories": behavior["favorite_categories"],
}
# 推理时使用
feature_service = FeatureService()
user_features = feature_service.get_user_features(user_id=12345)
prediction = model.predict(user_features)10.3 案例三：CDC 数据管道
10.3.1 业务需求
将 MySQL 业务数据库的变更实时同步到流处理系统：
- MySQL Binlog → Fluss → 多引擎消费
- 无需部署 Kafka Connect / Debezium / Schema Registry
- 利用 Fluss 原生
$changelog虚拟表
10.3.2 架构
┌──────────┐ ┌──────────────┐ ┌─────────────────────────┐
│ MySQL │────→│ Flink CDC │────→│ Apache Fluss │
│ (Binlog) │ │ Connector │ │ │
└──────────┘ └──────────────┘ │ users (PK Table) │
│ users$changelog (虚拟表) │
│ users$binlog (虚拟表) │
└──────────┬───────────────┘
│
┌────────────────────────────┼────────────────────┐
│ │ │
┌──────▼──────┐ ┌────────▼────────┐ ┌───────▼──────┐
│ Flink 流计算 │ │ Spark 批处理 │ │ 下游服务 │
│ 实时宽表 │ │ 离线分析 │ │ 实时查询 │
└─────────────┘ └─────────────────┘ └──────────────┘10.3.3 Flink CDC → Fluss 同步作业
sql
-- Step 1: 创建 Fluss 表（结构与 MySQL 表一致）
CREATE TABLE users_fluss (
user_id BIGINT,
name STRING,
email STRING,
city STRING,
status STRING,
created_at TIMESTAMP(3),
updated_at TIMESTAMP(3),
PRIMARY KEY (user_id) NOT ENFORCED
) WITH (
'bucket.num' = '8'
);
-- Step 2: 创建 MySQL CDC Source
CREATE TABLE users_mysql_cdc (
user_id BIGINT,
name STRING,
email STRING,
city STRING,
status STRING,
created_at TIMESTAMP(3),
updated_at TIMESTAMP(3),
PRIMARY KEY (user_id) NOT ENFORCED
) WITH (
'connector' = 'mysql-cdc',
'hostname' = 'mysql-host',
'port' = '3306',
'username' = 'cdc_user',
'password' = 'cdc_password',
'database-name' = 'business_db',
'table-name' = 'users',
'server-id' = '5400-5404'
);
-- Step 3: 实时同步
SET 'execution.runtime-mode' = 'streaming';
INSERT INTO users_fluss
SELECT * FROM users_mysql_cdc;
-- Step 4: 消费变更日志
-- 方法 1: 通过 $changelog 虚拟表获取变更
SELECT * FROM users_fluss$changelog;
-- 输出: +I (insert), -U (update_before), +U (update_after), -D (delete)
-- 方法 2: 通过 $binlog 虚拟表获取原始日志
SELECT * FROM users_fluss$binlog;10.3.4 下游多引擎消费
sql
-- Spark 消费 Fluss 中的 CDC 数据
CREATE CATALOG fluss_spark_catalog WITH (
'type' = 'fluss',
'bootstrap.servers' = 'coord-1:9123'
);
-- 批量分析
SELECT status, COUNT(*)
FROM fluss_spark_catalog.business_db.users_fluss
GROUP BY status;
-- 也可直接读 Iceberg Cold Tier（通过 Iceberg Catalog）
SELECT * FROM iceberg_catalog.business_db.users_fluss
WHERE dt = '2026-08-08';10.4 案例四：实时风控系统
10.4.1 业务需求
实时检测异常交易行为：
- 单用户短期内高频交易检测
- 单用户大额交易检测
- 设备/IP 关联风险检测
- 规则引擎实时决策（< 10ms）
10.4.2 表设计
sql
-- 交易流水表
CREATE TABLE transactions (
txn_id BIGINT,
user_id BIGINT,
device_id STRING,
ip_address STRING,
amount DECIMAL(12, 2),
merchant_id STRING,
txn_type STRING, -- 'payment', 'transfer', 'withdrawal'
txn_time TIMESTAMP(3),
dt STRING,
PRIMARY KEY (txn_id, dt) NOT ENFORCED
) PARTITIONED BY (dt)
WITH (
'bucket.num' = '32',
'table.merge-engine' = 'deduplicate'
);
-- 用户风控画像表（Aggregation Engine）
CREATE TABLE user_risk_profile (
user_id BIGINT,
txn_count_1h INT,
txn_count_24h INT,
total_amount_1h DECIMAL(14, 2),
total_amount_24h DECIMAL(14, 2),
distinct_devices_24h INT,
distinct_ips_24h INT,
risk_score INT,
risk_level STRING, -- 'low', 'medium', 'high', 'blocked'
last_txn_time TIMESTAMP(3),
PRIMARY KEY (user_id) NOT ENFORCED
) WITH (
'bucket.num' = '16',
'table.merge-engine' = 'aggregation',
'fields.txn_count_1h.aggregate-function' = 'sum',
'fields.txn_count_24h.aggregate-function' = 'sum',
'fields.total_amount_1h.aggregate-function' = 'sum',
'fields.total_amount_24h.aggregate-function' = 'sum',
'fields.distinct_devices_24h.aggregate-function' = 'max',
'fields.distinct_ips_24h.aggregate-function' = 'max',
'fields.risk_score.aggregate-function' = 'max',
'fields.last_txn_time.aggregate-function' = 'last_value'
);
-- 风控规则结果表
CREATE TABLE risk_alerts (
alert_id BIGINT,
txn_id BIGINT,
user_id BIGINT,
rule_type STRING, -- 'high_frequency', 'large_amount', 'device_risk'
risk_score INT,
decision STRING, -- 'pass', 'review', 'block'
alert_time TIMESTAMP(3),
PRIMARY KEY (alert_id) NOT ENFORCED
) WITH (
'bucket.num' = '8',
'table.merge-engine' = 'deduplicate'
);10.4.3 风控规则 Flink SQL
sql
-- 规则 1: 高频交易检测（1 小时内 > 10 笔）
INSERT INTO risk_alerts
SELECT
CONCAT(t.user_id, '_', 'high_freq_', UNIX_TIMESTAMP()) AS alert_id,
t.txn_id,
t.user_id,
'high_frequency' AS rule_type,
50 AS risk_score,
CASE WHEN u.txn_count_1h > 20 THEN 'block'
WHEN u.txn_count_1h > 10 THEN 'review'
ELSE 'pass' END AS decision,
NOW() AS alert_time
FROM transactions AS t
LEFT JOIN user_risk_profile FOR SYSTEM_TIME AS OF t.txn_time AS u
ON t.user_id = u.user_id
WHERE u.txn_count_1h > 10;
-- 规则 2: 大额交易检测（单笔 > 50000）
INSERT INTO risk_alerts
SELECT
CONCAT(t.user_id, '_', 'large_amt_', UNIX_TIMESTAMP()) AS alert_id,
t.txn_id,
t.user_id,
'large_amount' AS rule_type,
80 AS risk_score,
'review' AS decision,
NOW() AS alert_time
FROM transactions AS t
WHERE t.amount > 50000;
-- 规则 3: 多设备/IP 风险
INSERT INTO risk_alerts
SELECT
CONCAT(t.user_id, '_', 'device_risk_', UNIX_TIMESTAMP()) AS alert_id,
t.txn_id,
t.user_id,
'device_risk' AS rule_type,
90 AS risk_score,
'block' AS decision,
NOW() AS alert_time
FROM transactions AS t
LEFT JOIN user_risk_profile FOR SYSTEM_TIME AS OF t.txn_time AS u
ON t.user_id = u.user_id
WHERE u.distinct_devices_24h > 5 OR u.distinct_ips_24h > 10;10.5 案例五：客户 360
10.5.1 业务需求
构建客户统一视图，融合多个数据源：
- 交易数据（订单、支付）
- 行为数据（浏览、点击、搜索）
- 客服数据（工单、满意度）
- 营销数据（优惠券、活动参与）
10.5.2 表设计（Partial Update 多源写入）
sql
-- 客户 360 宽表（Partial Update 支持多源独立写入）
CREATE TABLE customer_360 (
customer_id BIGINT,
-- 来自交易系统
last_order_time TIMESTAMP(3),
lifetime_value DECIMAL(12, 2),
total_orders INT,
favorite_category INT,
-- 来自行为系统
last_active_time TIMESTAMP(3),
preferred_device STRING,
avg_session_duration INT,
search_keywords STRING,
-- 来自客服系统
last_ticket_time TIMESTAMP(3),
total_tickets INT,
satisfaction_score DECIMAL(3, 2),
customer_segment STRING, -- 'new', 'active', 'at_risk', 'churned'
PRIMARY KEY (customer_id) NOT ENFORCED
) WITH (
'bucket.num' = '32',
'table.merge-engine' = 'partial-update'
);
-- 交易系统写入（只写交易相关字段）
INSERT INTO customer_360(
customer_id, last_order_time, lifetime_value,
total_orders, favorite_category
)
SELECT
user_id,
MAX(order_time),
SUM(amount),
COUNT(*),
LAST_VALUE(category_id)
FROM orders
GROUP BY user_id;
-- 行为系统写入（只写行为相关字段）
INSERT INTO customer_360(
customer_id, last_active_time, preferred_device,
avg_session_duration, search_keywords
)
SELECT
user_id,
MAX(event_time),
LAST_VALUE(device_type),
AVG(duration),
LAST_VALUE(keyword)
FROM user_behavior
GROUP BY user_id;
-- 客服系统写入（只写客服相关字段）
INSERT INTO customer_360(
customer_id, last_ticket_time, total_tickets,
satisfaction_score, customer_segment
)
SELECT
user_id,
MAX(ticket_time),
COUNT(*),
AVG(satisfaction),
LAST_VALUE(segment)
FROM cs_tickets
GROUP BY user_id;
-- 最终所有字段自动合并为一张宽表！10.6 从 Kafka 迁移到 Fluss
10.6.1 迁移策略
Phase 1: 双写验证（1-2 周）
├── Kafka 正常写入
├── Fluss 通过 fluss-kafka 兼容模块启动镜像消费
└── 对比 Kafka 和 Fluss 的数据一致性
Phase 2: 灰度切换（2-4 周）
├── 将 20% 的 Flink 任务切换到 Fluss
├── 监控延迟、吞吐、错误率
└── 逐步增加到 100%
Phase 3: 全量切换（1 周）
├── 所有任务切换到 Fluss
├── 下线 Kafka Connect 和 Debezium
└── 关闭 Kafka 集群10.6.2 Kafka 兼容层
sql
-- Fluss 支持 Kafka 协议，现有的 Kafka 客户端无需大改动
-- 只需将 bootstrap.servers 指向 Fluss Coordinator
-- Kafka Consumer → Fluss Consumer
Properties props = new Properties();
// props.put("bootstrap.servers", "kafka:9092"); // 旧
props.put("bootstrap.servers", "fluss-coordinator:9123"); // 新
props.put("key.deserializer", "...");
props.put("value.deserializer", "...");10.6.3 迁移 Checklist
☐ 确认 Fluss 版本与现有 Flink 版本兼容
☐ 搭建 Fluss 测试集群，验证功能
☐ 在 Fluss 中创建与 Kafka Topic 对应的表
☐ 启动双写，持续 1 周验证数据一致性
☐ 准备回滚方案（保留 Kafka 集群 1 个月）
☐ 选择性迁移 1-2 个低风险 Flink 任务
☐ 监控关键指标 24 小时
☐ 逐步迁移所有任务
☐ 配置 Fluss Tiering 到 Iceberg/Paimon
☐ 下线 Kafka 集群10.7 性能调优总结
| 调优方向 | 关键参数 | 建议值 |
|---|---|---|
| Bucket 数量 | bucket.num |
节点数 × 4 ~ 节点数 × 8 |
| RocksDB Block Cache | kv.store.block.cache.size |
可用内存的 40% |
| Log Segment 大小 | log.segment.size |
1GB-4GB |
| Tiering 间隔 | table.tiering.commit-interval |
5min-10min |
| Flink 并行度 | parallelism.default |
TabletServer 数 × 2 |
| Checkpoint 间隔 | execution.checkpointing.interval |
60s-180s |
| JVM 堆 | Xmx | TabletServer: 16g-32g, Coordinator: 8g-16g |
| ZK 超时 | zookeeper.session.timeout |
60000ms-120000ms |
Fluss 生产就绪评估清单
架构设计
☐ 表类型选择正确（Log Table vs PK Table）
☐ 分区策略合理（PK 表分区列是主键子集）
☐ Bucket 数量满足未来 12 个月的数据增长
☐ Merge Engine 配置匹配业务需求
高可用
☐ CoordinatorServer ≥ 3 个节点
☐ TabletServer ≥ 3 个节点
☐ LogTablet 副本数 ≥ 3
☐ ZooKeeper ≥ 3 个节点（或 Raft 模式）
存储
☐ 数据目录使用 SSD
☐ Remote Storage 已配置（S3/Iceberg/Paimon）
☐ Tiering 已启用并验证
☐ 磁盘容量预留 30% 缓冲
监控
☐ Prometheus + Grafana 已部署
☐ 关键告警规则已配置：
☐ TabletServer Down
☐ KV 延迟 > 100ms
☐ 磁盘使用 > 85%
☐ Tiering 延迟 > 1000 万条
运维
☐ 备份策略已制定
☐ 灾难恢复流程已测试
☐ 扩缩容流程已文档化
☐ 日志保留策略已配置10.8 总结
| 案例 | 核心技术 | 关键收益 |
|---|---|---|
| 电商大屏 | 多时间窗口聚合 + Lookup Join | 亚秒级实时指标，品类 TOP N |
| 特征存储 | Aggregation + Partial Update | 亚毫秒特征查询，统一 ML 数据层 |
| CDC 管道 | Flink CDC + Fluss Changelog | 无需 Debezium/Kafka Connect |
| 实时风控 | 多规则并行检测 + 亚毫秒查询 | 10ms 内完成风控决策 |
| 客户 360 | Partial Update 多源写入 | 一张表融合 5 个数据源 |
Fluss 的核心价值一句话总结：将消息队列、KV 存储、状态后端、湖仓统一到一个基座，让流计算真正无状态、让实时分析真正实时、让数据架构真正简化。
系列完。本文基于 Apache Fluss 0.9.1。项目 GitHub: https://github.com/apache/fluss | 官网: https://fluss.apache.org/