Kafka 消息可靠性：发送确认、acks、副本保存与Offset手动提交

Kafka 经常被用在日志采集、用户行为、订单事件、数据同步和异步解耦场景。它的吞吐量很高，但高吞吐不等于天然不丢消息。面试里问"Kafka 如何保证消息不丢"，要沿着生产者、Broker、消费者三段链路讲。

一句话概括：Kafka 可靠性要从生产者发送、Broker 副本保存、消费者 Offset 提交三个位置兜住；生产者用回调和重试确认发送结果，Broker 用 acks=all 和副本机制保证保存，消费者关闭自动提交，处理成功后再提交 Offset。

#mermaid-svg-Y0tgAllk54O2IASs{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-Y0tgAllk54O2IASs .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-Y0tgAllk54O2IASs .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-Y0tgAllk54O2IASs .error-icon{fill:#552222;}#mermaid-svg-Y0tgAllk54O2IASs .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-Y0tgAllk54O2IASs .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-Y0tgAllk54O2IASs .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-Y0tgAllk54O2IASs .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-Y0tgAllk54O2IASs .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-Y0tgAllk54O2IASs .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-Y0tgAllk54O2IASs .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-Y0tgAllk54O2IASs .marker{fill:#333333;stroke:#333333;}#mermaid-svg-Y0tgAllk54O2IASs .marker.cross{stroke:#333333;}#mermaid-svg-Y0tgAllk54O2IASs svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-Y0tgAllk54O2IASs p{margin:0;}#mermaid-svg-Y0tgAllk54O2IASs .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-Y0tgAllk54O2IASs .cluster-label text{fill:#333;}#mermaid-svg-Y0tgAllk54O2IASs .cluster-label span{color:#333;}#mermaid-svg-Y0tgAllk54O2IASs .cluster-label span p{background-color:transparent;}#mermaid-svg-Y0tgAllk54O2IASs .label text,#mermaid-svg-Y0tgAllk54O2IASs span{fill:#333;color:#333;}#mermaid-svg-Y0tgAllk54O2IASs .node rect,#mermaid-svg-Y0tgAllk54O2IASs .node circle,#mermaid-svg-Y0tgAllk54O2IASs .node ellipse,#mermaid-svg-Y0tgAllk54O2IASs .node polygon,#mermaid-svg-Y0tgAllk54O2IASs .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-Y0tgAllk54O2IASs .rough-node .label text,#mermaid-svg-Y0tgAllk54O2IASs .node .label text,#mermaid-svg-Y0tgAllk54O2IASs .image-shape .label,#mermaid-svg-Y0tgAllk54O2IASs .icon-shape .label{text-anchor:middle;}#mermaid-svg-Y0tgAllk54O2IASs .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-Y0tgAllk54O2IASs .rough-node .label,#mermaid-svg-Y0tgAllk54O2IASs .node .label,#mermaid-svg-Y0tgAllk54O2IASs .image-shape .label,#mermaid-svg-Y0tgAllk54O2IASs .icon-shape .label{text-align:center;}#mermaid-svg-Y0tgAllk54O2IASs .node.clickable{cursor:pointer;}#mermaid-svg-Y0tgAllk54O2IASs .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-Y0tgAllk54O2IASs .arrowheadPath{fill:#333333;}#mermaid-svg-Y0tgAllk54O2IASs .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-Y0tgAllk54O2IASs .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-Y0tgAllk54O2IASs .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Y0tgAllk54O2IASs .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-Y0tgAllk54O2IASs .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Y0tgAllk54O2IASs .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-Y0tgAllk54O2IASs .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-Y0tgAllk54O2IASs .cluster text{fill:#333;}#mermaid-svg-Y0tgAllk54O2IASs .cluster span{color:#333;}#mermaid-svg-Y0tgAllk54O2IASs div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-Y0tgAllk54O2IASs .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-Y0tgAllk54O2IASs rect.text{fill:none;stroke-width:0;}#mermaid-svg-Y0tgAllk54O2IASs .icon-shape,#mermaid-svg-Y0tgAllk54O2IASs .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Y0tgAllk54O2IASs .icon-shape p,#mermaid-svg-Y0tgAllk54O2IASs .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-Y0tgAllk54O2IASs .icon-shape .label rect,#mermaid-svg-Y0tgAllk54O2IASs .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Y0tgAllk54O2IASs .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-Y0tgAllk54O2IASs .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-Y0tgAllk54O2IASs :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} callback + retry
acks=all
处理成功后
发送失败
副本不足
提交过早
Producer
Broker Leader
Followers/ISR
Consumer
手动提交 Offset
记录日志或重发
拒绝确认或等待同步
可能丢消息

Kafka 消息会丢在哪里

课件里把 Kafka 消息丢失拆成三段：

位置	可能问题	解决方向
生产者到 Broker	发送失败但业务没感知	异步回调、失败重试
Broker 存储	Leader 收到但副本没同步就故障	acks=all、副本机制
消费者消费	Offset 先提交，业务后失败	关闭自动提交，手动提交

所以 Kafka 的可靠性不是一个参数，而是一条端到端链路。

生产者发送可靠性

生产者发送消息有同步和异步两种方式。同步发送可以直接拿到结果，但吞吐较低；异步发送吞吐更高，必须在回调里处理失败。

// 异步发送，失败时要记录日志、告警或进入补偿重试。
kafkaProducer.send(record, new Callback() {
    @Override
    public void onCompletion(RecordMetadata metadata, Exception e) {
        if (e != null) {
            System.out.println("消息发送失败，记录日志或入库补偿");
            return;
        }
        long offset = metadata.offset();
        int partition = metadata.partition();
        String topic = metadata.topic();
    }
});

同时可以配置重试次数：

props.put(ProducerConfig.RETRIES_CONFIG, 10);

但要注意：重试可能导致重复消息或顺序变化。所以业务消费者仍然要做幂等，顺序敏感场景还要配合 key、分区和生产者相关配置一起设计。

如果使用较新的 Kafka 客户端，生产者幂等通常也要一起考虑：

props.put(ProducerConfig.ACKS_CONFIG, "all");
props.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true);
props.put(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE);
props.put(ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, 5);

生产者幂等解决的是"生产者重试导致同一条消息被写入多次"的问题。它不是业务幂等的替代品，因为业务重复消费、消费者重试、人工补偿仍然可能让同一业务事件被处理多次。

Broker 存储可靠性：acks

acks 决定生产者什么时候认为消息发送成功。

配置	含义	可靠性	性能
acks=0	不等待服务器响应	最差	最快
acks=1	Leader 收到就返回成功	中等	较快
acks=all	ISR 中副本都确认后才返回成功	最高	较慢

Follower 2 Follower 1 Leader Producer Follower 2 Follower 1 Leader Producer #mermaid-svg-7by766P9Bv20Q5XF{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-7by766P9Bv20Q5XF .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-7by766P9Bv20Q5XF .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-7by766P9Bv20Q5XF .error-icon{fill:#552222;}#mermaid-svg-7by766P9Bv20Q5XF .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-7by766P9Bv20Q5XF .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-7by766P9Bv20Q5XF .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-7by766P9Bv20Q5XF .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-7by766P9Bv20Q5XF .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-7by766P9Bv20Q5XF .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-7by766P9Bv20Q5XF .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-7by766P9Bv20Q5XF .marker{fill:#333333;stroke:#333333;}#mermaid-svg-7by766P9Bv20Q5XF .marker.cross{stroke:#333333;}#mermaid-svg-7by766P9Bv20Q5XF svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-7by766P9Bv20Q5XF p{margin:0;}#mermaid-svg-7by766P9Bv20Q5XF .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-7by766P9Bv20Q5XF text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-7by766P9Bv20Q5XF .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-7by766P9Bv20Q5XF .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-7by766P9Bv20Q5XF .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-7by766P9Bv20Q5XF .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-7by766P9Bv20Q5XF #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-7by766P9Bv20Q5XF .sequenceNumber{fill:white;}#mermaid-svg-7by766P9Bv20Q5XF #sequencenumber{fill:#333;}#mermaid-svg-7by766P9Bv20Q5XF #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-7by766P9Bv20Q5XF .messageText{fill:#333;stroke:none;}#mermaid-svg-7by766P9Bv20Q5XF .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-7by766P9Bv20Q5XF .labelText,#mermaid-svg-7by766P9Bv20Q5XF .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-7by766P9Bv20Q5XF .loopText,#mermaid-svg-7by766P9Bv20Q5XF .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-7by766P9Bv20Q5XF .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-7by766P9Bv20Q5XF .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-7by766P9Bv20Q5XF .noteText,#mermaid-svg-7by766P9Bv20Q5XF .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-7by766P9Bv20Q5XF .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-7by766P9Bv20Q5XF .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-7by766P9Bv20Q5XF .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-7by766P9Bv20Q5XF .actorPopupMenu{position:absolute;}#mermaid-svg-7by766P9Bv20Q5XF .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-7by766P9Bv20Q5XF .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-7by766P9Bv20Q5XF .actor-man circle,#mermaid-svg-7by766P9Bv20Q5XF line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-7by766P9Bv20Q5XF :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 发送消息复制消息复制消息同步完成同步完成acks=all 成功响应

如果业务更关注可靠性，优先选择 acks=all。它的意思不是"所有 Broker 都保存"，而是所有需要同步的副本确认后才返回成功。

不过，acks=all 还需要和 min.insync.replicas 一起看。

配置	位置	作用
replication.factor	Topic	一个分区有多少副本
acks=all	Producer	Leader 等待 ISR 副本确认后再返回
min.insync.replicas	Broker 或 Topic	写入成功至少需要多少个同步副本

常见可靠配置是：replication.factor=3、min.insync.replicas=2、生产者 acks=all。这样至少要有 2 个同步副本确认，生产者才认为写入成功。
#mermaid-svg-OvsTXInJArvQrDqg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-OvsTXInJArvQrDqg .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-OvsTXInJArvQrDqg .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-OvsTXInJArvQrDqg .error-icon{fill:#552222;}#mermaid-svg-OvsTXInJArvQrDqg .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-OvsTXInJArvQrDqg .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-OvsTXInJArvQrDqg .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-OvsTXInJArvQrDqg .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-OvsTXInJArvQrDqg .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-OvsTXInJArvQrDqg .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-OvsTXInJArvQrDqg .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-OvsTXInJArvQrDqg .marker{fill:#333333;stroke:#333333;}#mermaid-svg-OvsTXInJArvQrDqg .marker.cross{stroke:#333333;}#mermaid-svg-OvsTXInJArvQrDqg svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-OvsTXInJArvQrDqg p{margin:0;}#mermaid-svg-OvsTXInJArvQrDqg .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-OvsTXInJArvQrDqg .cluster-label text{fill:#333;}#mermaid-svg-OvsTXInJArvQrDqg .cluster-label span{color:#333;}#mermaid-svg-OvsTXInJArvQrDqg .cluster-label span p{background-color:transparent;}#mermaid-svg-OvsTXInJArvQrDqg .label text,#mermaid-svg-OvsTXInJArvQrDqg span{fill:#333;color:#333;}#mermaid-svg-OvsTXInJArvQrDqg .node rect,#mermaid-svg-OvsTXInJArvQrDqg .node circle,#mermaid-svg-OvsTXInJArvQrDqg .node ellipse,#mermaid-svg-OvsTXInJArvQrDqg .node polygon,#mermaid-svg-OvsTXInJArvQrDqg .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-OvsTXInJArvQrDqg .rough-node .label text,#mermaid-svg-OvsTXInJArvQrDqg .node .label text,#mermaid-svg-OvsTXInJArvQrDqg .image-shape .label,#mermaid-svg-OvsTXInJArvQrDqg .icon-shape .label{text-anchor:middle;}#mermaid-svg-OvsTXInJArvQrDqg .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-OvsTXInJArvQrDqg .rough-node .label,#mermaid-svg-OvsTXInJArvQrDqg .node .label,#mermaid-svg-OvsTXInJArvQrDqg .image-shape .label,#mermaid-svg-OvsTXInJArvQrDqg .icon-shape .label{text-align:center;}#mermaid-svg-OvsTXInJArvQrDqg .node.clickable{cursor:pointer;}#mermaid-svg-OvsTXInJArvQrDqg .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-OvsTXInJArvQrDqg .arrowheadPath{fill:#333333;}#mermaid-svg-OvsTXInJArvQrDqg .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-OvsTXInJArvQrDqg .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-OvsTXInJArvQrDqg .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-OvsTXInJArvQrDqg .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-OvsTXInJArvQrDqg .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-OvsTXInJArvQrDqg .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-OvsTXInJArvQrDqg .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-OvsTXInJArvQrDqg .cluster text{fill:#333;}#mermaid-svg-OvsTXInJArvQrDqg .cluster span{color:#333;}#mermaid-svg-OvsTXInJArvQrDqg div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-OvsTXInJArvQrDqg .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-OvsTXInJArvQrDqg rect.text{fill:none;stroke-width:0;}#mermaid-svg-OvsTXInJArvQrDqg .icon-shape,#mermaid-svg-OvsTXInJArvQrDqg .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-OvsTXInJArvQrDqg .icon-shape p,#mermaid-svg-OvsTXInJArvQrDqg .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-OvsTXInJArvQrDqg .icon-shape .label rect,#mermaid-svg-OvsTXInJArvQrDqg .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-OvsTXInJArvQrDqg .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-OvsTXInJArvQrDqg .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-OvsTXInJArvQrDqg :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 是
否
Producer

acks=all
Leader
Follower 1
Follower 2
ISR 数是否 >= min.insync.replicas
返回成功
抛出副本不足异常

如果 ISR 数量不足还继续写入，就可能在 Leader 故障时丢消息。所以强可靠场景宁愿让生产者收到异常并重试或降级，也不要静默写入一个没有足够副本保护的数据。

消费者 Offset 提交

Kafka 消费者通过 Offset 记录自己消费到哪里。默认情况下，消费者会自动定期提交 Offset，比如每隔 5 秒提交一次。

自动提交的问题是：Offset 可能已经提交了，但业务还没处理成功。
业务处理 Kafka Consumer 业务处理 Kafka Consumer #mermaid-svg-dCHo3G0Inb7fuGhV{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-dCHo3G0Inb7fuGhV .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-dCHo3G0Inb7fuGhV .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-dCHo3G0Inb7fuGhV .error-icon{fill:#552222;}#mermaid-svg-dCHo3G0Inb7fuGhV .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-dCHo3G0Inb7fuGhV .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-dCHo3G0Inb7fuGhV .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-dCHo3G0Inb7fuGhV .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-dCHo3G0Inb7fuGhV .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-dCHo3G0Inb7fuGhV .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-dCHo3G0Inb7fuGhV .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-dCHo3G0Inb7fuGhV .marker{fill:#333333;stroke:#333333;}#mermaid-svg-dCHo3G0Inb7fuGhV .marker.cross{stroke:#333333;}#mermaid-svg-dCHo3G0Inb7fuGhV svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-dCHo3G0Inb7fuGhV p{margin:0;}#mermaid-svg-dCHo3G0Inb7fuGhV .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-dCHo3G0Inb7fuGhV text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-dCHo3G0Inb7fuGhV .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-dCHo3G0Inb7fuGhV .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-dCHo3G0Inb7fuGhV .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-dCHo3G0Inb7fuGhV .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-dCHo3G0Inb7fuGhV #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-dCHo3G0Inb7fuGhV .sequenceNumber{fill:white;}#mermaid-svg-dCHo3G0Inb7fuGhV #sequencenumber{fill:#333;}#mermaid-svg-dCHo3G0Inb7fuGhV #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-dCHo3G0Inb7fuGhV .messageText{fill:#333;stroke:none;}#mermaid-svg-dCHo3G0Inb7fuGhV .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-dCHo3G0Inb7fuGhV .labelText,#mermaid-svg-dCHo3G0Inb7fuGhV .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-dCHo3G0Inb7fuGhV .loopText,#mermaid-svg-dCHo3G0Inb7fuGhV .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-dCHo3G0Inb7fuGhV .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-dCHo3G0Inb7fuGhV .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-dCHo3G0Inb7fuGhV .noteText,#mermaid-svg-dCHo3G0Inb7fuGhV .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-dCHo3G0Inb7fuGhV .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-dCHo3G0Inb7fuGhV .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-dCHo3G0Inb7fuGhV .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-dCHo3G0Inb7fuGhV .actorPopupMenu{position:absolute;}#mermaid-svg-dCHo3G0Inb7fuGhV .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-dCHo3G0Inb7fuGhV .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-dCHo3G0Inb7fuGhV .actor-man circle,#mermaid-svg-dCHo3G0Inb7fuGhV line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-dCHo3G0Inb7fuGhV :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Kafka 认为 100 已消费，下次从 101 开始，消息丢失 拉取 offset=100 的消息自动提交 offset=101处理业务业务失败或消费者宕机

更稳的方式是关闭自动提交，业务处理成功后再手动提交 Offset。

常见提交方式：

方式	特点
同步提交	确认提交成功再继续，可靠但会阻塞
异步提交	不阻塞，性能好，但失败处理弱
同步 + 异步组合	正常异步，关闭或再均衡前同步兜底

重复消费怎么处理

关闭自动提交后，消息丢失风险降低，但重复消费仍然可能出现。

比如消费者业务处理成功，还没来得及提交 Offset 就宕机。重启后会从旧 Offset 再消费一次。
#mermaid-svg-IbrS7vjHpouMWxPO{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-IbrS7vjHpouMWxPO .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-IbrS7vjHpouMWxPO .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-IbrS7vjHpouMWxPO .error-icon{fill:#552222;}#mermaid-svg-IbrS7vjHpouMWxPO .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-IbrS7vjHpouMWxPO .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-IbrS7vjHpouMWxPO .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-IbrS7vjHpouMWxPO .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-IbrS7vjHpouMWxPO .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-IbrS7vjHpouMWxPO .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-IbrS7vjHpouMWxPO .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-IbrS7vjHpouMWxPO .marker{fill:#333333;stroke:#333333;}#mermaid-svg-IbrS7vjHpouMWxPO .marker.cross{stroke:#333333;}#mermaid-svg-IbrS7vjHpouMWxPO svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-IbrS7vjHpouMWxPO p{margin:0;}#mermaid-svg-IbrS7vjHpouMWxPO .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-IbrS7vjHpouMWxPO .cluster-label text{fill:#333;}#mermaid-svg-IbrS7vjHpouMWxPO .cluster-label span{color:#333;}#mermaid-svg-IbrS7vjHpouMWxPO .cluster-label span p{background-color:transparent;}#mermaid-svg-IbrS7vjHpouMWxPO .label text,#mermaid-svg-IbrS7vjHpouMWxPO span{fill:#333;color:#333;}#mermaid-svg-IbrS7vjHpouMWxPO .node rect,#mermaid-svg-IbrS7vjHpouMWxPO .node circle,#mermaid-svg-IbrS7vjHpouMWxPO .node ellipse,#mermaid-svg-IbrS7vjHpouMWxPO .node polygon,#mermaid-svg-IbrS7vjHpouMWxPO .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-IbrS7vjHpouMWxPO .rough-node .label text,#mermaid-svg-IbrS7vjHpouMWxPO .node .label text,#mermaid-svg-IbrS7vjHpouMWxPO .image-shape .label,#mermaid-svg-IbrS7vjHpouMWxPO .icon-shape .label{text-anchor:middle;}#mermaid-svg-IbrS7vjHpouMWxPO .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-IbrS7vjHpouMWxPO .rough-node .label,#mermaid-svg-IbrS7vjHpouMWxPO .node .label,#mermaid-svg-IbrS7vjHpouMWxPO .image-shape .label,#mermaid-svg-IbrS7vjHpouMWxPO .icon-shape .label{text-align:center;}#mermaid-svg-IbrS7vjHpouMWxPO .node.clickable{cursor:pointer;}#mermaid-svg-IbrS7vjHpouMWxPO .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-IbrS7vjHpouMWxPO .arrowheadPath{fill:#333333;}#mermaid-svg-IbrS7vjHpouMWxPO .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-IbrS7vjHpouMWxPO .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-IbrS7vjHpouMWxPO .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-IbrS7vjHpouMWxPO .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-IbrS7vjHpouMWxPO .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-IbrS7vjHpouMWxPO .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-IbrS7vjHpouMWxPO .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-IbrS7vjHpouMWxPO .cluster text{fill:#333;}#mermaid-svg-IbrS7vjHpouMWxPO .cluster span{color:#333;}#mermaid-svg-IbrS7vjHpouMWxPO div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-IbrS7vjHpouMWxPO .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-IbrS7vjHpouMWxPO rect.text{fill:none;stroke-width:0;}#mermaid-svg-IbrS7vjHpouMWxPO .icon-shape,#mermaid-svg-IbrS7vjHpouMWxPO .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-IbrS7vjHpouMWxPO .icon-shape p,#mermaid-svg-IbrS7vjHpouMWxPO .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-IbrS7vjHpouMWxPO .icon-shape .label rect,#mermaid-svg-IbrS7vjHpouMWxPO .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-IbrS7vjHpouMWxPO .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-IbrS7vjHpouMWxPO .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-IbrS7vjHpouMWxPO :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 是
消费消息 offset=100
业务处理成功
提交 Offset 前宕机
重启后再次消费 offset=100
业务幂等判断

Kafka 的基本语义更接近"至少一次"。如果要业务正确，消费者必须幂等：

幂等方式	示例
业务唯一键	订单 ID、支付 ID、流水号
去重表	保存已处理 messageId
数据库唯一索引	重复插入直接失败
状态机判断	已支付、已发货、已完成则不重复处理

面试回答模板

可以这样答：

Kafka 保证消息不丢要从三个层面讲。生产者发送到 Broker 可能失败，所以我们会用异步发送回调判断结果，失败时记录日志或补偿重发，并配置 retries；为了避免生产者重试写出重复消息，还可以开启 enable.idempotence=true。Broker 存储阶段通过副本机制保证可靠性，生产端配置 acks=all，Topic 通常配置多副本，并配合 min.insync.replicas，比如 3 副本下至少 2 个 ISR 确认后才算写入成功。消费者侧默认自动提交 Offset 可能导致业务没处理完 Offset 已提交，所以一般关闭自动提交，业务处理成功后手动提交 Offset，可以用同步提交或同步加异步组合。最后要补充 Kafka 仍然可能重复消费，比如处理成功但 Offset 没提交就宕机，所以业务必须做幂等。

小结

Kafka 可靠性回答要抓住三句话：
#mermaid-svg-NPpKxulAzOculftv{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-NPpKxulAzOculftv .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-NPpKxulAzOculftv .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-NPpKxulAzOculftv .error-icon{fill:#552222;}#mermaid-svg-NPpKxulAzOculftv .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-NPpKxulAzOculftv .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-NPpKxulAzOculftv .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-NPpKxulAzOculftv .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-NPpKxulAzOculftv .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-NPpKxulAzOculftv .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-NPpKxulAzOculftv .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-NPpKxulAzOculftv .marker{fill:#333333;stroke:#333333;}#mermaid-svg-NPpKxulAzOculftv .marker.cross{stroke:#333333;}#mermaid-svg-NPpKxulAzOculftv svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-NPpKxulAzOculftv p{margin:0;}#mermaid-svg-NPpKxulAzOculftv .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-NPpKxulAzOculftv .cluster-label text{fill:#333;}#mermaid-svg-NPpKxulAzOculftv .cluster-label span{color:#333;}#mermaid-svg-NPpKxulAzOculftv .cluster-label span p{background-color:transparent;}#mermaid-svg-NPpKxulAzOculftv .label text,#mermaid-svg-NPpKxulAzOculftv span{fill:#333;color:#333;}#mermaid-svg-NPpKxulAzOculftv .node rect,#mermaid-svg-NPpKxulAzOculftv .node circle,#mermaid-svg-NPpKxulAzOculftv .node ellipse,#mermaid-svg-NPpKxulAzOculftv .node polygon,#mermaid-svg-NPpKxulAzOculftv .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-NPpKxulAzOculftv .rough-node .label text,#mermaid-svg-NPpKxulAzOculftv .node .label text,#mermaid-svg-NPpKxulAzOculftv .image-shape .label,#mermaid-svg-NPpKxulAzOculftv .icon-shape .label{text-anchor:middle;}#mermaid-svg-NPpKxulAzOculftv .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-NPpKxulAzOculftv .rough-node .label,#mermaid-svg-NPpKxulAzOculftv .node .label,#mermaid-svg-NPpKxulAzOculftv .image-shape .label,#mermaid-svg-NPpKxulAzOculftv .icon-shape .label{text-align:center;}#mermaid-svg-NPpKxulAzOculftv .node.clickable{cursor:pointer;}#mermaid-svg-NPpKxulAzOculftv .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-NPpKxulAzOculftv .arrowheadPath{fill:#333333;}#mermaid-svg-NPpKxulAzOculftv .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-NPpKxulAzOculftv .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-NPpKxulAzOculftv .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NPpKxulAzOculftv .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-NPpKxulAzOculftv .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NPpKxulAzOculftv .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-NPpKxulAzOculftv .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-NPpKxulAzOculftv .cluster text{fill:#333;}#mermaid-svg-NPpKxulAzOculftv .cluster span{color:#333;}#mermaid-svg-NPpKxulAzOculftv div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-NPpKxulAzOculftv .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-NPpKxulAzOculftv rect.text{fill:none;stroke-width:0;}#mermaid-svg-NPpKxulAzOculftv .icon-shape,#mermaid-svg-NPpKxulAzOculftv .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NPpKxulAzOculftv .icon-shape p,#mermaid-svg-NPpKxulAzOculftv .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-NPpKxulAzOculftv .icon-shape .label rect,#mermaid-svg-NPpKxulAzOculftv .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NPpKxulAzOculftv .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-NPpKxulAzOculftv .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-NPpKxulAzOculftv :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 生产者知道发没发成功
Broker 确认真正保存
acks=all + min.insync.replicas
消费者处理成功后再提交
重复消费靠业务幂等