kafka-消费者组偏移量重置

文章目录

1、消费者组偏移量重置

1.1、列出所有的消费者组

clike 复制代码
[root@localhost ~]# kafka-consumer-groups.sh --bootstrap-server 192.168.74.148:9092 --list
my_group2
my_group1

1.2、查看 my_group1 组的详细信息

clike 复制代码
[root@localhost ~]# kafka-consumer-groups.sh --bootstrap-server 192.168.74.148:9092 --group my_group1 --describe

GROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                               HOST            CLIENT-ID
my_group1       my_topic1       1          1               1               0               consumer-my_group1-1-c2ff5a19-af5c-47fc-9ad9-d6028844f86c /192.168.74.148 consumer-my_group1-1
my_group1       my_topic1       2          3               3               0               consumer-my_group1-1-c6a31cdb-c924-49bb-99da-cf45ffdbefb6 /192.168.74.148 consumer-my_group1-1
my_group1       my_topic1       0          2               2               0               consumer-my_group1-1-19852a4a-9a4e-4b41-b605-0a78530d0cd8 /192.168.74.148 consumer-my_group1-1

1.3、获取 kafka-consumer-groups.sh 的帮助信息

clike 复制代码
[root@localhost ~]# kafka-consumer-groups.sh --help
Missing required argument "[bootstrap-server]"
Option                                  Description                            
------                                  -----------                            
--all-groups                            Apply to all consumer groups.          
--all-topics                            Consider all topics assigned to a      
                                          group in the `reset-offsets` process.
--bootstrap-server <String: server to   REQUIRED: The server(s) to connect to. 
  connect to>                                                                  
--by-duration <String: duration>        Reset offsets to offset by duration    
                                          from current timestamp. Format:      
                                          'PnDTnHnMnS'                         
--command-config <String: command       Property file containing configs to be 
  config property file>                   passed to Admin Client and Consumer. 
--delete                                Pass in groups to delete topic         
                                          partition offsets and ownership      
                                          information over the entire consumer 
                                          group. For instance --group g1 --    
                                          group g2                             
--delete-offsets                        Delete offsets of consumer group.      
                                          Supports one consumer group at the   
                                          time, and multiple topics.           
--describe                              Describe consumer group and list       
                                          offset lag (number of messages not   
                                          yet processed) related to given      
                                          group.                               
--dry-run                               Only show results without executing    
                                          changes on Consumer Groups.          
                                          Supported operations: reset-offsets. 
--execute                               Execute operation. Supported           
                                          operations: reset-offsets.           
--export                                Export operation execution to a CSV    
                                          file. Supported operations: reset-   
                                          offsets.                             
--from-file <String: path to CSV file>  Reset offsets to values defined in CSV 
                                          file.                                
--group <String: consumer group>        The consumer group we wish to act on.  
--help                                  Print usage information.               
--list                                  List all consumer groups.              
--members                               Describe members of the group. This    
                                          option may be used with '--describe' 
                                          and '--bootstrap-server' options     
                                          only.                                
                                        Example: --bootstrap-server localhost: 
                                          9092 --describe --group group1 --    
                                          members                              
--offsets                               Describe the group and list all topic  
                                          partitions in the group along with   
                                          their offset lag. This is the        
                                          default sub-action of and may be     
                                          used with '--describe' and '--       
                                          bootstrap-server' options only.      
                                        Example: --bootstrap-server localhost: 
                                          9092 --describe --group group1 --    
                                          offsets                              
--reset-offsets                         Reset offsets of consumer group.       
                                          Supports one consumer group at the   
                                          time, and instances should be        
                                          inactive                             
                                        Has 2 execution options: --dry-run     
                                          (the default) to plan which offsets  
                                          to reset, and --execute to update    
                                          the offsets. Additionally, the --    
                                          export option is used to export the  
                                          results to a CSV format.             
                                        You must choose one of the following   
                                          reset specifications: --to-datetime, 
                                          --by-period, --to-earliest, --to-    
                                          latest, --shift-by, --from-file, --  
                                          to-current.                          
                                        To define the scope use --all-topics   
                                          or --topic. One scope must be        
                                          specified unless you use '--from-    
                                          file'.                               
--shift-by <Long: number-of-offsets>    Reset offsets shifting current offset  
                                          by 'n', where 'n' can be positive or 
                                          negative.                            
--state [String]                        When specified with '--describe',      
                                          includes the state of the group.     
                                        Example: --bootstrap-server localhost: 
                                          9092 --describe --group group1 --    
                                          state                                
                                        When specified with '--list', it       
                                          displays the state of all groups. It 
                                          can also be used to list groups with 
                                          specific states.                     
                                        Example: --bootstrap-server localhost: 
                                          9092 --list --state stable,empty     
                                        This option may be used with '--       
                                          describe', '--list' and '--bootstrap-
                                          server' options only.                
--timeout <Long: timeout (ms)>          The timeout that can be set for some   
                                          use cases. For example, it can be    
                                          used when describing the group to    
                                          specify the maximum amount of time   
                                          in milliseconds to wait before the   
                                          group stabilizes (when the group is  
                                          just created, or is going through    
                                          some changes). (default: 5000)       
--to-current                            Reset offsets to current offset.       
--to-datetime <String: datetime>        Reset offsets to offset from datetime. 
                                          Format: 'YYYY-MM-DDTHH:mm:SS.sss'    
--to-earliest                           Reset offsets to earliest offset.      
--to-latest                             Reset offsets to latest offset.        
--to-offset <Long: offset>              Reset offsets to a specific offset.    
--topic <String: topic>                 The topic whose consumer group         
                                          information should be deleted or     
                                          topic whose should be included in    
                                          the reset offset process. In `reset- 
                                          offsets` case, partitions can be     
                                          specified using this format: `topic1:
                                          0,1,2`, where 0,1,2 are the          
                                          partition to be included in the      
                                          process. Reset-offsets also supports 
                                          multiple topic inputs.               
--verbose                               Provide additional information, if     
                                          any, when describing the group. This 
                                          option may be used with '--          
                                          offsets'/'--members'/'--state' and   
                                          '--bootstrap-server' options only.   
                                        Example: --bootstrap-server localhost: 
                                          9092 --describe --group group1 --    
                                          members --verbose                    
--version                               Display Kafka version.  

1.4、 偏移量重置

clike 复制代码
[root@localhost ~]# kafka-consumer-groups.sh --bootstrap-server 192.168.74.148:9092 --topic my_topic1 --group my_group1 --reset-offsets --to-offset 1 --execute

GROUP                          TOPIC                          PARTITION  NEW-OFFSET     
my_group1                      my_topic1                      0          1              
my_group1                      my_topic1                      1          1              
my_group1                      my_topic1                      2          1   

1.5、再次查看 my_group1 组的详细信息

clike 复制代码
[root@localhost ~]# kafka-consumer-groups.sh --bootstrap-server 192.168.74.148:9092 --group my_group1 --describe

GROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                               HOST            CLIENT-ID
my_group1       my_topic1       0          2               2               0               consumer-my_group1-1-1306dc19-4077-4622-ae3b-ed6d077c4206 /192.168.74.148 consumer-my_group1-1
my_group1       my_topic1       2          3               3               0               consumer-my_group1-1-c4e3e804-e469-4f1a-ac0c-5b57907356f9 /192.168.74.148 consumer-my_group1-1
my_group1       my_topic1       1          1               1               0               consumer-my_group1-1-ad5fc413-e598-4e4b-b44c-0ae976a615c8 /192.168.74.148 consumer-my_group1-1
相关推荐
明达智控技术10 小时前
MR30分布式I/O在面机装备中的应用
分布式·物联网·自动化
JAVA学习通12 小时前
【RabbitMQ】---RabbitMQ 工作流程和 web 界面介绍
分布式·rabbitmq
cg.family13 小时前
Doris 消费kafka消息
kafka·doris
安卓开发者13 小时前
鸿蒙NEXT应用数据持久化全面解析:从用户首选项到分布式数据库
数据库·分布式·harmonyos
趴着喝可乐14 小时前
openEuler2403安装部署Kafka
kafka·openeuler
JAVA学习通16 小时前
【RabbitMQ】如何在 Ubuntu 安装 RabbitMQ
分布式·rabbitmq
Lansonli16 小时前
大数据Spark(六十三):RDD-Resilient Distributed Dataset
大数据·分布式·spark
BYSJMG17 小时前
计算机毕业设计选题:基于Spark+Hadoop的健康饮食营养数据分析系统【源码+文档+调试】
大数据·vue.js·hadoop·分布式·spark·django·课程设计
JAVA学习通17 小时前
【RabbitMQ】----RabbitMQ 的7种工作模式
分布式·rabbitmq
励志成为糕手18 小时前
Hadoop进程:深入理解分布式计算引擎的核心机制
大数据·hadoop·分布式·mapreduce·yarn