1 问题描述
1.1 基本信息
所属环境:CN-PT
问题时间:2023-11-21
所属程序: Flink Job(XXXPT_dimDeviceLogEventRi)
- 作业类型: Flink SQL Job
- 数据流 : 业务MySQL ==> Flink Job( Flink Cdc Connector(mysql) + Flink SQL) ==> Bigdata Kafka ==> Bigdata OLAP ==> 业务系统
- 作业备注: 关键技术---- 利用 Flink Cdc Connector(mysql) 监听业务库表进行实时数据同步业务维度数据至kafka
1.2 问题日志
Flink Job(XXXPT_dimDeviceLogEventRi) 运行过程中,出现错误:
log
...
2023-11-21 15:25:28,263 INFO org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer [] - Starting FlinkKafkaInternalProducer (1/1) to produce into default topic dim_device_log_event_ri
2023-11-21 15:25:28,266 INFO org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate [] - Converting recovered input channels (1 channels)
2023-11-21 15:25:28,325 INFO io.debezium.connector.common.BaseSourceTask [] - Stopping down connector
2023-11-21 15:25:28,325 INFO io.debezium.connector.mysql.MySqlConnectorTask [] - Stopping MySQL connector task
2023-11-21 15:25:28,326 ERROR com.alibaba.ververica.cdc.debezium.DebeziumSourceFunction [] - Reporting error:
org.apache.kafka.connect.errors.ConnectException: Error reading MySQL variables: Access denied for user 'root '@'123.45.67.89' (using password: YES)
at io.debezium.connector.mysql.MySqlJdbcContext.querySystemVariables(MySqlJdbcContext.java:342) ~[blob_p-b4fb1883b8b45408debf574a19853181362f64e4-1be92b6c42d0969e1e10ed136b1c5fc9:?]
at io.debezium.connector.mysql.MySqlJdbcContext.readMySqlSystemVariables(MySqlJdbcContext.java:321) ~[blob_p-b4fb1883b8b45408debf574a19853181362f64e4-1be92b6c42d0969e1e10ed136b1c5fc9:?]
at io.debezium.connector.mysql.MySqlTaskContext.<init>(MySqlTaskContext.java:79) ~[blob_p-b4fb1883b8b45408debf574a19853181362f64e4-1be92b6c42d0969e1e10ed136b1c5fc9:?]
at io.debezium.connector.mysql.MySqlTaskContext.<init>(MySqlTaskContext.java:52) ~[blob_p-b4fb1883b8b45408debf574a19853181362f64e4-1be92b6c42d0969e1e10ed136b1c5fc9:?]
at io.debezium.connector.mysql.MySqlConnectorTask.createAndStartTaskContext(MySqlConnectorTask.java:350) ~[blob_p-b4fb1883b8b45408debf574a19853181362f64e4-1be92b6c42d0969e1e10ed136b1c5fc9:?]
at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:143) ~[blob_p-b4fb1883b8b45408debf574a19853181362f64e4-1be92b6c42d0969e1e10ed136b1c5fc9:?]
at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:106) ~[blob_p-b4fb1883b8b45408debf574a19853181362f64e4-1be92b6c42d0969e1e10ed136b1c5fc9:?]
at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:758) [blob_p-b4fb1883b8b45408debf574a19853181362f64e4-1be92b6c42d0969e1e10ed136b1c5fc9:?]
at io.debezium.embedded.ConvertingEngineBuilder$2.run(ConvertingEngineBuilder.java:171) [blob_p-b4fb1883b8b45408debf574a19853181362f64e4-1be92b6c42d0969e1e10ed136b1c5fc9:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_362]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_362]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_362]
Caused by: java.sql.SQLException: Access denied for user 'root '@'123.45.67.89' (using password: YES)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129) ~[blob_p-
...
2 问题分析
2.1 Flink SQL Job
这是1个 Flink SQL Job:
其仅使用了8(5+3)个外部变量(放在NACOS配置文件中的):
经检查,无误。
那么,咱们来看看启动参数是否有错误:
2.2 作业启动参数
启动参数(nacos地址等)均无误。
再看看NACOS的配置:
果然有问题!
2.3 再细究日志
再仔细看看日志:赫然发现了这个空格!
2.X 问题结论
- 本次问题出在 MYSQL数据库用户名中含【空格】;
- 针对Flink CDC错误"ConnectException: Error reading MySQL variables: Access denied for user 'root '@'123.45.67.89' (using password: YES)"共有如下情况,做以小结:
shell
| 结论1 |
数据库链接(HOST/PORT/USER/PASSWORD)【配置错误】,包括可能意想不到的【空格】;
| 结论2 |
当出现 Access denied for user 'xx'@'xxxx' (using password: YES),只且仅有2类原因:
1)数据库【用户不存在】、或【密码错误】;
注:用户不存在,自然也包括了上面说的配置错误(比如空格)的情况;
2)【无操作权限】;
注1:通过 show grants for {user}@'{host}' 查看用户的权限授予情况;
注2:在使用 grant 授权用户后,应使用 flush privileges 刷新权限;否则,有最新权限尚未生效的小概率事件发生。
3 解决方法
- 本问题:去除 root 用户后面的空格符!
- 本错误的全部情况,参见:2.X 章节 问题结论
X 推荐与参考文献
[1] Custom Payload in Debezium - copyprogramming.com 【推荐】
[2] Multiple debezium connector for same MySql database not working - stackoom
[3] 大数据-业务数据采集-FlinkCDC DebeziumSourceFunction via the 'serverTimezone' configuration property - 51CTO
[4] 报错:Access denied for user 'xxx'@'localhost' - segementfault
[5] [数据库] MYSQL之授予/查验binlog权限 - 博客园/千千寰宇 【推荐】
[6] 一文看尽MySQL用户权限管理,真香 - Zhihu 【推荐】