mysql异常数据损坏处理,报错:Operating system error number 2 in a file operation

一、问题描述

某次一线反应,某主库表全部丢失,查看为空,登陆主机查看mysqld.log后报错:Operating system error number 2 in a file operation数据目录OS重装后修改过,但只是指向方式不同,目录还是同一目录,另怀疑被入侵遭删除,但也没发现相关异常;

关联资源 :superuser、mariadb-kb、mysql5.7

二、分析如下

1、手动执行mysql_safe启动,查看日志报错如下:

bash 复制代码
2024-07-08T06:33:43.608829Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2024-07-08T06:33:43.608961Z 0 [Note] --secure-file-priv is set to NULL. Operations related to importing and exporting data are disabled
2024-07-08T06:33:43.609008Z 0 [Note] ./bin/mysqld-debug (mysqld 5.7.44-debug-log) starting as process 2575072 ...
2024-07-08T06:33:43.616778Z 0 [Note] InnoDB: PUNCH HOLE support available
2024-07-08T06:33:43.616839Z 0 [Note] InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
2024-07-08T06:33:43.616850Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2024-07-08T06:33:43.616857Z 0 [Note] InnoDB: Uses event mutexes
2024-07-08T06:33:43.616865Z 0 [Note] InnoDB: GCC builtin __sync_synchronize() is used for memory barrier
2024-07-08T06:33:43.616873Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.13
2024-07-08T06:33:43.616880Z 0 [Note] InnoDB: Using Linux native AIO
2024-07-08T06:33:43.619317Z 0 [Note] InnoDB: Number of pools: 1
2024-07-08T06:33:43.619521Z 0 [Note] InnoDB: Using CPU crc32 instructions
2024-07-08T06:33:43.621668Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2024-07-08T06:33:43.683417Z 0 [Note] InnoDB: Completed initialization of buffer pool
2024-07-08T06:33:43.686321Z 0 [Note] InnoDB: page_cleaner coordinator priority: -20
2024-07-08T06:33:43.700000Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2024-07-08T06:33:43.766525Z 0 [ERROR] InnoDB: Operating system error number 2 in a file operation.
2024-07-08T06:33:43.766588Z 0 [ERROR] InnoDB: The error means the system cannot find the path specified.
2024-07-08T06:33:43.766599Z 0 [ERROR] InnoDB: If you are installing InnoDB, remember that you must create directories yourself, InnoDB does not creat
e them.
2024-07-08T06:33:43.766608Z 0 [ERROR] InnoDB: Cannot open datafile for read-only: './mysql/engine_cost.ibd' OS error: 71
2024-07-08T06:33:43.766639Z 0 [ERROR] InnoDB: Operating system error number 2 in a file operation.
2024-07-08T06:33:43.766648Z 0 [ERROR] InnoDB: The error means the system cannot find the path specified.
2024-07-08T06:33:43.766655Z 0 [ERROR] InnoDB: If you are installing InnoDB, remember that you must create directories yourself, InnoDB does not creat
e them.
2024-07-08T06:33:43.766665Z 0 [ERROR] InnoDB: Could not find a valid tablespace file for `mysql/engine_cost`. Please refer to http://dev.mysql.com/do
c/refman/5.7/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
2024-07-08T06:33:43.766675Z 0 [Warning] InnoDB: Ignoring tablespace `mysql/engine_cost` because it could not be opened.
2024-07-08T06:33:43.766785Z 0 [ERROR] InnoDB: Operating system error number 2 in a file operation.
2024-07-08T06:33:43.766803Z 0 [ERROR] InnoDB: The error means the system cannot find the path specified.
2024-07-08T06:33:43.766811Z 0 [ERROR] InnoDB: If you are installing InnoDB, remember that you must create directories yourself, InnoDB does not creat
e them.
2024-07-08T06:33:43.766819Z 0 [ERROR] InnoDB: Cannot open datafile for read-only: './mysql/gtid_executed.ibd' OS error: 71
2024-07-08T06:33:43.766827Z 0 [ERROR] InnoDB: Operating system error number 2 in a file operation.
............
mysqld: Table 'mysql.plugin' doesn't exist
2024-07-08T06:34:15.781770Z 0 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
2024-07-08T06:34:15.781880Z 0 [ERROR] Too many arguments (first extra is 'start').
2024-07-08T06:34:15.781889Z 0 [Note] Use --verbose --help to get a list of available options!
2024-07-08T06:34:15.781893Z 0 [ERROR] Aborting

如上所示,找不到innodb 数据字典文件*.ibd的,关于此类报错,官网解释如下:

With innodb_file_per_table enabled (the default), the following messages may appear at startup if a file-per-table tablespace file (.ibd file) is missing:

bash 复制代码
[ERROR] InnoDB: Operating system error number 2 in a file operation.
[ERROR] InnoDB: The error means the system cannot find the path specified.
[ERROR] InnoDB: Cannot open datafile for read-only: './test/t1.ibd' OS error: 71
[Warning] InnoDB: Ignoring tablespace `test/t1` because it could not be opened.

另外,ibdata1文件(660)作为记录InnoDB的共有表空间;数据库突然崩溃导致的该文件损坏也会导致如上报错;对应的独享表空间mysqld增加innodb_file_per_table;

2、强制innodb恢复,用以备份表

bash 复制代码
[mysqld]
innodb_force_recovery = 1  #值从0-6,默认0,安全值建议为1即可,不大于3,

3、mysql普通用户启动,参见Run MySQL as a Normal User

4、恢复

因现场环境被别人重装,损坏的数据删除,未能进一步排查恢复;但可参考如下:

1、备份原有的数据文件data为data_old

2、重启数据库,重新初始化重建,mysql表空间,

3、恢复源数据库数据目录到新的data下面,重启数据库验证

5、附录:

另外如果是多主环境,可配置如下,避免多主写入,带有自增列的记录导致自增列冲突的问题

bash 复制代码
auto_increment_offset = 2  #自增列偏移量,默认为1,即起始值
auto_increment_increment = 2  #自增量
#另Innodb参数
innodb_data_home_dir = /usr/local/mysql/data/
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = /usr/local/mysql/data/
innodb_log_arch_dir = /usr/local/mysql/data/
innodb_additional_mem_pool_size = 16M
innodb_buffer_pool_size = 512M
相关推荐
千里马学框架3 天前
一起学 Android 14:ShellTransition 屏幕旋转过程深度剖析
android·智能手机·性能优化·framework·性能·屏幕旋转·rotation
美狐美颜SDK开放平台3 天前
开发直播APP时如何接入视频美颜SDK?开发流程与注意事项
android·人工智能·计算机视觉·音视频·直播美颜sdk
AFinalStone3 天前
Android7 SystemUI源码解析(七)Keyguard锁屏模块深度解析
android·systemui
小白男神3 天前
MySQL进阶学习四(存储过程、游标、触发器)
mysql
这个DBA有点耶3 天前
MVCC深入:Read View、版本链与快照读——InnoDB并发控制的内核
数据库·mysql·架构
DBA_G3 天前
从地面到云霄:GBase数据库在民航三大场景的落地实践
数据库
致远ccc3 天前
Google Play 上架前如何测试 App?多国家 Android 环境测试
android·app测试·googleplay·多国家应用测试
自由能燃气设备3 天前
商用全预混低氮冷凝锅炉免费方案vs付费方案对比+选型避坑指南
大数据·数据库·人工智能
科创致远3 天前
科创致远 ESOP 系统核心效能与实战价值展示
大数据·数据库·人工智能·精益工程
ttyyttemo3 天前
Kotlin 协程中的 Job 结构化并发与取消
android