mysql> change master to master_host='172.25.254.10',master_user='repl',master_password='123',master_log_file='mysql-bin.000001',master_log_pos=1151;------设置master
Sun Aug 25 22:41:22 2024 - info Checking replication health on 172.25.254.20..
Sun Aug 25 22:41:22 2024 - info ok.
Sun Aug 25 22:41:22 2024 - info Checking replication health on 172.25.254.30..
Sun Aug 25 22:41:22 2024 - info ok.
Sun Aug 25 22:41:22 2024 - warning master_ip_failover_script is not defined.
Sun Aug 25 22:41:22 2024 - warning shutdown_script is not defined.
Sun Aug 25 22:41:22 2024 - info Got exit code 0 (Not master dead).
MySQL Replication Health is OK.
6.6MHA的切换方式
6.6.1无故障手动切换
root@mysql-mha conf# masterha_master_switch \
--conf=/etc/masterha/app1.cnf \
--master_state=alive \
--new_master_host=172.25.254.20 \
--new_master_port=3306 \
--orig_master_is_new_slave \
--running_updates_limit=10000
Mon Aug 26 03:05:30 2024 - info MHA::MasterRotate version 0.58.
Mon Aug 26 03:05:30 2024 - info Starting online master switch..
Mon Aug 26 03:05:30 2024 - info
Mon Aug 26 03:05:30 2024 - info * Phase 1: Configuration Check Phase..
Mon Aug 26 03:05:30 2024 - info
Mon Aug 26 03:05:30 2024 - warning Global configuration file /etc/masterha_default.cnf not found. Skipping.
Mon Aug 26 03:05:30 2024 - info Reading application default configuration from /etc/masterha/app1.cnf..
Mon Aug 26 03:05:30 2024 - info Reading server configuration from /etc/masterha/app1.cnf..
Mon Aug 26 03:05:31 2024 - info GTID failover mode = 1
Mon Aug 26 03:05:31 2024 - info Current Alive Master: 172.25.254.10(172.25.254.10:3306)
Mon Aug 26 03:05:31 2024 - info Alive Slaves:
Mon Aug 26 03:05:31 2024 - info 172.25.254.20(172.25.254.20:3306) Version=5.7.44-log (oldest major version between slaves) log-bin:enabled
Mon Aug 26 03:05:31 2024 - info GTID ON
Mon Aug 26 03:05:31 2024 - info Replicating from 172.25.254.10(172.25.254.10:3306)
Mon Aug 26 03:05:31 2024 - info Primary candidate for the new Master (candidate_master is set)
Mon Aug 26 03:05:31 2024 - info 172.25.254.30(172.25.254.30:3306) Version=5.7.44-log (oldest major version between slaves) log-bin:enabled
Mon Aug 26 03:05:31 2024 - info GTID ON
Mon Aug 26 03:05:31 2024 - info Replicating from 172.25.254.10(172.25.254.10:3306)
Mon Aug 26 03:05:31 2024 - info Not candidate for the new Master (no_master is set)
It is better to execute FLUSH NO_WRITE_TO_BINLOG TABLES on the master before switching. Is it ok to execute on 172.25.254.10(172.25.254.10:3306)? (YES/no): yes
Mon Aug 26 03:05:34 2024 - info Executing FLUSH NO_WRITE_TO_BINLOG TABLES. This may take long time..
Mon Aug 26 03:05:34 2024 - info ok.
Mon Aug 26 03:05:34 2024 - info Checking MHA is not monitoring or doing failover..
Mon Aug 26 03:05:34 2024 - info Checking replication health on 172.25.254.20..
Mon Aug 26 03:05:34 2024 - info ok.
Mon Aug 26 03:05:34 2024 - info Checking replication health on 172.25.254.30..
Mon Aug 26 03:05:34 2024 - info ok.
Mon Aug 26 03:05:34 2024 - info 172.25.254.20 can be new master.
Starting master switch from 172.25.254.10(172.25.254.10:3306) to 172.25.254.20(172.25.254.20:3306)? (yes/NO): yes
Mon Aug 26 03:05:35 2024 - info Checking whether 172.25.254.20(172.25.254.20:3306) is ok for the new master..
Mon Aug 26 03:05:35 2024 - info ok.
Mon Aug 26 03:05:35 2024 - info ** Phase 1: Configuration Check Phase completed.
Mon Aug 26 03:05:35 2024 - info
Mon Aug 26 03:05:35 2024 - info * Phase 2: Rejecting updates Phase..
Mon Aug 26 03:05:35 2024 - info
master_ip_online_change_script is not defined. If you do not disable writes on the current master manually, applications keep writing on the current master. Is it ok to proceed? (yes/NO): yes
Mon Aug 26 03:05:36 2024 - info Locking all tables on the orig master to reject updates from everybody (including root):
Mon Aug 26 03:05:36 2024 - info Executing FLUSH TABLES WITH READ LOCK..
Mon Aug 26 03:05:36 2024 - info ok.
Mon Aug 26 03:05:36 2024 - info Orig master binlog:pos is mysql-bin.000004:194.
Mon Aug 26 03:05:36 2024 - info Waiting to execute all relay logs on 172.25.254.20(172.25.254.20:3306)..
Mon Aug 26 03:05:36 2024 - info master_pos_wait(mysql-bin.000004:194) completed on 172.25.254.20(172.25.254.20:3306). Executed 0 events.
Mon Aug 26 03:05:36 2024 - info done.
Mon Aug 26 03:05:36 2024 - info Getting new master's binlog name and position..
Mon Aug 26 03:05:36 2024 - info mysql-bin.000004:194
Mon Aug 26 03:05:36 2024 - info All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='172.25.254.20', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='jcl', MASTER_PASSWORD='xxx';
Mon Aug 26 03:05:36 2024 - info
Mon Aug 26 03:05:36 2024 - info * Switching slaves in parallel..
Mon Aug 26 03:05:36 2024 - info
Mon Aug 26 03:05:36 2024 - info -- Slave switch on host 172.25.254.30(172.25.254.30:3306) started, pid: 72716
Mon Aug 26 03:05:36 2024 - info
Mon Aug 26 03:05:38 2024 - info Log messages from 172.25.254.30 ...
Mon Aug 26 03:05:38 2024 - info
Mon Aug 26 03:05:36 2024 - info Waiting to execute all relay logs on 172.25.254.30(172.25.254.30:3306)..
Mon Aug 26 03:05:36 2024 - info master_pos_wait(mysql-bin.000004:194) completed on 172.25.254.30(172.25.254.30:3306). Executed 0 events.
Mon Aug 26 03:05:36 2024 - info done.
Mon Aug 26 03:05:36 2024 - info Resetting slave 172.25.254.30(172.25.254.30:3306) and starting replication from the new master 172.25.254.20(172.25.254.20:3306)..
Mon Aug 26 03:05:36 2024 - info Executed CHANGE MASTER.
Mon Aug 26 03:05:37 2024 - info Slave started.
Mon Aug 26 03:05:38 2024 - info End of log messages from 172.25.254.30 ...
Mon Aug 26 03:05:38 2024 - info
Mon Aug 26 03:05:38 2024 - info -- Slave switch on host 172.25.254.30(172.25.254.30:3306) succeeded.
Mon Aug 26 03:05:38 2024 - info Unlocking all tables on the orig master:
Mon Aug 26 03:05:38 2024 - info Executing UNLOCK TABLES..
Mon Aug 26 03:05:38 2024 - info ok.
Mon Aug 26 03:05:38 2024 - info Starting orig master as a new slave..
Mon Aug 26 03:05:38 2024 - info Resetting slave 172.25.254.10(172.25.254.10:3306) and starting replication from the new master 172.25.254.20(172.25.254.20:3306)..
Mon Aug 26 03:05:38 2024 - info Executed CHANGE MASTER.
Mon Aug 26 03:05:39 2024 - info Slave started.
Mon Aug 26 03:05:39 2024 - info All new slave servers switched successfully.
Mon Aug 26 03:05:39 2024 - info
Mon Aug 26 03:05:39 2024 - info * Phase 5: New master cleanup phase..
Mon Aug 26 03:05:39 2024 - info
Mon Aug 26 03:05:39 2024 - info 172.25.254.20: Resetting slave info succeeded.
Mon Aug 26 03:05:39 2024 - info Switching master to 172.25.254.20(172.25.254.20:3306) completed successfully.
可以看到master切换到20上:
6.6.2故障手动切换
root@mysql-node2 \~# /etc/init.d/mysqld stop
Shutting down MySQL............ SUCCESS!
root@mysql-mha conf# masterha_master_switch \
--master_state=dead \
--conf=/etc/masterha/app1.cnf \
--dead_master_host=172.25.254.20 \
--dead_master_port=3306 \
--new_master_host=172.25.254.10 \
--new_master_port=3306 \
--ignore_last_failover
--dead_master_ip=<dead_master_ip> is not set. Using 172.25.254.20.
--dead_master_port=<dead_master_port> is not set. Using 3306.
Mon Aug 26 03:10:10 2024 - warning Global configuration file /etc/masterha_default.cnf not found. Skipping.
Mon Aug 26 03:10:10 2024 - info Reading application default configuration from /etc/masterha/app1.cnf..
Mon Aug 26 03:10:10 2024 - info Reading server configuration from /etc/masterha/app1.cnf..
Mon Aug 26 03:10:10 2024 - info MHA::MasterFailover version 0.58.
Mon Aug 26 03:10:10 2024 - info Starting master failover.
Mon Aug 26 03:10:10 2024 - info
Mon Aug 26 03:10:10 2024 - info * Phase 1: Configuration Check Phase..
Mon Aug 26 03:10:10 2024 - info
Mon Aug 26 03:10:11 2024 - info GTID failover mode = 1
Mon Aug 26 03:10:11 2024 - info Dead Servers:
Mon Aug 26 03:10:11 2024 - info 172.25.254.20(172.25.254.20:3306)
Mon Aug 26 03:10:11 2024 - info Checking master reachability via MySQL(double check)...
Mon Aug 26 03:10:11 2024 - info ok.
Mon Aug 26 03:10:11 2024 - info Alive Servers:
Mon Aug 26 03:10:11 2024 - info 172.25.254.10(172.25.254.10:3306)
Mon Aug 26 03:10:11 2024 - info 172.25.254.30(172.25.254.30:3306)
Mon Aug 26 03:10:11 2024 - info Alive Slaves:
Mon Aug 26 03:10:11 2024 - info 172.25.254.10(172.25.254.10:3306) Version=5.7.44-log (oldest major version between slaves) log-bin:enabled
Mon Aug 26 03:10:11 2024 - info GTID ON
Mon Aug 26 03:10:11 2024 - info Replicating from 172.25.254.20(172.25.254.20:3306)
Mon Aug 26 03:10:11 2024 - info Primary candidate for the new Master (candidate_master is set)
Mon Aug 26 03:10:11 2024 - info 172.25.254.30(172.25.254.30:3306) Version=5.7.44-log (oldest major version between slaves) log-bin:enabled
Mon Aug 26 03:10:11 2024 - info GTID ON
Mon Aug 26 03:10:11 2024 - info Replicating from 172.25.254.20(172.25.254.20:3306)
Mon Aug 26 03:10:11 2024 - info Not candidate for the new Master (no_master is set)
Master 172.25.254.20(172.25.254.20:3306) is dead. Proceed? (yes/NO): yes
Mon Aug 26 03:10:14 2024 - info Starting GTID based failover.
Mon Aug 26 03:10:14 2024 - info
Mon Aug 26 03:10:14 2024 - info ** Phase 1: Configuration Check Phase completed.
Mon Aug 26 03:10:14 2024 - info
Mon Aug 26 03:10:14 2024 - info * Phase 2: Dead Master Shutdown Phase..
Mon Aug 26 03:10:14 2024 - info
Mon Aug 26 03:10:14 2024 - info HealthCheck: SSH to 172.25.254.20 is reachable.
Mon Aug 26 03:10:14 2024 - info Forcing shutdown so that applications never connect to the current master..
Mon Aug 26 03:10:14 2024 - warning master_ip_failover_script is not set. Skipping invalidating dead master IP address.
Mon Aug 26 03:10:14 2024 - warning shutdown_script is not set. Skipping explicit shutting down of the dead master.
Mon Aug 26 03:10:14 2024 - info * Phase 2: Dead Master Shutdown Phase completed.
Mon Aug 26 03:10:14 2024 - info
Mon Aug 26 03:10:14 2024 - info * Phase 3: Master Recovery Phase..
Mon Aug 26 03:10:14 2024 - info
Mon Aug 26 03:10:14 2024 - info * Phase 3.1: Getting Latest Slaves Phase..
Mon Aug 26 03:10:14 2024 - info
Mon Aug 26 03:10:14 2024 - info The latest binary log file/position on all slaves is mysql-bin.000004:194
Mon Aug 26 03:10:14 2024 - info Retrieved Gtid Set: 347da1c1-6303-11ef-93da-000c2972a3ac:1
Mon Aug 26 03:10:14 2024 - info Latest slaves (Slaves that received relay log files to the latest):
Mon Aug 26 03:10:14 2024 - info 172.25.254.10(172.25.254.10:3306) Version=5.7.44-log (oldest major version between slaves) log-bin:enabled
Mon Aug 26 03:10:14 2024 - info GTID ON
Mon Aug 26 03:10:14 2024 - info Replicating from 172.25.254.20(172.25.254.20:3306)
Mon Aug 26 03:10:14 2024 - info Primary candidate for the new Master (candidate_master is set)
Mon Aug 26 03:10:14 2024 - info 172.25.254.30(172.25.254.30:3306) Version=5.7.44-log (oldest major version between slaves) log-bin:enabled
Mon Aug 26 03:10:14 2024 - info GTID ON
Mon Aug 26 03:10:14 2024 - info Replicating from 172.25.254.20(172.25.254.20:3306)
Mon Aug 26 03:10:14 2024 - info Not candidate for the new Master (no_master is set)
Mon Aug 26 03:10:14 2024 - info The oldest binary log file/position on all slaves is mysql-bin.000004:194
Mon Aug 26 03:10:14 2024 - info Retrieved Gtid Set: 347da1c1-6303-11ef-93da-000c2972a3ac:1
Mon Aug 26 03:10:14 2024 - info Oldest slaves:
Mon Aug 26 03:10:14 2024 - info 172.25.254.10(172.25.254.10:3306) Version=5.7.44-log (oldest major version between slaves) log-bin:enabled
Mon Aug 26 03:10:14 2024 - info GTID ON
Mon Aug 26 03:10:14 2024 - info Replicating from 172.25.254.20(172.25.254.20:3306)
Mon Aug 26 03:10:14 2024 - info Primary candidate for the new Master (candidate_master is set)
Mon Aug 26 03:10:14 2024 - info 172.25.254.30(172.25.254.30:3306) Version=5.7.44-log (oldest major version between slaves) log-bin:enabled
Mon Aug 26 03:10:14 2024 - info GTID ON
Mon Aug 26 03:10:14 2024 - info Replicating from 172.25.254.20(172.25.254.20:3306)
Mon Aug 26 03:10:14 2024 - info Not candidate for the new Master (no_master is set)
Mon Aug 26 03:10:14 2024 - info
Mon Aug 26 03:10:14 2024 - info * Phase 3.3: Determining New Master Phase..
Mon Aug 26 03:10:14 2024 - info
Mon Aug 26 03:10:14 2024 - info 172.25.254.10 can be new master.
Mon Aug 26 03:10:14 2024 - info New master is 172.25.254.10(172.25.254.10:3306)
Mon Aug 26 03:10:14 2024 - info Starting master failover..
Starting master switch from 172.25.254.20(172.25.254.20:3306) to 172.25.254.10(172.25.254.10:3306)? (yes/NO): yes
Mon Aug 26 03:10:16 2024 - info New master decided manually is 172.25.254.10(172.25.254.10:3306)
Mon Aug 26 03:10:16 2024 - info
Mon Aug 26 03:10:16 2024 - info * Phase 3.3: New Master Recovery Phase..
Mon Aug 26 03:10:16 2024 - info
Mon Aug 26 03:10:16 2024 - info Waiting all logs to be applied..
Mon Aug 26 03:10:16 2024 - info done.
Mon Aug 26 03:10:16 2024 - info Getting new master's binlog name and position..
Mon Aug 26 03:10:16 2024 - info mysql-bin.000004:438
Mon Aug 26 03:10:16 2024 - info All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='172.25.254.10', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='jcl', MASTER_PASSWORD='xxx';
Mon Aug 26 03:10:16 2024 - info Master Recovery succeeded. File:Pos:Exec_Gtid_Set: mysql-bin.000004, 438, 347da1c1-6303-11ef-93da-000c2972a3ac:1,
b1367cca-6302-11ef-a3aa-000c291d64ff:1-6
Mon Aug 26 03:10:16 2024 - warning master_ip_failover_script is not set. Skipping taking over new master IP address.
Mon Aug 26 03:10:16 2024 - info Setting read_only=0 on 172.25.254.10(172.25.254.10:3306)..
Mon Aug 26 03:10:16 2024 - info ok.
Mon Aug 26 03:10:16 2024 - info ** Finished master recovery successfully.
Mon Aug 26 03:10:16 2024 - info * Phase 3: Master Recovery Phase completed.
Mon Aug 26 03:10:16 2024 - info
Mon Aug 26 03:10:16 2024 - info * Phase 4: Slaves Recovery Phase..
Mon Aug 26 03:10:16 2024 - info
Mon Aug 26 03:10:16 2024 - info
Mon Aug 26 03:10:16 2024 - info * Phase 4.1: Starting Slaves in parallel..
Mon Aug 26 03:10:16 2024 - info
Mon Aug 26 03:10:16 2024 - info -- Slave recovery on host 172.25.254.30(172.25.254.30:3306) started, pid: 73590. Check tmp log /var/log/masterha/app1/172.25.254.30_3306_20240826031010.log if it takes time..
Mon Aug 26 03:10:18 2024 - info
Mon Aug 26 03:10:18 2024 - info Log messages from 172.25.254.30 ...
Mon Aug 26 03:10:18 2024 - info
Mon Aug 26 03:10:16 2024 - info Resetting slave 172.25.254.30(172.25.254.30:3306) and starting replication from the new master 172.25.254.10(172.25.254.10:3306)..
Mon Aug 26 03:10:16 2024 - info Executed CHANGE MASTER.
Mon Aug 26 03:10:17 2024 - info Slave started.
Mon Aug 26 03:10:17 2024 - info gtid_wait(347da1c1-6303-11ef-93da-000c2972a3ac:1,
b1367cca-6302-11ef-a3aa-000c291d64ff:1-6) completed on 172.25.254.30(172.25.254.30:3306). Executed 0 events.
Mon Aug 26 03:10:18 2024 - info End of log messages from 172.25.254.30.
Mon Aug 26 03:10:18 2024 - info -- Slave on host 172.25.254.30(172.25.254.30:3306) started.
Mon Aug 26 03:10:18 2024 - info All new slave servers recovered successfully.
Mon Aug 26 03:10:18 2024 - info
Mon Aug 26 03:10:18 2024 - info * Phase 5: New master cleanup phase..
Mon Aug 26 03:10:18 2024 - info
Mon Aug 26 03:10:18 2024 - info Resetting slave info on the new master..
Mon Aug 26 03:10:18 2024 - info 172.25.254.10: Resetting slave info succeeded.
Mon Aug 26 03:10:18 2024 - info Master failover to 172.25.254.10(172.25.254.10:3306) completed successfully.
Mon Aug 26 03:10:18 2024 - info
----- Failover Report -----
app1: MySQL Master failover 172.25.254.20(172.25.254.20:3306) to 172.25.254.10(172.25.254.10:3306) succeeded
Master 172.25.254.20(172.25.254.20:3306) is down!
Check MHA Manager logs at mysql-mha for details.
Started manual(interactive) failover.
Selected 172.25.254.10(172.25.254.10:3306) as a new master.
172.25.254.10(172.25.254.10:3306): OK: Applying all logs succeeded.
172.25.254.30(172.25.254.30:3306): OK: Slave started, replicating from 172.25.254.10(172.25.254.10:3306)
172.25.254.10(172.25.254.10:3306): Resetting slave info succeeded.
Master failover to 172.25.254.10(172.25.254.10:3306) completed successfully.
切换到10上
恢复故障20自动成为salve
root@mysql-node2 \~# /etc/init.d/mysqld start
Starting MySQL. SUCCESS!
root@mysql-node2 \~# mysql -p123
mysql> stop slave ;
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> CHANGE MASTER TO MASTER_HOST='172.25.254.10', MASTER_USER='jcl',MASTER_PASSWORD='123', MASTER_AUTO_POSITION=1;
Query OK, 0 rows affected, 2 warnings (0.01 sec)
mysql> start slave;
Query OK, 0 rows affected (0.01 sec)
mysql> show slave status\G;
6.6.3自动检测切换
将master停掉模拟故障
root@mysql-node1 \~# /etc/init.d/mysqld stop
Shutting down MySQL........... SUCCESS!
root@mysql-mha masterha# cd /var/log/masterha/app1