MySQL

MySQL

• MySQL的源码编译
• 步骤
• • 数据库的安装
  • 开始编译
• MySQL的安装及初始化
• • 生成启动脚本
  • 添加用户
  • 生成配置文件
  • 修改环境变量
  • 数据库初始化建立mysql基本数据
  • 开机启动
  • 数据库安全初始化
• MySQL的主复制
• MySQL主从集群的slave添加及延迟回放
• mysql的并行复制
• MySQL组从复制原理
• MySQL的gitd日志模式
• MySQL组复制功能的实现

---

MySQL的源码编译

步骤

准备两台rhel7的环境

数据库的安装

官网：http://www.mysql.com

（rhel7压缩包中）libtirpc 下载并xftp上传到/root下

[root@mysql-1 ~]# yum install libtirpc-devel-0.2.4-0.16.el7.x86_64.rpm -y

[root@mysql-1 ~]# tar zxf mysql-boost-5.7.44.tar.gz 

[root@mysql-1 ~]# cd mysql-5.7.44/
[root@mysql-1 mysql-5.7.44]# ls

[root@mysql-1 mysql-5.7.44]# yum install cmake -y

[root@mysql-1 mysql-5.7.44]# yum install gcc gcc-c++ -y

[root@mysql-1 mysql-5.7.44]# yum install openssl-devel -y

[root@mysql-1 mysql-5.7.44]# yum install ncurses-devel -y

（如果还有报错删除缓存文件------> rm -rf CMakeCache.txt）

开始编译

[root@mysql-1 mysql-5.7.44]# cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql 
-DMYSQL_DATADIR=/data/mysql 
-DMYSQL_UNIX_ADDR=/data/mysql/mysql.sock 
-DWITH_INNOBASE_STORAGE_ENGINE=1 
-DWITH_EXTRA_CHARSETS=all 
-DDEFAULT_CHARSET=utf8mb4 
-DDEFAULT_COLLATION=utf8mb4_unicode_ci 
-DWITH_BOOST=/root/mysql-5.7.44/boost/boost_1_59_0/

root@mysql-1 mysql-5.7.44\]# make install ## MySQL的安装及初始化 ### 生成启动脚本 ```c [root@mysql-1 ~]# yum install initscripts -y [root@mysql-1 ~]# cd /usr/local/mysql/ [root@mysql-1 mysql]# ls ``` ### 添加用户 ```c [root@mysql-1 mysql]# useradd -s /sbin/nologin -M mysql [root@mysql-1 mysql]# mkdir /data/mysql -p [root@mysql-1 mysql]# chown mysql.mysql -R /data/mysql [root@mysql-1 mysql]# ls [root@mysql-1 mysql]# cd support-files/ [root@mysql-1 support-files]# cp mysql.server /etc/init.d/mysqld [root@mysql-1 support-files]# ls ``` ### 生成配置文件 \[root@mysql-1 support-files\]# vim /etc/my.cnf > \[mysqld

datadir=/data/mysql

socket=/data/mysql/mysql.sock

symbolic-links=0

修改环境变量

root@mysql-1 support-files\]# vim \~/.bash_profile ```c # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs PATH=$PATH:$HOME/bin:/usr/local/mysql/bin export PAT ``` \[root@mysql-1 support-files\]# source \~/.bash_profile ### 数据库初始化建立mysql基本数据 \[root@mysql-1 \~\]# mysqld --user mysql --initialize ***生成数据库的密码*** > 1s\&Vgg:(IOdn \[root@mysql-1 \~\]# vim passwd > 初始化有问题：rm -rf /data/mysql/\* > > 重新开始初始化：\[root@mysql-1 \~\]# mysqld --user mysql --initialize > > 把密码重新写入passwd里 echo ... \> passwd ### 开机启动 \[root@mysql-1 \~\]# /etc/init.d/mysqld start SUCCESS! 等等等等等等等等等等等等 *** ** * ** *** 如果有下面的问题： ```c [root@mysql-1 mysql]# /etc/init.d/mysqld restart ERROR! MySQL server PID file could not be found! Starting MySQL.. ERROR! The server quit without updating PID file (/data/mysql/mysql-1.hh.org.pid). ``` 1、检查防火墙、selinux有没有关闭，可能被占用端口 > systemctl status firewalld.service > > systemctl stop firewalld.service > > systemctl mask firewalld.service > > getenforce > > vim /etc/sysconfig/selinux > > disabled 2、查看进程是否存在 解决方法： > \[root@mysql-1 mysql\]# ps -aux \| grep mysqld > > ...mysqld > > \[root@mysql-1 mysql\]# pkill -f mysql > > \[root@mysql-1 mysql\]# ps -aux \| grep mysqld > > #重新启动 > > \[root@mysql-1 mysql\]# /etc/init.d/mysqld start 其他的看看这个：[Mysql 启动报错解析：Starting MySQL... ERROR! The server quit without updating PID file (/usr/local/mysql/dat_starting mysql. error! the server quit without upd-CSDN博客](https://blog.csdn.net/Qevery678/article/details/96422599) *** ** * ** *** ### 数据库安全初始化 \[root@mysql-1 \~\]# mysql_secure_installation \[root@mysql-1 \~\]# mysql -uroot -p123 mysql\> SHOW DATABASES; mysql-2一样 ## MySQL的主复制 主：mysql-1 172.25.254.10 副：mysql-2 172.25.254.20 mysql-1 \[root@mysql-2 \~\]# vim /etc/my.cnf ```c [mysqld] datadir=/data/mysql socket=/data/mysql/mysql.sock symbolic-links=0 server-id=10 log-bin=mysql-bin ``` ```c [root@mysql-1 ~]# /etc/init.d/mysqld restart [root@mysql-1 ~]# mysql -uroot -p123 mysql> SELECT @@server_id; ``` 远程用户可以登陆查询 ```bash mysql> CREATE USER hh@'%' IDENTIFIED BY '123'; mysql> GRANT REPLICATION SLAVE ON *.* TO hh@'%'; mysql> show master status; mysql> show master status\G; mysql> create database hhhoo; mysql> create table hhhoo.userlist( -> username varchar(10) not null, -> password varchar(50) not null -> ); mysql> INSERT INTO hhhoo.userlist values ('hui1','123'); mysql> SELECT * FROM hhhoo.userlist; mysql> INSERT INTO hhhoo.userlist values ('hui2','444'); mysql> SELECT * FROM hhhoo.userlist; ``` *** ** * ** *** mysql-2（不能写入数据，只读） ```bash [root@mysql-2 ~]# vim /etc/my.cnf [mysqld] datadir=/data/mysql socket=/data/mysql/mysql.sock symbolic-links=0 server-id=20 ``` ```bash Slave_IO_Running: Yes #数据传输 Slave_SQL_Running: Yes #数据回放 #看到这俩就表示成功咯 ``` ## MySQL主从集群的slave添加及延迟回放 克隆一个rhel7：172.25.254.30 在mysql-1上 ```bash [root@mysql-1 ~]# rsync -al /usr/local/mysql [email protected]:/usr/local ``` 生成启动脚本 ```bash [root@mysql-3 ~]# yum install initscripts -y [root@mysql-3 ~]# cd /usr/local/mysql/ [root@mysql-3 mysql]# ls ``` 添加用户 ```bash [root@mysql-3 mysql]# useradd -s /sbin/nologin -M mysql [root@mysql-3 mysql]# mkdir /data/mysql -p [root@mysql-3 mysql]# chown mysql.mysql -R /data/mysql [root@mysql-3 mysql]# ls [root@mysql-3 mysql]# cd support-files/ [root@mysql-3 support-files]# cp mysql.server /etc/init.d/mysqld [root@mysql-3 support-files]# ls ``` **生成配置文件** ```bash [root@mysql-3 support-files]# vim /etc/my.cnf [mysqld] datadir=/data/mysql socket=/data/mysql/mysql.sock symbolic-links=0 server_id=30或者super_read_only=on ``` ```bash [root@mysql-3 support-files]# vim ~/.bash_profile # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs PATH=$PATH:$HOME/bin:/usr/local/mysql/bin export PAT [root@mysql-3 support-files]# source ~/.bash_profile ``` 数据库初始化建立mysql基本数据 ```bash [root@mysql-3 ~]# mysqld --user mysql --initialize 生成数据库的密码 [root@mysql-3 ~]# ll /data/mysql [root@mysql-3 ~]# /etc/init.d/mysqld start ``` 回mysql-1 ```bash [root@mysql-1 ~]# mysqldump -uroot -p hhhoo > hhhoo.sql Enter password: [root@mysql-1 ~]# cat hhhoo.sql [root@mysql-1 ~]# scp hhhoo.sql [email protected]:/mnt/ ``` 回mysql-3 ```bash [root@mysql-3 ~]# cd /mnt/ [root@mysql-3 mnt]# ls hhhoo.sql [root@mysql-3 mnt]# mysql -uroot -p123 -e "CREATE DATABASE hhhoo;" [root@mysql-3 mnt]# mysql -uroot -p123 hhhoo < hhhoo.sql [root@mysql-3 mnt]# mysql -uroot -p123 mysql> change master to master_host='172.25.254.100',master_user='hh',master_password='123',master_log_file=" mysql-bin.000003 ",master_log_pos=154; mysql> start slave; mysql> SHOW SLAVE STATUS\G; Slave_IO_Running: Connecting Slave_SQL_Running: Yes Last_IO_Error: error connecting to master '[email protected]:3306' - retry-time: 60 retries: 2（有报错） mysql> stop slave; [root@mysql-2 ~]# mysql -uhh -p123 -h 172.25.254.10（看看用户名能不能连上） mysql> change master to master_host='172.25.254.10',master_user='hh',master_password='123',master_log_file=" mysql-bin.000004",master_log_pos=154; ``` *** ** * ** *** 解决报错： > Slave_IO_Running: No > > Slave_SQL_Running: Yes > > Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name ```bash mysql> stop slave; ``` 去master上查看状态 ```bash mysql> show master status\G; *************************** 1. row *************************** File: mysql-bin.000003 #一样的 Position: 154 Binlog_Do_DB: Binlog_Ignore_DB: Executed_Gtid_Set: 1 row in set (0.00 sec) ERROR: No query specified ``` 刷新一下日志： ```bash mysql> flush logs; mysql> show master status\G; *************************** 1. row *************************** File: mysql-bin.000004 #copy Position: 154 #copy Binlog_Do_DB: Binlog_Ignore_DB: Executed_Gtid_Set: 1 row in set (0.00 sec) ``` 在slave中写上 ```bash mysql> CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000004',MASTER_LOG_POS=154;（更新后的） mysql> start slave; mysql> SHOW SLAVE STATUS\G; ``` over *** ** * ** *** **延迟复制:** 延迟复制时用来控制sql线程的，和i/o线程无关 ```bash mysql> SHOW SLAVE STATUS\G; SQL_Delay: 0 #数据延迟 修改： mysql> STOP SLAVE SQL_THREAD; mysql> change master to master_delay=60; mysql> start slave sql_thread; mysql> SHOW SLAVE STATUS\G; SQL_Delay: 60 ``` 完成！ 在master把hui1删除 ```bash mysql> delete from hhhoo.userlist where username='hui1'; mysql> select * from hhhoo.userlist; ``` #hui1没了 回slave看hui1还在，等60s后hui1没了； 为什么要延迟复制？防止出错找不回数据； **慢查询日志** 执行SQL很慢的查询，SQL语句就是需要优化的 慢查询被记录在慢查询日志里 如果需要优化SQL语句，就可以开启这个功能 在master上做 ```bash mysql> SHOW variables like "slow%";（查看） mysql> SET GLOBAL slow_query_log=on; mysql> SHOW variables like "slow%"; mysql> SHOW VARIABLES like "long%"; ``` 设定慢查询的时间值 ```bash mysql> SET long_query_time=4; ``` 测试： ```bash mysql> select sleep (10); [root@mysql-node1 ~]# cat /data/mysql/mysql-node1-slow.log ``` ## mysql的并行复制 默认情况下slave中使用的是sql单线程回放 在slave2中写 ```bash [root@mysql-2 ~]# vim /etc/my.cnf [mysqld] datadir=/data/mysql socket=/data/mysql/mysql.sock symbolic-links=0 server-id=20 gtid_mode=ON enforce-gtid-consistency=ON slave-parallel-type=LOGICAL_CLOCK slave-parallel-workers=16 master_info_repository=TABLE relay_log_recovery=ON ``` 重启 ```bash [root@mysql-2 ~]# /etc/init.d/mysqld restart ```  ## MySQL组从复制原理 slave会开启两个线程 I/O线程：用于处理与外部系统之间的信息交互，如输入输出操作 SQL线程：用于执行并发任务的线程，允许它在多个处理器上同时执行任务，从而提高性能和资源利用率 master： dump线程：用于实现数据库之间的数据复制同步 主从架构采用的是异步机制 master更新完成后直接发送二进制日志到slave，但是slaves是否真正保存了数据master端不会检测 master端直接保存二进制日志到磁盘 当master端到slave端的网络出现问题时或者master端直接挂掉，二进制日志可能根本没有到达slave master出现问题slave端接管master，这个过程中数据就丢失了 这样的问题出现就无法达到数据的强一致性，零数据丢失 ## MySQL的gitd日志模式 在master端的写入时多用户读写，在slave端的复制时单线程日志回放，所以slave端一定会延迟 master端 ```bash [root@mysql-1 ~]# vim /etc/my.cnf [mysqld] datadir=/data/mysql socket=/data/mysql/mysql.sock symbolic-links=0 server-id=10 log-bin=mysql-bin gtid_mode=ON enforce-gtid-consistency=ON [root@mysql-1 ~]# /etc/init.d/mysqld restart ``` 登陆mysql上查看进程show master status\\G; ```bash [root@mysql-node1 ~]# mysqlbinlog -vv /data/mysql/mysql-bin.0000？进程号 ``` 在slave上 ```bash [root@mysql-3 ~]# vim /etc/my.cnf [mysqld] datadir=/data/mysql socket=/data/mysql/mysql.sock symbolic-links=0 server_id=30 gtid_mode=ON enforce-gtid-consistency=ON [root@mysql-3 ~]# /etc/init.d/mysqld restart ``` slave2: ```bash [root@mysql-2 ~]# mysql -uroot -p123 mysql> stop slave; `mysql> change master to -> master_host='172.25.254.10', -> master_user='hh', -> master_password='123', -> master_auto_position=1;` mysql> start slave; mysql> SHOW SLAVE STATUS\G; ``` slave3:一样的 ```bash [root@mysql-3 ~]# mysql -uroot -p123 mysql> stop slave; `mysql> change master to -> master_host='172.25.254.10', -> master_user='hh', -> master_password='123', -> master_auto_position=1;` mysql> start slave; mysql> SHOW SLAVE STATUS\G; ``` MySQL的半同步复制  在master上写： ```bash [root@mysql-1 ~]# vim /etc/my.cnf [mysqld] datadir=/data/mysql socket=/data/mysql/mysql.sock symbolic-links=0 server-id=10 log-bin=mysql-bin gtid_mode=ON enforce-gtid-consistency=ON rpl_semi_sync_master_enabled=1 ``` 登陆mysql安装插件 ```bash mysql> INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so'; mysql> SELECT PLUGIN_NAME, PLUGIN_STATUS -> FROM INFORMATION_SCHEMA.PLUGINS -> WHERE PLUGIN_NAME LIKE '%semi%'; mysql> SET GLOBAL rpl_semi_sync_master_enabled = 1; mysql> SHOW VARIABLES LIKE 'rpl_semi_sync%'; mysql> SHOW STATUS LIKE 'Rpl_semi_sync%'; ``` 在slave2.3上 ```bash root@mysql-2 ~]# vim /etc/my.cnf [mysqld] datadir=/data/mysql socket=/data/mysql/mysql.sock symbolic-links=0 server-id=20 gtid_mode=ON enforce-gtid-consistency=ON rpl_semi_sync_master_enabled=1 ``` 登陆数据库安装插件 ```bash mysql> INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so'; mysql> SET GLOBAL rpl_semi_sync_master_enabled = 1; [root@mysql-2 ~]# /etc/init.d/mysqld restart ``` *** ** * ** *** 如果服务是off 重启I/O线程： ```bash mysql> STOP SLAVE IO_THREAD; mysql> START SLAVE IO_THREAD; ``` 查看有没有开了 ```bash mysql> SHOW VARIABLES LIKE 'rpl_semi_sync%'; ``` *** ** * ** *** 测试： #在slave端 ```bash [root@mysql-node2 ~]# mysql -p123 mysql> STOP SLAVE IO_THREAD; [root@mysql-node3 ~]# mysql -p123 mysql> STOP SLAVE IO_THREAD; ``` #在master端插入数据 ```bash mysql> insert into lee.userlist values ('user5','555'); Query OK, 1 row affected (10.00 sec) #10秒超时 ``` 解决： 在slave端开启 ```bash mysql> start SLAVE IO_THREAD; ``` ## MySQL组复制功能的实现 组复制流程  组复制分为： 单组：单写模式 group 内只有一台节点可写可读，其他节点只可以读。当主服务器失败时，会自动选择新的主服务器。  多组：组内的所有机器都是 primary 节点，同时可以进行读写操作，并且数据是最终一致的。  为了避免出错，在所有节点中从新生成数据库数据 ```bash [root@mysql-1 ~]# /etc/init.d/mysqld stop Shutting down MySQL........... SUCCESS! [root@mysql-2 ~]# /etc/init.d/mysqld stop [root@mysql-3 ~]# /etc/init.d/mysqld stop ``` 把数据也删掉（企业记得备份） ```bash [root@mysql-1 ~]# rm -fr /data/mysql/* [root@mysql-2 ~]# rm -fr /data/mysql/* [root@mysql-3 ~]# rm -fr /data/mysql/* ``` 在master里写 ```bash [root@mysql-1 ~]# vim /etc/my.cnf [mysqld] datadir=/data/mysql socket=/data/mysql/mysql.sock symbolic-links=0 server-id=10 log-bin=mysql-bin gtid_mode=ON enforce-gtid-consistency=ON disabled_storage_engines="MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY" master_info_repository=TABLE relay_log_info_repository=TABLE binlog_checksum=NONE log_slave_updates=ON log_bin=binlog binlog_format=ROW plugin_load_add='group_replication.so' transaction_write_set_extraction=XXHASH64 group_replication_group_name="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa" group_replication_start_on_boot=off group_replication_local_address="172.25.254.10:33061" group_replication_group_seeds="172.25.254.10:33061,172.25.254.20:33061,172.25.254.30:33061" group_replication_ip_whitelist="172.25.254.0/24,127.0.0.1/8" group_replication_bootstrap_group=off group_replication_single_primary_mode=OFF group_replication_enforce_update_everywhere_checks=ON group_replication_allow_local_disjoint_gtids_join=1 ``` ```bash [root@mysql-1 ~]# mysqld --user=mysql --initialize password is generated for root@localhost: DhflEwCuu7.) [root@mysql-1 ~]# /etc/init.d/mysqld start ``` 登陆MySQL ```bash [root@mysql-1 ~]# mysql -uroot -p'DhflEwCuu7.)' ``` 改密码 ```bash mysql> alter user root@localhost identified by '123'; ``` 日志信息关闭 ```bash mysql> SET SQL_LOG_BIN=0; mysql> CREATE USER hhhoo@'%' IDENTIFIED BY 'hh'; mysql> grant replication slave on *.* TO hhhoo@'%'; ``` 打开日志功能 ```bash mysql> SET SQL_LOG_BIN=1; mysql>CHANGE MASTER TO MASTER_USER='hhhoo', MASTER_PASSWORD='123' FOR CHANNEL'group_replication_recovery'; mysql> SET GLOBAL group_replication_bootstrap_group=ON; mysql> START GROUP_REPLICATION; mysql> SET GLOBAL group_replication_bootstrap_group=OFF; mysql> show databases; mysql> SELECT * FROM performance_schema.replication_group_members; ``` 做解析（3个MySQL都做） ```bash vim /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.25.254.30 mysql-3.hh.org 172.25.254.20 mysql-2.hh.org 172.25.254.10 mysql-1.hh.org ``` 复制文件： ```bash [root@mysql-1 ~]# scp /etc/my.cnf [email protected]:/etc/my.cnf [root@mysql-1 ~]# scp /etc/my.cnf [email protected]:/etc/my.cnf ``` slave2 ```bash [root@mysql-2 ~]# vim /etc/my.cnf [mysqld] datadir=/data/mysql socket=/data/mysql/mysql.sock symbolic-links=0 server-id=20 log-bin=mysql-bin gtid_mode=ON enforce-gtid-consistency=ON disabled_storage_engines="MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY" master_info_repository=TABLE relay_log_info_repository=TABLE binlog_checksum=NONE log_slave_updates=ON log_bin=binlog binlog_format=ROW plugin_load_add='group_replication.so' transaction_write_set_extraction=XXHASH64 group_replication_group_name="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa" group_replication_start_on_boot=off group_replication_local_address="172.25.254.20:33061" group_replication_group_seeds="172.25.254.10:33061,172.25.254.20:33061,172.25.254.30:33061" group_replication_ip_whitelist="172.25.254.0/24,127.0.0.1/8" group_replication_bootstrap_group=off group_replication_single_primary_mode=OFF group_replication_enforce_update_everywhere_checks=ON group_replication_allow_local_disjoint_gtids_join=1 ``` ```bash [root@mysql-2 ~]# rm -fr /data/mysql/* [root@mysql-2 ~]# mysqld --user=mysql --initialize [root@mysql-2 ~]# /etc/init.d/mysqld start [root@mysql-2 ~]# mysql -uroot -p'=,f3%mLVtRNu' mysql> alter user root@localhost identified by '123'; mysql> SET SQL_LOG_BIN=0; mysql> CREATE USER hhhoo@'%' IDENTIFIED BY 'hh'; mysql> GRANT REPLICATION SLAVE ON *.* TO hhhoo@'%'; mysql> FLUSH PRIVILEGES; mysql> SET SQL_LOG_BIN=1; mysql> CHANGE MASTER TO MASTER_USER='hhhoo', MASTER_PASSWORD='123' FOR CHANNEL'group_replication_recovery'; mysql> START GROUP_REPLICATION; mysql> SELECT * FROM performance_schema.replication_group_members; ```