【Kingbase FlySync】命令模式:安装部署同步软件，实现KES到KES实现同步迁移
概述
- Kingbase FlySync是面向同城/异地灾备、数据库平滑升级替换、数据集中共享与分发、应用上云迁移、数据库负载均衡等场景的数据同步产品。该产品基于增量日志解析技术，性能高、时延低、资源占用极少，能够实现异构数据源之间大规模增量数据的任意方向流转和实时同步，帮助用户打破数据孤岛，轻松共享分发数据。Kingbase FlySync支持丰富的软硬件平台和数据源，同步拓扑可以一对一、一对多、多对一、级联等方式任意组合延伸，并可在数据同步过程中保证端到端的事务级数据完整性和高可用性。
准备环境
| 虚拟机 |
IP |
数据库软件版本 |
角色 |
操作系统 |
| K2node1 |
192.168.40.111 |
(Kingbase) KingbaseES V8R6 |
主库 |
CenterOS7.2 |
| K2node2 |
192.168.40.112 |
(Kingbase) KingbaseES V8R6 |
备库 |
CenterOS7.2 |
| 项目 |
node1 |
node2 |
| 数据库版本 |
KingbaseES V8R6 |
KingbaseES V8R6 |
| 安装路径 |
/KingbaseES/V8/ |
/KingbaseES/V8/ |
| data路径 |
/data |
/data |
| 端口 |
54321 |
54321 |
| 用户名 |
system |
system |
| 密码 |
kingbase |
kingbase |
目标
- 同步程序安装部署
- 使用命令方式，完成数据迁移，并保证数据一致。
- 按照以下规范完成部署
| 类别 |
源端 |
目标端 |
| replicater安装路径 |
/home/flysync/replicator |
/home/flysync/replicator |
| replicator端口 |
rmi-port：11000/kufl-port:3112 |
rmi-port:11000/kufl-port：3112 |
| kfs服务名 |
kingbase8_source |
kingbase8_target |
| KFS 同步所使用的用户 |
flysync/flysync |
flysync/flysync |
| 映射规则1 |
exam |
exam[0017] |
| 映射规则2 |
exam.course |
exam0017.course0017 |
资源
1.测试虚拟机下载地址包含node1,node2
- 虚拟机快照都还原至：优化数据库
- root用户密码为kingbase.123
- kingbase用户密码为kingbase
点击下载：百度网盘
2.同步工具下载地址
点击下载：百度网盘
3.临时授权下载地址
点击下载：百度网盘
4.ruby工具下载地址
点击下载：百度网盘
5.EXAMv0.11.sql下载地址
点击下载：百度网盘
实操：同步软件安装部署
1.node1准备安装环境
(1)增加flysync 用户并设置密码
bash
复制代码
[root@node1 ~]# adduser flysync
[root@node1 ~]# usermod -G kingbase flysync
[root@node1 ~]# echo flysync | passwd --stdin flysync
更改用户 flysync 的密码 。
passwd：所有的身份验证令牌已经成功更新。
[root@node1 ~]#
(2)调整flysync的最大文件句柄数（open files）、允许创建的最大进程数量（max user processes）
bash
复制代码
[root@node1 ~]# cat >> /etc/security/limits.conf << EOF
flysync - nofile 65535
flysync - nproc 8096
EOF
[root@node1 ~]# cat /etc/security/limits.conf
# /etc/security/limits.conf
#
#This file sets the resource limits for the users logged in via PAM.
#It does not affect resource limits of the system services.
#
#Also note that configuration files in /etc/security/limits.d directory,
#which are read in alphabetical order, override the settings in this
#file in case the domain is the same or more specific.
#That means for example that setting a limit for wildcard domain here
#can be overriden with a wildcard setting in a config file in the
#subdirectory, but a user specific setting here can be overriden only
#with a user specific setting in the subdirectory.
#
#Each line describes a limit for a user in the form:
#
#<domain> <type> <item> <value>
#
#Where:
#<domain> can be:
# - a user name
# - a group name, with @group syntax
# - the wildcard *, for default entry
# - the wildcard %, can be also used with %group syntax,
# for maxlogin limit
#
#<type> can have the two values:
# - "soft" for enforcing the soft limits
# - "hard" for enforcing hard limits
#
#<item> can be one of the following:
# - core - limits the core file size (KB)
# - data - max data size (KB)
# - fsize - maximum filesize (KB)
# - memlock - max locked-in-memory address space (KB)
# - nofile - max number of open file descriptors
# - rss - max resident set size (KB)
# - stack - max stack size (KB)
# - cpu - max CPU time (MIN)
# - nproc - max number of processes
# - as - address space limit (KB)
# - maxlogins - max number of logins for this user
# - maxsyslogins - max number of logins on the system
# - priority - the priority to run user process with
# - locks - max number of file locks the user can hold
# - sigpending - max number of pending signals
# - msgqueue - max memory used by POSIX message queues (bytes)
# - nice - max nice priority allowed to raise to values: [-20, 19]
# - rtprio - max realtime priority
#
#<domain> <type> <item> <value>
#
#* soft core 0
#* hard rss 10000
#@student hard nproc 20
#@faculty soft nproc 20
#@faculty hard nproc 50
#ftp hard nproc 0
#@student - maxlogins 4
#add by kingbase
kingbase soft nproc 65536
kingbase hard nproc 65536
kingbase soft nofile 65536
kingbase hard nofile 65536
#kingbase soft stack 10240
#kingbase hard stack 32768
kingbase soft core unlimited
kingbase hard core unlimited
flysync - nofile 65535
flysync - nproc 8096
[root@node1 ~]#
(3)hosts 文件配置
bash
复制代码
[root@node1 ~]# cat >> /etc/hosts << EOF
192.168.40.111 node1
192.168.40.112 node2
EOF
[root@node1 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.40.111 node1
192.168.40.112 node2
[root@node1 ~]#
(4)network配置
bash
复制代码
[root@node1 ~]# cat >> /etc/sysconfig/network << EOF
> NETWORKING = yes
> EOF
[root@node1 ~]# cat /etc/sysconfig/network
# Created by anaconda
NETWORKING = yes
HOSTNAME = node1
[root@node1 ~]#
(5)上传并安装ruby工具
- 使用FTP工具上传rbenv_2.2.2_linux_x64.zip到/home/flysync目录下
bash
复制代码
[root@node1 flysync]# ls -l
总用量 6832
-rw-r--r-- 1 root root 6993781 11月 16 10:28 rbenv_2.2.2_linux_x64.zip
[root@node1 flysync]# pwd
/home/flysync
[root@node1 flysync]# [root@node1 flysync]# chown flysync:flysync /home/flysync/rbenv_2.2.2_linux_x64.zip
[root@node1 flysync]#
- 解压rbenv_2.2.2_linux_x64.zip包
bash
复制代码
[root@node1 flysync]su flysync
[flysync@node1 flysync]# unzip -q /home/flysync/rbenv_2.2.2_linux_x64.zip
[flysync@node1 flysync]# ls -l
总用量 6836
-rwxr-xr-x 1 root root 340 11月 4 2019 fspm
drwxrwxr-x 3 flysync flysync 17 11月 4 2019 rbenv
-rw-r--r-- 1 flysync flysync 6993781 11月 16 10:28 rbenv_2.2.2_linux_x64.zip
[root@node1 flysync]#
bash
复制代码
[flysync@node1 flysync]# vi ~/.bash_profile
[flysync@node1 flysync]# source ~/.bash_profile
[flysync@node1 flysync]# cat ~/.bash_profile
[flysync@node1 ~]$ cat ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH
export PATH
[flysync@node1 flysync]# ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysync@node1 flysync]#
(6)node1数据库配置
bash
复制代码
[kingbase@node1 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node1 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=#
bash
复制代码
test=# CREATE USER flysync PASSWORD 'flysync';
CREATE ROLE
test=# ALTER USER flysync REPLICATION;
ALTER ROLE
test=# GRANT ALL PRIVILEGES ON DATABASE TEST TO flysync ;
GRANT
test=# ALTER USER flysync SUPERUSER;
ALTER ROLE
test=#
bash
复制代码
test=# ALTER SYSTEM SET wal_level ='logical';
ALTER SYSTEM
test=# ALTER SYSTEM SET wal_keep_segments = 4;
ALTER SYSTEM
test=# ALTER SYSTEM SET max_wal_senders =10;
ALTER SYSTEM
test=# ALTER SYSTEM SET max_replication_slots =10;
ALTER SYSTEM
test=#
bash
复制代码
test=# \q
[kingbase@node1 bin]$ cat >> /data/sys_hba.conf << EOF
host replication all 0.0.0.0/0 scram-sha-256
EOF
[kingbase@node1 bin]$ cat /data/sys_hba.conf
# Kingbase Client Authentication Configuration File
# ===================================================
#
# Refer to the "Client Authentication" section in the Kingbase
# documentation for a complete description of this file. A short
# synopsis follows.
#
# This file controls: which hosts are allowed to connect, how clients
# are authenticated, which Kingbase user names they can use, which
# databases they can access. Records take one of these forms:
#
# local DATABASE USER METHOD [OPTIONS]
# host DATABASE USER ADDRESS METHOD [OPTIONS]
# hostssl DATABASE USER ADDRESS METHOD [OPTIONS]
# hostnossl DATABASE USER ADDRESS METHOD [OPTIONS]
#
# (The uppercase items must be replaced by actual values.)
#
# The first field is the connection type: "local" is a Unix-domain
# socket, "host" is either a plain or SSL-encrypted TCP/IP socket,
# "hostssl" is an SSL-encrypted TCP/IP socket, and "hostnossl" is a
# plain TCP/IP socket.
#
# DATABASE can be "all", "sameuser", "samerole", "replication", a
# database name, or a comma-separated list thereof. The "all"
# keyword does not match "replication". Access to replication
# must be enabled in a separate record (see example below).
#
# USER can be "all", a user name, a group name prefixed with "+", or a
# comma-separated list thereof. In both the DATABASE and USER fields
# you can also write a file name prefixed with "@" to include names
# from a separate file.
#
# ADDRESS specifies the set of hosts the record matches. It can be a
# host name, or it is made up of an IP address and a CIDR mask that is
# an integer (between 0 and 32 (IPv4) or 128 (IPv6) inclusive) that
# specifies the number of significant bits in the mask. A host name
# that starts with a dot (.) matches a suffix of the actual host name.
# Alternatively, you can write an IP address and netmask in separate
# columns to specify the set of hosts. Instead of a CIDR-address, you
# can write "samehost" to match any of the server's own IP addresses,
# or "samenet" to match any address in any subnet that the server is
# directly connected to.
#
# METHOD can be "trust", "reject", "md5", "password", "scram-sha-256",
# "gss", "sspi", "ident", "peer", "pam", "ldap", "radius" or "cert".
# Note that "password" sends passwords in clear text; "md5" or
# "scram-sha-256" are preferred since they send encrypted passwords.
#
# OPTIONS are a set of options for the authentication in the format
# NAME=VALUE. The available options depend on the different
# authentication methods -- refer to the "Client Authentication"
# section in the documentation for a list of which options are
# available for which authentication methods.
#
# Database and user names containing spaces, commas, quotes and other
# special characters must be quoted. Quoting one of the keywords
# "all", "sameuser", "samerole" or "replication" makes the name lose
# its special character, and just match a database or username with
# that name.
#
# This file is read on server startup and when the server receives a
# SIGHUP signal. If you edit the file on a running system, you have to
# SIGHUP the server for the changes to take effect, run "sys_ctl reload",
# or execute "SELECT pg_reload_conf()".
#
# Put your actual configuration here
# ----------------------------------
#
# If you want to allow non-local connections, you need to add more
# "host" records. In that case you will also need to make Kingbase
# listen on a non-local interface via the listen_addresses
# configuration parameter, or via the -i or -h command line switches.
# CAUTION: Configuring the system for local "trust" authentication
# allows any local user to connect as any Kingbase user, including
# the database superuser. If you do not trust all your local users,
# use another authentication method.
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 scram-sha-256
host all all 0.0.0.0/0 scram-sha-256
# IPv6 local connections:
host all all ::1/128 scram-sha-256
host all all ::0/0 scram-sha-256
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 scram-sha-256
host replication all ::1/128 scram-sha-256
host replication all 0.0.0.0/32 scram-sha-256
host replication all 0.0.0.0/0 scram-sha-256
[kingbase@node1 bin]$
(7)重启数据库使配置生效
bash
复制代码
[kingbase@node1 bin]$ sys_ctl restart -D /data
waiting for server to shut down.... done
server stopped
waiting for server to start....2023-11-16 03:43:14.182 GMT [5233] 警告: max_connections should be less than or equal than 10 (restricted by license)
2023-11-16 03:43:14.182 GMT [5233] 提示: the value of max_connect is set 10
2023-11-16 03:43:14.183 GMT [5233] 警告: max_connections should be less than or equal than 10 (restricted by license)
2023-11-16 03:43:14.183 GMT [5233] 提示: the value of max_connect is set 10
2023-11-16 11:43:14.189 CST [5233] 日志: sepapower extension initialized
2023-11-16 11:43:14.191 CST [5233] 日志: 太多后台工作进程
2023-11-16 11:43:14.191 CST [5233] 详细信息: 当前设置里最多可以注册2个后台工作进程.
2023-11-16 11:43:14.191 CST [5233] 提示: 考虑增大配置参数 "max_worker_processes"的值.
2023-11-16 11:43:14.191 CST [5233] 日志: 太多后台工作进程
2023-11-16 11:43:14.191 CST [5233] 详细信息: 当前设置里最多可以注册2个后台工作进程.
2023-11-16 11:43:14.191 CST [5233] 提示: 考虑增大配置参数 "max_worker_processes"的值.
2023-11-16 11:43:14.192 CST [5233] 日志: 正在启动 KingbaseES V008R006C005B0023 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46), 64-bit
2023-11-16 11:43:14.192 CST [5233] 日志: 正在监听IPv4地址"0.0.0.0"，端口 54321
2023-11-16 11:43:14.192 CST [5233] 日志: 正在监听IPv6地址"::"，端口 54321
2023-11-16 11:43:14.193 CST [5233] 日志: 在Unix套接字 "/tmp/.s.KINGBASE.54321"上侦听
2023-11-16 11:43:14.244 CST [5233] 日志: 日志输出重定向到日志收集进程
2023-11-16 11:43:14.244 CST [5233] 提示: 后续的日志输出将出现在目录 "sys_log"中.
done
server started
[kingbase@node1 bin]$
(8)导入测试数据
bash
复制代码
[kingbase@node1 install]$ ksql -U flysync -d test -f /install/EXAM_v0.11.sql
ksql:/install/EXAM_v0.11.sql:8: 注意: 串联删除10个其它对象
描述: 递归删除 序列 exam.seq_student_sno
递归删除 表 exam.student
递归删除 表 exam.course
递归删除 表 exam.instructor
递归删除 表 exam.instructor_history
递归删除 表 exam.score
递归删除 视图 exam.v_score_information
递归删除 function exam.get_avg_score(numeric)
递归删除 procedure exam.retire_instructor(numeric)
递归删除 表 exam.tmpbak01
DROP SCHEMA
CREATE SCHEMA
SET
CREATE SEQUENCE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
CREATE VIEW
CREATE FUNCTION
CREATE PROCEDURE
CREATE TRIGGER
ksql:/install/EXAM_v0.11.sql:202: 警告: 没有事物在运行中
COMMIT
[kingbase@node1 install]$
(9)最后检查下环境
bash
复制代码
[kingbase@node1 install]$ exit
登出
[root@node2 flysync]# uname -n
node2
bash
复制代码
[root@node1 ~]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 11210
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 8096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
bash
复制代码
[root@node1 flysync]# hostname --ip-address
192.168.40.111 192.168.40.111
[root@node1 flysync]#
bash
复制代码
[root@node1 ~]$ ping -c 4 node1
PING node1 (192.168.40.111) 56(84) bytes of data.
64 bytes from node1 (192.168.40.111): icmp_seq=1 ttl=64 time=0.036 ms
64 bytes from node1 (192.168.40.111): icmp_seq=2 ttl=64 time=0.110 ms
64 bytes from node1 (192.168.40.111): icmp_seq=3 ttl=64 time=0.036 ms
64 bytes from node1 (192.168.40.111): icmp_seq=4 ttl=64 time=0.037 ms
--- node1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.036/0.054/0.110/0.033 ms
[flysync@node1 ~]$ ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq=1 ttl=64 time=1.65 ms
64 bytes from node2 (192.168.40.112): icmp_seq=2 ttl=64 time=1.43 ms
64 bytes from node2 (192.168.40.112): icmp_seq=3 ttl=64 time=0.262 ms
64 bytes from node2 (192.168.40.112): icmp_seq=4 ttl=64 time=0.301 ms
--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 0.262/0.913/1.656/0.637 ms
bash
复制代码
[root@node1 ~]$ date
2023年 11月 16日 星期四 11:48:34 CST
2.node2准备安装环境
(1)增加flysync 用户并设置密码
bash
复制代码
[root@node2 ~]# userdel -r flysync
[root@node2 ~]# adduser flysync
[root@node2 ~]# usermod -G kingbase flysync
[root@node2 ~]# echo flysync |passwd --stdin flysync
更改用户 flysync 的密码 。
passwd：所有的身份验证令牌已经成功更新。
[root@node2 ~]#
(2)调整flysync的最大文件句柄数（open files）、允许创建的最大进程数量（max user processes）
bash
复制代码
[root@node2 ~]# cat >> /etc/security/limits.conf << EOF
flysync - nofile 65535
flysync - nproc 8096
EOF
[root@node2 ~]# cat /etc/security/limits.conf
# /etc/security/limits.conf
#
#This file sets the resource limits for the users logged in via PAM.
#It does not affect resource limits of the system services.
#
#Also note that configuration files in /etc/security/limits.d directory,
#which are read in alphabetical order, override the settings in this
#file in case the domain is the same or more specific.
#That means for example that setting a limit for wildcard domain here
#can be overriden with a wildcard setting in a config file in the
#subdirectory, but a user specific setting here can be overriden only
#with a user specific setting in the subdirectory.
#
#Each line describes a limit for a user in the form:
#
#<domain> <type> <item> <value>
#
#Where:
#<domain> can be:
# - a user name
# - a group name, with @group syntax
# - the wildcard *, for default entry
# - the wildcard %, can be also used with %group syntax,
# for maxlogin limit
#
#<type> can have the two values:
# - "soft" for enforcing the soft limits
# - "hard" for enforcing hard limits
#
#<item> can be one of the following:
# - core - limits the core file size (KB)
# - data - max data size (KB)
# - fsize - maximum filesize (KB)
# - memlock - max locked-in-memory address space (KB)
# - nofile - max number of open file descriptors
# - rss - max resident set size (KB)
# - stack - max stack size (KB)
# - cpu - max CPU time (MIN)
# - nproc - max number of processes
# - as - address space limit (KB)
# - maxlogins - max number of logins for this user
# - maxsyslogins - max number of logins on the system
# - priority - the priority to run user process with
# - locks - max number of file locks the user can hold
# - sigpending - max number of pending signals
# - msgqueue - max memory used by POSIX message queues (bytes)
# - nice - max nice priority allowed to raise to values: [-20, 19]
# - rtprio - max realtime priority
#
#<domain> <type> <item> <value>
#
#* soft core 0
#* hard rss 10000
#@student hard nproc 20
#@faculty soft nproc 20
#@faculty hard nproc 50
#ftp hard nproc 0
#@student - maxlogins 4
# End of file
#add by kingbase
kingbase soft nproc 65536
kingbase hard nproc 65536
kingbase soft nofile 65536
kingbase hard nofile 65536
kingbase soft stack 10240
kingbase hard stack 32768
kingbase soft core unlimited
kingbase hard core unlimited
flysync - nofile 65535
flysync - nproc 8096
[root@node2 ~]#
(3)hosts 文件配置
bash
复制代码
[root@node2 ~]# cat >> /etc/hosts << EOF
192.168.40.111 node1
192.168.40.112 node2
EOF
[root@node2 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.40.111 node1
192.168.40.112 node2
[root@node2 ~]#
(4)network配置
bash
复制代码
[root@node1 ~]# cat >> /etc/sysconfig/network << EOF
NETWORKING = yes
EOF
[root@node2 ~]# cat /etc/sysconfig/network
# Created by anaconda
NETWORKING = yes
HOSTNAME = node2
[root@node2 ~]#
(5)拷贝并安装ruby工具
bash
复制代码
```bash
[flysync@node2 ~]# scp root@node1:/home/flysync/rbenv_2.2.2_linux_x64.zip /home/flysync/
The authenticity of host 'node1 (192.168.40.111)' can't be established.
ECDSA key fingerprint is b8:54:e0:74:c5:e2:33:46:0d:7d:39:91:e3:b5:5a:6d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node1,192.168.40.111' (ECDSA) to the list of known hosts.
root@node1's password:
rbenv_2.2.2_linux_x64.zip 100% 6830KB 6.7MB/s 00:00
[flysync@node2 ~]# cd /home/flysync/
[flysync@node2 flysync]# ls rbenv_2.2.2_linux_x64.zip
rbenv_2.2.2_linux_x64.zip
[flysync@node2 flysync]#
- 解压rbenv_2.2.2_linux_x64.zip包
bash
复制代码
[root@node2 flysync]su flysync
[flysync@node2 flysync]# unzip -q /home/flysync/rbenv_2.2.2_linux_x64.zip
[flysync@node2 flysync]# ls -l
总用量 6836
-rwxr-xr-x 1 root root 340 11月 4 2019 fspm
drwxrwxr-x 3 flysync flysync 17 11月 4 2019 rbenv
-rw-r--r-- 1 flysync flysync 6993781 11月 16 10:28 rbenv_2.2.2_linux_x64.zip
[root@node1 flysync]#
bash
复制代码
[flysync@node2 ~]$ vi ~/.bash_profile
[flysync@node2 ~]$ source ~/.bash_profile
[flysync@node2 ~]$ cat ~/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
export RUBY_HOME=/home/flysync/rbenv/ruby
export PATH=$RUBY_HOME/bin:$PATH
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export PATH
[flysync@node2 flysync]# ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysync@node2 flysync]#
(6)node2数据库添加用户并授权
bash
复制代码
[flysync@node2 ~]$ exit
exit
[root@node2 flysync]# su kingbase
[kingbase@node2 flysync]$
[kingbase@node2 flysync]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node2 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=#
bash
复制代码
test=# CREATE USER flysync PASSWORD 'flysync';
CREATE ROLE
ttest=# GRANT ALL PRIVILEGES ON DATABASE TEST TO flysync ;
GRANT
test=# ALTER USER flysync SUPERUSER;
ALTER ROLE
test=#
(7)最后检查下环境
bash
复制代码
[kingbase@node1 install]$ exit
登出
[root@node2 flysync]# uname -n
node2
bash
复制代码
[root@node2 flysync]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 9838
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 9838
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[root@node2 flysync]#
bash
复制代码
[root@node2 flysync]# hostname --ip-address
192.168.40.112
[root@node2 flysync]#
bash
复制代码
[root@node2 flysync]# ping -c 4 node1
PING node1 (192.168.40.111) 56(84) bytes of data.
64 bytes from node1 (192.168.40.111): icmp_seq=1 ttl=64 time=0.296 ms
64 bytes from node1 (192.168.40.111): icmp_seq=2 ttl=64 time=0.274 ms
64 bytes from node1 (192.168.40.111): icmp_seq=3 ttl=64 time=0.370 ms
64 bytes from node1 (192.168.40.111): icmp_seq=4 ttl=64 time=0.325 ms
--- node1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.274/0.316/0.370/0.038 ms
[root@node2 flysync]# ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq=1 ttl=64 time=0.028 ms
64 bytes from node2 (192.168.40.112): icmp_seq=2 ttl=64 time=0.043 ms
64 bytes from node2 (192.168.40.112): icmp_seq=3 ttl=64 time=0.038 ms
64 bytes from node2 (192.168.40.112): icmp_seq=4 ttl=64 time=0.036 ms
--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.028/0.036/0.043/0.006 ms
[root@node2 flysync]#
bash
复制代码
[root@node2 flysync]# date
2023年 11月 16日 星期四 13:19:32 CST
[root@node2 flysync]#
3.node1节点部署replicator同步程序
(1).上传并解压replicator同步程序
bash
复制代码
[root@node1 ~]# su flysync
[flysync@node1 install]# cd /install/
[flysync@node1 install]# ls KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysync@node1 install]$ tar -zxf KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysync@node1 install]$ ls KingbaseFlySync-V002R002C004B20230320-replicator
cluster-home extensions flysync-replicator tools
[root@node1 install]#
(2).拷贝驱动包
bash
复制代码
[flysync@node1 install]$ cd KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/lib
[flysync@node1 lib]$ cp /install/KingbaseFlySync-V002R002C004B20230320-replicator/extensions/jdbc/kingbase8-8.6.0.jar .
[flysync@node1 lib]$ ls -l |grep kingbase8-8.6.0.jar
-rw-rw-r-- 1 flysync flysync 1019069 11月 16 13:35 kingbase8-8.6.0.jar
[flysync@node1 lib]$
(3).配置静默安装文件
bash
复制代码
[flysync@node1 lib]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysync@node1 ini]$ cp flysync_kingbase8_source.ini ~/flysync.ini
[flysync@node1 ini]$ vi ~/flysync.ini
参数如下：
bash
复制代码
[defaults]
# 安装目录
install-directory=~/replicator
# 环境变量脚本位置
profile-script=~/.bash_profile
# rmi端口,默认11000
rmi-port=11000
# 服务名
[kingbase8_source]
# 监听kufl获取的端口
kufl-port=3112
# master主机名
master=node1
# 当前主机名
members=node1
# 数据源类型
datasource-type=kingbase
# 同步数据库信息
replication-host=node1
replication-user=flysync
replication-password=flysync
replication-port=54321
# 同步服务角色
role=master
## 指定连接的数据库名称; 源端时，需指定解析方式
kingbase-dbname=test
kingbase-extractor-method=xlogical
## kingbase 版本号
datasource-version=8
## 过滤器开关配置
# svc_extractor_filters=replicate,casetransform,dropcolumn,rename
svc_extractor_filters=replicate
## 集群参数
# property=replicator.datasource.global.connectionSpec.use_cluster=true
# property=replicator.datasource.global.connectionSpec.slave_add=192.168.28.176,192.168.28.176
# property=replicator.datasource.global.connectionSpec.slave_port=1521,1521
### 过滤器详细配置
## replicate 表过滤
# property=replicator.filter.replicate.do=PUBLIC.*
# property=replicator.filter.replicate.ignore=PUBLIC.T1
## casetransform 大小写转换
# property=replicator.filter.casetransform.to_upper_case=true
property=replicator.filter.replicate.do=exam.*
## 优化参数
# 达到多少条记录后，分事务，默认 500
property=replicator.extractor.dbms.maxRowsByBlock=500
# 若多少秒没有新的事务产生，那么就直接返回该事务，不再等待合并
property=replicator.extractor.dbms.maxWaitingTime=2
# 大事务优化参数
property=replicator.extractor.dbms.minRowsPerBlock=4000
# 自动恢复尝试次数，系统 OFFLINE 时，尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts=0
# 系统 OFFLINE 时，尝试 ONLINE 之前的等待时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval=30s
# 系统自动恢复次数重置时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval=300s
(4).执行安装
bash
复制代码
[flysync@node1 ini]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
..
WARN >> node1 >> ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN >> node1 >> Linux swappiness is currently set to 30, on restart it will be 1, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
...License : /home/flysync/replicator/license.dat
.
#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the
cluster will be available.
/home/flysync/replicator/flysync/cluster-home/bin/startall
Wait a minute for the services to start up and configure themselves. After
that you may proceed.
We have added FlySync environment variables to ~/.bash_profile.
Run `source ~/.bash_profile` to rebuild your environment.
Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server.
/home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl services
Configuration is now complete. For further information, please consult
FlySync documentation.
NOTE >> Command successfully completed
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$
(5).上传并更新同步软件授权
bash
复制代码
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ cp /install/license.dat ~/replicator/license.dat
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls ~/replicator/license.dat
/home/flysync/replicator/license.dat
(6).让环境变量生效
bash
复制代码
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$
(7).配置源端过滤策略
bash
复制代码
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ vi ~/flysync.ini
cat ~/flysync.ini | grep -E "svc_extractor_filters|replicate.do"
# svc_extractor_filters=replicate,casetransform,dropcolumn,rename
svc_extractor_filters=replicate
# property=replicator.filter.replicate.do=PUBLIC.*
property=replicator.filter.replicate.do=exam.*
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$
(8).启动同步程序并置于offline 状态
bash
复制代码
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:23406
(9).查看同步软件的运行状态
bash
复制代码
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service kingbase8_source services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : master
serviceName : kingbase8_source
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$
4.node2节点部署replicator同步程序
(1).上传并解压replicator同步程序
bash
复制代码
[root@node2 ~]# su flysync
[flysync@node2 root]$ cd /install/
[flysync@node2 install]$ ls KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysync@node2 install]$ tar -zxf KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysync@node2 install]$ ls KingbaseFlySync-V002R002C004B20230320-replicator
cluster-home extensions flysync-replicator tools
[flysync@node2 install]$
(2).拷贝驱动包
bash
复制代码
[flysync@node2 install]$ cd KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/lib
[flysync@node2 lib]$ cp /install/KingbaseFlySync-V002R002C004B20230320-replicator/extensions/jdbc/kingbase8-8.6.0.jar .
[flysync@node2 lib]$ ls -l |grep kingbase8-8.6.0.jar
-rw-rw-r-- 1 flysync flysync 1019069 11月 16 22:30 kingbase8-8.6.0.jar
[flysync@node2 lib]$
(3).配置静默安装文件
bash
复制代码
[flysync@node2 lib]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysync@node2 ini]$ cp flysync_kingbase8_target.ini ~/flysync.ini
[flysync@node2 ini]$ vi ~/flysync.ini
[flysync@node2 ini]$ cat ~/flysync.ini
[defaults]
# 安装目录
install-directory=~/replicator
# 环境变量脚本位置
profile-script=~/.bash_profile
# rmi端口,默认11000
rmi-port=11000
# 服务名
[kingbase8_target]
# 监听kufl获取的端口
kufl-port=3112
# master主机名
master=node1
# 当前主机名
members=node2
##指定连接master监听的kufl端口
master-kufl-port=3112
# 数据源类型
datasource-type=kingbase
# 同步数据库信息
replication-host=node2
replication-user=flysync
replication-password=flysync
replication-port=54321
# 同步服务角色
role=slave
## 指定连接的数据库名称
kingbase-dbname=test
## kingbase 版本号
datasource-version=8
## 过滤器开关配置
# svc-remote-filters=casetransform,dropcolumn,rename
svc-remote-filters=rename
## 集群参数
# property=replicator.datasource.global.connectionSpec.use_cluster=true
# property=replicator.datasource.global.connectionSpec.slave_add=192.168.28.176,192.168.28.176
# property=replicator.datasource.global.connectionSpec.slave_port=1521,1521
### 过滤器详细配置
## replicate 表过滤
# property=replicator.filter.replicate.do=PUBLIC.*
# property=replicator.filter.replicate.ignore=PUBLIC.T1
## casetransform 大小写转换
# property=replicator.filter.casetransform.to_upper_case=true
## 优化参数
# 开启小事务合并入库
property=replicator.applier.dbms.optimizeRowEvents=true
# 指定单表一次入库的数据量
property=replicator.applier.dbms.maxRowBatchSize=5000
# 自动恢复尝试次数，系统 OFFLINE 时，尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts=0
# 系统 OFFLINE 时，尝试 ONLINE 之前的等待时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval=30s
# 系统自动恢复次数重置时间，s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval=300s
[flysync@node2 ini]$
(4).执行安装
bash
复制代码
[flysync@node2 ini]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
..
WARN >> node2 >> ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN >> node2 >> Linux swappiness is currently set to 30, on restart it will be 0, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
License : /home/flysync/replicator/license.dat
.
#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the
cluster will be available.
/home/flysync/replicator/flysync/cluster-home/bin/startall
Wait a minute for the services to start up and configure themselves. After
that you may proceed.
We have added FlySync environment variables to ~/.bash_profile.
Run `source ~/.bash_profile` to rebuild your environment.
Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server.
/home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl services
Configuration is now complete. For further information, please consult
FlySync documentation.
NOTE >> Command successfully completed
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$
(5).上传并更新同步软件授权
bash
复制代码
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cp /install/license.dat ~/replicator/license.dat
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls ~/replicator/license.dat
/home/flysync/replicator/license.dat
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$
(6).让环境变量生效
bash
复制代码
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$
(7).配置映射策略
- 将exam 模式下的增量数据映射到exam0017下。
- 将exam.course 表的增量数据映射到 exam0017.course0017下。
bash
复制代码
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ vi ~/replicator/filters-config/rename.csv
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cat ~/replicator/filters-config/rename.csv
#######################################
# RenameFilter sample definition file #
#######################################
# CSV format:
#
# originalSchema,originalTable,originalColumn,newSchema,newTable,newColumn
#
# Left side (first three columns) may contain a special character:
#
# * - match all [schemas/tables/columns] (part-string matching not supported)
#
# Right side (last three columns) may contain a special character:
#
# - - leave as original
# In case of multiple entries matching the same request, order of preference is
# important. Consider the following example:
#
# schemac,*,*,schemac1,-,-
# schemac,tablew,*,schemac2,-,-
#
# In this example all tables are moved to "schemac1" except "tablew", which is
# moved to "schemac2". Here's the rules being followed:
#
# Order of preference when looking up schema name:
#
# 1. schema.table
# 2. schema.*
#
# Order of preference when looking up table name:
#
# 1. schema.table
# 2. *.table
#
# Order of preference when looking up column name:
#
# 1. schema.table
# 2. schema.*
# 3. *.table
# 4. *.*
#
# By the way, order of preference is fixed in order to have a fast lookup.
# Notes:
#
# * Case sensitivity matters. If it is required to ignore case, use another
# filter in the pipeline before to transform case.
#
# * For renaming columns, make sure to have column names in KUFL events. For
# some DBMS types a separate filter (eg. ColumnNameFilter) might be needed.
############
# Examples #
############
# Rename schema for all tables.
schemac,*,*,schemac1,-,-
# This table moved to a different schema.
schemac,tablew,*,schemac2,-,-
# Table renamed and moved to a different schema.
schemav,tablee,*,schemav1,tableee,-
# Rename table in a particular schema.
schemab,tabler,*,-,tabler1,-
# Rename each table with a given name across all schemas.
*,tablet,*,-,tablet1,-
# Rename column of this particular table.
schemaz,tableq,cola,-,-,colaa # Column renaming doesn't change schema/table.
# Rename column of each table with a given name across all schemas.
*,tablew,colb,-,-,colbb
# Rename column occurring in any table across all schemas.
*,*,colc,-,-,colcc
# Rename column across all tables in the given schema.
schemax,*,cold,-,-,coldd
# Renamed schema, but one table is left in it and some of its columns renamed.
scheman,*,*,scheman2,-,-
scheman,tabley,*,-,tabley2,- # Leave table in original schema.
scheman,tabley,colj,-,-,colj2
scheman,tabley,colk,-,-,colk2
# Renamed schema and one table with one column in it.
schemam,*,*,schemam2,-,-
schemam,tableu,*,schemam2,tableu2,- # Move renamed table to new schema too.
schemam,tableu,colj,-,-,colj2 # Column renaming doesn't change schema/table.
# --以下为新增内容--
exam,*,*,exam0017,-,-
exam,course,*,exam0017,course0017,-
flysync_kingbase8_source,trep_commit_seqno,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,consistency,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,heartbeat,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,trep_shard,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,trep_shard_channel,*,flysync_kingbase8_target,-,-
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$
(8).启动同步程序并置于offline 状态
bash
复制代码
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:53248
(9).查看同步软件的运行状态
bash
复制代码
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : slave
serviceName : kingbase8_target
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysync@node2 KingbaseFlySync-V002R002C004B20230320-replicator]$
5.KFS存量数据搬迁
(1)检查状态
bash
复制代码
[flysync@node1 ~]$ fsrepctl -service kingbase8_source services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : master
serviceName : kingbase8_source
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysync@node1 ~]$
bash
复制代码
[flysync@node2 ~]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : slave
serviceName : kingbase8_target
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysync@node2 ~]$
ddlscan 和loader迁移模式选其中一种就好
(2).ddlscan结构搬迁
a.新建模式exam0017
bash
复制代码
[root@node2 ~]# su kingbase
[kingbase@node2 root]$ exit
exit
[root@node2 ~]# su - kingbase
上一次登录：四 11月 16 22:58:52 CST 2023pts/0 上
[kingbase@node2 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node2 bin]$ ./ksql -U flysync -d test
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=# CREATE SCHEMA exam0017;
CREATE SCHEMA
test=# \q
[kingbase@node2 bin]$
b.开始迁移数据
bash
复制代码
[kingbase@node2 bin]$ exit
[root@node2 ~]# su - flysync
上一次登录：四 11月 16 23:04:05 CST 2023pts/0 上
[flysync@node2 ~]$ ddlscan -target.service kingbase8_target -source.user flysync -source.pass flysync -source.db test -source.dbtype kingbase8 -source.host node1 -source.port 54321 -source.schema exam -target.db test -mgType 0
| [ - main] INFO ddlscan.DDLScanWorker Tables or TableFile not specified - extracting everything!
| [ - main] INFO ddlscan.DDLScanWorker Connect to source database with url: jdbc:kingbase8://node1:54321/test
| [ - main] INFO database.AbstractDatabase Loading database driver: com.kingbase8.Driver
| [ - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
| [ - main] INFO filter.RenameDefinitions schemac.*.* -> schemac1.-.-
| [ - main] INFO filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
| [ - main] INFO filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
| [ - main] INFO filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
| [ - main] INFO filter.RenameDefinitions *.tablet.* -> -.tablet1.-
| [ - main] INFO filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
| [ - main] INFO filter.RenameDefinitions *.tablew.colb -> -.-.colbb
| [ - main] INFO filter.RenameDefinitions *.*.colc -> -.-.colcc
| [ - main] INFO filter.RenameDefinitions schemax.*.cold -> -.-.coldd
| [ - main] INFO filter.RenameDefinitions scheman.*.* -> scheman2.-.-
| [ - main] INFO filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
| [ - main] INFO filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
| [ - main] INFO filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
| [ - main] INFO filter.RenameDefinitions schemam.*.* -> schemam2.-.-
| [ - main] INFO filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
| [ - main] INFO filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
| [ - main] INFO filter.RenameDefinitions exam.*.* -> exam0017.-.-
| [ - main] INFO filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
| [ - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
| [ - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
| [ - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
| [ - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
| [ - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
| [ - main] INFO ddlscan.DDLScanWorker There is 5 tables to migration.
| [ - main] INFO ddlscan.TargetDDLScanWorker Connect to target database with url: jdbc:kingbase8://node2:54321/test
| [ - main] INFO ddlscan.TargetDDLScanWorker Drop tables on target database.
| [ - main] INFO ddlscan.DDLScanCtrl
--------------------------------------------------------------------------------
TOTAL: 5
SUCCESS: 5
FAILURE: 0
TOTAL TIME: 1s
FINISHED AT: 2023-11-16 23:04:18
OUT: /tmp/ddlscan_20231116_230417.sql
ERROR:
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+
|NUM |SOURCE_TABLE |TARGET_TABLE |STATUS |MESSAGE |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+
|1 |exam.course |exam0017.course0017 |Success | |
|2 |exam.instructor |exam0017.instructor |Success | |
|3 |exam.instructor_history |exam0017.instructor_history |Success | |
|4 |exam.score |exam0017.score |Success | |
|5 |exam.student |exam0017.student |Success | |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------------------------------------+
[flysync@node2 ~]$
(3).loader数据搬迁(极速模式)
bash
复制代码
[flysync@node2 ~]$ loader -source.service kingbase8_source -source.user flysync -source.pass flysync \
> -source.db test -source.dbtype kingbase8 -source.host node1 \
> -source.port 54321 -source.schema exam \
> -source.rmiHost node1 -source.rmiPort 11000 \
> -target.rmiHost node2 -target.rmiPort 11000 \
> -target.service kingbase8_target -mgType 0 -clean
| [ - main] INFO database.AbstractDatabase Loading database driver: com.kingbase8.Driver
| [ - main] INFO loader.LoaderWorker Tables not specified - extracting everything!
| [ - main] INFO loader.LoaderWorker Import table exam.course
| [ - main] INFO loader.LoaderWorker Import table exam.instructor
| [ - main] INFO loader.LoaderWorker Import table exam.instructor_history
| [ - main] INFO loader.LoaderWorker Import table exam.score
| [ - main] INFO loader.LoaderWorker Import table exam.student
| [ - main] INFO conf.ReplicatorRuntime Replicator role: slave
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
| [kingbase8_target - main] INFO pipeline.Pipeline Configuring pipeline: slave
| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: name=global
| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.KingbaseDataSource
| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: name=extractor
| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: name=file_metadata
| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
| [kingbase8_target - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave
| [kingbase8_target - main] INFO pipeline.Pipeline Preparing pipeline: slave
| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: name=file_metadata
| [kingbase8_target - main] INFO datasource.FileDataSource Service directory does not exist, creating: /home/flysync/replicator/metadata/applier/kingbase8_target
| [kingbase8_target - main] INFO datasource.FileDataSource Initializing data source files: service=kingbase8_target directory=/home/flysync/replicator/metadata/applier
| [kingbase8_target - main] INFO datasource.FileCommitSeqno Initializing file-based seqno tracking: directory=/home/flysync/replicator/metadata/applier/kingbase8_target file=commitseqno.0
| [kingbase8_target - pool-1-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave
| [kingbase8_target - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: name=global
| [kingbase8_target - main] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: name=global
| [kingbase8_target - main] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: name=global
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms
| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* -> schemac1.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* -> -.tablet1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb -> -.-.colbb
| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc -> -.-.colcc
| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold -> -.-.coldd
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* -> scheman2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* -> schemam2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* -> exam0017.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source
| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave
| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position
| [kingbase8_target - main] INFO pipeline.Pipeline Starting pipeline: slave
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 6
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 6
| [kingbase8_target - pool-1-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave
| [kingbase8_target - main] INFO pipeline.Pipeline Shutting down pipeline: slave
| [kingbase8_target - main] INFO pipeline.Pipeline Releasing pipeline: slave
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Replicator role: slave
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
| [kingbase8_target - main] INFO pipeline.Pipeline Configuring pipeline: slave
| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: name=global
| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.KingbaseDataSource
| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: name=extractor
| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: name=file_metadata
| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
| [kingbase8_target - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave
| [kingbase8_target - main] INFO pipeline.Pipeline Preparing pipeline: slave
| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: name=file_metadata
| [kingbase8_target - main] INFO datasource.FileDataSource Initializing data source files: service=kingbase8_target directory=/home/flysync/replicator/metadata/applier
| [kingbase8_target - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: name=global
| [kingbase8_target - main] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: name=global
| [kingbase8_target - main] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: name=global
| [kingbase8_target - pool-5-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms
| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* -> schemac1.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* -> -.tablet1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb -> -.-.colbb
| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc -> -.-.colcc
| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold -> -.-.coldd
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* -> scheman2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* -> schemam2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* -> exam0017.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source
| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave
| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position
| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* -> schemac1.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* -> -.tablet1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb -> -.-.colbb
| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc -> -.-.colcc
| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold -> -.-.coldd
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* -> scheman2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* -> schemam2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* -> exam0017.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source
| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave
| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-1] Last event data not available; Setting extractor to current position
| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* -> schemac1.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* -> -.tablet1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb -> -.-.colbb
| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc -> -.-.colcc
| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold -> -.-.coldd
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* -> scheman2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* -> schemam2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* -> exam0017.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source
| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave
| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-2] Last event data not available; Setting extractor to current position
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position
| [kingbase8_target - main] INFO pipeline.Pipeline Starting pipeline: slave
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 6
| [kingbase8_target - main-q-to-dbms-0] INFO applier.JdbcApplier nologging: exam0017.course0017
| [kingbase8_target - main-q-to-dbms-0] INFO applier.JdbcApplier nologging: exam0017.score
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno=(unavailable)
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 3
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - main-q-to-dbms-1] INFO applier.JdbcApplier nologging: exam0017.instructor
| [kingbase8_target - main-q-to-dbms-2] INFO applier.JdbcApplier nologging: exam0017.instructor_history
| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - main-q-to-dbms-1] INFO applier.JdbcApplier nologging: exam0017.student
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno=(unavailable)
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task event count: 3
| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task event count: 2
| [kingbase8_target - pool-5-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave
| [kingbase8_target - main] INFO pipeline.Pipeline Shutting down pipeline: slave
| [kingbase8_target - main] INFO pipeline.Pipeline Releasing pipeline: slave
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms
| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Replicator role: slave
| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true
| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Configuring pipeline: slave
| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Configuring data source: name=global
| [kingbase8_target - pipeline-0] INFO datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.KingbaseDataSource
| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Configuring data source: name=extractor
| [kingbase8_target - pipeline-0] INFO datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Configuring data source: name=file_metadata
| [kingbase8_target - pipeline-0] INFO datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
| [kingbase8_target - pipeline-0] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false
| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id
| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms
| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave
| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Preparing pipeline: slave
| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Preparing and initializing data source: name=file_metadata
| [kingbase8_target - pipeline-0] INFO datasource.FileDataSource Initializing data source files: service=kingbase8_target directory=/home/flysync/replicator/metadata/applier
| [kingbase8_target - pipeline-0] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Preparing and initializing data source: name=global
| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: name=global
| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: name=global
| [kingbase8_target - pool-11-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave
| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemac.*.* -> schemac1.-.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions *.tablet.* -> -.tablet1.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions *.tablew.colb -> -.-.colbb
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions *.*.colc -> -.-.colcc
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemax.*.cold -> -.-.coldd
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.*.* -> scheman2.-.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemam.*.* -> schemam2.-.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions exam.*.* -> exam0017.-.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - pipeline-0] INFO applier.JdbcApplier Connecting to data source
| [kingbase8_target - pipeline-0] INFO applier.JdbcApplier Suppressing logging on privileged slave
| [kingbase8_target - pipeline-0] WARN pipeline.StageTaskGroup [pipeline-0-q-to-dbms-0] Last event data not available; Setting extractor to current position
| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
| [kingbase8_target - pipeline-0] WARN pipeline.StageTaskGroup [pipeline-0-loader-to-q-0] Last event data not available; Setting extractor to current position
| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Starting pipeline: slave
| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "exam"."course":0-3
| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "exam"."instructor":0-7
| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "exam"."instructor_history":0-1
| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "exam"."score":0-15
| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from "exam"."student":0-8
| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 6
| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 6
| [kingbase8_target - pool-11-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave
| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Shutting down pipeline: slave
| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Releasing pipeline: slave
| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms
| [kingbase8_target - main] INFO loader.TargetLoaderWorker Tables imported
| [kingbase8_target - main] INFO loader.LoaderWorker | -------------------------打印执行结果开始-------------------------
| [kingbase8_target - main] INFO loader.LoaderWorker | exam.instructor_history : success
| [kingbase8_target - main] INFO loader.LoaderWorker | exam.course : success
| [kingbase8_target - main] INFO loader.LoaderWorker | exam.student : success
| [kingbase8_target - main] INFO loader.LoaderWorker | exam.score : success
| [kingbase8_target - main] INFO loader.LoaderWorker | exam.instructor : success
| [kingbase8_target - main] INFO loader.LoaderWorker | -------------------------打印执行结果结束-------------------------
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Replicator role: slave
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn
| [kingbase8_target - main] INFO pipeline.Pipeline Configuring pipeline: slave
| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: name=global
| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: name=global className=com.kingbase.flysync.replicator.datasource.KingbaseDataSource
| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: name=extractor
| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: name=extractor className=com.kingbase.flysync.replicator.datasource.AliasDataSource
| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: name=file_metadata
| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: name=file_metadata className=com.kingbase.flysync.replicator.datasource.FileDataSource
| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings
| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter
| [kingbase8_target - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms
| [kingbase8_target - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave
| [kingbase8_target - main] INFO pipeline.Pipeline Preparing pipeline: slave
| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: name=file_metadata
| [kingbase8_target - main] INFO datasource.FileDataSource Initializing data source files: service=kingbase8_target directory=/home/flysync/replicator/metadata/applier
| [kingbase8_target - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files=1 channels=1
| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: name=global
| [kingbase8_target - main] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: name=global
| [kingbase8_target - main] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: name=global
| [kingbase8_target - pool-15-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms
| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* -> schemac1.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* -> -.tablet1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb -> -.-.colbb
| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc -> -.-.colcc
| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold -> -.-.coldd
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* -> scheman2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* -> schemam2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* -> exam0017.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source
| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave
| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position
| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* -> schemac1.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* -> -.tablet1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb -> -.-.colbb
| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc -> -.-.colcc
| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold -> -.-.coldd
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* -> scheman2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* -> schemam2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* -> exam0017.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source
| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave
| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-1] Last event data not available; Setting extractor to current position
| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* -> schemac1.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* -> schemac2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* -> schemav1.tableee.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* -> -.tabler1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* -> -.tablet1.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola -> -.-.colaa
| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb -> -.-.colbb
| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc -> -.-.colcc
| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold -> -.-.coldd
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* -> scheman2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* -> -.tabley2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk -> -.-.colk2
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* -> schemam2.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* -> schemam2.tableu2.-
| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj -> -.-.colj2
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* -> exam0017.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* -> exam0017.course0017.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* -> flysync_kingbase8_target.-.-
| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source
| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave
| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-2] Last event data not available; Setting extractor to current position
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter
| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position
| [kingbase8_target - main] INFO pipeline.Pipeline Starting pipeline: slave
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 6
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno=(unavailable)
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 3
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno=(unavailable)
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task event count: 3
| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Starting stage task thread
| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task has been cancelled
| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread
| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno=5 eventid=NIL
| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task event count: 2
| [kingbase8_target - pool-15-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave
| [kingbase8_target - main] INFO pipeline.Pipeline Shutting down pipeline: slave
| [kingbase8_target - main] INFO pipeline.Pipeline Releasing pipeline: slave
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q
| [kingbase8_target - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter
| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms
| [kingbase8_target - main] INFO loader.LoaderCtrl Total time:2
| [kingbase8_target - main] INFO loader.LoaderCtrl
--------------------------------------------------------------------------------
TOTAL: 5
SUCCESS: 5
FAILURE: 0
TOTAL TIME: 2s
FINISHED AT: 2023-11-16 23:07:02
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+
|NUM |SOURCE_TABLE |TARGET_TABLE |STATUS |TOTAL |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+
|1 |exam.score |exam0017.score |Success |15 |
|2 |exam.student |exam0017.student |Success |8 |
|3 |exam.instructor_history |exam0017.instructor_history |Success |1 |
|4 |exam.course |exam0017.course0017 |Success |3 |
|5 |exam.instructor |exam0017.instructor |Success |7 |
+-----+--------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+--------+----------+
[flysync@node2 ~]$
6.增量数据同步
(1).node1节点启动同步服务，数据开始同步
bash
复制代码
[flysync@node1 ~]$ fsrepctl -service kingbase8_source online
[flysync@node1 ~]$ fsrepctl -service kingbase8_source services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: 0
appliedLatency : 0.115
role : master
serviceName : kingbase8_source
serviceType : local
started : true
state : ONLINE
Finished services command...
[flysync@node1 ~]$
(2).node2节点启动同步服务，数据开始同步
bash
复制代码
[flysync@node2 ~]$ fsrepctl -service kingbase8_target online
[flysync@node2 ~]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: 0
appliedLatency : 99.334
role : slave
serviceName : kingbase8_target
serviceType : local
started : true
state : ONLINE
Finished services command...
[flysync@node2 ~]$
(3).同步进程状态：查看同步是否正常。
a.查看node1同步状态
bash
复制代码
[flysync@node1 ~]$ fsrepctl -service kingbase8_source status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : kb:31973688:31973840
appliedLastSeqno : 0
appliedLatency : 0.115
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : kingbase8_source
currentEventId : kb:31973688:31974112
currentTimeMillis : 1700147650924
dataServerHost : node1
extensions :
host : node1
latestEpochNumber : 0
masterConnectUri : kufl://localhost:/
masterListenUri : kufl://node1:3112/
maximumStoredSeqNo : 0
minimumStoredSeqNo : 0
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : UNKNOWN
relativeLatency : 234.457
resourcePrecedence : 99
rmiPort : 11000
role : master
seqnoType : java.lang.Long
serviceName : kingbase8_source
serviceType : local
simpleServiceName : kingbase8_source
siteName : default
sourceId : node1
state : ONLINE
timeInStateSeconds : 234.437
timezone : GMT+08:00
transitioningTo :
uptimeSeconds : 3541.458
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@node1 ~]$
b.查看node2同步状态
bash
复制代码
[flysync@node2 ~]$ fsrepctl -service kingbase8_target status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : kb:31973688:31973840
appliedLastSeqno : 0
appliedLatency : 99.334
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : kingbase8_target
currentEventId : NONE
currentTimeMillis : 1700147694828
dataServerHost : node2
extensions :
host : node2
latestEpochNumber : 0
masterConnectUri : kufl://node1:3112/
masterListenUri : kufl://node2:3112/
maximumStoredSeqNo : 0
minimumStoredSeqNo : 0
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : kufl://node1:3112/
relativeLatency : 278.361
resourcePrecedence : 99
rmiPort : 11000
role : slave
seqnoType : java.lang.Long
serviceName : kingbase8_target
serviceType : local
simpleServiceName : kingbase8_target
siteName : default
sourceId : node2
state : ONLINE
timeInStateSeconds : 179.158
timezone : GMT+08:00
transitioningTo :
uptimeSeconds : 1304.402
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysync@node2 ~]$
(4).数据同步验证
a.登录node1的数据库生成数据
bash
复制代码
[flysync@node1 ~]$ exit
exit
[root@node1 install]# su - kingbase
上一次登录：四 11月 16 18:09:03 CST 2023pts/1 上
[kingbase@node1 ~]$ cd /KingbaseES/V8/
ClientTools/ desktops/ DtsTools/ Help/ Interface/ jre/ Logs/ ManagerTools/ Scripts/ Server/ Uninstall/ xulrunner/
[kingbase@node1 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node1 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=# \d exam.course;
数据表 "exam.course"
栏位 | 类型 | 校对规则 | 可空的 | 预设
-----------+----------------------------+----------+----------+------
cno | numeric(2,0) | | not null |
cname | character varying(30 char) | | not null |
clevel | character varying(13 char) | | |
pass_mark | numeric(5,0) | | not null |
索引：
"cno_pk" PRIMARY KEY, btree (cno)
检查约束限制
"pass_mark_ck" CHECK (pass_mark > 0::numeric)
由引用：
TABLE "exam.score" CONSTRAINT "cno_fk" FOREIGN KEY (cno) REFERENCES exam.course(cno)
test=# INSERT INTO exam.course VALUES(40,'KFS','middle',80);
INSERT 0 1
test=# INSERT INTO exam.instructor VALUES(108,'fenxiaoyue','Intermediate Instructor');
INSERT 0 1
test=#
b.登录node2的数据库查询刚才在node1插入的数据
bash
复制代码
[flysync@node2 ~]$ exit
登出
[root@node2 ~]# su - kingbase
上一次登录：四 11月 16 23:03:45 CST 2023pts/0 上
[kingbase@node2 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbase@node2 bin]$ ./ksql -U system test
ksql (V8.0)
输入 "help" 来获取帮助信息.
test=# \d exam0017.course0017;
数据表 "exam0017.course0017"
栏位 | 类型 | 校对规则 | 可空的 | 预设
-----------+----------------------------+----------+----------+------
cno | numeric(2,0) | | not null |
cname | character varying(30 char) | | not null |
clevel | character varying(13 char) | | |
pass_mark | numeric(5,0) | | not null |
索引：
"course0017_pkey" PRIMARY KEY, btree (cno)
test=#SELECT * FROM exam0017.course0017 WHERE cno=40;
cno | cname | clevel | pass_mark
-----+-------+--------+-----------
40 | KFS | middle | 80
(1 行记录)
test=# SELECT * FROM exam0017.instructor WHERE ino=108;
ino | iname | title
-----+------------+-------------------------
108 | fenxiaoyue | Intermediate Instructor
(1 行记录)
test=#
到这里我们的本次教程就结束了
报错处理
1.启动报错
bash
复制代码
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ find /home/flysync/replicator/ -name *.log
/home/flysync/replicator/service_logs/replicator-user.log
/home/flysync/replicator/service_logs/xtrabackup.log
/home/flysync/replicator/service_logs/mysqldump.log
/home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/replicator-user.log
/home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/fsrepsvc.log
/home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/failure_record_file.log
[flysync@node1 KingbaseFlySync-V002R002C004B20230320-replicator]$ tail -100 /home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/fsrepsvc.log