系列文档参考 MYSQL系列-整体架构介绍
详细设计
表模型改动
当前表结构如下
point_shard1 | point_shard2 | 分库分表字段 | |
---|---|---|---|
point_balance | point_balance1 point_balance2 | point_balance1 point_balance2 | uid(long) |
point_balance | point_balance1 point_balance2 | point_balance1 point_balance2 | uid(long) |
point_balance_his | point_balance_his1 point_balance_his2 | point_balance_his1 point_balance_his2 | uid(long) |
coupon_info | coupon_info1 coupon_info2 | coupon_info1 coupon_info2 | coupon_code(String) |
上述改动主要是将不同类型分库分表、以及配置表落在单库都涉及到
相关表结构如下
sql
DROP TABLES IF EXISTS point_balance_his1;
CREATE TABLE IF NOT EXISTS `point_balance_his1` (
`id` BIGINT(20) NOT NULL AUTO_INCREMENT COMMENT '主键ID',
`uid` BIGINT(20) NOT NULL COMMENT 'uid',
`point_sum` BIGINT(20) NOT NULL COMMENT '积分总和',
`create_time` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`country` VARCHAR(20) NOT NULL DEFAULT 'CN' COMMENT '国家',
PRIMARY KEY (`id`),
INDEX idx_uid(`uid`)
) ENGINE=INNODB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COMMENT='用户积分变动历史表1';
DROP TABLES IF EXISTS point_balance_his2;
CREATE TABLE IF NOT EXISTS `point_balance_his2` (
`id` BIGINT(20) NOT NULL AUTO_INCREMENT COMMENT '主键ID',
`uid` BIGINT(20) NOT NULL COMMENT 'uid',
`point_sum` BIGINT(20) NOT NULL COMMENT '积分总和',
`create_time` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`country` VARCHAR(20) NOT NULL DEFAULT 'CN' COMMENT '国家',
PRIMARY KEY (`id`),
INDEX idx_uid(`uid`)
) ENGINE=INNODB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COMMENT='用户积分变动历史表2';
DROP TABLE IF EXISTS `coupon_info1`;
CREATE TABLE coupon_info1 (
`id` INT AUTO_INCREMENT NOT NULL COMMENT '自增ID',
`coupon_code` VARCHAR (20) NOT NULL COMMENT '券码CODE',
`rev_uid` BIGINT(20) DEFAULT NULL COMMENT '领券UID',
`create_time` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`modify_time` TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
`country` VARCHAR (20) NOT NULL DEFAULT 'CN' COMMENT '国家',
PRIMARY KEY (`id`),
INDEX idx_coupon_code_country (coupon_code, country)
) ENGINE = INNODB AUTO_INCREMENT = 1 DEFAULT CHARSET = utf8 COMMENT = '券码表1' ;
DROP TABLE IF EXISTS `coupon_info2`;
CREATE TABLE coupon_info2 (
`id` INT AUTO_INCREMENT NOT NULL COMMENT '自增ID',
`coupon_code` VARCHAR (20) NOT NULL COMMENT '券码CODE',
`rev_uid` BIGINT(20) DEFAULT NULL COMMENT '领券UID',
`create_time` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`modify_time` TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
`country` VARCHAR (20) NOT NULL DEFAULT 'CN' COMMENT '国家',
PRIMARY KEY (`id`),
INDEX idx_coupon_code_country (coupon_code, country)
) ENGINE = INNODB AUTO_INCREMENT = 1 DEFAULT CHARSET = utf8 COMMENT = '券码表2' ;
maven引用
主要涉及将sharding-jdbc等引用进来
注意,本期是基于sharding-jdbc 5.2来进行落地实践的
xml
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<version>2.7.9</version>
<exclusions>
<exclusion>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
<version>2.7.9</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.49</version>
</dependency>
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid</artifactId>
<version>1.2.16</version>
</dependency>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjrt</artifactId>
<version>1.9.5</version>
</dependency>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>1.9.5</version>
</dependency>
<dependency>
<groupId>cglib</groupId>
<artifactId>cglib-nodep</artifactId>
<version>3.2.10</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<version>2.7.9</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.hyw.study</groupId>
<artifactId>com.toby.logback</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc-core-spring-boot-starter</artifactId>
<version>5.2.0</version>
</dependency>
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>1.33</version>
</dependency>
<dependency>
<groupId>org.glassfish.jaxb</groupId>
<artifactId>jaxb-runtime</artifactId>
<version>2.3.8</version>
</dependency>
</dependencies>
关键技术点实现
整体框架
在META-INF
新建sharding-databases-tables.yaml配置文件,主要将druid数据源配置、分库分表算法、读写分离等放进去 然后使用JAVA代码加载创建datasource
java
@Configuration
public class ShardingSphereConfig {
@Bean
public DataSource dataSource() throws IOException, SQLException {
DataSource dataSource = YamlShardingSphereDataSourceFactory.createDataSource(
getFile("classpath:META-INF/sharding-databases-tables.yaml"));
return dataSource;
}
}
mybatics相关配置如下 application.yml
配置
yaml
spring:
main:
allow-bean-definition-overriding: true
mybatis:
config-location: classpath:sqlmap-config.xml
sqlmap-config.xml
如下
java
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE configuration
PUBLIC "-//mybatis.org//DTD Config 3.0//EN"
"http://mybatis.org/dtd/mybatis-3-config.dtd">
<configuration>
<settings>
<!-- 开启驼峰命名自动映射 -->
<setting name="mapUnderscoreToCamelCase" value="true"/>
<!-- 开启二级缓存 -->
<setting name="cacheEnabled" value="true"/>
<!-- 打印 SQL 语句 -->
<setting name="logImpl" value="STDOUT_LOGGING"/>
</settings>
</configuration>
Mapper文件本次采用注解的方式,去掉xml配置,具体如下:
java
public interface CouponInfoMapper {
@Insert("insert into coupon_info (coupon_code, rev_uid, country)\n" +
" values (#{couponCode}, #{recUid}, #{country})\n")
int addCouponCode(CouponInfo couponInfo);
@Select("select id,\n" +
" coupon_code,\n" +
" rev_uid,\n" +
" create_time,\n" +
" modify_time,\n" +
" country" +
" from coupon_info where coupon_code=#{couponCode} and country=#{country}")
List<CouponInfo> selectByCouponCode(@Param("couponCode") String couponCode, @Param("country") String country);
}
业务表point_balance1基于UID进行分库分表
yml文件中配置数据源
yml
dataSources:
ds_1:
dataSourceClassName: com.alibaba.druid.pool.DruidDataSource
driverClassName: com.mysql.jdbc.Driver
url: jdbc:mysql://172.17.108.172:3306/point_shard1?autoReconnect=true&characterEncoding=UTF-8&useUnicode=true&connectTimeout=3000&socketTimeout=3000
username: root
password: 1Qaz1Qaz
ds_2:
dataSourceClassName: com.alibaba.druid.pool.DruidDataSource
driverClassName: com.mysql.jdbc.Driver
url: jdbc:mysql://172.17.108.172:3306/point_shard2?autoReconnect=true&characterEncoding=UTF-8&useUnicode=true&connectTimeout=3000&socketTimeout=3000
username: root
password: 1Qaz1Qaz
主要druid是url,连接池是HikariCP的化是jdbcUrl
配置分库分表,具体值含义参考官网-数据源配置
yml
- !SHARDING
tables:
point_balance:
actualDataNodes: ds_${1..4}.point_balance${1..2}
databaseStrategy:
standard:
shardingColumn: uid
shardingAlgorithmName: d_uid_inline
tableStrategy:
standard:
shardingColumn: uid
shardingAlgorithmName: t_uid_inline
auditStrategy:
auditorNames:
- sharding_key_required_auditor
allowHintDisable: true
d_uid_inline
是分库算法
yml
shardingAlgorithms:
d_uid_inline:
type: CLASS_BASED
props:
algorithmClassName: com.toby.sharding.jdbc.source.start.db.sharding.UIDDbSharding
strategy: STANDARD
具体的分库逻辑在com.toby.sharding.jdbc.source.start.db.sharding.UIDDbSharding
java
public abstract class AbstractLongShardingAlgorithm implements StandardShardingAlgorithm<Long> {
Properties properties;
@Override
public String doSharding(Collection<String> collection, PreciseShardingValue<Long> preciseShardingValue) {
String name = routeTable(collection, getLongFromObject(preciseShardingValue.getValue()));
if (Objects.isNull(name)) {
throw new UnsupportedOperationException("cannot not find rout for" + preciseShardingValue);
}
//返回需要分库的yml中配置名字
return name;
}
@Override
public Collection<String> doSharding(Collection<String> collection, RangeShardingValue<Long> rangeShardingValue) {
Range<Long> range = rangeShardingValue.getValueRange();
Set<String> names = new HashSet<>();
for (long start = range.lowerEndpoint(); start <= range.upperEndpoint(); start++) {
String name = routeTable(collection, start);
if (Objects.isNull(name)) {
throw new UnsupportedOperationException("cannot not find range rout for " + start);
}
names.add(name);
}
return names;
}
@Override
public Properties getProps() {
return properties;
}
@Override
public void init(Properties properties) {
this.properties = properties;
}
private long getLongFromObject(Object obj) {
long value = -1L;
try {
if (obj instanceof Integer) {
value = ((Integer) obj).longValue();
} else if (obj instanceof BigInteger) {
value = ((BigInteger) obj).longValue();
} else if (obj instanceof String) {
value = Long.parseLong((String) obj);
} else {
value = Long.parseLong(obj.toString());
}
} catch (Exception e) {
//ingore
}
return value;
}
private String routeTable(Collection<String> collection, long value) {
long mod = getValue(value);
mod = getMode(mod);
for (String name : collection) {
if (name.endsWith(String.valueOf(mod))) {
return name;
}
}
return null;
}
protected abstract long getValue(long value);
protected long getMode(long mode) {
return mode;
}
}
java
public class UIDDbSharding extends AbstractLongShardingAlgorithm {
@Override
protected long getValue(long value) {
//具体分库算法
return value / 2 % 2 + 1;
}
@Override
protected long getMode(long mode) {
//影子库路由算法,后面讲
if (Context.isShadow()) {
mode += 2;
}
return mode;
}
}
t_uid_inline
是分表算法
yml
t_uid_inline:
type: CLASS_BASED
props:
algorithmClassName: com.toby.sharding.jdbc.source.start.db.sharding.UIDTableSharding
strategy: STANDARD
com.toby.sharding.jdbc.source.start.db.sharding.UIDTableSharding
存放具体分表逻辑
java
public class UIDTableSharding extends AbstractLongShardingAlgorithm {
@Override
protected long getValue(long value) {
return value % 2 + 1;
}
}
综上是point_balance基于shardingjdbc的分库分表配置项
配置表param_config默认走point_shard1库,不进行分库分表
可以在yml文件配置如下,代表既不分库也不分表
java
param_config:
actualDataNodes: ds_1.param_config
databaseStrategy:
none:
tableStrategy:
none:
当然也可以什么都不配置,sharding可以自己找到对应的数据库实例,这块没详细看,亲测如果我把配置表放在point_shard2,如果param_config啥都不配置,也是可以获取到的
未完待续