0103水平分片-jdbc-shardingsphere-中间件

gaog2zh2023-08-24 12:26

文章目录

1 准备服务器

随着系统业务的发展，t_order表数据快速增长，服务器压力增大，影响系统性能，我需要对server-order进行分库分表。

服务器规划：

  • 服务器：容器名server-order0，端口号3310
  • 服务器：容器名server-order1，端口号3311

1.1 创建server-order0容器

  • step1：创建挂载文件夹

    bash 复制代码 
    mkdir -p server-order0/conf/conf.d
mkdir server-order0/data

  • Step2：创建容器

    bash 复制代码 
    docker run -it -p 3310:3306 --name server-order0 --privileged=true -v /Users/gaogzhen/data/docker/mysql/mysql8/server-order0/conf/conf.d:/etc/mysql/conf.d -v /Users/gaogzhen/data/docker/mysql/mysql8/server-order0/data:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=123456 -d mysql
    • step3：登录MySQL服务器：
    shell 复制代码 
    #进入容器：
docker exec -it server-order0 env LANG=C.UTF-8 /bin/bash
#进入容器内的mysql命令行
mysql -uroot -p
#修改默认密码插件
ALTER USER 'root'@'%' IDENTIFIED WITH mysql_native_password BY '123456';
    • step4：创建数据库：

    注意：水平分片的id需要在业务层实现，不能依赖数据库的主键自增

    sql 复制代码 
    CREATE DATABASE db_order;
USE db_order;
CREATE TABLE t_order0 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) 
);
CREATE TABLE t_order1 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) 
);

1.2 创建server-order1容器

  • step1：创建挂载文件夹

    bash 复制代码 
    mkdir -p server-order1/conf/conf.d
mkdir server-order1/data

  • Step2：创建容器

    bash 复制代码 
    docker run -it -p 3311:3306 --name server-order1 --privileged=true -v /Users/gaogzhen/data/docker/mysql/mysql8/server-order1/conf/conf.d:/etc/mysql/conf.d -v /Users/gaogzhen/data/docker/mysql/mysql8/server-order1/data:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=123456 -d mysql
    • step3：登录MySQL服务器：
    shell 复制代码 
    #进入容器：
docker exec -it server-order0 env LANG=C.UTF-8 /bin/bash
#进入容器内的mysql命令行
mysql -uroot -p
#修改默认密码插件
ALTER USER 'root'@'%' IDENTIFIED WITH mysql_native_password BY '123456';
    • step4：创建数据库：

    注意：水平分片的id需要在业务层实现，不能依赖数据库的主键自增

    sql 复制代码 
    CREATE DATABASE db_order;
USE db_order;
CREATE TABLE t_order0 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) 
);
CREATE TABLE t_order1 (
  id BIGINT,
  order_no VARCHAR(30),
  user_id BIGINT,
  amount DECIMAL(10,2),
  PRIMARY KEY(id) 
);

2、基本水平分片

2.1、基本配置

properties 复制代码 
#========================基本配置
# 应用名称
spring.application.name=sharging-jdbc-demo
# 开发环境设置
spring.profiles.active=dev
# 内存模式
spring.shardingsphere.mode.type=Memory
# 打印SQl
spring.shardingsphere.props.sql-show=true

2.2、数据源配置

properties 复制代码 
#========================数据源配置
# 配置真实数据源
spring.shardingsphere.datasource.names=server-user,server-order0,server-order1

# 配置第 1 个数据源
spring.shardingsphere.datasource.server-user.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.server-user.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.server-user.jdbc-url=jdbc:mysql://192.168.100.201:3301/db_user
spring.shardingsphere.datasource.server-user.username=root
spring.shardingsphere.datasource.server-user.password=123456

# 配置第 2 个数据源
spring.shardingsphere.datasource.server-order0.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.server-order0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.server-order0.jdbc-url=jdbc:mysql://192.168.100.201:3310/db_order
spring.shardingsphere.datasource.server-order0.username=root
spring.shardingsphere.datasource.server-order0.password=123456

# 配置第 3 个数据源
spring.shardingsphere.datasource.server-order1.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.server-order1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.server-order1.jdbc-url=jdbc:mysql://192.168.100.201:3311/db_order
spring.shardingsphere.datasource.server-order1.username=root
spring.shardingsphere.datasource.server-order1.password=123456

2.3、标椎分片表配置

properties 复制代码 
#========================标准分片表配置（数据节点配置）
# spring.shardingsphere.rules.sharding.tables.<table-name>.actual-data-nodes=值
# 值由数据源名 + 表名组成，以小数点分隔。多个表以逗号分隔，支持 inline 表达式。
# <table-name>：逻辑表名
spring.shardingsphere.rules.sharding.tables.t_user.actual-data-nodes=server-user.t_user
spring.shardingsphere.rules.sharding.tables.t_order.actual-data-nodes=server-order0.t_order0,server-order0.t_order1,server-order1.t_order0,server-order1.t_order1

修改Order实体类的主键策略：

java 复制代码 
//@TableId(type = IdType.AUTO)//依赖数据库的主键自增策略
@TableId(type = IdType.ASSIGN_ID)//分布式id

测试：保留上面配置中的一个分片表节点分别进行测试，检查每个分片节点是否可用

java 复制代码 
/**
     * 水平分片：插入数据测试
     */
@Test
public void testInsertOrder(){

    Order order = new Order();
    order.setOrderNo("20230822001");
    order.setUserId(1L);
    order.setAmount(new BigDecimal(100));
    orderMapper.insert(order);
}

2.4、行表达式

优化上一步的分片表配置

https://shardingsphere.apache.org/document/5.1.1/cn/features/sharding/concept/inline-expression/

properties 复制代码 
#========================标准分片表配置（数据节点配置）
# spring.shardingsphere.rules.sharding.tables.<table-name>.actual-data-nodes=值
# 值由数据源名 + 表名组成，以小数点分隔。多个表以逗号分隔，支持 inline 表达式。
# <table-name>：逻辑表名
spring.shardingsphere.rules.sharding.tables.t_user.actual-data-nodes=server-user.t_user
spring.shardingsphere.rules.sharding.tables.t_order.actual-data-nodes=server-order$->{0..1}.t_order$->{0..1}

2.5、分片算法配置

水平分库：

分片规则：order表中user_id为偶数时，数据插入server-order0服务器，user_id为奇数时，数据插入server-order1服务器。这样分片的好处是，同一个用户的订单数据，一定会被插入到同一台服务器上，查询一个用户的订单时效率较高。

properties 复制代码 
#------------------------分库策略
# 分片列名称
spring.shardingsphere.rules.sharding.tables.t_order.database-strategy.standard.sharding-column=user_id
# 分片算法名称
spring.shardingsphere.rules.sharding.tables.t_order.database-strategy.standard.sharding-algorithm-name=alg_inline_userid

#------------------------分片算法配置
# 行表达式分片算法
# 分片算法类型
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_inline_userid.type=INLINE
# 分片算法属性配置
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_inline_userid.props.algorithm-expression=server-order$->{user_id % 2}

# 取模分片算法
# 分片算法类型
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_mod.type=MOD
# 分片算法属性配置
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_mod.props.sharding-count=2

为了方便测试，先设置只在 t_order0表上进行测试

properties 复制代码 
xxx.actual-data-nodes=server-order$->{0..1}.t_order0

测试：可以分别测试行表达式分片算法和取模分片算法

java 复制代码 
/**
     * 水平分片：分库插入数据测试
     */
@Test
public void testInsertOrderDatabaseStrategy(){

    for (long i = 0; i < 4; i++) {
        Order order = new Order();
        order.setOrderNo("20230821001");
        order.setUserId(i + 1);
        order.setAmount(new BigDecimal(100));
        orderMapper.insert(order);
    }

}

水平分表：

分片规则：order表中order_no的哈希值为偶数时，数据插入对应服务器的t_order0表，order_no的哈希值为奇数时，数据插入对应服务器的t_order1表。因为order_no是字符串形式，因此不能直接取模。

properties 复制代码 
#------------------------分表策略
# 分片列名称
spring.shardingsphere.rules.sharding.tables.t_order.table-strategy.standard.sharding-column=order_no
# 分片算法名称
spring.shardingsphere.rules.sharding.tables.t_order.table-strategy.standard.sharding-algorithm-name=alg_hash_mod


#------------------------分片算法配置
# 哈希取模分片算法
# 分片算法类型
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_hash_mod.type=HASH_MOD
# 分片算法属性配置
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_hash_mod.props.sharding-count=2

测试前不要忘记将如下节点改回原来的状态

properties 复制代码 
xxx.actual-data-nodes=server-order$->{0..1}.t_order$->{0..1}

测试：

java 复制代码 
/**
     * 水平分片：分表插入数据测试
     */
@Test
public void testInsertOrderTableStrategy(){

    for (long i = 1; i < 5; i++) {

        Order order = new Order();
        order.setOrderNo("gaogzhen" + i);
        order.setUserId(1L);
        order.setAmount(new BigDecimal(100));
        orderMapper.insert(order);
    }

    for (long i = 5; i < 9; i++) {

        Order order = new Order();
        order.setOrderNo("gaogzhen" + i);
        order.setUserId(2L);
        order.setAmount(new BigDecimal(100));
        orderMapper.insert(order);
    }
}

/**
     * 测试哈希取模
     */
@Test
public void testHash(){

    //注意hash取模的结果是整个字符串hash后再取模，和数值后缀是奇数还是偶数无关
    System.out.println("gaogzhen001".hashCode() % 2);
    System.out.println("gaogzhen0011".hashCode() % 2);
}

查询测试：

java 复制代码 
/**
     * 水平分片：查询所有记录
     * 查询了两个数据源，每个数据源中使用UNION ALL连接两个表
     */
@Test
public void testShardingSelectAll(){

    List<Order> orders = orderMapper.selectList(null);
    orders.forEach(System.out::println);
}

/**
     * 水平分片：根据user_id查询记录
     * 查询了一个数据源，每个数据源中使用UNION ALL连接两个表
     */
@Test
public void testShardingSelectByUserId(){

    QueryWrapper<Order> orderQueryWrapper = new QueryWrapper<>();
    orderQueryWrapper.eq("user_id", 1L);
    List<Order> orders = orderMapper.selectList(orderQueryWrapper);
    orders.forEach(System.out::println);
}

2.6、分布式序列算法

雪花算法：

https://shardingsphere.apache.org/document/5.1.1/cn/features/sharding/concept/key-generator/

水平分片需要关注全局序列，因为不能简单的使用基于数据库的主键自增。

这里有两种方案：一种是基于MyBatisPlus的id策略；一种是ShardingSphere-JDBC的全局序列配置。

基于MyBatisPlus的id策略：将Order类的id设置成如下形式

java 复制代码 
@TableId(type = IdType.ASSIGN_ID)
private Long id;

基于ShardingSphere-JDBC的全局序列配置：和前面的MyBatisPlus的策略二选一

properties 复制代码 
#------------------------分布式序列策略配置
# 分布式序列列名称
spring.shardingsphere.rules.sharding.tables.t_order.key-generate-strategy.column=id
# 分布式序列算法名称
spring.shardingsphere.rules.sharding.tables.t_order.key-generate-strategy.key-generator-name=alg_snowflake

# 分布式序列算法配置
# 分布式序列算法类型
spring.shardingsphere.rules.sharding.key-generators.alg_snowflake.type=SNOWFLAKE
# 分布式序列算法属性配置
#spring.shardingsphere.rules.sharding.key-generators.alg_snowflake.props.xxx=

此时，需要将实体类中的id策略修改成以下形式：

java 复制代码 
//当配置了shardingsphere-jdbc的分布式序列时，自动使用shardingsphere-jdbc的分布式序列
//当没有配置shardingsphere-jdbc的分布式序列时，自动依赖数据库的主键自增策略
@TableId(type = IdType.AUTO)

3、多表关联

3.1、创建关联表

在server-order0、server-order1服务器中分别创建两张订单详情表t_order_item0、t_order_item1

我们希望同一个用户的订单表和订单详情表中的数据都在同一个数据源中，避免跨库关联，因此这两张表我们使用相同的分片策略。

那么在t_order_item中我们也需要创建order_no和user_id这两个分片键

sql 复制代码 
CREATE TABLE t_order_item0(
    id BIGINT,
    order_no VARCHAR(30),
    user_id BIGINT,
    price DECIMAL(10,2),
    `count` INT,
    PRIMARY KEY(id)
);

CREATE TABLE t_order_item1(
    id BIGINT,
    order_no VARCHAR(30),
    user_id BIGINT,
    price DECIMAL(10,2),
    `count` INT,
    PRIMARY KEY(id)
);

3.2、创建实体类

java 复制代码 
package com.gaogzhen.shardingjdbcdemo.entity;

@TableName("t_order_item")
@Data
public class OrderItem {
    //当配置了shardingsphere-jdbc的分布式序列时，自动使用shardingsphere-jdbc的分布式序列
    @TableId(type = IdType.AUTO)
    private Long id;
    private String orderNo;
    private Long userId;
    private BigDecimal price;
    private Integer count;
}

3.3、创建Mapper

java 复制代码 
package com.gaogzhen.shargingjdbcdemo.mapper;

@Mapper
public interface OrderItemMapper extends BaseMapper<OrderItem> {

}

3.4、配置关联表

t_order_item的分片表、分片策略、分布式序列策略和t_order一致

properties 复制代码 
#------------------------标准分片表配置（数据节点配置）
spring.shardingsphere.rules.sharding.tables.t_order_item.actual-data-nodes=server-order$->{0..1}.t_order_item$->{0..1}

#------------------------分库策略
# 分片列名称
spring.shardingsphere.rules.sharding.tables.t_order_item.database-strategy.standard.sharding-column=user_id
# 分片算法名称
spring.shardingsphere.rules.sharding.tables.t_order_item.database-strategy.standard.sharding-algorithm-name=alg_mod

#------------------------分表策略
# 分片列名称
spring.shardingsphere.rules.sharding.tables.t_order_item.table-strategy.standard.sharding-column=order_no
# 分片算法名称
spring.shardingsphere.rules.sharding.tables.t_order_item.table-strategy.standard.sharding-algorithm-name=alg_hash_mod

#------------------------分布式序列策略配置
# 分布式序列列名称
spring.shardingsphere.rules.sharding.tables.t_order_item.key-generate-strategy.column=id
# 分布式序列算法名称
spring.shardingsphere.rules.sharding.tables.t_order_item.key-generate-strategy.key-generator-name=alg_snowflake

3.5、测试插入数据

同一个用户的订单表和订单详情表中的数据都在同一个数据源中，避免跨库关联

java 复制代码 
/**
     * 测试关联表插入
     */
@Test
public void testInsertOrderAndOrderItem(){

    for (long i = 1; i < 3; i++) {

        Order order = new Order();
        order.setOrderNo("gaogzhen" + i);
        order.setUserId(1L);
        orderMapper.insert(order);

        for (long j = 1; j < 3; j++) {
            OrderItem orderItem = new OrderItem();
            orderItem.setOrderNo("gaogzhen" + i);
            orderItem.setUserId(1L);
            orderItem.setPrice(new BigDecimal(10));
            orderItem.setCount(2);
            orderItemMapper.insert(orderItem);
        }
    }

    for (long i = 5; i < 7; i++) {

        Order order = new Order();
        order.setOrderNo("gaogzhen" + i);
        order.setUserId(2L);
        orderMapper.insert(order);

        for (long j = 1; j < 3; j++) {
            OrderItem orderItem = new OrderItem();
            orderItem.setOrderNo("gaogzhen" + i);
            orderItem.setUserId(2L);
            orderItem.setPrice(new BigDecimal(1));
            orderItem.setCount(3);
            orderItemMapper.insert(orderItem);
        }
    }

}

4、绑定表

**需求：**查询每个订单的订单号和总订单金额

4.1、创建VO对象

java 复制代码 
package com.gaogzhen.shardingjdbcdemo.entity;

@Data
public class OrderVo {
    private String orderNo;
    private BigDecimal amount;
}

4.2、添加Mapper方法

  • OrderMapper.java
java 复制代码 
package com.gaogzhen.shardingjdbcdemo.mapper;

@Mapper
public interface OrderMapper extends BaseMapper<Order> {

    /**
     * 计算订单金额
     * @return 订单金额列表
     */
    List<OrderVO> getOrderAmount();

}

  • OrderMapper.xml

    xml 复制代码 
        <select id="getOrderAmount" resultType="com.gaogzhen.shardingjdbcdemo.vo.OrderVO">
        select
            t1.order_no,
            sum(t2.price * t2.count) amount
        from
            t_order t1
            join t_order_item t2 on t2.order_no = t1.order_no
        group by
            t1.order_no
    </select>

4.3、测试关联查询

java 复制代码 
/**
     * 测试关联表查询
     */
@Test
public void testGetOrderAmount(){

    List<OrderVo> orderAmountList = orderMapper.getOrderAmount();
    orderAmountList.forEach(System.out::println);
}

查询结果

复制代码 
2023-08-23 20:10:40.015  INFO 27448 --- [           main] ShardingSphere-SQL                       : Logic SQL: select
            t1.order_no,
            sum(t2.price * t2.count) amount
        from
            t_order t1
            join t_order_item t2 on t2.order_no = t1.order_no
        group by
            t1.order_no
2023-08-23 20:10:40.015  INFO 27448 --- [           main] ShardingSphere-SQL                       : SQLStatement: MySQLSelectStatement(table=Optional.empty, limit=Optional.empty, lock=Optional.empty, window=Optional.empty)
2023-08-23 20:10:40.015  INFO 27448 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: select
            t1.order_no,
            sum(t2.price * t2.count) amount
        from
            t_order0 t1
            join t_order_item0 t2 on t2.order_no = t1.order_no
        group by
            t1.order_no ORDER BY t1.order_no ASC 
2023-08-23 20:10:40.015  INFO 27448 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: select
            t1.order_no,
            sum(t2.price * t2.count) amount
        from
            t_order1 t1
            join t_order_item0 t2 on t2.order_no = t1.order_no
        group by
            t1.order_no ORDER BY t1.order_no ASC 
2023-08-23 20:10:40.015  INFO 27448 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: select
            t1.order_no,
            sum(t2.price * t2.count) amount
        from
            t_order0 t1
            join t_order_item1 t2 on t2.order_no = t1.order_no
        group by
            t1.order_no ORDER BY t1.order_no ASC 
2023-08-23 20:10:40.015  INFO 27448 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order1 ::: select
            t1.order_no,
            sum(t2.price * t2.count) amount
        from
            t_order1 t1
            join t_order_item1 t2 on t2.order_no = t1.order_no
        group by
            t1.order_no ORDER BY t1.order_no ASC 
2023-08-23 20:10:40.015  INFO 27448 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: select
            t1.order_no,
            sum(t2.price * t2.count) amount
        from
            t_order0 t1
            join t_order_item0 t2 on t2.order_no = t1.order_no
        group by
            t1.order_no ORDER BY t1.order_no ASC 
2023-08-23 20:10:40.015  INFO 27448 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: select
            t1.order_no,
            sum(t2.price * t2.count) amount
        from
            t_order1 t1
            join t_order_item0 t2 on t2.order_no = t1.order_no
        group by
            t1.order_no ORDER BY t1.order_no ASC 
2023-08-23 20:10:40.015  INFO 27448 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: select
            t1.order_no,
            sum(t2.price * t2.count) amount
        from
            t_order0 t1
            join t_order_item1 t2 on t2.order_no = t1.order_no
        group by
            t1.order_no ORDER BY t1.order_no ASC 
2023-08-23 20:10:40.015  INFO 27448 --- [           main] ShardingSphere-SQL                       : Actual SQL: server-order0 ::: select
            t1.order_no,
            sum(t2.price * t2.count) amount
        from
            t_order1 t1
            join t_order_item1 t2 on t2.order_no = t1.order_no
        group by
            t1.order_no ORDER BY t1.order_no ASC 
OrderVO(orderNo=gaogzhen1, amount=40.00)
OrderVO(orderNo=gaogzhen2, amount=40.00)
OrderVO(orderNo=gaogzhen5, amount=6.00)
OrderVO(orderNo=gaogzhen6, amount=6.00)

4.4、配置绑定表

在原来水平分片配置的基础上添加如下配置：

properties 复制代码 
#------------------------绑定表
spring.shardingsphere.rules.sharding.binding-tables[0]=t_order,t_order_item

配置完绑定表后再次进行关联查询的测试：

  • **如果不配置绑定表：测试的结果为8个SQL。**多表关联查询会出现笛卡尔积关联。

  • 如果配置绑定表：测试的结果为4个SQL。 多表关联查询不会出现笛卡尔积关联，关联查询效率将大大提升。

绑定表：指分片规则一致的一组分片表。 使用绑定表进行多表关联查询时，必须使用分片键进行关联，否则会出现笛卡尔积关联或跨库关联，从而影响查询效率。

目前测试还是查询8个SQL， 配置未生效，暂时没找到解决方法

5、广播表

4.1、什么是广播表

指所有的分片数据源中都存在的表，表结构及其数据在每个数据库中均完全一致。 适用于数据量不大且需要与海量数据的表进行关联查询的场景，例如：字典表。

广播具有以下特性：

（1）插入、更新操作会实时在所有节点上执行，保持各个分片的数据一致性

（2）查询操作，只从一个节点获取

（3）可以跟任何一个表进行 JOIN 操作

4.2、创建广播表

在server-order0、server-order1和server-user服务器中分别创建t_dict表

sql 复制代码 
CREATE TABLE t_dict(
    id BIGINT,
    dict_type VARCHAR(200),
    PRIMARY KEY(id)
);

4.3、程序实现

4.3.1、创建实体类

java 复制代码 
package com.gaogzhen.shardingjdbcdemo.entity;

@TableName("t_dict")
@Data
public class Dict {
    //可以使用MyBatisPlus的雪花算法
    @TableId(type = IdType.ASSIGN_ID)
    private Long id;
    private String dictType;
}

4.3.2、创建Mapper

java 复制代码 
package com.gaogzhen.shardingjdbcdemo.mapper;

@Mapper
public interface DictMapper extends BaseMapper<Dict> {
}

4.3.3、配置广播表

properties 复制代码 
#数据节点可不配置，默认情况下，向所有数据源广播
spring.shardingsphere.rules.sharding.tables.t_dict.actual-data-nodes=server-user.t_dict,server-order$->{0..1}.t_dict

# 广播表
spring.shardingsphere.rules.sharding.broadcast-tables[0]=t_dict

4.4、测试广播表

java 复制代码 
@Autowired
private DictMapper dictMapper;

/**
     * 广播表：每个服务器中的t_dict同时添加了新数据
     */
@Test
public void testBroadcast(){

    Dict dict = new Dict();
    dict.setDictType("type1");
    dictMapper.insert(dict);
}

/**
     * 查询操作，只从一个节点获取数据
     * 随机负载均衡规则
     */
@Test
public void testSelectBroadcast(){

    List<Dict> dicts = dictMapper.selectList(null);
    dicts.forEach(System.out::println);
}

5 配置文件方式

  • application.properties
properties 复制代码 
#----------------------- 基础配置
# 项目名称
spring.application.name=sharding-jdbc-demo
spring.profiles.active=dev
# shardingsphere 配置
# 模式
spring.shardingsphere.mode.type=Memory

# 数据源名称
spring.shardingsphere.datasource.names=server-user,server-order0,server-order1

#------------------------ 数据源配置
# 配置第1个数据源
spring.shardingsphere.datasource.server-user.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.server-user.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.server-user.jdbc-url=jdbc:mysql://127.0.0.1:3301/db_user?allowPublicKeyRetrieval=true&useSSL=false
spring.shardingsphere.datasource.server-user.username=root
spring.shardingsphere.datasource.server-user.password=123456

# 配置第2个数据源
spring.shardingsphere.datasource.server-order0.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.server-order0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.server-order0.jdbc-url=jdbc:mysql://127.0.0.1:3310/db_order?allowPublicKeyRetrieval=true&useSSL=false
spring.shardingsphere.datasource.server-order0.username=root
spring.shardingsphere.datasource.server-order0.password=123456

# 配置第3个数据源
spring.shardingsphere.datasource.server-order1.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.server-order1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.server-order1.jdbc-url=jdbc:mysql://127.0.0.1:3311/db_order?allowPublicKeyRetrieval=true&useSSL=false
spring.shardingsphere.datasource.server-order1.username=root
spring.shardingsphere.datasource.server-order1.password=123456

#------------------------数据节点配置
## 标准分配表配置
# spring.shardingsphere.rules.sharding.tables.<table-name>.actual-data-nodes=值
# 值由数据源名 + 表名组成，以小数点分隔。多个表以逗号分隔，支持 inline 表达式。
# <table-name>：逻辑表名
spring.shardingsphere.rules.sharding.tables.t_user.actual-data-nodes=server-user.t_user
spring.shardingsphere.rules.sharding.tables.t_order.actual-data-nodes=server-order$->{0..1}.t_order$->{0..1}
#spring.shardingsphere.rules.sharding.tables.t_order.actual-data-nodes=server-order$->{0..1}.t_order0

#------------------------分库策略
# 分片列配置
spring.shardingsphere.rules.sharding.tables.t_order.database-strategy.standard.sharding-column=user_id
# 分片算法名称
spring.shardingsphere.rules.sharding.tables.t_order.database-strategy.standard.sharding-algorithm-name=alg_inline_userid

#------------------------分片算法配置
# 行表达式分片算法
# 分片算法类型
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_inline_userid.type=INLINE
# 分片算法属性配置
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_inline_userid.props.algorithm-expression=server-order$->{user_id % 2}

# 取模分片算法
# 分片算法类型
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_mod.type=MOD
# 分片算法属性配置
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_mod.props.sharding-count=2

#------------------------分表策略
# 分片列名称
spring.shardingsphere.rules.sharding.tables.t_order.table-strategy.standard.sharding-column=order_no
# 分片算法名称
spring.shardingsphere.rules.sharding.tables.t_order.table-strategy.standard.sharding-algorithm-name=alg_hash_mod


#------------------------分片算法配置
# 哈希取模分片算法
# 分片算法类型
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_hash_mod.type=HASH_MOD
# 分片算法属性配置
spring.shardingsphere.rules.sharding.sharding-algorithms.alg_hash_mod.props.sharding-count=2

#------------------------分布式序列策略配置
# 分布式序列列名称
spring.shardingsphere.rules.sharding.tables.t_order.key-generate-strategy.column=id
# 分布式序列算法名称
spring.shardingsphere.rules.sharding.tables.t_order.key-generate-strategy.key-generator-name=alg_snowflake

# 分布式序列算法配置
# 分布式序列算法类型
spring.shardingsphere.rules.sharding.key-generators.alg_snowflake.type=SNOWFLAKE
# 分布式序列算法属性配置
#spring.shardingsphere.rules.sharding.key-generators.alg_snowflake.props.xxx=

#------------------------标准分片表配置（数据节点配置）
spring.shardingsphere.rules.sharding.tables.t_order_item.actual-data-nodes=server-order$->{0..1}.t_order_item$->{0..1}

#------------------------分库策略
# 分片列名称
spring.shardingsphere.rules.sharding.tables.t_order_item.database-strategy.standard.sharding-column=user_id
# 分片算法名称
spring.shardingsphere.rules.sharding.tables.t_order_item.database-strategy.standard.sharding-algorithm-name=alg_mod

#------------------------分表策略
# 分片列名称
spring.shardingsphere.rules.sharding.tables.t_order_item.table-strategy.standard.sharding-column=order_no
# 分片算法名称
spring.shardingsphere.rules.sharding.tables.t_order_item.table-strategy.standard.sharding-algorithm-name=alg_hash_mod

#------------------------分布式序列策略配置
# 分布式序列列名称
spring.shardingsphere.rules.sharding.tables.t_order_item.key-generate-strategy.column=id
# 分布式序列算法名称
spring.shardingsphere.rules.sharding.tables.t_order_item.key-generate-strategy.key-generator-name=alg_snowflake

#------------------------绑定表
spring.shardingsphere.rules.sharding.binding-tables=t_order,t_order_item

# 广播表
spring.shardingsphere.rules.sharding.broadcast-tables[0]=t_dict

# 打印日志
spring.shardingsphere.props.sql-show=true

# mybatis plus 配置
mybatis.mapper-locations=classpath:mapper/*.xml
mybatis.type-aliases-package=com.gaogzhen.shardingjdbcdemo.entity

  • application.properties+applicaton-dev.yml

    properties 复制代码 
    #----------------------- 基础配置
# 项目名称
spring.application.name=sharding-jdbc-demo
spring.profiles.active=dev

# mybatis plus 配置
mybatis.mapper-locations=classpath:mapper/*.xml
mybatis.type-aliases-package=com.gaogzhen.shardingjdbcdemo.entity
    yaml 复制代码 
    spring:
  shardingSphere:
    mode:
      type: Memory
    schema:
      name: horizontal-sharding
    datasource:
      names: server_user,server-order0,server-order1
      server-user:
        type: com.zaxxer.hikari.HikariDataSource
        driverClassName: com.mysql.jdbc.Driver
        url: jdbc:mysql://127.0.0.1:3301/db_user?allowPublicKeyRetrieval=true&useSSL=false
        username: root
        password: 123456
      server-order0:
        type: com.zaxxer.hikari.HikariDataSource
        driverClassName: com.mysql.jdbc.Driver
        url: jdbc:mysql://127.0.0.1:3310/db_order?allowPublicKeyRetrieval=true&useSSL=false
        username: root
        password: 123456
      server-order1:
        type: com.zaxxer.hikari.HikariDataSource
        driverClassName: com.mysql.jdbc.Driver
        url: jdbc:mysql://127.0.0.1:3311/db_order?allowPublicKeyRetrieval=true&useSSL=false
        username: root
        password: 123456
    rules:
      sharding:
        tables:
          t_user:
            actualDataNodes: server-user.t_user
          t_order:
            actualDataNodes: server-order$->{0..1}.t_order$->{0..1}
            databaseStrategy:
              standard:
                shardingColumn: user_id
                shardingAlgorithmName: alg-inline-userid
            tableStrategy:
              standard:
                shardingColumn: order_no
                shardingAlgorithmName: alg-hash-mod
            keyGenerateStrategy:
              column: id
              keyGeneratorName: alg-snowflake
          t_order_item:
            actualDataNodes: server-order$->{0..1}.t_order_item$->{0..1}
            databaseStrategy:
              standard:
                shardingColumn: user_id
                shardingAlgorithmName: alg-mod
            tableStrategy:
              standard:
                shardingColumn: order_no
                shardingAlgorithmName: alg-hash-mod
            keyGenerateStrategy:
              column: id
              keyGeneratorName: alg-snowflake
        keyGenerators:
          alg-snowflake:
            type: SNOWFLAKE
        shardingAlgorithms:
          alg-inline-userid:
            type: INLINE
            props:
              algorithm-expression: server-order$->{user_id % 2}
          alg-mod:
            type: MOD
            props:
              sharding-count: 2
          alg-hash-mod:
            type: HASH_MOD
            props:
              sharding-count: 2
		binding-tables: t_order,t_order_item
		broadcast-tables: t_dict
		
    props:
      sqlShow: true

6 问题集

6.1 简述

sharding-jdbc 报错多半报错因为配置文件引起的，除了个人粗心大意外，多半和官方给的配置字段名有关。官方文档配置字段名有的给驼峰形式，有的给"-"连接形式，这里建议统一用"-"连接的形式。

  • props下的所有配置需要使用"-"连接的形式，不然报错或者不生效

6.1 Parameter index out of range

报错内容如下：

复制代码 
### Error updating database.  Cause: java.sql.SQLException: Parameter index out of range (1 > number of parameters, which is 0).
### The error may exist in com/gaogzhen/shardingjdbcdemo/mapper/OrderItemMapper.java (best guess)
### The error may involve com.gaogzhen.shardingjdbcdemo.mapper.OrderItemMapper.insert-Inline
### The error occurred while setting parameters
### SQL: INSERT INTO t_order_item0  ( order_no, user_id, price, count )  VALUES  ( ?, ?, ?, ? )
### Cause: java.sql.SQLException: Parameter index out of range (1 > number of parameters, which is 0).
; Parameter index out of range (1 > number of parameters, which is 0).; nested exception is java.sql.SQLException: Parameter index out of range (1 > number of parameters, which is 0).
	at com.gaogzhen.shardingjdbcdemo.HorizontalShardingTest.testInsertOrderAndOrderItem(HorizontalShardingTest.java:102)
Caused by: java.sql.SQLException: Parameter index out of range (1 > number of parameters, which is 0).
	at com.gaogzhen.shardingjdbcdemo.HorizontalShardingTest.testInsertOrderAndOrderItem(HorizontalShardingTest.java:102)

可能出现问题原因

  1. 首选确保官网文档固定的配置项不出现错误，比如table-strategy 大小写，下划线或驼峰形式
  2. 对于自定义的数据源名称、逻辑表名称注意前后一致
  3. 然后MybatisPlus实体类表名注解@TableName(value ="t_order_item") 其中表名为配置的逻辑表名，非真实表名

6.2 No implementation class load from SPI

  • 报错内容：

    org.apache.shardingsphere.spi.exception.ServiceProviderNotFoundException: No implementation class load from SPI org.apache.shardingsphere.sharding.spi.ShardingAlgorithm with type null.

6.3 Error creating bean with name 'org.apache.shardingsphere.spring.boot.ShardingSphereAutoConfiguration'

  • 报错内容：

    java.lang.IllegalStateException: Failed to load ApplicationContext
    Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.shardingsphere.spring.boot.ShardingSphereAutoConfiguration': Initialization of bean failed; nested exception is java.lang.NullPointerException
    Caused by: java.lang.NullPointerException

  • 出错原因

    yaml 复制代码 
    # 按照官网文档配置的数据源如下
    datasource:
      names: server_user,server_order0,server_order1
      server_user:
        dataSourceClassName: com.zaxxer.hikari.HikariDataSource
        driverClassName: com.mysql.jdbc.Driver
        jdbcUrl: jdbc:mysql://127.0.0.1:3301/db_user?allowPublicKeyRetrieval=true&useSSL=false
        username: root
        password: 123456

  • 解决方案：

    yaml 复制代码 
        datasource:
      names: server_user,server_order0,server_order1
      server_user:
        type: com.zaxxer.hikari.HikariDataSource
        driverClassName: com.mysql.jdbc.Driver
        url: jdbc:mysql://127.0.0.1:3301/db_user?allowPublicKeyRetrieval=true&useSSL=false
        username: root
        password: 123456
    • dataSourceClassName替换为type
    • jdbcUrl替换为url

6.4 could not determine a constructor for the tag !SHARDING:

  • 报错内容

    java.lang.IllegalStateException: Failed to load ApplicationContext
    Caused by: org.yaml.snakeyaml.constructor.ConstructorException:
    could not determine a constructor for the tag !SHARDING:
    in 'reader', line 28, column 7:
    - !SHARDING:

我的shardingsphere 版本5.1.1 按照官网sharding-jdbc yaml配置会报上述错误，不识别- !SHARDING

  • 解决方案

    yaml 复制代码 
    把- !SHARDING替换为sharding

    如下图所示：

6.5 Data sources cannot be empty

  • 报错内容

    复制代码 
    Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'shardingSphereDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Data sources cannot be empty.
Caused by: java.lang.IllegalArgumentException: Data sources cannot be empty.

  • 出错原因

    yml 复制代码 
    # 按照官网文档配置
    dataSources:
      server_user:
        dataSourceClassName: com.zaxxer.hikari.HikariDataSource
        driverClassName: com.mysql.jdbc.Driver
        jdbcUrl: jdbc:mysql://127.0.0.1:3301/db_user?allowPublicKeyRetrieval=true&useSSL=false
        username: root
        password: 123456

  • 解决方案

    yml 复制代码 
        datasource:
      names: server_user,server_order0,server_order1
      server_user:
        type: com.zaxxer.hikari.HikariDataSource
        driverClassName: com.mysql.jdbc.Driver
        url: jdbc:mysql://127.0.0.1:3301/db_user?allowPublicKeyRetrieval=true&useSSL=false
        username: root
        password: 123456
    • dataSources替换为datasource
    • 添加names属性，值为逻辑属性源

6.6 Insert statement does not support sharding table routing to multiple data nodes

  • 报错内容

    复制代码 
    org.mybatis.spring.MyBatisSystemException: 
nested exception is org.apache.ibatis.exceptions.PersistenceException: 
### Error updating database.  Cause: java.lang.IllegalStateException: Insert statement does not support sharding table routing to multiple data nodes.
### The error may exist in com/gaogzhen/shardingjdbcdemo/mapper/OrderMapper.java (best guess)
### The error may involve com.gaogzhen.shardingjdbcdemo.mapper.OrderMapper.insert-Inline
### The error occurred while setting parameters
### SQL: INSERT INTO t_order  ( order_no, user_id )  VALUES  ( ?, ? )
### Cause: java.lang.IllegalStateException: Insert statement does not support sharding table routing to multiple data nodes.
	at com.gaogzhen.shardingjdbcdemo.HorizontalShardingTest.testInsertOrderAndOrderItem(HorizontalShardingTest.java:94)
Caused by: org.apache.ibatis.exceptions.PersistenceException:

  • 报错原因

    yaml 复制代码 
        rules:
      sharding:
        tables:
          t_order_item:
            actualDataNodes: server-order$->{0..1}.t_order_item$->{0..1}
            databaseStrategy:
              standard:
                shardingColumn: user_id
                shardingAlgorithmName: alg-mod
            tableStrategy:
              standard:
                shardingColumn: order_id
                shardingAlgorithmName: alg-hash-mod
            keyGenerateStrategy:
              column: id
              keyGeneratorName: alg-snowflake
        keyGenerators:
          alg-snowflake:
            type: SNOWFLAKE
        shardingAlgorithms:
          alg-inline-userid:
            type: INLINE
            props:
              algorithm-expression: server-order$->{user_id % 2}
          alg-mod:
            type: MOD
            props:
              sharding-count: 2
          alg-hash-mod:
            type: HASH_MOD
            props:
              sharding-count: 2
    • 分库或者分表算法名称不能使用"_"下划线分割 ，用"-"代替

    6.7 Inline sharding algorithm expression cannot be null or empty

    • 报错原因

      yaml 复制代码 
              shardingAlgorithms:
          alg-inline-userid:
            type: INLINE
            props:
              algorithmExpression: server-order$->{user_id % 2}
      • algorithmExpression不能为驼峰命名

    • 解决方案

      yaml 复制代码 
      algorithmExpression改为algorithm-expression

结语

如果小伙伴什么问题或者指教，欢迎交流。

QQ:806797785

仓库源代码地址：https://gitee.com/gaogzhen/shardingsphere-jdbc-demo.git

参考链接:

1ShardingSphere5实战教程CP/OL.2022-09-14.p18-23.

20101读写分离测试-jdbc-shardingsphere-中间件CP/OL.

30102垂直分片-jdbc-shardingsphereCP/OL.

相关推荐
阿昌喜欢吃黄桃7 天前
RocketMq事务消息原理
java·中间件·消息队列·rocketmq·mq
半夜修仙8 天前
延迟队列的介绍及常见问题
java·数据库·中间件·rabbitmq
手握风云-8 天前
一条消息的旅程：RabbitMQ 学习与实践（一）
中间件·rabbitmq
RH2312119 天前
2026.6.8Linux
java·数据库·中间件
理人综艺好会10 天前
双Token机制在实际项目中的应用与实践
中间件·token
番茄去哪了10 天前
神领物流面试题（一）
java·大数据·中间件
念何架构之路10 天前
消息中间件
中间件
都说名字长不会被发现10 天前
Spring Boot Starter 中间件账号密码加密方案设计与实现
java·spring boot·后端·中间件
瀚高PG实验室11 天前
java中间件无法连接数据库
java·数据库·中间件·瀚高数据库
之歆11 天前
Day11_Express 深入解析：从中间件到项目实战
中间件·express
热门推荐
012026年6月AI大模型全景报告：GPT-5.6、Claude Opus 4.8、Gemini 3.5，中美AI三足鼎立谁主沉浮？022026年6月AI行业全景：从百模大战到Agent元年，这30天发生了什么？03【AI】2026 年具身智能模型和世界模型总结042026 年 AI 编程工具终极横评：Cursor vs Claude Code vs Copilot vs Windsurf05Claude Code、Codex、Cursor三分天下：2026年AI编程Agent生态全景剖析06GitHub 镜像站点07【AI总结】2026年6月 主流国内外大模型总结08AI科技热点日报 | 2026年6月1日09AI科技热点日报 | 2026年6月22日10AI一周事件 · 2026-06-03 至 2026-06-09