yudaoCloud引入ShardingJDBC5.3.X

yudaoCloud引入ShardingJDBC

完整配置

版本前瞻

在开头我先把完整配置给出来，心急的小伙伴可以直接照搬节省时间，当然了还是希望要是时间允许或者过程中遇到问题一定要继续往下面读，去寻找所以然。

公司项目，用的是yudao_cloud版本，其中平台版本如下：

xml 复制代码

<spring.boot.version>3.3.1</spring.boot.version>
<spring.cloud.version>2023.0.3</spring.cloud.version>
<spring.cloud.alibaba.version>2023.0.1.2</spring.cloud.alibaba.version>
<druid.version>1.2.23</druid.version>
<mybatis.version>3.5.16</mybatis.version>
<mybatis-plus.version>3.5.7</mybatis-plus.version>
<mybatis-plus-generator.version>3.5.7</mybatis-plus-generator.version>
<dynamic-datasource.version>4.3.1</dynamic-datasource.version>
<mybatis-plus-join.version>1.4.13</mybatis-plus-join.version>

从上面可以看出，在当前这是时间哈，版本还是用的比较新的，也导致后面集成ShardingSphere的过程中遇到了非常多的问题，其中有一大半是关于版本相关的问题，所以这里我也想跟各位技术人提一嘴，架构并不是版本越新越好，越新表示很多坑需要自己去踩，无法站在巨人的肩膀上（其他人的踩坑笔记中）去快速落地！

然后，继续正文，在上面的版本基础上，ShardingSphere（实验很多次之后）我采用了如下版本：

xml 复制代码

        <!-- ShardingSphere 5.3.2 -->
        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>shardingsphere-jdbc-core</artifactId>
            <version>5.3.2</version>
        </dependency>
        <dependency>
            <groupId>org.yaml</groupId>
            <artifactId>snakeyaml</artifactId>
            <version>2.3</version> <!-- 可以根据 ShardingSphere 文档调整版本 小于2.0会导致项目无法启动 -->
        </dependency>

这里引入了适合shardingsphere-jdbc-core[5.3.2]版本的 org.yaml[2.3]注意了版本对应很重要，踩了好多坑。

排除相斥依赖

因为我们自己引入了org.yaml但是好多jar包会包含这个依赖，这里我们需要使用

借助mvn dependency:tree命令查看依赖树 然后在控制台 ctrl+f 查找org.yaml 然后使用将其排除。

例如，这里我在spring-boot-starter-validation springdoc-openapi-starter-webmvc-api找到了包含该Jar包的将其排除

xml 复制代码

       <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-validation</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>org.yaml</groupId>
                    <artifactId>snakeyaml</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.springdoc</groupId> <!-- 接口文档：使用最新版本的 Swagger 模型 -->
            <artifactId>springdoc-openapi-starter-webmvc-api</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>org.yaml</groupId>
                    <artifactId>snakeyaml</artifactId>
                </exclusion>
            </exclusions>
            <scope>provided</scope>
        </dependency>

Sharding.yml配置

sharding.yml配置文件

yaml 复制代码

schemaName: logic_db
mode:
  type: STANDALONE
dataSources:
  sharding_ds0:
    url: jdbc:mysql://localhost:3306/quan?useUnicode=true&allowMultiQueries=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&rewriteBatchedStatements=true
    driver-class-name: com.mysql.jdbc.Driver
#    dataSourceClassName: com.alibaba.druid.pool.DruidDataSource
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    username: root
    password: root
    #          url: jdbc:mysql://127.0.0.1:3306/quanquan?useUnicode=true&allowMultiQueries=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
    #          driver-class-name: com.mysql.jdbc.Driver
    #          username: root
    #          password: root

rules:
  - !SHARDING
    shardingAlgorithms:
      table-createTime-inline:
        type: INTERVAL
        props:
          datetime-pattern: "yyyy-MM-dd HH:mm:ss" # 分片字段格式
          datetime-lower: "2025-01-01 00:00:00" # 范围下限
          datetime-upper: "2025-12-30 23:59:59"
          sharding-suffix-pattern: "yyyyMM"
          datetime-interval-amount: 1  # 分片间隔，这里指一个月
          datetime-interval-unit: "MONTHS" # 分片间隔单位
          # 强制按完整字符串解析
          datetime-interval-step-unit: NANOSECONDS

    tables:
      # 逻辑表
      member_money_active_log:
        # 实际表
        actualDataNodes: sharding_ds0.member_money_active_log_${202501..202512}
        #分表策略
        tableStrategy:
          standard:
            # 分片列名称：用create_time进行分表
            shardingColumn: create_time
            # 分片算法（与算法组中的名称对应）
            shardingAlgorithmName: table-createTime-inline


# 展示shardingSphere对SQL的处理
props:
  sql:
    show: true
    simple: false
    check-table-metadata-enabled: true  # 自动检查表结构一致性
  database-type: mysql

application-prod.yml配置

application-prod.yml

yml 复制代码

--- #################### 注册中心 + 配置中心相关配置 ####################

spring:
  cloud:
    nacos:
      server-addr: 127.0.0.1:8848 # Nacos 服务器地址
      username: # Nacos 账号
      password: # Nacos 密码
      discovery: # 【配置中心】配置项
        namespace: 417d7fcc-6b41-4473-a1ba-c5385f507e70 # 命名空间。这里使用 dev 开发环境
        group: DEFAULT_GROUP # 使用的 Nacos 配置分组，默认为 DEFAULT_GROUP
        metadata:
          version: 1.0.0 # 服务实例的版本号，可用于灰度发布
      config: # 【注册中心】配置项
        namespace: 417d7fcc-6b41-4473-a1ba-c5385f507e70 # 命名空间。这里使用 dev 开发环境
        group: DEFAULT_GROUP # 使用的 Nacos 配置分组，默认为 DEFAULT_GROUP

--- #################### 数据库相关配置 ####################
spring:
  # 数据源配置项
  autoconfigure:
    exclude:
  datasource:
    druid: # Druid 【监控】相关的全局配置
      web-stat-filter:
        enabled: true
      stat-view-servlet:
        enabled: true
        allow: # 设置白名单，不填则允许所有访问
        url-pattern: /druid/*
        login-username: # 控制台管理用户名和密码
        login-password:
      filter:
        stat:
          enabled: true
          log-slow-sql: true # 慢 SQL 记录
          slow-sql-millis: 100
          merge-sql: true
        wall:
          config:
            multi-statement-allow: true
    dynamic: # 多数据源配置
      druid: # Druid 【连接池】相关的全局配置
        initial-size: 5 # 初始连接数
        min-idle: 10 # 最小连接池数量
        max-active: 20 # 最大连接池数量
        max-wait: 600000 # 配置获取连接等待超时的时间，单位：毫秒
        time-between-eviction-runs-millis: 60000 # 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位：毫秒
        min-evictable-idle-time-millis: 300000 # 配置一个连接在池中最小生存的时间，单位：毫秒
        max-evictable-idle-time-millis: 900000 # 配置一个连接在池中最大生存的时间，单位：毫秒
        validation-query: SELECT 1 FROM DUAL # 配置检测连接是否有效
        test-while-idle: true
        test-on-borrow: false
        test-on-return: false
      primary: master
      strict: false
      datasource:
        master:
          url: jdbc:mysql://localhost:3306/quanquan?useUnicode=true&allowMultiQueries=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&rewriteBatchedStatements=true
          driver-class-name: com.mysql.jdbc.Driver
          username: root
          password: root
          #          url: jdbc:mysql://127.0.0.1:3306/quanquan?useUnicode=true&allowMultiQueries=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
          #          driver-class-name: com.mysql.jdbc.Driver
          #          username: root
          #          password: root
        slave: # 模拟从库，可根据自己需要修改
          url: jdbc:mysql://localhost:3306/quan?allowMultiQueries=true&useUnicode=true&useSSL=false&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&autoReconnect=true&nullCatalogMeansCurrent=true&rewriteBatchedStatements=true # MySQL Connector/J 8.X 连接的示例
          driver-class-name: com.mysql.jdbc.Driver
          username: root
          password: root
        # 分表数据源
        sharding:
          driver-class-name: org.apache.shardingsphere.driver.ShardingSphereDriver
          url: jdbc:shardingsphere:classpath:sharding.yaml



  # Redis 配置。Redisson 默认的配置足够使用，一般不需要进行调优
  data:
    redis:
      host: 127.0.0.1 # 地址
      port: 6379 # 端口
      database: 1 # 数据库索引
#      password: 123456 # 密码，建议生产环境开启

--- #################### MQ 消息队列相关配置 ####################

--- #################### 定时任务相关配置 ####################
xxl:
  job:
    admin:
      addresses: http://192.168.1.231:9090/xxl-job-admin  # 调度中心部署跟地址

--- #################### 服务保障相关配置 ####################

# Lock4j 配置项
lock4j:
  acquire-timeout: 3000 # 获取分布式锁超时时间，默认为 3000 毫秒
  expire: 30000 # 分布式锁的超时时间，默认为 30 毫秒

--- #################### 监控相关配置 ####################

# Actuator 监控端点的配置项
management:
  endpoints:
    web:
      base-path: /actuator # Actuator 提供的 API 接口的根目录。默认为 /actuator
      exposure:
        include: '*' # 需要开放的端点。默认值只打开 health 和 info 两个端点。通过设置 * ，可以开放所有端点。

# Spring Boot Admin 配置项
spring:
  boot:
    admin:
      # Spring Boot Admin Client 客户端的相关配置
      client:
        instance:
          service-host-type: IP # 注册实例时，优先使用 IP [IP, HOST_NAME, CANONICAL_HOST_NAME]
      # Spring Boot Admin Server 服务端的相关配置
      context-path: /admin # 配置 Spring
# 日志文件配置
logging:
  level:
    # 配置自己写的 MyBatis Mapper 打印日志
    cn.quan.yiqihui.module.member.dal.mysql: debug
    org.springframework.context.support.PostProcessorRegistrationDelegate: ERROR # TODO 芋艿：先禁用，Spring Boot 3.X 存在部分错误的 WARN 提示

--- #################### 芋道相关配置 ####################

# 芋道配置项，设置当前项目所有自定义的配置
quan:
  demo: false # 开启演示模式
  security:
    mock-enable: true
    mock-secret: test # Token 模拟机制的 Token 前缀


springdoc:
  api-docs:
    enabled: true # 1. 是否开启 Swagger 接文档的元数据
    path: /v3/api-docs
  swagger-ui:
    enabled: true # 2.1 是否开启 Swagger 文档的官方 UI 界面
    path: /swagger-ui
  default-flat-param-object: false # 参见 https://doc.xiaominfo.com/docs/faq/v4/knife4j-parameterobject-flat-param 文档

knife4j:
  enable: true # 2.2 是否开启 Swagger 文档的 Knife4j UI 界面
  setting:
    language: zh_cn

spring:
  jackson:
    date-format: yyyy-MM-dd HH:mm:ss
    time-zone: Asia/Shanghai

其实集成ShardingSphere很容易，难点在于yudao_cloud作为一个成熟的框架，库很多，版本之间，中间件之间的协作和兼容很不好处理，这里的Dynamic多数据源和ShardingSphere数据源之间，我踩了很多坑，所以如下配置很重要：

yaml 复制代码

    # 分表数据源
    sharding:
      driver-class-name: org.apache.shardingsphere.driver.ShardingSphereDriver
      url: jdbc:shardingsphere:classpath:sharding.yaml

这里也是Dynamic与ShardingSphere之间的桥梁。

让Dynamic识别到ShardingSphere的数据源，并进行分库分表的写入。

添加Representer.java文件

添加Representer.java文件

java 复制代码

package org.yaml.snakeyaml.representer;

import org.yaml.snakeyaml.DumperOptions;
import org.yaml.snakeyaml.DumperOptions.FlowStyle;
import org.yaml.snakeyaml.TypeDescription;
import org.yaml.snakeyaml.introspector.Property;
import org.yaml.snakeyaml.introspector.PropertyUtils;
import org.yaml.snakeyaml.nodes.*;

import java.util.*;

/**
 * Represent JavaBeans
 */
public class Representer extends SafeRepresenter {

    protected Map<Class<? extends Object>, TypeDescription> typeDefinitions = Collections.emptyMap();

    public Representer() {
        super(new DumperOptions());
        this.representers.put(null, new RepresentJavaBean());
    }

    public Representer(DumperOptions options) {
        super(options);
        this.representers.put(null, new RepresentJavaBean());
    }

    public TypeDescription addTypeDescription(TypeDescription td) {
        if (Collections.EMPTY_MAP == typeDefinitions) {
            typeDefinitions = new HashMap<Class<? extends Object>, TypeDescription>();
        }
        if (td.getTag() != null) {
            addClassTag(td.getType(), td.getTag());
        }
        td.setPropertyUtils(getPropertyUtils());
        return typeDefinitions.put(td.getType(), td);
    }

    @Override
    public void setPropertyUtils(PropertyUtils propertyUtils) {
        super.setPropertyUtils(propertyUtils);
        Collection<TypeDescription> tds = typeDefinitions.values();
        for (TypeDescription typeDescription : tds) {
            typeDescription.setPropertyUtils(propertyUtils);
        }
    }

    protected class RepresentJavaBean implements Represent {

        public Node representData(Object data) {
            return representJavaBean(getProperties(data.getClass()), data);
        }
    }

    /**
     * Tag logic: - explicit root tag is set in serializer - if there is a predefined class tag it is
     * used - a global tag with class name is always used as tag. The JavaBean parent of the specified
     * JavaBean may set another tag (tag:yaml.org,2002:map) when the property class is the same as
     * runtime class
     *
     * @param properties JavaBean getters
     * @param javaBean instance for Node
     * @return Node to get serialized
     */
    protected MappingNode representJavaBean(Set<Property> properties, Object javaBean) {
        List<NodeTuple> value = new ArrayList<NodeTuple>(properties.size());
        Tag tag;
        Tag customTag = classTags.get(javaBean.getClass());
        tag = customTag != null ? customTag : new Tag(javaBean.getClass());
        // flow style will be chosen by BaseRepresenter
        MappingNode node = new MappingNode(tag, value, FlowStyle.AUTO);
        representedObjects.put(javaBean, node);
        DumperOptions.FlowStyle bestStyle = FlowStyle.FLOW;
        for (Property property : properties) {
            Object memberValue = property.get(javaBean);
            Tag customPropertyTag = memberValue == null ? null : classTags.get(memberValue.getClass());
            NodeTuple tuple =
                    representJavaBeanProperty(javaBean, property, memberValue, customPropertyTag);
            if (tuple == null) {
                continue;
            }
            if (!((ScalarNode) tuple.getKeyNode()).isPlain()) {
                bestStyle = FlowStyle.BLOCK;
            }
            Node nodeValue = tuple.getValueNode();
            if (!(nodeValue instanceof ScalarNode && ((ScalarNode) nodeValue).isPlain())) {
                bestStyle = FlowStyle.BLOCK;
            }
            value.add(tuple);
        }
        if (defaultFlowStyle != FlowStyle.AUTO) {
            node.setFlowStyle(defaultFlowStyle);
        } else {
            node.setFlowStyle(bestStyle);
        }
        return node;
    }

    /**
     * Represent one JavaBean property.
     *
     * @param javaBean - the instance to be represented
     * @param property - the property of the instance
     * @param propertyValue - value to be represented
     * @param customTag - user defined Tag
     * @return NodeTuple to be used in a MappingNode. Return null to skip the property
     */
    protected NodeTuple representJavaBeanProperty(Object javaBean, Property property,
                                                  Object propertyValue, Tag customTag) {
        ScalarNode nodeKey = (ScalarNode) representData(property.getName());
        // the first occurrence of the node must keep the tag
        boolean hasAlias = this.representedObjects.containsKey(propertyValue);

        Node nodeValue = representData(propertyValue);

        if (propertyValue != null && !hasAlias) {
            NodeId nodeId = nodeValue.getNodeId();
            if (customTag == null) {
                if (nodeId == NodeId.scalar) {
                    // generic Enum requires the full tag
                    if (property.getType() != java.lang.Enum.class) {
                        if (propertyValue instanceof Enum<?>) {
                            nodeValue.setTag(Tag.STR);
                        }
                    }
                } else {
                    if (nodeId == NodeId.mapping) {
                        if (property.getType() == propertyValue.getClass()) {
                            if (!(propertyValue instanceof Map<?, ?>)) {
                                if (!nodeValue.getTag().equals(Tag.SET)) {
                                    nodeValue.setTag(Tag.MAP);
                                }
                            }
                        }
                    }
                    checkGlobalTag(property, nodeValue, propertyValue);
                }
            }
        }

        return new NodeTuple(nodeKey, nodeValue);
    }

    /**
     * Remove redundant global tag for a type safe (generic) collection if it is the same as defined
     * by the JavaBean property
     *
     * @param property - JavaBean property
     * @param node - representation of the property
     * @param object - instance represented by the node
     */
    @SuppressWarnings("unchecked")
    protected void checkGlobalTag(Property property, Node node, Object object) {
        // Skip primitive arrays.
        if (object.getClass().isArray() && object.getClass().getComponentType().isPrimitive()) {
            return;
        }

        Class<?>[] arguments = property.getActualTypeArguments();
        if (arguments != null) {
            if (node.getNodeId() == NodeId.sequence) {
                // apply map tag where class is the same
                Class<? extends Object> t = arguments[0];
                SequenceNode snode = (SequenceNode) node;
                Iterable<Object> memberList = Collections.EMPTY_LIST;
                if (object.getClass().isArray()) {
                    memberList = Arrays.asList((Object[]) object);
                } else if (object instanceof Iterable<?>) {
                    // list
                    memberList = (Iterable<Object>) object;
                }
                Iterator<Object> iter = memberList.iterator();
                if (iter.hasNext()) {
                    for (Node childNode : snode.getValue()) {
                        Object member = iter.next();
                        if (member != null) {
                            if (t.equals(member.getClass())) {
                                if (childNode.getNodeId() == NodeId.mapping) {
                                    childNode.setTag(Tag.MAP);
                                }
                            }
                        }
                    }
                }
            } else if (object instanceof Set) {
                Class<?> t = arguments[0];
                MappingNode mnode = (MappingNode) node;
                Iterator<NodeTuple> iter = mnode.getValue().iterator();
                Set<?> set = (Set<?>) object;
                for (Object member : set) {
                    NodeTuple tuple = iter.next();
                    Node keyNode = tuple.getKeyNode();
                    if (t.equals(member.getClass())) {
                        if (keyNode.getNodeId() == NodeId.mapping) {
                            keyNode.setTag(Tag.MAP);
                        }
                    }
                }
            } else if (object instanceof Map) { // NodeId.mapping ends-up here
                Class<?> keyType = arguments[0];
                Class<?> valueType = arguments[1];
                MappingNode mnode = (MappingNode) node;
                for (NodeTuple tuple : mnode.getValue()) {
                    resetTag(keyType, tuple.getKeyNode());
                    resetTag(valueType, tuple.getValueNode());
                }
            } else {
                // the type for collection entries cannot be
                // detected
            }
        }
    }

    private void resetTag(Class<? extends Object> type, Node node) {
        Tag tag = node.getTag();
        if (tag.matches(type)) {
            if (Enum.class.isAssignableFrom(type)) {
                node.setTag(Tag.STR);
            } else {
                node.setTag(Tag.MAP);
            }
        }
    }

    /**
     * Get JavaBean properties to be serialised. The order is respected. This method may be overridden
     * to provide custom property selection or order.
     *
     * @param type - JavaBean to inspect the properties
     * @return properties to serialise
     */
    protected Set<Property> getProperties(Class<? extends Object> type) {
        if (typeDefinitions.containsKey(type)) {
            return typeDefinitions.get(type).getProperties();
        }
        return getPropertyUtils().getProperties(type);
    }
}

为什么要添加这个类的原因：新版本没有无参构造器

org.yaml snakeyaml

2.2

解决办法：

这里感谢blog.csdn.net/weixin_4369... 帖子给予的帮助!

最后

最后，在对应的Service方法写上 @DS("sharding") 就可以愉快的玩耍了。