spring cloud + nacos 配置刷新 jasypt 未解密 数据库连接失败1045

现象

1. 修改了nacos配置，重新发布
2. 过了一段时间发现，数据库偶尔会连接失败报异常

- create connection SQLException, url: jdbc:mysql://*******:3306/settlement?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&zeroDateTimeBehavior=convertToNull&serverTimezone=Asia/Shanghai, errorCode 1045, state 28000java.sql.SQLException: Access denied for user '****'@'ip' (using password: YES)
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
	at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:836)
	at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:456)
	at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:246)
	at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:199)
	at com.alibaba.druid.filter.FilterChainImpl.connection_connect(FilterChainImpl.java:156)
	at com.alibaba.druid.filter.FilterAdapter.connection_connect(FilterAdapter.java:787)
	at com.alibaba.druid.filter.FilterEventAdapter.connection_connect(FilterEventAdapter.java:38)
	at com.alibaba.druid.filter.FilterChainImpl.connection_connect(FilterChainImpl.java:150)
	at com.alibaba.druid.filter.stat.StatFilter.connection_connect(StatFilter.java:218)
	at com.alibaba.druid.filter.FilterChainImpl.connection_connect(FilterChainImpl.java:150)
	at com.alibaba.druid.filter.FilterAdapter.connection_connect(FilterAdapter.java:787)
	at com.alibaba.druid.filter.FilterChainImpl.connection_connect(FilterChainImpl.java:150)
	at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1646)
	at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1710)
	at com.alibaba.druid.pool.DruidDataSource$CreateConnectionThread.run(DruidDataSource.java:2753)

环境

1. mysql 5.7.28-log
2. maven pom.xml

<dependency>
    <groupId>com.github.ulisesbocchio</groupId>
    <artifactId>jasypt-spring-boot-starter</artifactId>
    <version>2.1.2</version>
</dependency>

 <dependency>
        <groupId>com.alibaba.cloud</groupId>
        <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
        <version>2.2.0.RELEASE</version>
 </dependency>    

<dependency>
     <groupId>com.alibaba</groupId>
     <artifactId>druid-spring-boot-starter</artifactId>
      <version>1.1.21</version>
</dependency>

3. bootstrap.yml 配置

spring:
  application:
    name: test
  cloud:
    nacos:
      serveraddr: ***
      namespace: loc
      config:
        server-addr: ${spring.cloud.nacos.serveraddr}
        namespace: ${spring.cloud.nacos.namespace}
        prefix: ${spring.application.name}
        file-extension: yml
      discovery:
        server-addr: ${spring.cloud.nacos.serveraddr}
        namespace: ${spring.cloud.nacos.namespace}

4. nacos配置 test.yml

jasypt:
  encryptor:
    password: hello

分析

发布nacos配置会导致配置刷新吗？

如下代码可以清楚看到和bootstrap.yml对应的NacosConfigProperties ，默认是开启刷新的。

com.alibaba.cloud.nacos.NacosConfigProperties

	/**
	 * the master switch for refresh configuration, it default opened(true).
	 */
	private boolean refreshEnabled = true;

数据库为啥连接失败？

启动时数据库创建是成功的，为啥现在偶尔失败。

打个断点看一看

com.alibaba.druid.pool.DruidAbstractDataSource#createPhysicalConnection()

       // 断点
        String password = getPassword();
        PasswordCallback passwordCallback = getPasswordCallback();

断点处发现 密码变成了 ENC()加密数据 ，密码根本未解密才导致连接失败。

什么时候密码被变成未解密的了？

修改nacos上的配置，重新发布 com.alibaba.druid.pool.DruidAbstractDataSource#setPassword 打个断点看一看

    public void setPassword(String password) {
     // 断点
        if (StringUtils.equals(this.password, password)) {
            return;
        }

        if (inited) {
            LOG.info("password changed");
        }

        this.password = password;
    }

堆栈如下，收到NacosContextRefresher事件后，动态刷新触发了密码的修改

setPassword:1134, DruidAbstractDataSource (com.alibaba.druid.pool)
invoke:-1, GeneratedMethodAccessor502 (sun.reflect)
invoke:43, DelegatingMethodAccessorImpl (sun.reflect)
invoke:498, Method (java.lang.reflect)
setValue:346, JavaBeanBinder$BeanProperty (org.springframework.boot.context.properties.bind)
bind:96, JavaBeanBinder (org.springframework.boot.context.properties.bind)
bind:79, JavaBeanBinder (org.springframework.boot.context.properties.bind)
bind:56, JavaBeanBinder (org.springframework.boot.context.properties.bind)
lambda$bindDataObject$5:452, Binder (org.springframework.boot.context.properties.bind)
get:-1, 540325452 (org.springframework.boot.context.properties.bind.Binder$$Lambda$42)
withIncreasedDepth:570, Binder$Context (org.springframework.boot.context.properties.bind)
withDataObject:556, Binder$Context (org.springframework.boot.context.properties.bind)
access$400:513, Binder$Context (org.springframework.boot.context.properties.bind)
bindDataObject:450, Binder (org.springframework.boot.context.properties.bind)
bindObject:391, Binder (org.springframework.boot.context.properties.bind)
bind:320, Binder (org.springframework.boot.context.properties.bind)
bind:308, Binder (org.springframework.boot.context.properties.bind)
bind:238, Binder (org.springframework.boot.context.properties.bind)
bind:225, Binder (org.springframework.boot.context.properties.bind)
bind:89, ConfigurationPropertiesBinder (org.springframework.boot.context.properties)
bind:107, ConfigurationPropertiesBindingPostProcessor (org.springframework.boot.context.properties)
postProcessBeforeInitialization:96, ConfigurationPropertiesBindingPostProcessor (org.springframework.boot.context.properties)
applyBeanPostProcessorsBeforeInitialization:416, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
initializeBean:1795, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
initializeBean:407, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
rebind:108, ConfigurationPropertiesRebinder (org.springframework.cloud.context.properties)
rebind:84, ConfigurationPropertiesRebinder (org.springframework.cloud.context.properties)
onApplicationEvent:142, ConfigurationPropertiesRebinder (org.springframework.cloud.context.properties)
onApplicationEvent:51, ConfigurationPropertiesRebinder (org.springframework.cloud.context.properties)
doInvokeListener:172, SimpleApplicationEventMulticaster (org.springframework.context.event)
invokeListener:165, SimpleApplicationEventMulticaster (org.springframework.context.event)
multicastEvent:139, SimpleApplicationEventMulticaster (org.springframework.context.event)
publishEvent:403, AbstractApplicationContext (org.springframework.context.support)
publishEvent:360, AbstractApplicationContext (org.springframework.context.support)
refreshEnvironment:96, ContextRefresher (org.springframework.cloud.context.refresh)
refresh:85, ContextRefresher (org.springframework.cloud.context.refresh)
handle:72, RefreshEventListener (org.springframework.cloud.endpoint.event)
onApplicationEvent:61, RefreshEventListener (org.springframework.cloud.endpoint.event)
doInvokeListener:172, SimpleApplicationEventMulticaster (org.springframework.context.event)
invokeListener:165, SimpleApplicationEventMulticaster (org.springframework.context.event)
multicastEvent:139, SimpleApplicationEventMulticaster (org.springframework.context.event)
publishEvent:403, AbstractApplicationContext (org.springframework.context.support)
publishEvent:360, AbstractApplicationContext (org.springframework.context.support)
innerReceive:133, NacosContextRefresher$1 (com.alibaba.cloud.nacos.refresh)
receiveConfigInfo:38, AbstractSharedListener (com.alibaba.nacos.api.config.listener)
run:203, CacheData$1 (com.alibaba.nacos.client.config.impl)
safeNotifyListener:233, CacheData (com.alibaba.nacos.client.config.impl)
checkListenerMd5:174, CacheData (com.alibaba.nacos.client.config.impl)
run:552, ClientWorker$LongPollingRunnable (com.alibaba.nacos.client.config.impl)
call:511, Executors$RunnableAdapter (java.util.concurrent)
run$$$capture:266, FutureTask (java.util.concurrent)
run:-1, FutureTask (java.util.concurrent)

如何解决？

1. 不刷新

bootstrap.yml 配置 spring.cloud.nacos.config.refreshEnabled 为false

spring:
  application:
    name: test
  cloud:
    nacos:
      serveraddr: ***
      namespace: loc
      config:
        server-addr: ${spring.cloud.nacos.serveraddr}
        namespace: ${spring.cloud.nacos.namespace}
        prefix: ${spring.application.name}
        file-extension: yml
        refreshEnabled: false
      discovery:
        server-addr: ${spring.cloud.nacos.serveraddr}
        namespace: ${spring.cloud.nacos.namespace}

2. 让获取的property是解密后。

梳理一下事件和动作点

时刻	动作
T1	我们点击发布新的nacos配置
T2	触发ContextRefresh，更新本地的属性
T3	连接池重新连接数据库

复现

在T1，T2时间之间，我kill掉了当前的connection。

因为是连接池，可能原来connection继续使用，这样无法看到报错，我需要kill掉之前的connection。

为了方便kill，我将druid初始化连接数，最小连接数都改为1。

mysql kill connection

// 查看当前connection
show processlist;
// 杀掉当前连接
kill pid;

refresh() 刷新

org.springframework.cloud.context.refresh.ContextRefresher

public synchronized Set<String> refresh() {
   Set<String> keys = refreshEnvironment();
   this.scope.refreshAll();
   return keys;
}

public synchronized Set<String> refreshEnvironment() {
   Map<String, Object> before = extract(
         this.context.getEnvironment().getPropertySources());
   addConfigFilesToEnvironment();
   // 哪些属性修改了，哪些Bean需要重新创建
   Set<String> keys = changes(before,
         extract(this.context.getEnvironment().getPropertySources())).keySet();
   this.context.publishEvent(new EnvironmentChangeEvent(this.context, keys));
   return keys;
}

debug看一下,可以看到是加密的PropertySource

在调用 addConfigFilesToEnvironment 之后，可以看到此时是不加密的PropertySource了。

nacos Property 如何刷新

com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadNacosDataIfPresent

private void loadNacosDataIfPresent(final CompositePropertySource composite,
      final String dataId, final String group, String fileExtension,
      boolean isRefreshable) {
   if (null == dataId || dataId.trim().length() < 1) {
      return;
   }
   if (null == group || group.trim().length() < 1) {
      return;
   }
   NacosPropertySource propertySource = this.loadNacosPropertySource(dataId, group,
         fileExtension, isRefreshable);
   this.addFirstPropertySource(composite, propertySource, false);
}

可以从图上看到核心的问题是NacosPropertySource没有被 EncryptableEnumerablePropertySourceWrapper 装饰，导致了获取到属性是未加密的。

可以从 org.springframework.cloud.context.refresh.ContextRefresher 看到触发了 EnvironmentChangeEvent 事件，所以解决方法是我们先处理EnvironmentChangeEvent事件，将NacosPropertySource装饰为EncryptableEnumerablePropertySourceWrapper

官方解决方案

jasypt-spring-boot-parent-3.0.3

1. 升级版本到v3.0.3

<dependency> 
    <groupId>com.github.ulisesbocchio</groupId> 
    <artifactId>jasypt-spring-boot-starter</artifactId> 
    <version>3.0.3<version> 
</dependency>

2. nacos test.yml 新增配置

jasypt:
  encryptor:
    password: hello
    # 新增配置
    algorithm: PBEWithMD5AndDES
    iv-generator-classname: org.jasypt.iv.NoIvGenerator

源码解析

我拉了v3.0.3和v3.0.3对比，

RefreshScopeRefreshedEventListener 是一个处理ApplicationEvent的Listener，

v3.0.3 新增了对 org.springframework.cloud.context.environment.EnvironmentChangeEvent的处理

V3.0.3 RefreshScopeRefreshedEventListener 代码如下

package com.ulisesbocchio.jasyptspringboot.caching;

import com.ulisesbocchio.jasyptspringboot.EncryptablePropertySource;
import com.ulisesbocchio.jasyptspringboot.EncryptablePropertySourceConverter;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.ApplicationEvent;
import org.springframework.context.ApplicationListener;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.core.env.*;
import org.springframework.util.ClassUtils;

@Order(Ordered.HIGHEST_PRECEDENCE)
@Slf4j
public class RefreshScopeRefreshedEventListener implements ApplicationListener<ApplicationEvent> {

    public static final String REFRESHED_EVENT_CLASS = "org.springframework.cloud.context.scope.refresh.RefreshScopeRefreshedEvent";
    public static final String ENVIRONMENT_EVENT_CLASS = "org.springframework.cloud.context.environment.EnvironmentChangeEvent";
    private final ConfigurableEnvironment environment;
    private final EncryptablePropertySourceConverter converter;

    public RefreshScopeRefreshedEventListener(ConfigurableEnvironment environment, EncryptablePropertySourceConverter converter) {
        this.environment = environment;
        this.converter = converter;
    }

    @Override
    @SneakyThrows
    public void onApplicationEvent(ApplicationEvent event) {
        if (isAssignable(ENVIRONMENT_EVENT_CLASS, event) || isAssignable(REFRESHED_EVENT_CLASS, event)) {
            log.info("Refreshing cached encryptable property sources");
            refreshCachedProperties();
            decorateNewSources();
        }
    }

    private void decorateNewSources() {
        // 将新的PropertySource转为EncryptablePropertySource

        MutablePropertySources propSources = environment.getPropertySources();
        converter.convertPropertySources(propSources);
    }

    boolean isAssignable(String className, Object value) {
        try {
            return ClassUtils.isAssignableValue(ClassUtils.forName(className, null), value);
        } catch (ClassNotFoundException e) {
            return false;
        }
    }

    private void refreshCachedProperties() {
        PropertySources propertySources = environment.getPropertySources();
        propertySources.forEach(this::refreshPropertySource);
    }

    @SuppressWarnings("rawtypes")
    private void refreshPropertySource(PropertySource<?> propertySource) {
        if (propertySource instanceof CompositePropertySource) {
            CompositePropertySource cps = (CompositePropertySource) propertySource;
            cps.getPropertySources().forEach(this::refreshPropertySource);
        } else if (propertySource instanceof EncryptablePropertySource) {
            EncryptablePropertySource eps = (EncryptablePropertySource) propertySource;
            eps.refresh();
        }
    }
}

参考资料

jasypt-spring-boot-parent-3.0.3