1.前言
类似于订单超时自动关闭之类的场景,除了MQ的延迟消息,还可采用Redis
的键空间通知来实现。
开启键通知后,操作Redis
key时,Redis
以Pub/Sub形式发布消息,channel格式__keyspace@<db>__ prefix
。
客户度订阅即可收到消息。
例:订阅失效键空间通知。
shell
127.0.0.1:6383> psubscribe '__keyevent@*__:expired'
shell
10.0.2.15:6383> set a 123 EX 5
shell
127.0.0.1:6383> psubscribe '__keyevent@*__:expired'
1) "psubscribe"
2) "__keyevent@*__:expired"
3) (integer) 1
1) "pmessage"
2) "__keyevent@*__:expired"
3) "__keyevent@0__:expired"
4) "a"
2.Spring Boot实现------单机Redis
java
@Bean
public RedisMessageListenerContainer redisMessageListenerContainer(
RedisConnectionFactory connectionFactory) throws Exception {
RedisMessageListenerContainer redisMessageListenerContainer = new RedisMessageListenerContainer();
redisMessageListenerContainer.setConnectionFactory(connectionFactory);
// 默认为SimpleAsyncTaskExecutor, 每次new Thread
redisMessageListenerContainer.setTaskExecutor(redisKeyEventExecutor);
return redisMessageListenerContainer;
}
@Bean
public KeyExpirationEventMessageListener keyExpirationEventMessageListener(
RedisMessageListenerContainer redisMessageListenerContainer) {
KeyExpirationEventMessageListener keyExpirationEventMessageListener = new KeyExpirationEventMessageListener(
redisMessageListenerContainer);
return keyExpirationEventMessageListener;
}
java
@Component
public class RedisKeyExpiredEventApplicationListener implements ApplicationListener<RedisKeyExpiredEvent> {
@Override
public void onApplicationEvent(RedisKeyExpiredEvent event) {
// TODO close order
}
}
KeyExpirationEventMessageListener
是Spring Boot已集成的,开发者只需要配置相关类,即可实现功能,从开发实现角度,还是比较便捷的。
但是此方案有几个问题,除非认为这些问题的影响,在当前业务下是可接受的。否则不推荐使用此方案。
3.方案缺点分析
3.1消息丢失
键空间通知是以Pub/Sub形式,消息是不持久化的,如果客户端断开连接并稍后重新连接,则客户端断开连接期间的所有消息都会丢失。
因此还需要再实现一个低频率的定时任务做补偿,来关闭断开连接期间超时的订单。
3.2任务延迟
Redis Expire Key的策略是惰性删除+定时删除,那么对expire key就有2个时刻。
键失效的时刻 vs 键删除的时刻
键失效的时刻:这是键被标记为过期的时间点。Redis
不会立即删除这个键,而是仅标记它为已过期。该键在失效之后仍然可能存在于数据库中,直到某个客户端试图访问它或Redis
执行定期清理操作。
键删除的时刻:这是Redis
实际将过期的键从数据库中删除的时间点。删除操作可能发生在客户端访问过期键时,或者在Redis
的后台清理任务运行时。
Expired (expired) events are generated when the Redis server deletes the key and not when the time to live theoretically reaches the value of zero
而失效键通知则发生在键删除的时刻,因此执行订单关闭会存在一些延时。
3.3消费幂等
Pub/Sub为广播模式,如果服务为集群模式,那么每个服务节点都会收到消息。需要额外编码做处理。
3.4Redis Cluster只能收到单节点通知
常规的publish channel messgae
, redis-cli -c -h host -p port -a password
连接至Cluster下任意主节点都收到通知。Redis会将publish的消息广播到其他主节点。
但是键空间通知不会广播,因此客户端只能收到当前连接主节点下的通知。
根据Redis作者antirez
的回复
Hello. I'm not sure we are able to provide non-local events in Redis Cluster. This would require to broadcast the events cluster-wide, which is quite bandwidth intensive... Isn't it better that the client just subscribes to all the master nodes instead, merging all the received events? However one problem with this is that from time to time it should check the cluster configuration to make sure to connect to other masters added during the live of the cluster.
集群下广播键事件通知会占用大量带宽。
并且Lettuce
, Redisson
连接池都是连接到Cluster中的随机一个节点订阅。
因此,需要额外编码连接至每个主节点开启订阅。
除此之外,还需要定时读取cluster nodes
信息,更新订阅节点。
因为Redis Cluster
是分布式架构,当发生故障转移,进行主从切换时,订阅节点需要变化(键通知只发生在主节点上);当Cluster
节点增加、减少时,订阅节点也需要变化。否则会丢失这些新节点上产生的消息。
同时,由于读取Redis
拓扑信息是定时的,必然与实际Cluster
拓扑结构发生变化的时刻存在一定延时。 此时,必须需要一个定时任务做补偿。
Redis Cluster
键通知样例代码
java
@Bean
public ConcurrentMap<String, RedisMessageListenerContainer> redisMessageListenerContainers(
RedisConnectionFactory redisConnectionFactory,
@Qualifier("redisSubscribeExecutor") ThreadPoolTaskExecutor redisSubscribeExecutor) throws Exception {
if (redisConnectionFactory.getConnection() instanceof LettuceClusterConnection) {
subscribeClusterKeyExpirationEvent(redisConnectionFactory, redisSubscribeExecutor);
scheduledExecutorService.scheduleAtFixedRate(() -> {
try {
subscribeClusterKeyExpirationEvent(redisConnectionFactory, redisSubscribeExecutor);
} catch (Exception e) {
log.error("subscribe error", e);
}
}, 5, 5, TimeUnit.MINUTES);
}
return containers;
}
private void subscribeClusterKeyExpirationEvent(RedisConnectionFactory redisConnectionFactory,
ThreadPoolTaskExecutor redisSubscribeExecutor) throws Exception {
String password = ((LettuceConnectionFactory) redisConnectionFactory).getPassword();
// 获取集群节点信息
Partitions partitions = getClusterTopology(redisConnectionFactory);
if (partitions == null) {
return;
}
// 在每个主节点上开启订阅
doSubscribeClusterKeyExpirationEvent(partitions, redisSubscribeExecutor, password);
// 移除失效节点
removeFailedNode(partitions);
}
private Partitions getClusterTopology(RedisConnectionFactory redisConnectionFactory) {
if (redisConnectionFactory instanceof LettuceConnectionFactory) {
AbstractRedisClient nativeClient = ((LettuceConnectionFactory) redisConnectionFactory).getNativeClient();
if (nativeClient instanceof RedisClusterClient) {
return ((RedisClusterClient) nativeClient).getPartitions();
}
}
return null;
}
private void doSubscribeClusterKeyExpirationEvent(Partitions partitions, ThreadPoolTaskExecutor executor,
String password) throws Exception {
for (RedisClusterNode redisClusterNode : partitions.getPartitions()) {
RedisURI uri = redisClusterNode.getUri();
if (redisClusterNode.is(NodeFlag.UPSTREAM) && !containers.containsKey(
uri.getHost() + ":" + uri.getPort())) {
LettuceConnectionFactory factory = createLettuceConnectionFactory(uri, password);
RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory(factory);
// 默认为SimpleAsyncTaskExecutor, 每次new Thread
container.setTaskExecutor(executor);
container.afterPropertiesSet();
container.start();
KeyExpirationEventMessageListener keyExpirationEventMessageListener = new KeyExpirationEventMessageListener(
container);
keyExpirationEventMessageListener.setApplicationEventPublisher(applicationEventPublisher);
keyExpirationEventMessageListener.afterPropertiesSet();
containers.put(uri.getHost() + ":" + uri.getPort(), container);
}
}
}
private LettuceConnectionFactory createLettuceConnectionFactory(RedisURI redisURI, String password) {
RedisStandaloneConfiguration singleNodeConfig = new RedisStandaloneConfiguration();
singleNodeConfig.setHostName(redisURI.getHost());
singleNodeConfig.setPort(redisURI.getPort());
if (StringUtils.hasText(password)) {
singleNodeConfig.setPassword(password);
}
LettuceConnectionFactory factory = new LettuceConnectionFactory(singleNodeConfig, lettuceClientConfiguration);
factory.afterPropertiesSet();
return factory;
}
private void removeFailedNode(Partitions partitions) throws Exception {
for (Entry<String, RedisMessageListenerContainer> entry : containers.entrySet()) {
String[] split = entry.getKey().split(":");
RedisClusterNode redisClusterNode = partitions.getPartition(split[0], Integer.parseInt(split[1]));
if (redisClusterNode == null) {
RedisMessageListenerContainer container = entry.getValue();
container.stop();
((LettuceConnectionFactory) container.getConnectionFactory()).destroy();
container.destroy();
containers.remove(entry.getKey());
}
}
}
4.总结
如果业务量较小且是Redis单机下,可以采用此方案实现订单超时关闭。