Redis缓存预热-缓存穿透-缓存雪崩-缓存击穿

biubiubiu07062024-03-09 23:32

什么叫缓存穿透?

模拟一个场景:

前端用户发送请求获取数据,后端首先会在缓存Redis中查询,如果能查到数据,则直接返回.如果缓存中查不到数据,则要去数据库查询,如果数据库有,将数据保存到Redis缓存中并且返回用户数据.如果数据库没有则返回null;

这个缓存穿透的问题就是这个返回的null上面,如果客户端恶意频繁的发起Redis不存在的Key,且数据库中也不存在的数据,返回永远是null.当洪流式的请求过来,给数据库造成极大压力,甚至压垮数据库.它永远越过Redis缓存而直接访问数据库,这个过程就是缓存穿透.

其实是个设计上的缺陷.

缓存穿透解决方案

业界比较成熟的一种解决方案:当越过缓存,且数据库没有该数据返回客户端null并且存到Redis,数据是为"",看实际情况并给这个Key设置过期时间.这种方案一定程度上减少数据库频繁查询的压力.

实战过程

CREATE TABLE `item` (

`id` int(11) NOT NULL AUTO_INCREMENT,

`code` varchar(255) DEFAULT NULL COMMENT '商品编号',

`name` varchar(255) CHARACTER SET utf8mb4 DEFAULT NULL COMMENT '商品名称',

`create_time` datetime DEFAULT NULL,

PRIMARY KEY (`id`)

) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8 COMMENT='商品信息表';

复制代码 
INSERT INTO `item` VALUES ('1', 'book_10010', 'Redis缓存穿透实战', '2019-03-17 17:21:16');

项目整体结构

依赖

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.2</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example</groupId>
    <artifactId>redis1</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>redis1</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>2.3.0</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter-test</artifactId>
            <version>3.0.3</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

启动类

@SpringBootApplication
@MapperScan({"com.example.redis1.mapper"})
public class Redis1Application {

    public static void main(String[] args) {
        SpringApplication.run(Redis1Application.class, args);
    }

}

application.yml

server:
  port: 80
spring:
  application:
    name: redis-test
  redis:
    ##redis 单机环境配置
    ##将docker脚本部署的redis服务映射为宿主机ip
    ##生产环境推荐使用阿里云高可用redis服务并设置密码
    host: 127.0.0.1
    port: 6379
    password:
    database: 0
    ssl: false
    ##redis 集群环境配置
    #cluster:
    #  nodes: 127.0.0.1:7001,127.0.0.1:7002,127.0.0.1:7003
    #  commandTimeout: 5000
  datasource:
    driver-class-name: com.mysql.cj.jdbc.Driver
    url: jdbc:mysql://1111111:3306/redis-test?useSSL=false&useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=GMT%2B8&useCursorFetch=true
    username: xxxx
    password: xxxxxxxx
mybatis:
  mapper-locations: classpath:mappers/*Mapper.xml  # 指定mapper文件位置
  type-aliases-package: com.example.redis1.pojo
  configuration:
    map-underscore-to-camel-case: true
logging:
  level:
    com.example.redis1.mapper: debug

数据库映射xml

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace="com.example.redis1.mapper.ItemMapper" >
  <resultMap id="BaseResultMap" type="com.example.redis1.pojo.Item" >
    <id column="id" property="id" jdbcType="INTEGER" />
    <result column="code" property="code" jdbcType="VARCHAR" />
    <result column="name" property="name" jdbcType="VARCHAR" />
    <result column="create_time" property="createTime" jdbcType="TIMESTAMP" />
  </resultMap>
  <sql id="Base_Column_List" >
    id, code, name, create_time
  </sql>
  <select id="selectByPrimaryKey" resultMap="BaseResultMap" parameterType="java.lang.Integer" >
    select
    <include refid="Base_Column_List" />
    from item
    where id = #{id,jdbcType=INTEGER}
  </select>
  <delete id="deleteByPrimaryKey" parameterType="java.lang.Integer" >
    delete from item
    where id = #{id,jdbcType=INTEGER}
  </delete>
  <insert id="insert" parameterType="item" >
    insert into item (id, code, name,
      create_time)
    values (#{id,jdbcType=INTEGER}, #{code,jdbcType=VARCHAR}, #{name,jdbcType=VARCHAR},
      #{createTime,jdbcType=TIMESTAMP})
  </insert>
  <insert id="insertSelective" parameterType="item" >
    insert into item
    <trim prefix="(" suffix=")" suffixOverrides="," >
      <if test="id != null" >
        id,
      </if>
      <if test="code != null" >
        code,
      </if>
      <if test="name != null" >
        name,
      </if>
      <if test="createTime != null" >
        create_time,
      </if>
    </trim>
    <trim prefix="values (" suffix=")" suffixOverrides="," >
      <if test="id != null" >
        #{id,jdbcType=INTEGER},
      </if>
      <if test="code != null" >
        #{code,jdbcType=VARCHAR},
      </if>
      <if test="name != null" >
        #{name,jdbcType=VARCHAR},
      </if>
      <if test="createTime != null" >
        #{createTime,jdbcType=TIMESTAMP},
      </if>
    </trim>
  </insert>
  <update id="updateByPrimaryKeySelective" parameterType="item" >
    update item
    <set >
      <if test="code != null" >
        code = #{code,jdbcType=VARCHAR},
      </if>
      <if test="name != null" >
        name = #{name,jdbcType=VARCHAR},
      </if>
      <if test="createTime != null" >
        create_time = #{createTime,jdbcType=TIMESTAMP},
      </if>
    </set>
    where id = #{id,jdbcType=INTEGER}
  </update>
  <update id="updateByPrimaryKey" parameterType="item" >
    update item
    set code = #{code,jdbcType=VARCHAR},
      name = #{name,jdbcType=VARCHAR},
      create_time = #{createTime,jdbcType=TIMESTAMP}
    where id = #{id,jdbcType=INTEGER}
  </update>

  <!--根据商品编码查询-->
  <select id="selectByCode" resultType="item">
    select
    <include refid="Base_Column_List" />
    from item
    where code = #{code}
  </select>

</mapper>

pojo

package com.example.redis1.pojo;

import com.fasterxml.jackson.annotation.JsonFormat;
import lombok.Data;

import java.util.Date;

@Data
public class Item {
    private Integer id;

    private String code;

    private String name;

    @JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss",timezone = "GMT+8")
    private Date createTime;

}

mapper

package com.example.redis1.mapper;


import com.example.redis1.pojo.Item;
import org.apache.ibatis.annotations.Param;

public interface ItemMapper {
    int deleteByPrimaryKey(Integer id);

    int insert(Item record);

    int insertSelective(Item record);

    Item selectByPrimaryKey(Integer id);

    int updateByPrimaryKeySelective(Item record);

    int updateByPrimaryKey(Item record);

    Item selectByCode(@Param("code") String code);
}

service

package com.example.redis1.service;


import com.example.redis1.mapper.ItemMapper;
import com.example.redis1.pojo.Item;
import com.fasterxml.jackson.databind.ObjectMapper;


import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.ValueOperations;
import org.springframework.stereotype.Service;

import java.util.concurrent.TimeUnit;

/**
 * 缓存穿透service
 * Created by Administrator on 2019/3/17.
 */
@Service
public class CachePassService {

    private static final Logger log= LoggerFactory.getLogger(CachePassService.class);

    @Autowired
    private ItemMapper itemMapper;

    @Autowired
    private RedisTemplate redisTemplate;

    @Autowired
    private ObjectMapper objectMapper;

    private static final String keyPrefix="item:";

    /**
     * 获取商品详情-如果缓存有，则从缓存中获取；如果没有，则从数据库查询，并将查询结果塞入缓存中
     * @param itemCode
     * @return
     * @throws Exception
     */
    public Item getItemInfo(String itemCode) throws Exception{
        Item item=null;

        final String key=keyPrefix+itemCode;
        ValueOperations valueOperations=redisTemplate.opsForValue();
        if (redisTemplate.hasKey(key)){
            log.info("---获取商品详情-缓存中存在该商品---商品编号为：{} ",itemCode);

            //从缓存中查询该商品详情
            Object res=valueOperations.get(key);
            if (res!=null&&!(res.equals(""))){
                item=objectMapper.readValue(res.toString(),Item.class);
            }
        }else{
            log.info("---获取商品详情-缓存中不存在该商品-从数据库中查询---商品编号为：{} ",itemCode);

            //从数据库中获取该商品详情
            item=itemMapper.selectByCode(itemCode);
            if (item!=null){
                valueOperations.set(key,objectMapper.writeValueAsString(item));
            }else{
                //过期失效时间TTL设置为30分钟-当然实际情况要根据实际业务决定
                valueOperations.set(key,"",30L, TimeUnit.MINUTES);
            }
        }
        return item;
    }
}

controller

package com.example.redis1.controller;

import com.example.redis1.service.CachePassService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import java.util.HashMap;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;


/**
 * 缓存穿透实战
 * @Author:debug (SteadyJack)
 * @Date: 2019/3/17 18:33
 **/
@RestController
public class CachePassController {

    private static final Logger log= LoggerFactory.getLogger(CachePassController.class);

    private static final String prefix="cache/pass";

    @Autowired
    private CachePassService cachePassService;


    /**
     * 获取热销商品信息
     * @param itemCode
     * @return
     */
    @RequestMapping(value = prefix+"/item/info",method = RequestMethod.GET)
    public Map<String,Object> getItem(@RequestParam String itemCode){
        Map<String,Object> resMap=new HashMap<>();
        resMap.put("code",0);
        resMap.put("msg","成功");

        try {
            resMap.put("data",cachePassService.getItemInfo(itemCode));
        }catch (Exception e){
            resMap.put("code",-1);
            resMap.put("msg","失败"+e.getMessage());
        }
        return resMap;
    }
}

第一次访问

localhost/cache/pass/item/info?itemCode=book_10010

查看日志输出

用个数据库不存在的

localhost/cache/pass/item/info?itemCode=book_10012

后端的处理是将不存在的key存到redis并指定过期时间

其他典型问题介绍

缓存雪崩:指的的某个时间点,缓存中的Key集体发生过期失效,导致大量查询的请求落到数据库上,导致数据库负载过高,压力暴增的现象

解决方案:设置错开不同的过期时间

缓存击穿:指缓存中某个频繁被访问的Key(热点Key),突然过期时间到了失效了,持续的高并发访问瞬间就像击破缓存一样瞬间到达数据库。

解决办法:设置热点Key永不过期

缓存预热:一般指应用启动前,提前加载数据到缓存中

相关推荐
猿小喵16 分钟前
DBA之路，始于足下
数据库·dba
tyler_download25 分钟前
golang 实现比特币内核:实现基于椭圆曲线的数字签名和验证
开发语言·数据库·golang
编程、小哥哥40 分钟前
设计模式之抽象工厂模式(替换Redis双集群升级，代理类抽象场景)
redis·设计模式·抽象工厂模式
weixin_449310841 小时前
高效集成：聚水潭采购数据同步到MySQL
android·数据库·mysql
Cachel wood2 小时前
Github配置ssh key原理及操作步骤
运维·开发语言·数据库·windows·postgresql·ssh·github
standxy2 小时前
如何将钉钉新收款单数据高效集成到MySQL
数据库·mysql·钉钉
Narutolxy3 小时前
MySQL 权限困境：从权限丢失到权限重生的完整解决方案20241108
数据库·mysql
Venchill3 小时前
安装和卸载Mysql（压缩版）
数据库·mysql
Humbunklung3 小时前
一种EF（EntityFramework） MySQL修改表名去掉dbo前缀的方法
数据库·mysql·c#
PGCCC4 小时前
【PGCCC】postgresql 缓存池并发设计
数据库·缓存·postgresql
热门推荐
01如何才能让手机厂商主动拥抱华为，接入鸿蒙系统？02【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总03组基轨迹建模 GBTM的介绍与实现（Stata 或 R）04基于YOLOv10深度学习的CT扫描图像肾结石智能检测系统【python源码+Pyqt5界面+数据集+训练代码】深度学习实战、目标检测05RAG 实践- Ollama+RagFlow 部署本地知识库06【经验分享】Ubuntu22.04安装微信(linux官方版)07Macbook pro M1 安装Ubuntu教程08【Java】Ruoyi-Vue-Plus 整合 文件分片上传至 minio、阿里云、七牛云等OSS-【后端篇】09Windows10安装PCL1.14.0及点云配准10文件或文件夹名称中有空格如何批量去除