Redis List 实现的点对点任务队列实现排队获取IP对应的省市地址(解决访问频率过高导致被禁用问题)

背景:

我司原有的IP地址通过访问第三方链接获取具体的中文省市地址,近期业务发生变化,获取的频率变高,被第三方链接禁用了24小时。

采用List 队列模式原因:

1、没有采用rabbitmq或者RocketMq(当前项目业务量不大,引入这两个组件会增加业务维护复杂性)

2、没有采用Redis 的 Pub/Sub(发布订阅) 模式(不符合需求),消息发布后,所有在线订阅者都会实时收到,消息不持久化、不积压,消费者离线就会丢消息,更适合实时通知场景

3、List 队列模式:消息会持久化积压在列表里,只有一个消费者能取到,天然支持排队等待,更适配你「降频、排队处理、不丢任务」的核心需求

项目代码:

IpGeoConstants

IpGeoHttpUtil

java 复制代码
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.charset.StandardCharsets;

/**
 * IP属地查询专用HTTP工具(超时强制生效)
 */
public class IpGeoHttpUtil {
    /**
     * 发送GET请求,模拟浏览器请求头,显式指定超时
     */
    public static String sendGet(String url, int connectTimeout, int readTimeout) throws IOException {
        HttpURLConnection conn = null;
        InputStream is = null;
        ByteArrayOutputStream baos = null;
        try {
            URL realUrl = new URL(url);
            conn = (HttpURLConnection) realUrl.openConnection();
            conn.setRequestMethod("GET");
            conn.setConnectTimeout(connectTimeout);
            conn.setReadTimeout(readTimeout);
            conn.setUseCaches(false);
            conn.setInstanceFollowRedirects(true);

            // ========== 模拟Chrome浏览器请求头,消除爬虫特征 ==========
            conn.setRequestProperty("Host", "api.iping.cc");
            conn.setRequestProperty("Accept", "application/json, text/plain, */*");
            conn.setRequestProperty("Accept-Language", "zh-CN,zh;q=0.9,en;q=0.8");
            // 修复:声明不使用压缩,避免乱码解析失败
            conn.setRequestProperty("Accept-Encoding", "identity");
            conn.setRequestProperty("User-Agent",
                    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36");
            conn.setRequestProperty("Referer", "https://www.iping.cc/");
            conn.setRequestProperty("Connection", "keep-alive");
            conn.setRequestProperty("Sec-Fetch-Dest", "empty");
            conn.setRequestProperty("Sec-Fetch-Mode", "cors");
            conn.setRequestProperty("Sec-Fetch-Site", "same-site");

            conn.connect();

            int responseCode = conn.getResponseCode();
            if (responseCode != 200) {
                throw new IOException("HTTP状态码异常:" + responseCode);
            }

            is = conn.getInputStream();
            baos = new ByteArrayOutputStream();
            byte[] buffer = new byte[1024];
            int len;
            while ((len = is.read(buffer)) != -1) {
                baos.write(buffer, 0, len);
            }
            return baos.toString(StandardCharsets.UTF_8.name());
        } finally {
            if (is != null) { try { is.close(); } catch (IOException ignored) {} }
            if (baos != null) { try { baos.close(); } catch (IOException ignored) {} }
            if (conn != null) { conn.disconnect(); }
        }
    }
}