【Nginx系列】关于一次请求超时的思考

一.基础信息

1.问题背景

在服务器部署了一个接口,这个接口处理时间较长,超过了 1 分钟,使用如下 curl 请求接口

apl 复制代码

curl -X GET "https://cloud-test/imchat/userInfo/login?password=qyj123&userName=qyj123" -H "accept: */*"

稳定在 60s 时返回 504 超时错误

2.nginx 配置

apl 复制代码

user root;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;


include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;

    location /imchat {
    proxy_pass  http://0.0.0.0:8127$1;

    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_redirect off;
 }

3.请求链路

我理解的链路

客户端 curl 请求 nginx
nginx 请求代理服务器
代理服务器内部处理逻辑

二.逐步排查

1.原因分析

nginx 超时
内部服务器超时
curl 客户端超时

2.nginx 超时

apl 复制代码

user root;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;


include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    proxy_connect_timeout 300s; #单位秒
    proxy_send_timeout 300s; #单位秒
    proxy_read_timeout 300s; #单位秒
    keepalive_timeout 180s;  # 设置为180s
    client_header_timeout 180s;  # 设置为180s
    client_body_timeout 180s;  # 设置为180s
    send_timeout 180s; #设置为180s
 }

nginx 如图所示,配置后还是在在 60s 返回 504,并且查看 nginx 的日志返回 499,关于 nginx 499 单独作为一个章节说明,从这里可以说明超时不是 nginx 的问题

3.另一种排除 nginx 的方法

4.排查步骤

apl 复制代码

curl -v -X GET "https://www.zhanmeng.net/imchat/userInfo/login?password=qyj123&userName=qyj123" -H "accept: */*"

apl 复制代码

curl -v -X GET "https://cloud-test/imchat/userInfo/login?password=qyj123&userName=qyj123" -H "accept: */*"

三.nginx 499

二.精进学习

apl 复制代码

curl -v -X  POST 'https://cloud-test/v1/chat-messages' \
--header 'Authorization: Bearer app-v2N21o4qPwdBdeIOTNxxT3gi' \
--header 'Content-Type: application/json' \
--data-raw '{
    "inputs": {},
    "query": "高度为36米的单层厂房是否属于高层建筑？",
    "response_mode": "blocking",
    "conversation_id": "",
    "user": "abc-123"
}'

apl 复制代码

# curl不用设置超时时间
# flow的超时时间也是ok的
# nginx转发的时候是否设置正确?
curl -v -X POST 'http://10.201.2.165:5001/v1/chat-messages' \
--header 'Authorization: Bearer app-v2N21o4qPwdBdeIOTNxxT3gi' \
--header 'Content-Type: application/json' \
--data-raw '{
    "inputs": {},
    "query": "高度为36米的单层厂房是否属于高层建筑？",
    "response_mode": "blocking",
    "conversation_id": "",
    "user": "abc-123"
}'

apl 复制代码

tail -f /var/log/nginx/error.log


tail -f /var/log/nginx/access.log


"POST /v1/chat-messages HTTP/1.1" 499 0 "-" "curl/8.7.1" "14.29.124.4"

nginx -s reload

使用 curl 测试,在 60s 时会返回 499,报错信息如下(我在 login 接口中让线程睡眠了 75 秒),我希望客户端在 60s 时不断开,问题在哪里?如何修改?

apl 复制代码

curl -X GET "https://cloud-test/imchat/userInfo/login?password=qyj123&userName=qyj123" -H "accept: */*"
<html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
</body>
</html>

apl 复制代码

curl -v -X GET "https://cloud-test/imchat/userInfo/login?password=qyj123&userName=qyj123" -H "accept: */*"

apl 复制代码

curl -v -X GET "https://www.zhanmeng.net/imchat/userInfo/login?password=qyj123&userName=qyj123" -H "accept: */*"

apl 复制代码

user root;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;


include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    proxy_connect_timeout 300s; #单位秒
    proxy_send_timeout 300s; #单位秒
    proxy_read_timeout 300s; #单位秒
    keepalive_timeout 180s;  # 设置为180s
    client_header_timeout 180s;  # 设置为180s
    client_body_timeout 180s;  # 设置为180s
    send_timeout 180s; #设置为180s
 }

apl 复制代码

location /v1 {
    proxy_pass  http://0.0.0.0:5001$1;

    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_redirect off;

    proxy_read_timeout 120s;      # 等待后端响应的时间
    client_header_timeout 120s;    # 等待客户端请求头的时间
    client_body_timeout 120s;      # 等待客户端请求体的时间
    keepalive_timeout 120s;        # 保持连接的时间
    send_timeout 120s;             # 等待客户端接收数据的时间
}

apl 复制代码

    #proxy_ignore_client_abort  on;