[python]requests VS httpx VS aiohttp

花酒锄作田2025-06-11 23:01

前言

前段时间想着把一个python服务的接口逐渐改成异步的，其中用到requests的地方就要改成httpx或者aiohttp，有点好奇异步请求相较于同步请求有哪些提升，遂做了点小实验。

首先有个服务A提供接口，这个接口会停顿1秒，模拟数据库操作。服务B去请求服务A的这个接口，并把响应返回给客户端C。服务B提供4个接口，这4个接口分别用requests、httpx同步、httpx异步和aiohttp去请求服务A。

客户端使用wrk做请求测试。

实现服务A

服务A使用Go编写，用标准库即可完成

go 复制代码 
package main

import (
	"net/http"
	"time"
)


func main() {
	mux := http.NewServeMux()
	mux.HandleFunc("GET /a", func(w http.ResponseWriter, r *http.Request) {
		time.Sleep(1 * time.Second)
		w.Write([]byte("ok"))
	})
	if err := http.ListenAndServe("127.0.0.1:8000", mux); err != nil {
		panic(err)
	}
}

先用wrk直接请求试试，以此作为基准

bash 复制代码 
$ wrk -t8 -c1000 -d30s http://127.0.0.1:8000/a
Running 30s test @ http://127.0.0.1:8000/a
  8 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.00s     7.62ms   1.17s    97.34%
    Req/Sec   194.17    266.07     1.24k    86.82%
  29491 requests in 30.10s, 3.32MB read
Requests/sec:    979.85
Transfer/sec:    112.91KB

服务B: FastAPI

先用FastAPI做服务B试试

python 复制代码 
from fastapi import FastAPI
import httpx
import aiohttp
import uvicorn
import requests

app = FastAPI()
url = "http://127.0.0.1:8000/a"

@app.get("/sync1")
def sync1():
    try:
        resp = requests.get(url, timeout=2)
        resp.raise_for_status()
    except Exception as e:
        print(f"sync request failed, {e}")
    else:
        return resp.text
    
@app.get("/sync2")
def sync2():
    try:
        with httpx.Client() as client:
            resp = client.get(url, timeout=2)
            resp.raise_for_status()
    except Exception as e:
        print(f"sync2 request failed, {e}")
    else:
        return resp.text
    
@app.get("/async1")
async def async1():
    try:
        async with httpx.AsyncClient(timeout=2) as client:
            resp = await client.get(url)
            resp.raise_for_status()
    except Exception as e:
        print(f"async1 request failed, {e}")
    else:
        return resp.text
    
@app.get("/async2")
async def async2():
    try:
        async with aiohttp.ClientSession() as session:
            async with session.get(url, timeout=2) as resp:
                resp.raise_for_status()
                content = await resp.text()
    except Exception as e:
        print(f"async2 request failed, {e}")
    else:
        return content
    
if __name__ == "__main__":
    uvicorn.run("server1:app", host="127.0.0.1", port=8001, workers=4, access_log=False)

wrk请求结果。httpx不仅同步请求性能不如requests，没想到连异步请求性能也不如requests。而aiohttp以五倍多第二名的性能冠绝群雄。

API Total request QPS timeout comment
/sync1 4640 154.15 4480 requests 同步请求
/sync2 3631 120.87 3570 httpx 同步请求
/async1 4313 143.40 4254 httpx 异步请求
/async2 25379 843.35 0 aiohttp 异步请求

异步比同步性能还差，着实有点费解，遂找大模型问了下，大模型回复说httpx默认配置参数不高，可以额外指定参数，还需要避免反复创建http client。似乎有点道理，但是同步性能不如开箱即用的requests，异步性能不如开箱即用的aiohttp，我为什么还要折腾httpx呢？

服务B: Flask

Flask 2.0 也支持异步接口，但是之前测试性能并不是很好，拉出来一并测试瞧瞧实力。

Flask 版本：3.1.1。因为gunicorn运行异步接口会报错，所以用的flask内置webserver。

python 复制代码 
from flask import Flask
import requests
import httpx
import logging
import aiohttp

app = Flask(__name__)
url = "http://127.0.0.1:8000/a"

@app.get("/sync1")
def sync1():
    try:
        resp = requests.get(url, timeout=2)
        resp.raise_for_status()
    except Exception as e:
        print("request failed")
        return "request failed"
    else:
        return resp.text
    
@app.get("/sync2")
def sync2():
    try:
        with httpx.Client() as client:
            resp = client.get(url, timeout=2)
            resp.raise_for_status()
    except Exception as e:
        print("request failed")
        return "request failed"
    else:
        return resp.text

@app.get("/async1")
async def async1():
    try:
        async with httpx.AsyncClient(timeout=2) as client:
            resp = await client.get(url, timeout=2)
            resp.raise_for_status()
    except Exception as e:
        print("request failed")
        return "request failed"
    else:
        return resp.text
    
@app.get("/async2")
async def async2():
    try:
        async with aiohttp.ClientSession() as session:
            async with session.get(url, timeout=2) as response:
                response.raise_for_status()
                resp = await response.text()
    except Exception as e:
        print("request failed")
        return "request failed"
    else:
        return resp
    
if __name__ == "__main__":
    werkzeug_logger = logging.getLogger("werkzeug")
    werkzeug_logger.disabled = True
    app.run(host="127.0.0.1", port=8001)

测试结果。看来flask还是跟requests更搭，异步还不如同步。

API Total request QPS timeout comment
/sync1 13279 441.27 248 requests 同步请求
/sync2 2324 77.26 2323 httpx 同步请求
/async1 2330 77.46 2330 httpx 异步请求
/async2 8277 275.03 6887 aiohttp 异步请求

服务B: Sanic

再用Sanic测试一遍

python 复制代码 
from sanic import Sanic
from sanic.response import text
import requests
import httpx
import aiohttp

app = Sanic(__name__)
url = "http://127.0.0.1:8000/a"

@app.get("/sync1")
def sync1(request):
    try:
        resp = requests.get(url, timeout=2)
        resp.raise_for_status()
    except Exception as e:
        print(f"sync1 request failed, {e}")
    else:
        return text(resp.text)

@app.get("/sync2")
def sync2(request):
    try:
        with httpx.Client() as client:
            response = client.get(url, timeout=2)
            response.raise_for_status()
    except Exception as e:
        print(f"sync2 request failed, {e}")
    else:
        return text(response.text)

@app.get("/async1")
async def async1(request):
    try:
        async with httpx.AsyncClient() as client:
            response = await client.get(url, timeout=2)
            response.raise_for_status()
    except Exception as e:
        print(f"async1 request failed, {e}")
    else:
        return text(response.text)

@app.get("/async2")
async def async2(request):
    try:
        async with aiohttp.ClientSession() as session:
            async with session.get(url, timeout=2) as response:
                response.raise_for_status()
                content = await response.text()
    except Exception as e:
        print(f"async2 request failed, {e}")
    else:
        return text(content)

if __name__ == "__main__":
    app.run(host="127.0.0.1", port=8001, debug=False, access_log=False, workers=4)

可能是我对Sanic了解不多，单就这个测试结果来看，Sanic根本不适合编写同步API。而且使用httpx异步请求的时候有大量报错，wrk结果显示 Non-2xx or 3xx responses: 1244

API Total request QPS timeout comment
/sync1 37 1.23 35 requests 同步请求
/sync2 16 0.53 16 httpx 同步请求
/async1 5481 182.09 5339 httpx 异步请求
/async2 28116 934.67 0 aiohttp 异步请求

服务B: Go

最后再用Go实现下请求

go 复制代码 
func GetA(w http.ResponseWriter, r *http.Request) {
	resp, err := http.Get("http://127.0.0.1:8000/a")
	if err != nil {
		log.Println(err)
	}
	defer resp.Body.Close()
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Println(err)
	}
	w.Write(body)
}

func main() {
	mux := http.NewServeMux()
	mux.HandleFunc("GET /b", GetA)
	if err := http.ListenAndServe("127.0.0.1:8001", mux); err != nil {
		panic(err)
	}
}

测试结果，和直接请求服务A差别不大。

bash 复制代码 
$ wrk -t8 -c1000 -d30s http://127.0.0.1:8001/b
Running 30s test @ http://127.0.0.1:8001/b
  8 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.02s    85.69ms   1.66s    96.55%
    Req/Sec   210.49    175.05   740.00     63.65%
  29000 requests in 30.08s, 3.26MB read
Requests/sec:    963.97
Transfer/sec:    111.08KB

小结

用python编写同步请求还是老老实实用requests，异步接口应该用aiohttp，httpx的性能只能说能用。

相关推荐
大米2H4 小时前
Jupyter lab 配置两个python环境
ide·python·jupyter
猎嘤一号6 小时前
使用 PyTorch 和 TensorBoard 实时可视化模型训练
人工智能·pytorch·python
Takina~6 小时前
python打卡day49
python
Frankabcdefgh7 小时前
Python基础数据类型与运算符全面解析
开发语言·数据结构·python·面试
是梦终空7 小时前
Python毕业设计226—基于python+爬虫+html的豆瓣影视数据可视化系统(源代码+数据库+万字论文)
爬虫·python·html·毕业设计·毕业论文·源代码·豆瓣影视数据可视化
kaiaaaa7 小时前
算法训练第十五天
开发语言·python·算法
小玺玺7 小时前
[RDK X5] MJPG编解码开发实战：从官方API到OpenWanderary库的C++/Python实现
c++·python·opencv·rdk x5
zhuiQiuMX8 小时前
力扣LFU460
python·leetcode
noravinsc8 小时前
django 获取当前时间 格式 YYYY-MM-DD HH:Mm:ss
python·django·sqlite
谢李由202303220818 小时前
网络爬虫学习心得
爬虫·python
热门推荐
01Coze扣子平台完整体验和实践（附国内和国际版对比）02【图像处理与机器视觉】XJTU期末考点03KGG转MP3工具|非KGM文件|解密音频04坚果投影仪J10如何用苹果Siri开关机并和米家联动05海康Visionmaster-常见问题排查方法-启动阶段06YOLOv8入门 | 重要性能衡量指标、训练结果评价及分析及影响mAP的因素【发论文关注的指标】07Anaconda下的pkgs占用空间13G，如何安全的清理（已解决）08从零安装 LLaMA-Factory 微调 Qwen 大模型成功及所有的坑09【SpeedAI科研小助手】2分钟极速解决知网维普重复率、AIGC率过高，一键全文降！文件格式不变，公式都保留的！10【云原生】动态资源分配（DRA）深度洞察报告