目录
- FastAPI性能优化全攻略：构建高性能API服务
-
- 引言
- [1. FastAPI性能基础](#1. FastAPI性能基础)
-
- [1.1 FastAPI性能优势的来源](#1.1 FastAPI性能优势的来源)
- [1.2 性能基准测试](#1.2 性能基准测试)
- [2. 异步编程优化](#2. 异步编程优化)
-
- [2.1 正确使用async/await](#2.1 正确使用async/await)
- [2.2 使用异步数据库驱动](#2.2 使用异步数据库驱动)
- [3. 数据库优化策略](#3. 数据库优化策略)
-
- [3.1 连接池优化](#3.1 连接池优化)
- [3.2 查询优化技巧](#3.2 查询优化技巧)
- [4. 缓存策略优化](#4. 缓存策略优化)
-
- [4.1 多级缓存架构](#4.1 多级缓存架构)
- [4.2 Redis缓存实现](#4.2 Redis缓存实现)
- [5. 请求处理优化](#5. 请求处理优化)
-
- [5.1 中间件优化](#5.1 中间件优化)
- [5.2 请求限流与队列](#5.2 请求限流与队列)
- [6. 并发与并行优化](#6. 并发与并行优化)
-
- [6.1 异步任务处理](#6.1 异步任务处理)
- [6.2 连接复用与Keep-Alive](#6.2 连接复用与Keep-Alive)
- [7. 内存与资源管理](#7. 内存与资源管理)
-
- [7.1 内存优化](#7.1 内存优化)
- [7.2 数据库连接管理](#7.2 数据库连接管理)
- [8. 监控与性能分析](#8. 监控与性能分析)
-
- [8.1 性能监控仪表板](#8.1 性能监控仪表板)
- [8.2 集成Prometheus监控](#8.2 集成Prometheus监控)
- [9. 部署与生产环境优化](#9. 部署与生产环境优化)
-
- [9.1 Gunicorn/Uvicorn配置优化](#9.1 Gunicorn/Uvicorn配置优化)
- [9.2 Docker优化配置](#9.2 Docker优化配置)
- [10. 性能优化检查清单](#10. 性能优化检查清单)
-
- [10.1 部署前检查清单](#10.1 部署前检查清单)
- [11. 总结](#11. 总结)
- 附录：性能优化参考指标
『宝藏代码胶囊开张啦！』------ 我的 CodeCapsule 来咯！✨写代码不再头疼！我的新站点 CodeCapsule 主打一个 "白菜价"+"量身定制 "！无论是卡脖子的毕设/课设/文献复现 ，需要灵光一现的算法改进 ，还是想给项目加个"外挂"，这里都有便宜又好用的代码方案等你发现！低成本，高适配，助你轻松通关！速来围观 👉 CodeCapsule官网
FastAPI性能优化全攻略：构建高性能API服务
引言
FastAPI作为现代Python Web框架，以其卓越的性能和开发者友好性而闻名。然而，要充分发挥其性能潜力，需要深入了解其内部工作原理并应用恰当的优化策略。本文将深入探讨FastAPI性能优化的各个方面，从基础配置到高级技巧，帮助您构建真正高性能的API服务。
1. FastAPI性能基础
1.1 FastAPI性能优势的来源
FastAPI的高性能主要源于以下几个方面：
- 基于Starlette：使用异步Web框架Starlette构建
- Pydantic模型：使用Rust实现的类型验证
- 自动文档生成：不牺牲性能的开箱即用文档
- 异步支持：原生支持async/await语法
性能对比公式：
FastAPI性能 = α × 异步处理 + β × 类型验证优化 + γ × 中间件效率 \text{FastAPI性能} = \alpha \times \text{异步处理} + \beta \times \text{类型验证优化} + \gamma \times \text{中间件效率} FastAPI性能=α×异步处理+β×类型验证优化+γ×中间件效率
其中 α , β , γ \alpha, \beta, \gamma α,β,γ为权重系数。
1.2 性能基准测试
python
# benchmarks/performance_test.py
import asyncio
import time
import statistics
from fastapi import FastAPI, Depends
from fastapi.testclient import TestClient
import httpx
import pandas as pd
import matplotlib.pyplot as plt
app = FastAPI()
@app.get("/ping")
async def ping():
return {"message": "pong"}
@app.get("/compute")
async def compute_sync():
# 模拟CPU密集型计算
result = 0
for i in range(10000):
result += i * i
return {"result": result}
async def performance_benchmark():
"""性能基准测试"""
results = []
with TestClient(app) as client:
# 测试1000次请求
for i in range(1000):
start = time.perf_counter()
response = client.get("/ping")
end = time.perf_counter()
results.append(end - start)
# 计算统计指标
stats = {
"total_requests": len(results),
"avg_response_time": statistics.mean(results),
"p95_response_time": sorted(results)[int(len(results) * 0.95)],
"p99_response_time": sorted(results)[int(len(results) * 0.99)],
"min_response_time": min(results),
"max_response_time": max(results),
"requests_per_second": 1 / statistics.mean(results)
}
return stats
if __name__ == "__main__":
stats = asyncio.run(performance_benchmark())
print("性能基准测试结果:")
for key, value in stats.items():
print(f"{key}: {value:.6f}" if isinstance(value, float) else f"{key}: {value}")2. 异步编程优化
2.1 正确使用async/await
python
# 优化前：错误的异步使用
@app.get("/users/{user_id}")
async def get_user(user_id: int):
# 错误：在异步函数中执行同步阻塞操作
time.sleep(1) # 阻塞整个事件循环
return {"user_id": user_id}
# 优化后：正确的异步使用
import asyncio
@app.get("/users/{user_id}")
async def get_user(user_id: int):
# 使用异步休眠
await asyncio.sleep(1) # 非阻塞
return {"user_id": user_id}2.2 使用异步数据库驱动
python
# 使用异步数据库连接
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy import select
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
# 创建异步引擎（注意postgresql使用asyncpg驱动）
DATABASE_URL = "postgresql+asyncpg://user:password@localhost/dbname"
engine = create_async_engine(
DATABASE_URL,
echo=False,
pool_size=20, # 连接池大小
max_overflow=10, # 最大溢出连接数
pool_pre_ping=True, # 连接前检查
pool_recycle=3600, # 连接回收时间（秒）
)
AsyncSessionLocal = sessionmaker(
engine,
class_=AsyncSession,
expire_on_commit=False
)
# 异步数据库模型
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key=True, index=True)
username = Column(String, unique=True, index=True)
email = Column(String, unique=True, index=True)
# 异步CRUD操作
async def get_user_by_id(user_id: int, db: AsyncSession):
"""异步获取用户"""
result = await db.execute(
select(User).where(User.id == user_id)
)
return result.scalar_one_or_none()
@app.get("/users/{user_id}")
async def read_user(user_id: int, db: AsyncSession = Depends(get_db)):
user = await get_user_by_id(user_id, db)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
return user3. 数据库优化策略
3.1 连接池优化
python
# config/database.py
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine, async_sessionmaker
from contextlib import asynccontextmanager
class DatabaseManager:
"""数据库管理器，优化连接池配置"""
def __init__(self, database_url: str):
self.engine = create_async_engine(
database_url,
# 连接池配置
pool_size=20, # 保持在池中的连接数
max_overflow=30, # 超过pool_size时允许的最大连接数
pool_timeout=30, # 获取连接的超时时间（秒）
pool_recycle=1800, # 连接回收时间，避免数据库断开
pool_pre_ping=True, # 每次连接前检查
echo=False, # 生产环境设为False
# PostgreSQL特定优化
connect_args={
"server_settings": {
"jit": "off", # 对于短查询，关闭JIT可能更快
"statement_timeout": "5000", # 5秒超时
}
}
)
self.async_session = async_sessionmaker(
self.engine,
expire_on_commit=False,
class_=AsyncSession,
)
@asynccontextmanager
async def get_session(self):
"""获取数据库会话的上下文管理器"""
async with self.async_session() as session:
try:
yield session
await session.commit()
except Exception:
await session.rollback()
raise
finally:
await session.close()
async def optimize_queries(self):
"""执行数据库优化"""
async with self.engine.begin() as conn:
# 更新统计信息
await conn.execute("ANALYZE users;")
# 清理膨胀
await conn.execute("VACUUM ANALYZE users;")
# 使用示例
db_manager = DatabaseManager(DATABASE_URL)
@app.get("/optimized-users")
async def get_optimized_users(page: int = 1, page_size: int = 50):
"""使用优化的分页查询"""
async with db_manager.get_session() as session:
# 使用offset/limit分页（对于小型数据集）
query = select(User).offset((page - 1) * page_size).limit(page_size)
# 对于大型数据集，使用keyset分页（游标分页）
# query = select(User).where(User.id > last_id).limit(page_size)
result = await session.execute(query)
users = result.scalars().all()
# 获取总数（考虑缓存此值）
count_query = select(func.count()).select_from(User)
total_result = await session.execute(count_query)
total = total_result.scalar()
return {
"users": users,
"page": page,
"page_size": page_size,
"total": total,
"total_pages": (total + page_size - 1) // page_size
}3.2 查询优化技巧
python
# optimizations/query_optimization.py
from sqlalchemy import select, func, exists
from sqlalchemy.orm import selectinload, joinedload
from sqlalchemy.sql.expression import and_, or_
class QueryOptimizer:
"""查询优化器"""
@staticmethod
async def optimize_relationship_loading(session, user_id: int):
"""优化关联关系加载"""
# 优化前：N+1查询问题
# users = await session.execute(select(User))
# for user in users.scalars():
# posts = await session.execute(select(Post).where(Post.user_id == user.id))
# 优化后：使用join或selectinload
query = (
select(User)
.options(selectinload(User.posts)) # 适合一对多关系
.where(User.id == user_id)
)
result = await session.execute(query)
return result.scalar_one()
@staticmethod
async def use_index_hints(session):
"""使用索引提示（数据库特定）"""
# PostgreSQL示例
query = select(User).where(
User.username.ilike('john%')
)
# 对于复杂查询，考虑使用CTE或物化视图
return await session.execute(query)
@staticmethod
async def batch_operations(session, user_ids: list[int]):
"""批量操作优化"""
# 批量查询
query = select(User).where(User.id.in_(user_ids))
result = await session.execute(query)
# 批量更新
update_stmt = (
update(User)
.where(User.id.in_(user_ids))
.values(is_active=True)
.execution_options(synchronize_session="fetch")
)
await session.execute(update_stmt)
return result.scalars().all()
# 索引优化建议
INDEX_OPTIMIZATIONS = """
-- 创建合适索引
CREATE INDEX CONCURRENTLY idx_users_email ON users(email);
CREATE INDEX CONCURRENTLY idx_users_username ON users(username);
-- 复合索引
CREATE INDEX CONCURRENTLY idx_users_status_created
ON users(status, created_at DESC);
-- 部分索引
CREATE INDEX CONCURRENTLY idx_active_users
ON users(id) WHERE is_active = true;
-- 表达式索引
CREATE INDEX CONCURRENTLY idx_users_lower_email
ON users(LOWER(email));
"""4. 缓存策略优化
4.1 多级缓存架构
是 否 是 否 客户端请求 CDN缓存 命中? 返回CDN缓存 FastAPI应用 内存缓存
Redis/Memcached 命中? 返回缓存 数据库查询 数据库缓存 返回数据 更新Redis缓存 设置CDN缓存 返回客户端
4.2 Redis缓存实现
python
# cache/redis_manager.py
import redis.asyncio as redis
from typing import Any, Optional, Union
import pickle
import json
from datetime import timedelta
import hashlib
class RedisCacheManager:
"""Redis缓存管理器"""
def __init__(self, redis_url: str = "redis://localhost:6379/0"):
self.redis = redis.from_url(
redis_url,
encoding="utf-8",
decode_responses=False, # 保留二进制数据
max_connections=20,
socket_keepalive=True,
)
# 默认TTL配置
self.default_ttl = {
"short": timedelta(minutes=5),
"medium": timedelta(hours=1),
"long": timedelta(hours=24),
"session": timedelta(days=7),
}
def generate_cache_key(self, prefix: str, *args, **kwargs) -> str:
"""生成缓存键"""
key_parts = [prefix]
# 添加位置参数
for arg in args:
if isinstance(arg, (str, int, float, bool)):
key_parts.append(str(arg))
# 添加关键字参数
for key, value in sorted(kwargs.items()):
if isinstance(value, (str, int, float, bool)):
key_parts.append(f"{key}:{value}")
# 创建哈希
key_str = ":".join(key_parts)
return f"cache:{hashlib.md5(key_str.encode()).hexdigest()}"
async def get_or_set(
self,
key: str,
fetch_func,
ttl: Optional[timedelta] = None,
force_refresh: bool = False
) -> Any:
"""
获取或设置缓存
:param key: 缓存键
:param fetch_func: 获取数据的函数
:param ttl: 过期时间
:param force_refresh: 强制刷新缓存
:return: 数据
"""
# 如果强制刷新，直接获取新数据
if force_refresh:
data = await fetch_func()
await self.set(key, data, ttl)
return data
# 尝试从缓存获取
cached_data = await self.get(key)
if cached_data is not None:
return cached_data
# 缓存未命中，获取数据并缓存
data = await fetch_func()
await self.set(key, data, ttl)
return data
async def get(self, key: str) -> Optional[Any]:
"""获取缓存"""
try:
data = await self.redis.get(key)
if data:
return pickle.loads(data)
except (pickle.PickleError, redis.RedisError) as e:
print(f"Cache get error: {e}")
return None
async def set(self, key: str, value: Any, ttl: Optional[timedelta] = None) -> bool:
"""设置缓存"""
try:
serialized = pickle.dumps(value)
if ttl:
await self.redis.setex(key, int(ttl.total_seconds()), serialized)
else:
await self.redis.set(key, serialized)
return True
except (pickle.PickleError, redis.RedisError) as e:
print(f"Cache set error: {e}")
return False
async def delete_pattern(self, pattern: str) -> int:
"""删除匹配模式的缓存键"""
keys = await self.redis.keys(pattern)
if keys:
return await self.redis.delete(*keys)
return 0
async def invalidate_user_cache(self, user_id: int):
"""使指定用户的缓存失效"""
patterns = [
f"cache:*:user:{user_id}:*",
f"cache:*:user_id:{user_id}:*",
f"user:{user_id}:*"
]
for pattern in patterns:
await self.delete_pattern(pattern)
# 缓存装饰器
def cache_response(ttl: timedelta = None, key_prefix: str = "api"):
"""缓存API响应装饰器"""
def decorator(func):
async def wrapper(*args, **kwargs):
# 获取缓存管理器
cache_manager = get_cache_manager()
# 生成缓存键
cache_key = cache_manager.generate_cache_key(
key_prefix,
func.__name__,
*args,
**{k: v for k, v in kwargs.items()
if k not in ['request', 'db', 'current_user']}
)
# 尝试从缓存获取
cached_response = await cache_manager.get(cache_key)
if cached_response is not None:
return cached_response
# 执行函数
result = await func(*args, **kwargs)
# 缓存结果
await cache_manager.set(cache_key, result, ttl)
return result
return wrapper
return decorator
# 使用示例
cache_manager = RedisCacheManager()
@app.get("/users/{user_id}")
@cache_response(ttl=timedelta(minutes=5))
async def get_user_cached(
user_id: int,
db: AsyncSession = Depends(get_db),
cache: RedisCacheManager = Depends(get_cache_manager)
):
"""带缓存的用户获取接口"""
async with db_manager.get_session() as session:
user = await get_user_by_id(user_id, session)
if not user:
raise HTTPException(status_code=404)
# 转换响应格式
return {
"id": user.id,
"username": user.username,
"email": user.email,
"cached": True
}5. 请求处理优化
5.1 中间件优化
python
# middleware/optimization_middleware.py
from fastapi import FastAPI, Request
from fastapi.middleware.cors import CORSMiddleware
from fastapi.middleware.gzip import GZipMiddleware
from fastapi.middleware.httpsredirect import HTTPSRedirectMiddleware
import time
import gzip
from typing import Callable
from starlette.types import ASGIApp
class OptimizationMiddleware:
"""优化中间件集合"""
@staticmethod
def add_standard_middlewares(app: FastAPI):
"""添加标准优化中间件"""
# CORS中间件
app.add_middleware(
CORSMiddleware,
allow_origins=["https://example.com"], # 生产环境指定具体域名
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
max_age=600, # 预检请求缓存时间
)
# GZIP压缩中间件
app.add_middleware(
GZipMiddleware,
minimum_size=1000, # 只压缩大于1KB的响应
)
# 添加自定义优化中间件
@app.middleware("http")
async def add_security_headers(request: Request, call_next):
"""添加安全头信息"""
response = await call_next(request)
# 安全头
response.headers["X-Content-Type-Options"] = "nosniff"
response.headers["X-Frame-Options"] = "DENY"
response.headers["X-XSS-Protection"] = "1; mode=block"
response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains"
# 性能头
response.headers["Connection"] = "keep-alive"
response.headers["Keep-Alive"] = "timeout=5, max=100"
return response
@app.middleware("http")
async def add_performance_metrics(request: Request, call_next):
"""性能监控中间件"""
start_time = time.time()
# 添加请求ID用于追踪
request_id = request.headers.get("X-Request-ID", str(time.time()))
try:
response = await call_next(request)
# 计算处理时间
process_time = time.time() - start_time
# 添加性能头
response.headers["X-Process-Time"] = str(process_time)
response.headers["X-Request-ID"] = request_id
# 日志记录（生产环境应该使用结构化日志）
if process_time > 1.0: # 慢请求警告
print(f"Slow request: {request.method} {request.url.path} "
f"took {process_time:.3f}s")
return response
except Exception as e:
# 异常处理
process_time = time.time() - start_time
print(f"Request failed: {request.method} {request.url.path} "
f"error: {str(e)} time: {process_time:.3f}s")
raise
class ResponseCompressor:
"""自定义响应压缩器（针对特定类型优化）"""
@staticmethod
async def compress_json_response(response):
"""压缩JSON响应"""
if response.status_code == 200:
body = await response.body()
# 只压缩特定大小的响应
if len(body) > 1024: # 大于1KB
compressed = gzip.compress(body, compresslevel=6) # 中等压缩级别
# 更新响应
response.body = compressed
response.headers["Content-Encoding"] = "gzip"
response.headers["Content-Length"] = str(len(compressed))
return response5.2 请求限流与队列
python
# security/rate_limiter.py
import asyncio
from collections import defaultdict
from datetime import datetime, timedelta
from typing import Dict, Tuple
import heapq
class TokenBucketRateLimiter:
"""令牌桶限流器"""
def __init__(self, rate: float, capacity: int):
"""
:param rate: 令牌填充速率（个/秒）
:param capacity: 桶容量
"""
self.rate = rate
self.capacity = capacity
self.tokens = capacity
self.last_update = datetime.now()
self.lock = asyncio.Lock()
async def acquire(self, tokens: int = 1) -> bool:
"""获取令牌"""
async with self.lock:
now = datetime.now()
elapsed = (now - self.last_update).total_seconds()
# 填充令牌
self.tokens = min(
self.capacity,
self.tokens + elapsed * self.rate
)
self.last_update = now
# 检查是否有足够令牌
if self.tokens >= tokens:
self.tokens -= tokens
return True
return False
class SlidingWindowRateLimiter:
"""滑动窗口限流器"""
def __init__(self, max_requests: int, window_seconds: int):
self.max_requests = max_requests
self.window_seconds = window_seconds
self.requests = defaultdict(list)
self.lock = asyncio.Lock()
async def is_allowed(self, client_id: str) -> Tuple[bool, float]:
"""检查是否允许请求"""
async with self.lock:
now = datetime.now()
window_start = now - timedelta(seconds=self.window_seconds)
# 清理过期请求
self.requests[client_id] = [
req_time for req_time in self.requests[client_id]
if req_time > window_start
]
# 检查请求数量
if len(self.requests[client_id]) < self.max_requests:
self.requests[client_id].append(now)
return True, 0.0
# 计算需要等待的时间
oldest_request = min(self.requests[client_id])
wait_time = (window_start - oldest_request).total_seconds()
return False, max(0.0, wait_time)
class RequestQueue:
"""请求队列，防止系统过载"""
def __init__(self, max_concurrent: int = 100):
self.max_concurrent = max_concurrent
self.current_requests = 0
self.waiting_queue = []
self.lock = asyncio.Lock()
self.condition = asyncio.Condition()
async def enter(self) -> bool:
"""进入队列"""
async with self.lock:
if self.current_requests < self.max_concurrent:
self.current_requests += 1
return True
# 添加到等待队列
future = asyncio.Future()
heapq.heappush(self.waiting_queue, (datetime.now(), future))
return await future
async def exit(self):
"""离开队列"""
async with self.lock:
self.current_requests -= 1
# 唤醒等待的请求
if self.waiting_queue and self.current_requests < self.max_concurrent:
_, future = heapq.heappop(self.waiting_queue)
self.current_requests += 1
future.set_result(True)
# 使用限流器
global_rate_limiter = TokenBucketRateLimiter(rate=100, capacity=200) # 100请求/秒
user_rate_limiter = SlidingWindowRateLimiter(max_requests=10, window_seconds=60) # 10请求/分钟
request_queue = RequestQueue(max_concurrent=500)
@app.middleware("http")
async def rate_limit_middleware(request: Request, call_next):
"""限流中间件"""
client_ip = request.client.host if request.client else "unknown"
# 全局限流
if not await global_rate_limiter.acquire():
return JSONResponse(
status_code=429,
content={"detail": "Too many requests"},
headers={"Retry-After": "1"}
)
# 用户级限流
allowed, wait_time = await user_rate_limiter.is_allowed(client_ip)
if not allowed:
return JSONResponse(
status_code=429,
content={"detail": f"Rate limit exceeded. Try again in {wait_time:.1f}s"},
headers={"Retry-After": str(int(wait_time))}
)
# 请求队列
if not await request_queue.enter():
return JSONResponse(
status_code=503,
content={"detail": "Service temporarily overloaded"}
)
try:
response = await call_next(request)
return response
finally:
await request_queue.exit()6. 并发与并行优化
6.1 异步任务处理
python
# workers/async_worker.py
import asyncio
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
from typing import List, Any, Callable
import time
class AsyncWorkerPool:
"""异步工作池"""
def __init__(self, max_workers: int = 4):
self.max_workers = max_workers
self.thread_pool = ThreadPoolExecutor(max_workers=max_workers)
self.process_pool = ProcessPoolExecutor(max_workers=max_workers)
async def run_io_bound(self, func: Callable, *args) -> Any:
"""运行I/O密集型任务"""
loop = asyncio.get_event_loop()
return await loop.run_in_executor(self.thread_pool, func, *args)
async def run_cpu_bound(self, func: Callable, *args) -> Any:
"""运行CPU密集型任务"""
loop = asyncio.get_event_loop()
return await loop.run_in_executor(self.process_pool, func, *args)
async def batch_process(self, items: List, process_func: Callable,
batch_size: int = 10) -> List:
"""批量处理"""
results = []
semaphore = asyncio.Semaphore(self.max_workers)
async def process_with_semaphore(item):
async with semaphore:
return await process_func(item)
tasks = [process_with_semaphore(item) for item in items]
# 分批处理
for i in range(0, len(tasks), batch_size):
batch = tasks[i:i + batch_size]
batch_results = await asyncio.gather(*batch, return_exceptions=True)
results.extend(batch_results)
return results
# CPU密集型任务优化示例
def optimize_image_cpu(image_data: bytes) -> bytes:
"""CPU密集型图像处理（使用多进程）"""
# 这里使用PIL进行图像处理
from PIL import Image
import io
image = Image.open(io.BytesIO(image_data))
# 优化处理
image = image.convert('RGB')
# 调整大小（保持宽高比）
max_size = (1920, 1080)
image.thumbnail(max_size, Image.Resampling.LANCZOS)
# 优化JPEG质量
output = io.BytesIO()
image.save(output, format='JPEG', quality=85, optimize=True)
return output.getvalue()
@app.post("/optimize-images")
async def optimize_images_batch(files: List[UploadFile]):
"""批量优化图像（并行处理）"""
worker_pool = AsyncWorkerPool(max_workers=4)
async def process_file(file: UploadFile):
# 读取文件
content = await file.read()
# 使用进程池处理CPU密集型任务
optimized = await worker_pool.run_cpu_bound(optimize_image_cpu, content)
return {
"filename": file.filename,
"original_size": len(content),
"optimized_size": len(optimized),
"compression_ratio": len(optimized) / len(content) if content else 0
}
# 并行处理所有文件
results = await asyncio.gather(
*[process_file(file) for file in files],
return_exceptions=True
)
# 处理结果
successful = []
failed = []
for i, result in enumerate(results):
if isinstance(result, Exception):
failed.append({"filename": files[i].filename, "error": str(result)})
else:
successful.append(result)
return {
"successful": successful,
"failed": failed,
"total_files": len(files)
}6.2 连接复用与Keep-Alive
python
# config/http_client.py
import httpx
from typing import Optional
class OptimizedHTTPClient:
"""优化的HTTP客户端"""
_instance: Optional[httpx.AsyncClient] = None
@classmethod
async def get_client(cls) -> httpx.AsyncClient:
"""获取或创建HTTP客户端（单例模式）"""
if cls._instance is None:
cls._instance = httpx.AsyncClient(
# 连接池配置
limits=httpx.Limits(
max_connections=100, # 最大连接数
max_keepalive_connections=50, # 保持活动的连接数
keepalive_expiry=5.0, # 保持活动时间
),
# 超时配置
timeout=httpx.Timeout(
connect=5.0, # 连接超时
read=30.0, # 读取超时
write=30.0, # 写入超时
pool=1.0, # 从池获取连接的超时
),
# 传输配置
transport=httpx.AsyncHTTPTransport(
retries=2, # 重试次数
http2=True, # 启用HTTP/2
),
# 其他配置
follow_redirects=True,
max_redirects=5,
)
return cls._instance
@classmethod
async def close(cls):
"""关闭客户端"""
if cls._instance:
await cls._instance.aclose()
cls._instance = None
# 使用优化客户端
@app.get("/fetch-external")
async def fetch_external_data(url: str):
"""获取外部数据（使用优化客户端）"""
client = await OptimizedHTTPClient.get_client()
try:
# 设置请求头
headers = {
"User-Agent": "FastAPI-Optimized/1.0",
"Accept": "application/json",
"Accept-Encoding": "gzip, deflate, br",
}
# 发送请求
response = await client.get(url, headers=headers)
response.raise_for_status()
# 处理响应
data = response.json()
return {
"status": "success",
"data": data,
"response_time": response.elapsed.total_seconds(),
"content_length": len(response.content)
}
except httpx.RequestError as e:
raise HTTPException(
status_code=503,
detail=f"External service error: {str(e)}"
)
except httpx.HTTPStatusError as e:
raise HTTPException(
status_code=e.response.status_code,
detail=f"External service returned error: {str(e)}"
)7. 内存与资源管理
7.1 内存优化
python
# optimizations/memory_optimization.py
import gc
import tracemalloc
from typing import List, Dict, Any
import numpy as np
from pympler import asizeof
from collections import defaultdict
class MemoryOptimizer:
"""内存优化器"""
@staticmethod
def optimize_pydantic_models():
"""优化Pydantic模型配置"""
from pydantic import BaseModel
class OptimizedUser(BaseModel):
id: int
username: str
email: str
class Config:
# 性能优化配置
anystr_strip_whitespace = True # 自动去除空格
validate_assignment = True # 赋值时验证
arbitrary_types_allowed = False # 不允许任意类型
use_enum_values = True # 使用枚举值
# 禁用额外字段
extra = 'forbid' # 生产环境使用'forbid'
# 启用ORM模式
orm_mode = True
return OptimizedUser
@staticmethod
def measure_memory_usage(obj: Any) -> int:
"""测量对象内存使用"""
return asizeof.asizeof(obj)
@staticmethod
def find_memory_leaks():
"""查找内存泄漏"""
# 开始追踪内存
tracemalloc.start()
# 执行一些操作...
# 获取内存快照
snapshot = tracemalloc.take_snapshot()
# 分析内存使用
top_stats = snapshot.statistics('lineno')
print("[ Top 10 memory usage ]")
for stat in top_stats[:10]:
print(stat)
# 停止追踪
tracemalloc.stop()
@staticmethod
def optimize_data_structures(data_list: List[Dict]) -> List:
"""优化数据结构"""
# 使用生成器减少内存
def process_items(items):
for item in items:
# 只保留必要字段
yield {
'id': item.get('id'),
'name': item.get('name'),
# 移除不需要的字段
}
return list(process_items(data_list))
@staticmethod
def use_slots_for_classes():
"""使用__slots__减少内存"""
class OptimizedUser:
__slots__ = ['id', 'username', 'email', '__dict__']
def __init__(self, user_id: int, username: str, email: str):
self.id = user_id
self.username = username
self.email = email
return OptimizedUser
@staticmethod
def force_garbage_collection():
"""强制垃圾回收"""
# 收集第0代和第1代
collected = gc.collect(0)
collected += gc.collect(1)
# 打印统计信息
print(f"Garbage collected: {collected} objects")
# 获取GC统计
gc_stats = gc.get_stats()
for gen, stats in enumerate(gc_stats):
print(f"Generation {gen}: {stats}")
# 大文件处理优化
@app.post("/upload-large-file")
async def upload_large_file(file: UploadFile):
"""处理大文件上传（内存优化）"""
# 使用流式处理
total_size = 0
chunk_size = 1024 * 1024 # 1MB chunks
# 临时文件处理
import tempfile
import shutil
try:
# 创建临时文件
with tempfile.NamedTemporaryFile(delete=False, suffix='.tmp') as tmp:
# 分块读取和写入
while True:
chunk = await file.read(chunk_size)
if not chunk:
break
tmp.write(chunk)
total_size += len(chunk)
# 每处理10MB检查一次内存
if total_size % (10 * 1024 * 1024) == 0:
MemoryOptimizer.force_garbage_collection()
tmp_path = tmp.name
# 处理文件
# ...
return {
"filename": file.filename,
"size": total_size,
"status": "processed"
}
finally:
# 清理临时文件
if 'tmp_path' in locals():
import os
os.unlink(tmp_path)7.2 数据库连接管理
python
# config/connection_pool.py
from sqlalchemy.pool import QueuePool, StaticPool
import asyncio
from contextlib import asynccontextmanager
class ConnectionPoolManager:
"""数据库连接池管理器"""
def __init__(self, database_url: str, min_connections: int = 5,
max_connections: int = 20):
self.database_url = database_url
self.min_connections = min_connections
self.max_connections = max_connections
# 连接池统计
self.stats = {
'total_connections': 0,
'active_connections': 0,
'waiting_requests': 0,
}
self.stats_lock = asyncio.Lock()
async def get_optimized_pool(self):
"""获取优化连接池"""
from sqlalchemy.ext.asyncio import create_async_engine
return create_async_engine(
self.database_url,
poolclass=QueuePool, # 使用队列池
pool_size=self.min_connections,
max_overflow=self.max_connections - self.min_connections,
pool_timeout=30, # 等待连接的超时时间
pool_recycle=3600, # 1小时后回收连接
pool_pre_ping=True, # 连接前检查
pool_use_lifo=True, # LIFO模式，提高缓存命中率
# PostgreSQL优化参数
connect_args={
"server_settings": {
"statement_timeout": "30000", # 30秒语句超时
"idle_in_transaction_session_timeout": "10000", # 10秒空闲超时
"lock_timeout": "10000", # 10秒锁超时
}
}
)
@asynccontextmanager
async def get_connection(self):
"""获取数据库连接（带统计）"""
async with self.stats_lock:
self.stats['waiting_requests'] += 1
try:
# 获取连接...
async with self.get_optimized_pool().connect() as conn:
async with self.stats_lock:
self.stats['active_connections'] += 1
self.stats['waiting_requests'] -= 1
yield conn
finally:
async with self.stats_lock:
self.stats['active_connections'] -= 1
async def monitor_pool_health(self):
"""监控连接池健康状态"""
while True:
await asyncio.sleep(60) # 每分钟检查一次
async with self.stats_lock:
stats = self.stats.copy()
# 检查健康状态
utilization = (stats['active_connections'] / self.max_connections) * 100
if utilization > 80:
print(f"警告：连接池使用率 {utilization:.1f}%")
if stats['waiting_requests'] > 10:
print(f"警告：{stats['waiting_requests']} 个请求在等待连接")
# 记录统计信息
print(f"连接池统计: {stats}")
# 启动健康监控
@app.on_event("startup")
async def startup_event():
"""应用启动事件"""
pool_manager = ConnectionPoolManager(DATABASE_URL)
# 启动监控任务
asyncio.create_task(pool_manager.monitor_pool_health())8. 监控与性能分析
8.1 性能监控仪表板
FastAPI应用 Prometheus指标 结构化日志 分布式追踪 Grafana仪表板 ELK堆栈 Jaeger/Zipkin 实时监控 日志分析 请求追踪 性能警报 错误分析 瓶颈识别
8.2 集成Prometheus监控
python
# monitoring/prometheus_integration.py
from prometheus_client import Counter, Histogram, Gauge, generate_latest
from prometheus_client.core import CollectorRegistry
import time
from typing import Callable
from fastapi import FastAPI, Request, Response
from fastapi.routing import APIRoute
import asyncio
class MetricsCollector:
"""指标收集器"""
def __init__(self):
self.registry = CollectorRegistry()
# HTTP请求指标
self.request_count = Counter(
'http_requests_total',
'Total HTTP requests',
['method', 'endpoint', 'status'],
registry=self.registry
)
self.request_duration = Histogram(
'http_request_duration_seconds',
'HTTP request duration',
['method', 'endpoint'],
buckets=[0.01, 0.05, 0.1, 0.5, 1.0, 2.0, 5.0],
registry=self.registry
)
self.request_size = Histogram(
'http_request_size_bytes',
'HTTP request size',
['method', 'endpoint'],
buckets=[100, 1000, 10000, 100000, 1000000],
registry=self.registry
)
# 业务指标
self.user_registrations = Counter(
'user_registrations_total',
'Total user registrations',
registry=self.registry
)
self.active_users = Gauge(
'active_users_current',
'Current active users',
registry=self.registry
)
# 系统指标
self.memory_usage = Gauge(
'process_memory_usage_bytes',
'Process memory usage',
registry=self.registry
)
self.cpu_usage = Gauge(
'process_cpu_usage_percent',
'Process CPU usage',
registry=self.registry
)
def record_request(self, method: str, endpoint: str,
status_code: int, duration: float,
request_size: int = 0):
"""记录请求指标"""
self.request_count.labels(
method=method,
endpoint=endpoint,
status=status_code
).inc()
self.request_duration.labels(
method=method,
endpoint=endpoint
).observe(duration)
if request_size > 0:
self.request_size.labels(
method=method,
endpoint=endpoint
).observe(request_size)
def update_system_metrics(self):
"""更新系统指标"""
import psutil
import os
process = psutil.Process(os.getpid())
# 内存使用
memory_info = process.memory_info()
self.memory_usage.set(memory_info.rss)
# CPU使用率
cpu_percent = process.cpu_percent(interval=0.1)
self.cpu_usage.set(cpu_percent)
# 监控中间件
metrics = MetricsCollector()
@app.middleware("http")
async def metrics_middleware(request: Request, call_next):
"""指标收集中间件"""
start_time = time.time()
# 计算请求大小
request_size = 0
if request.method in ["POST", "PUT", "PATCH"]:
body = await request.body()
request_size = len(body)
# 恢复body以供后续使用
async def receive():
return {"type": "http.request", "body": body}
request._receive = receive
# 处理请求
response = await call_next(request)
# 计算处理时间
duration = time.time() - start_time
# 记录指标
endpoint = request.url.path
metrics.record_request(
method=request.method,
endpoint=endpoint,
status_code=response.status_code,
duration=duration,
request_size=request_size
)
return response
# Prometheus端点
@app.get("/metrics")
async def get_metrics():
"""Prometheus指标端点"""
# 更新系统指标
metrics.update_system_metrics()
return Response(
content=generate_latest(metrics.registry),
media_type="text/plain"
)
# 自定义路由类，添加指标支持
class InstrumentedAPIRoute(APIRoute):
"""支持指标收集的自定义路由"""
def get_route_handler(self) -> Callable:
original_handler = super().get_route_handler()
async def instrumented_handler(request: Request):
start_time = time.time()
try:
response = await original_handler(request)
# 记录业务指标
if self.path == "/api/auth/register":
metrics.user_registrations.inc()
return response
except Exception as e:
# 记录错误指标
metrics.request_count.labels(
method=request.method,
endpoint=self.path,
status="error"
).inc()
raise
return instrumented_handler
# 使用自定义路由
app.router.route_class = InstrumentedAPIRoute
# 健康检查端点
@app.get("/health")
async def health_check():
"""健康检查端点，包含性能指标"""
import psutil
import os
process = psutil.Process(os.getpid())
return {
"status": "healthy",
"timestamp": time.time(),
"memory_usage_mb": process.memory_info().rss / 1024 / 1024,
"cpu_percent": process.cpu_percent(interval=0.1),
"thread_count": process.num_threads(),
"open_files": len(process.open_files()),
"connections": len(process.connections()),
"metrics": {
"request_count": metrics.request_count._value.get(),
"active_connections": 0, # 从连接池管理器获取
}
}9. 部署与生产环境优化
9.1 Gunicorn/Uvicorn配置优化
yaml
# gunicorn_config.py
import multiprocessing
import os
# 工作进程数
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = "uvicorn.workers.UvicornWorker"
# 绑定地址
bind = "0.0.0.0:8000"
# 工作进程配置
worker_connections = 1000
keepalive = 5
timeout = 120
graceful_timeout = 30
# 日志配置
accesslog = "-"
errorlog = "-"
loglevel = "info"
# 进程名称
proc_name = "fastapi_app"
# 安全配置
limit_request_line = 4096
limit_request_fields = 100
limit_request_field_size = 8190
# 性能优化
preload_app = True # 预加载应用
max_requests = 1000 # 每个工作进程处理的最大请求数
max_requests_jitter = 50 # 随机抖动，防止所有工作进程同时重启
# 环境变量
raw_env = [
"PYTHONPATH=/app",
"PYTHONUNBUFFERED=true",
]
# Uvicorn特定配置
uvicorn_options = {
"http": "h11", # HTTP协议实现
"loop": "uvloop", # 使用uvloop（如果可用）
"interface": "asgi3",
"lifespan": "on",
"access_log": False, # 禁用Uvicorn访问日志，使用Gunicorn的
"proxy_headers": True,
"forwarded_allow_ips": "*",
}
# Docker优化
if os.getenv("IN_DOCKER"):
# Docker特定优化
workers = min(workers, 4) # 限制Docker中的工作进程数
preload_app = True9.2 Docker优化配置
dockerfile
# Dockerfile
# 多阶段构建
FROM python:3.9-slim as builder
# 安装编译依赖
RUN apt-get update && apt-get install -y \
gcc \
g++ \
libpq-dev \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
# 复制依赖文件
COPY requirements.txt .
# 安装依赖到虚拟环境
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
RUN pip install --no-cache-dir -r requirements.txt
# 生产阶段
FROM python:3.9-slim
# 安装运行时依赖
RUN apt-get update && apt-get install -y \
libpq5 \
&& rm -rf /var/lib/apt/lists/*
# 复制虚拟环境
COPY --from=builder /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
# 创建非root用户
RUN useradd -m -u 1000 fastapi
USER fastapi
WORKDIR /app
# 复制应用代码
COPY --chown=fastapi:fastapi . .
# 环境变量
ENV PYTHONUNBUFFERED=1
ENV PYTHONPATH=/app
ENV PORT=8000
# 健康检查
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:$PORT/health')"
# 启动命令
CMD ["gunicorn", "-c", "gunicorn_config.py", "main:app"]
yaml
# docker-compose.prod.yml
version: '3.8'
services:
api:
build:
context: .
dockerfile: Dockerfile
ports:
- "8000:8000"
environment:
- DATABASE_URL=postgresql+asyncpg://user:pass@db:5432/app
- REDIS_URL=redis://redis:6379/0
- LOG_LEVEL=info
- IN_DOCKER=true
deploy:
resources:
limits:
cpus: '2'
memory: 1G
reservations:
cpus: '0.5'
memory: 512M
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
networks:
- backend
depends_on:
db:
condition: service_healthy
redis:
condition: service_healthy
db:
image: postgres:14-alpine
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=pass
- POSTGRES_DB=app
volumes:
- postgres_data:/var/lib/postgresql/data
- ./init.sql:/docker-entrypoint-initdb.d/init.sql
command: >
postgres
-c shared_buffers=256MB
-c effective_cache_size=1GB
-c maintenance_work_mem=64MB
-c checkpoint_completion_target=0.9
-c wal_buffers=16MB
-c default_statistics_target=100
healthcheck:
test: ["CMD-SHELL", "pg_isready -U user -d app"]
interval: 10s
timeout: 5s
retries: 5
networks:
- backend
redis:
image: redis:7-alpine
command: redis-server --maxmemory 256mb --maxmemory-policy allkeys-lru
volumes:
- redis_data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
networks:
- backend
nginx:
image: nginx:alpine
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
- ./ssl:/etc/nginx/ssl
depends_on:
- api
networks:
- backend
networks:
backend:
driver: bridge
volumes:
postgres_data:
redis_data:10. 性能优化检查清单
10.1 部署前检查清单
python
# checklist/performance_checklist.py
import asyncio
import psutil
import socket
from typing import List, Dict
class PerformanceChecklist:
"""性能优化检查清单"""
@staticmethod
async def run_all_checks() -> Dict[str, bool]:
"""运行所有性能检查"""
checks = {
"async_usage": await PerformanceChecklist.check_async_usage(),
"database_pool": await PerformanceChecklist.check_database_pool(),
"caching_enabled": await PerformanceChecklist.check_caching(),
"compression_enabled": await PerformanceChecklist.check_compression(),
"rate_limiting": await PerformanceChecklist.check_rate_limiting(),
"monitoring_setup": await PerformanceChecklist.check_monitoring(),
"connection_reuse": await PerformanceChecklist.check_connection_reuse(),
"memory_usage": await PerformanceChecklist.check_memory_usage(),
"cpu_usage": await PerformanceChecklist.check_cpu_usage(),
"network_latency": await PerformanceChecklist.check_network_latency(),
}
return checks
@staticmethod
async def check_async_usage() -> bool:
"""检查异步使用情况"""
# 检查是否有阻塞调用
import inspect
import main # 假设主应用模块
async_functions = 0
sync_functions = 0
for name, obj in inspect.getmembers(main):
if inspect.iscoroutinefunction(obj):
async_functions += 1
elif inspect.isfunction(obj) and not name.startswith('_'):
sync_functions += 1
ratio = async_functions / (async_functions + sync_functions) if (async_functions + sync_functions) > 0 else 0
return ratio > 0.7 # 70%以上的函数应该是异步的
@staticmethod
async def check_database_pool() -> bool:
"""检查数据库连接池配置"""
try:
from sqlalchemy import inspect
from main import engine # 假设engine在main中定义
inspector = inspect(engine)
pool = inspector.pool
return (
pool.size() >= 5 and
pool.checkedin() > 0 and
hasattr(pool, '_overflow') and
pool._overflow >= 0
)
except:
return False
@staticmethod
async def check_caching() -> bool:
"""检查缓存配置"""
try:
# 检查Redis连接
import redis
r = redis.Redis(host='localhost', port=6379, socket_connect_timeout=1)
return r.ping()
except:
return False
@staticmethod
async def check_compression() -> bool:
"""检查压缩配置"""
from fastapi.middleware.gzip import GZipMiddleware
from main import app # 假设app在main中定义
for middleware in app.user_middleware:
if middleware.cls == GZipMiddleware:
return True
return False
@staticmethod
async def check_rate_limiting() -> bool:
"""检查限流配置"""
# 检查是否有限流中间件
import inspect
from main import app
for middleware in app.user_middleware:
if 'rate' in str(middleware.cls).lower() or 'limit' in str(middleware.cls).lower():
return True
return False
@staticmethod
async def check_monitoring() -> bool:
"""检查监控配置"""
from main import app
# 检查是否有监控端点
routes = [route.path for route in app.routes]
monitoring_endpoints = {'/metrics', '/health', '/status'}
return any(endpoint in routes for endpoint in monitoring_endpoints)
@staticmethod
async def check_connection_reuse() -> bool:
"""检查连接复用"""
import httpx
from main import OptimizedHTTPClient
try:
client = await OptimizedHTTPClient.get_client()
return client._transport._pool._max_keepalive > 0
except:
return False
@staticmethod
async def check_memory_usage() -> bool:
"""检查内存使用"""
process = psutil.Process()
memory_percent = process.memory_percent()
# 内存使用率应小于70%
return memory_percent < 70
@staticmethod
async def check_cpu_usage() -> bool:
"""检查CPU使用"""
cpu_percent = psutil.cpu_percent(interval=1)
# CPU使用率应小于80%
return cpu_percent < 80
@staticmethod
async def check_network_latency() -> bool:
"""检查网络延迟"""
try:
# 测试本地数据库连接延迟
start = asyncio.get_event_loop().time()
reader, writer = await asyncio.open_connection('localhost', 5432)
writer.close()
await writer.wait_closed()
end = asyncio.get_event_loop().time()
latency = (end - start) * 1000 # 转换为毫秒
return latency < 10 # 延迟应小于10ms
except:
return False
# 运行检查清单
@app.get("/performance-check")
async def performance_check():
"""性能检查端点"""
checks = await PerformanceChecklist.run_all_checks()
passed = sum(checks.values())
total = len(checks)
score = (passed / total) * 100 if total > 0 else 0
recommendations = []
if not checks["async_usage"]:
recommendations.append("增加异步函数的使用比例")
if not checks["database_pool"]:
recommendations.append("优化数据库连接池配置")
if not checks["caching_enabled"]:
recommendations.append("启用Redis缓存")
if not checks["compression_enabled"]:
recommendations.append("启用GZIP压缩")
if not checks["rate_limiting"]:
recommendations.append("添加请求限流")
if not checks["monitoring_setup"]:
recommendations.append("设置监控端点")
return {
"score": f"{score:.1f}%",
"passed": f"{passed}/{total}",
"checks": checks,
"recommendations": recommendations,
"status": "good" if score > 80 else "needs_improvement" if score > 60 else "poor"
}11. 总结
通过本文介绍的FastAPI性能优化技巧，您应该能够构建出高性能、可扩展的API服务。关键点总结：
- 异步优先：充分利用async/await，避免阻塞操作
- 连接池优化：合理配置数据库和HTTP连接池
- 缓存策略：实现多级缓存，减少数据库压力
- 资源管理：监控和优化内存、CPU使用
- 监控报警：建立完善的监控体系
- 生产优化：优化部署配置，使用CDN和负载均衡
记住，性能优化是一个持续的过程，需要根据实际使用情况不断调整和优化。使用本文提供的工具和技巧作为起点，结合具体的业务需求，您将能够构建出卓越的FastAPI应用。
附录：性能优化参考指标
| 指标 | 优秀 | 良好 | 需要优化 |
|---|---|---|---|
| API响应时间 | < 100ms | 100-500ms | > 500ms |
| 数据库查询时间 | < 10ms | 10-50ms | > 50ms |
| 内存使用率 | < 60% | 60-80% | > 80% |
| CPU使用率 | < 70% | 70-90% | > 90% |
| 缓存命中率 | > 90% | 70-90% | < 70% |
| 错误率 | < 0.1% | 0.1-1% | > 1% |
| 吞吐量 | > 1000 req/s | 500-1000 req/s | < 500 req/s |
定期监控这些指标，并根据实际情况调整优化策略，是保持API高性能的关键。