Jetson CSI/USB 摄像头配置与多路视频流处理

1. Jetson 摄像头接口概览

Jetson 支持两种摄像头接口：

复制代码

摄像头接口对比：
┌─────────────┬──────────────────┬──────────────────┐
│ 特性         │ CSI 摄像头        │ USB 摄像头        │
├─────────────┼──────────────────┼──────────────────┤
│ 接口         │ MIPI CSI-2       │ USB 2.0/3.0      │
│ 带宽         │ 高（4 Lane）      │ 中等              │
│ 延迟         │ 极低（<10ms）     │ 中等（30-100ms）   │
│ CPU 占用     │ 低（ISP 硬件加速）│ 高（软解码）        │
│ 多路支持     │ 最多 6 路         │ 受 USB 带宽限制    │
│ 热插拔       │ ❌ 不支持         │ ✅ 支持          │
│ 价格         │ 中等              │ 低               │
│ 推荐场景     │ 实时检测、机器人  │ 通用开发、调试       │
└─────────────┴──────────────────┴──────────────────┘

2. CSI 摄像头配置

2.1 常见 CSI 摄像头型号

复制代码

支持的 CSI 摄像头：
├── Raspberry Pi Camera Module v2 (IMX219)
│   ├── 分辨率：3280×2464 @ 15fps / 1920×1080 @ 30fps
│   ├── 接口：15-pin FPC
│   └── 价格：~$25
├── Raspberry Pi HQ Camera (IMX477)
│   ├── 分辨率：4056×3040 @ 30fps / 1920×1080 @ 60fps
│   ├── 接口：22-pin FPC（需转接板）
│   └── 价格：~$50
├── ArduCam IMX219 双目摄像头
│   ├── 双路 IMX219
│   └── 适合立体视觉
└── Leopard Imaging LI-IMX477
    ├── 工业级 IMX477
    └── 适合工业部署

2.2 硬件连接

复制代码

CSI 摄像头连接步骤：
1. 断开 Jetson 电源
2. 找到载板上的 CSI 接口（通常标有 CAM0/CAM1）
3. 拉起 FPC 连接器的卡扣
4. 将排线金手指朝下插入（蓝色面朝 PCB 板边）
5. 按下卡扣固定
6. 接通电源

注意：
- Orin NX Developer Kit 有 2 个 CSI 接口（CAM0 + CAM1）
- 第三方载板 CSI 接口位置可能不同
- IMX477 需要 22-pin 转 15-pin 转接板

2.3 检测 CSI 摄像头

bash 复制代码

# 检查摄像头是否被系统识别
ls /dev/video*
# 应该看到 /dev/video0 等设备

# 使用 v4l2 工具检查
v4l2-ctl --list-devices
# 输出示例：
# vi-output, imx219 9-0010 (platform:tegra-capture-vi:0):
#     /dev/video0

# 查看摄像头支持的格式
v4l2-ctl -d /dev/video0 --list-formats-ext
# 输出示例：
# [0]: 'RG10' (10-bit Bayer RGRG/GBGB)
#     Size: Discrete 3280x2464
#         Interval: Discrete 0.067s (15.000 fps)
#     Size: Discrete 1920x1080
#         Interval: Discrete 0.033s (30.000 fps)

2.4 测试 CSI 摄像头

bash 复制代码

# 使用 GStreamer 测试 CSI 摄像头
# IMX219 测试
gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! \
    'video/x-raw(memory:NVMM), width=1920, height=1080, framerate=30/1' ! \
    nvvidconv ! 'video/x-raw, format=BGRx' ! videoconvert ! \
    'video/x-raw, format=BGR' ! autovideosink

# IMX477 测试
gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! \
    'video/x-raw(memory:NVMM), width=4032, height=3040, framerate=20/1' ! \
    nvvidconv ! 'video/x-raw, format=BGRx' ! videoconvert ! \
    'video/x-raw, format=BGR' ! autovideosink

# 截图
gst-launch-1.0 nvarguscamerasrc num-buffers=1 ! \
    'video/x-raw(memory:NVMM), width=1920, height=1080' ! \
    nvvidconv ! 'video/x-raw, format=BGRx' ! videoconvert ! \
    'video/x-raw, format=BGR' ! pngenc ! filesink location=camera_test.png

3. USB 摄像头配置

3.1 检测 USB 摄像头

bash 复制代码

# 列出 USB 设备
lsusb | grep -i camera
# 输出示例：
# Bus 002 Device 003: ID 046d:0825 Logitech, Inc. Webcam C270

# 查看摄像头信息
v4l2-ctl -d /dev/video0 --all
# 输出：设备名、驱动、格式、分辨率等

# 测试摄像头
v4l2-ctl -d /dev/video0 --list-formats-ext

3.2 测试 USB 摄像头

bash 复制代码

# 方式一：GStreamer（推荐，低延迟）
gst-launch-1.0 v4l2src device=/dev/video0 ! \
    'video/x-raw, width=1280, height=720, framerate=30/1' ! \
    videoconvert ! 'video/x-raw, format=BGR' ! autovideosink

# 方式二：OpenCV（简单）
python3 -c "
import cv2
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
while True:
    ret, frame = cap.read()
    if ret:
        cv2.imshow('USB Camera', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
cap.release()
cv2.destroyAllWindows()
"

# 方式三：FFmpeg
ffmpeg -f v4l2 -video_size 1280x720 -i /dev/video0 -f sdl2 preview

3.3 USB 摄像头性能优化

python 复制代码

#!/usr/bin/env python3
"""usb_camera_optimized.py - USB 摄像头优化采集"""
import cv2
import threading
import queue
import time

class OptimizedUSBCamera:
    """优化的 USB 摄像头采集"""
    
    def __init__(self, device_id=0, width=1280, height=720, fps=30):
        self.device_id = device_id
        self.width = width
        self.height = height
        self.fps = fps
        self.frame_queue = queue.Queue(maxsize=2)
        self.running = False
        
        # 使用 GStreamer 管道（降低 CPU 占用）
        self.pipeline = (
            f"v4l2src device=/dev/video{device_id} ! "
            f"video/x-raw, width={width}, height={height}, framerate={fps}/1 ! "
            f"videoconvert ! video/x-raw, format=BGR ! appsink"
        )
    
    def start(self):
        """启动采集线程"""
        self.running = True
        self.thread = threading.Thread(target=self._capture_loop, daemon=True)
        self.thread.start()
        return self
    
    def _capture_loop(self):
        """采集循环"""
        cap = cv2.VideoCapture(self.pipeline, cv2.CAP_GSTREAMER)
        
        if not cap.isOpened():
            print(f"❌ 无法打开摄像头 /dev/video{self.device_id}")
            return
        
        while self.running:
            ret, frame = cap.read()
            if ret:
                # 丢弃旧帧，保持队列最新
                if self.frame_queue.full():
                    try:
                        self.frame_queue.get_nowait()
                    except queue.Empty:
                        pass
                self.frame_queue.put(frame)
            else:
                time.sleep(0.001)
        
        cap.release()
    
    def read(self, timeout=1.0):
        """获取最新帧"""
        try:
            return self.frame_queue.get(timeout=timeout)
        except queue.Empty:
            return None
    
    def stop(self):
        """停止采集"""
        self.running = False
        if hasattr(self, 'thread'):
            self.thread.join(timeout=2)

if __name__ == "__main__":
    camera = OptimizedUSBCamera(device_id=0, width=1280, height=720, fps=30)
    camera.start()
    
    fps_counter = 0
    fps_start = time.time()
    
    while True:
        frame = camera.read()
        if frame is not None:
            fps_counter += 1
            if time.time() - fps_start >= 1.0:
                print(f"FPS: {fps_counter}")
                fps_counter = 0
                fps_start = time.time()
            
            cv2.imshow("Optimized USB Camera", frame)
        
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    
    camera.stop()
    cv2.destroyAllWindows()

4. 多路视频流处理

4.1 双 CSI 摄像头

python 复制代码

#!/usr/bin/env python3
"""dual_csi.py - 双 CSI 摄像头实时显示"""
import cv2
import threading

class CSICamera:
    def __init__(self, sensor_id, width=1280, height=720, fps=30):
        self.sensor_id = sensor_id
        self.pipeline = (
            f"nvarguscamerasrc sensor-id={sensor_id} ! "
            f"video/x-raw(memory:NVMM), width={width}, height={height}, "
            f"format=NV12, framerate={fps}/1 ! "
            f"nvvidconv flip-method=0 ! "
            f"video/x-raw, width={width}, height={height}, format=BGRx ! "
            f"videoconvert ! video/x-raw, format=BGR ! appsink"
        )
        self.cap = cv2.VideoCapture(self.pipeline, cv2.CAP_GSTREAMER)
        self.frame = None
        self.running = True
        self.thread = threading.Thread(target=self._capture, daemon=True)
        self.thread.start()
    
    def _capture(self):
        while self.running:
            ret, frame = self.cap.read()
            if ret:
                self.frame = frame
    
    def read(self):
        return self.frame
    
    def release(self):
        self.running = False
        self.cap.release()

def main():
    cam0 = CSICamera(sensor_id=0, width=1280, height=720)
    cam1 = CSICamera(sensor_id=1, width=1280, height=720)
    
    print("🎯 按 'q' 退出")
    
    while True:
        frame0 = cam0.read()
        frame1 = cam1.read()
        
        if frame0 is not None and frame1 is not None:
            # 水平拼接
            combined = cv2.hconcat([frame0, frame1])
            cv2.imshow("Dual CSI Camera", combined)
        
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    
    cam0.release()
    cam1.release()
    cv2.destroyAllWindows()

if __name__ == "__main__":
    main()

4.2 四路 USB 摄像头

python 复制代码

#!/usr/bin/env python3
"""quad_usb.py - 四路 USB 摄像头"""
import cv2
import threading
import numpy as np
import time

class USBCameraThread(threading.Thread):
    def __init__(self, device_id, width=640, height=480):
        super().__init__(daemon=True)
        self.device_id = device_id
        self.frame = None
        self.running = True
        
        self.pipeline = (
            f"v4l2src device=/dev/video{device_id} ! "
            f"video/x-raw, width={width}, height={height}, framerate=30/1 ! "
            f"videoconvert ! video/x-raw, format=BGR ! appsink"
        )
    
    def run(self):
        cap = cv2.VideoCapture(self.pipeline, cv2.CAP_GSTREAMER)
        while self.running:
            ret, frame = cap.read()
            if ret:
                self.frame = frame
            else:
                time.sleep(0.01)
        cap.release()

def main():
    # 启动 4 路摄像头
    cameras = []
    for i in range(4):
        cam = USBCameraThread(device_id=i, width=640, height=480)
        cam.start()
        cameras.append(cam)
        print(f"摄像头 {i} 已启动")
    
    time.sleep(2)  # 等待摄像头初始化
    
    while True:
        frames = []
        for i, cam in enumerate(cameras):
            if cam.frame is not None:
                frame = cam.frame.copy()
                cv2.putText(frame, f"Camera {i}", (10, 30),
                           cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
                frames.append(frame)
            else:
                # 黑色占位
                frames.append(np.zeros((480, 640, 3), dtype=np.uint8))
        
        # 2x2 拼接
        top = cv2.hconcat([frames[0], frames[1]])
        bottom = cv2.hconcat([frames[2], frames[3]])
        grid = cv2.vconcat([top, bottom])
        
        cv2.imshow("Quad USB Cameras", grid)
        
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    
    for cam in cameras:
        cam.running = False
    cv2.destroyAllWindows()

if __name__ == "__main__":
    main()

4.3 混合 CSI + USB 摄像头

python 复制代码

#!/usr/bin/env python3
"""hybrid_cameras.py - CSI + USB 混合多路"""
import cv2
import threading
import queue
import time

class HybridCameraSystem:
    """混合摄像头系统"""
    
    def __init__(self):
        self.cameras = {}
        self.frame_queues = {}
    
    def add_csi_camera(self, name, sensor_id, width=1280, height=720, fps=30):
        """添加 CSI 摄像头"""
        pipeline = (
            f"nvarguscamerasrc sensor-id={sensor_id} ! "
            f"video/x-raw(memory:NVMM), width={width}, height={height}, "
            f"format=NV12, framerate={fps}/1 ! "
            f"nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! "
            f"video/x-raw, format=BGR ! appsink"
        )
        self.cameras[name] = pipeline
        self.frame_queues[name] = queue.Queue(maxsize=2)
    
    def add_usb_camera(self, name, device_id, width=1280, height=720, fps=30):
        """添加 USB 摄像头"""
        pipeline = (
            f"v4l2src device=/dev/video{device_id} ! "
            f"video/x-raw, width={width}, height={height}, framerate={fps}/1 ! "
            f"videoconvert ! video/x-raw, format=BGR ! appsink"
        )
        self.cameras[name] = pipeline
        self.frame_queues[name] = queue.Queue(maxsize=2)
    
    def _capture_thread(self, name, pipeline, frame_queue):
        """采集线程"""
        cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)
        while True:
            ret, frame = cap.read()
            if ret:
                if not frame_queue.full():
                    frame_queue.put(frame)
            else:
                time.sleep(0.01)
        cap.release()
    
    def start(self):
        """启动所有摄像头"""
        threads = []
        for name, pipeline in self.cameras.items():
            t = threading.Thread(
                target=self._capture_thread,
                args=(name, pipeline, self.frame_queues[name]),
                daemon=True
            )
            t.start()
            threads.append(t)
            print(f"✅ 摄像头 '{name}' 已启动")
        return threads
    
    def get_frame(self, name, timeout=1.0):
        """获取指定摄像头帧"""
        try:
            return self.frame_queues[name].get(timeout=timeout)
        except queue.Empty:
            return None

if __name__ == "__main__":
    system = HybridCameraSystem()
    
    # 添加 CSI 摄像头
    system.add_csi_camera("前视", sensor_id=0, width=1280, height=720)
    system.add_csi_camera("后视", sensor_id=1, width=1280, height=720)
    
    # 添加 USB 摄像头
    system.add_usb_camera("侧视", device_id=0, width=640, height=480)
    
    system.start()
    time.sleep(2)
    
    while True:
        frames = {}
        for name in system.cameras:
            frame = system.get_frame(name)
            if frame is not None:
                cv2.imshow(name, frame)
        
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    
    cv2.destroyAllWindows()

5. RTSP 视频流推流

5.1 摄像头 → RTSP 推流

bash 复制代码

# 使用 GStreamer 推流到 RTSP 服务器
# 需要安装 RTSP 服务器
sudo apt install -y gstreamer1.0-rtsp libgstrtspserver-1.0-dev

# 启动 RTSP 推流（CSI 摄像头）
gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 ! \
    'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
    nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' ! \
    nvv4l2h264enc bitrate=4000000 ! \
    h264parse ! rtph264pay ! \
    udpsink host=127.0.0.1 port=5400

# 播放 RTSP 流
ffplay rtsp://127.0.0.1:8554/camera0

5.2 Python RTSP 推流服务器

python 复制代码

#!/usr/bin/env python3
"""rtsp_server.py - 基于 GStreamer 的 RTSP 推流"""
import gi
gi.require_version('Gst', '1.0')
gi.require_version('GstRtspServer', '1.0')
from gi.repository import Gst, GstRtspServer, GLib

Gst.init(None)

class CSIStreamFactory(GstRtspServer.RTSPMediaFactory):
    """CSI 摄像头流工厂"""
    
    def __init__(self, sensor_id=0, **kwargs):
        super().__init__(**kwargs)
        self.sensor_id = sensor_id
    
    def do_create_element(self, url):
        pipeline_str = (
            f"( nvarguscamerasrc sensor-id={self.sensor_id} ! "
            f"video/x-raw(memory:NVMM), width=1280, height=720, "
            f"framerate=30/1, format=NV12 ! "
            f"nvv4l2h264enc bitrate=4000000 ! "
            f"h264parse ! rtph264pay name=pay0 pt=96 )"
        )
        return Gst.parse_launch(pipeline_str)

def main():
    server = GstRtspServer.RTSPServer()
    server.set_service("8554")
    
    # 多路摄像头
    for i, name in enumerate(["camera0", "camera1"]):
        factory = CSIStreamFactory(sensor_id=i)
        factory.set_shared(True)
        server.get_mount_points().add_factory(f"/{name}", factory)
        print(f"✅ RTSP 流: rtsp://0.0.0.0:8554/{name}")
    
    server.attach(None)
    print("🚀 RTSP 服务器已启动")
    
    loop = GLib.MainLoop()
    loop.run()

if __name__ == "__main__":
    main()

6. 录像与截图

python 复制代码

#!/usr/bin/env python3
"""recorder.py - 视频录制与截图"""
import cv2
import os
import time
from datetime import datetime

class VideoRecorder:
    """视频录制器"""
    
    def __init__(self, output_dir="recordings"):
        self.output_dir = output_dir
        os.makedirs(output_dir, exist_ok=True)
        self.writer = None
        self.recording = False
    
    def start_recording(self, width, height, fps=30, codec='mp4v'):
        """开始录制"""
        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
        filepath = os.path.join(self.output_dir, f"rec_{timestamp}.mp4")
        
        fourcc = cv2.VideoWriter_fourcc(*codec)
        self.writer = cv2.VideoWriter(filepath, fourcc, fps, (width, height))
        self.recording = True
        self.start_time = time.time()
        
        print(f"🔴 开始录制: {filepath}")
        return filepath
    
    def write_frame(self, frame):
        """写入帧"""
        if self.recording and self.writer:
            self.writer.write(frame)
    
    def stop_recording(self):
        """停止录制"""
        if self.writer:
            self.writer.release()
            self.writer = None
            self.recording = False
            duration = time.time() - self.start_time
            print(f"⏹ 停止录制: {duration:.1f} 秒")
    
    def take_screenshot(self, frame, prefix="screenshot"):
        """截图"""
        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S_%f")
        filepath = os.path.join(self.output_dir, f"{prefix}_{timestamp}.png")
        cv2.imwrite(filepath, frame)
        print(f"📸 截图: {filepath}")
        return filepath

if __name__ == "__main__":
    recorder = VideoRecorder()
    
    # CSI 摄像头
    pipeline = (
        "nvarguscamerasrc sensor-id=0 ! "
        "video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1 ! "
        "nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! "
        "video/x-raw, format=BGR ! appsink"
    )
    cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)
    
    print("按键说明:")
    print("  r - 开始/停止录制")
    print("  s - 截图")
    print("  q - 退出")
    
    while True:
        ret, frame = cap.read()
        if not ret:
            break
        
        # 显示录制状态
        if recorder.recording:
            elapsed = time.time() - recorder.start_time
            cv2.circle(frame, (30, 30), 10, (0, 0, 255), -1)
            cv2.putText(frame, f"REC {elapsed:.0f}s", (50, 40),
                       cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)
        
        cv2.imshow("Camera", frame)
        
        key = cv2.waitKey(1) & 0xFF
        if key == ord('q'):
            break
        elif key == ord('r'):
            if recorder.recording:
                recorder.stop_recording()
            else:
                recorder.start_recording(1280, 720)
        elif key == ord('s'):
            recorder.take_screenshot(frame)
        
        if recorder.recording:
            recorder.write_frame(frame)
    
    recorder.stop_recording()
    cap.release()
    cv2.destroyAllWindows()

总结

场景	推荐方案	预期 FPS
单路 CSI 实时检测	nvarguscamerasrc + TensorRT	30-60
双路 CSI 立体视觉	双线程采集 + 单线程推理	15-30 每路
4 路 USB 监控	GStreamer + 多线程	15-30 每路
混合 CSI+USB	统一采集框架	按需配置
RTSP 推流	nvv4l2h264enc 硬编码	30

核心要点：

CSI 优先：低延迟、低 CPU 占用
GStreamer 管道：Jetson 上最高效的视频处理方式
多线程采集：采集和推理分离，避免丢帧
硬件编码 ：推流使用 nvv4l2h264enc