GPU视频编解码：X86 DeepStream 视频编解码入门（三）

一.设备硬件信息

1.X86 auto 云端主机

bash 复制代码

video) (base) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries/encode_video# cat /etc/os-release 
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

2.python3 版本必须得是3.10

bash 复制代码

(video) (base) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries/encode_video# python3
Python 3.10.0 (default, Mar  3 2022, 09:58:08) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> eixt()

3.cuda 版本必须大于12

cpp 复制代码

(video) (base) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries/encode_video# nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Mon_Apr__3_17:16:06_PDT_2023
Cuda compilation tools, release 12.1, V12.1.105
Build cuda_12.1.r12.1/compiler.32688072_0

二. 生态环境安装

1.源码仓库clone：https://github.com/NVIDIA-AI-IOT/deepstream_libraries.git

2.whl文件下载：以下方式都行

cpp 复制代码

wget --content-disposition 'https://api.ngc.nvidia.com/v2/resources/org/nvidia/deepstream/7.1/files?redirect=true&path=deepstream_libraries-1.1-cp310-cp310-linux_x86_64.whl' -O deepstream_libraries-1.1-cp310-cp310-linux_x86_64.whl

cpp 复制代码

curl -LO 'https://api.ngc.nvidia.com/v2/resources/org/nvidia/deepstream/7.1/files?redirect=true&path=deepstream_libraries-1.1-cp310-cp310-linux_x86_64.whl'

3.安装 :必须是python3.10

cpp 复制代码

conda create -n video python=3.10 
conda activate video

cpp 复制代码

(video) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries# pip3 install de
decode_video/                                          deepstream_libraries-1.1-cp310-cp310-linux_x86_64.whl
(video) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries# pip3 install deepstream_libraries-1.1-cp310-cp310-linux_x86_64.whl 
Looking in indexes: http://mirrors.aliyun.com/pypi/simple
Processing ./deepstream_libraries-1.1-cp310-cp310-linux_x86_64.whl
Collecting cvcuda-cu12@ https://github.com/CVCUDA/CV-CUDA/releases/download/v0.10.1-beta/cvcuda_cu12-0.10.1b0-cp310-cp310-linux_x86_64.whl (from deepstream-libraries==1.1)
  Downloading https://github.com/CVCUDA/CV-CUDA/releases/download/v0.10.1-beta/cvcuda_cu12-0.10.1b0-cp310-cp310-linux_x86_64.whl (124.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.8/124.8 MB 9.1 MB/s eta 0:00:00
Collecting PyNvVideoCodec==1.0.1 (from deepstream-libraries==1.1)
  Downloading http://mirrors.aliyun.com/pypi/packages/24/ce/3789d47a2336319a02563e6071eb85eba643e9f9d22bdcf3415d0d2a0aa3/PyNvVideoCodec-1.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (56.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.8/56.8 MB 13.8 MB/s eta 0:00:00
Collecting nvidia-nvimgcodec-cu12==0.3.0.5 (from deepstream-libraries==1.1)
  Downloading http://mirrors.aliyun.com/pypi/packages/67/6d/749eb173622a0f269c967abc0f946970899e90970427b1b8446e8b7ba8d1/nvidia_nvimgcodec_cu12-0.3.0.5-py3-none-manylinux2014_x86_64.whl (10.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.9/10.9 MB 13.8 MB/s eta 0:00:00
Collecting numpy<2.0.0,>=1.23.5 (from cvcuda-cu12@ https://github.com/CVCUDA/CV-CUDA/releases/download/v0.10.1-beta/cvcuda_cu12-0.10.1b0-cp310-cp310-linux_x86_64.whl->deepstream-libraries==1.1)
  Downloading http://mirrors.aliyun.com/pypi/packages/4b/d7/ecf66c1cd12dc28b4040b15ab4d17b773b87fa9d29ca16125de01adb36cd/numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 15.3 MB/s eta 0:00:00
Installing collected packages: PyNvVideoCodec, nvidia-nvimgcodec-cu12, numpy, cvcuda-cu12, deepstream-libraries
Successfully installed PyNvVideoCodec-1.0.1 cvcuda-cu12-0.10.1b0 deepstream-libraries-1.1 numpy-1.26.4 nvidia-nvimgcodec-cu12-0.3.0.5
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
(video) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries#

三.依赖包讲解

1.CVCUDA是字节和NVIDIA开发的基于CUDA进行视频图像处理工具，代替OpenCV

2.PyNvVideoCodec继承自上一节用到的VPF框架：

cpp 复制代码

pip install PyNvVideoCodec==1.0.1

nvidia-nvimgcodec这是处理图片的工具

4.pycuda提供python cuda 接口

四.视频编码实例

1.视频编码：成功将mp4进行h264编码

cpp 复制代码

(video) (base) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries# cd encode_video/
(video) (base) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries/encode_video# python3 encode.py --gpu_id 0 --raw_file_path ../assets/videos/pexels-chiel-slotman-4423925-1920x1080-25fps.mp4  --encoded_file_path output.h264 --size 1920x1080 --format nv12 --codec h264 --config_file ../assets/configs/encode_config.json
context fetched=0x562094010510
stream created=0x562094977a90
(video) (base) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries/encode_video# ls
README.md  encode.py  output.h264
(video) (base) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries/encode_video#

2.编码前后文件大小对比，降低了75%+存储空间

cpp 复制代码

(video) (base) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries/encode_video# du -sh output.h264 
1.1M    output.h264
(video) (base) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries/encode_video# du -sh ../assets/videos/pexels-chiel-slotman-4423925-1920x1080-25fps.mp4
4.6M    ../assets/videos/pexels-chiel-slotman-4423925-1920x1080-25fps.mp4
(video) (base) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries/encode_video#

3.代码讲解(非常简洁，比前面几个系列硬编解码都nice!)

导入依赖

python 复制代码

import io
import sys
import json
import argparse
sys.path.append('../')
from pathlib import Path
import PyNvVideoCodec as nvc
from common.nvc_utils import FetchGPUFrame, FetchCPUFrame, AppFrame

total_num_frames = 1000

函数参数列表解析

python 复制代码

def encode(gpu_id, dec_file_path, enc_file_path, width, height, fmt, config_params):
    """
                This function illustrates encoding of frames using CUDA device buffers as input.

                The application reads the image data from file and loads it to CUDA input buffers using
                FetchGPUFrame(). The encoder subsequently copies the CUDA buffers and submits them to NVENC hardware
                for encoding as part of Encode() function. Video memory buffer allocated
                by the application to get the NVENC hardware output. This application copies the NVENC output
                from video memory buffer to host memory buffer in order to dump to a file, but this
                is not needed if application choose to use it in some other way.

                Parameters:
                    - gpu_id (int): Ordinal of GPU to use [Parameter not in use]
                    - dec_file_path (str): Path to
                file to be decoded
                    - enc_file_path (str): Path to output file into which raw frames are stored
                    - width (int): width of encoded frame
                    - height (int): height of encoded frame
                    - fmt (str) : surface format string in uppercase, for e.g. NV12
                    - config_params(key value pairs) : key value pairs providing fine-grained control on encoding

                Returns: - None.

函数调用实例

python 复制代码

Example:
                >>> encode(0, "path/to/input/yuv/file","path/to/output/elementary/bitstream",1920,1080,"NV12")
                Encode 1080p NV12 raw YUV into elementary bitstream using H.264 codec and P4 preset

pipline说明

python 复制代码

 with open(dec_file_path, "rb") as decFile, open(enc_file_path, "wb") as encFile:
        nvenc = nvc.CreateEncoder(width, height, fmt, False, **config_params)  # create encoder object
        input_frame_list = list([AppFrame(width, height, fmt) for x in range(1, 5)])
        for input_gpu_frame in FetchGPUFrame(input_frame_list,
                                             FetchCPUFrame(decFile, input_frame_list[0].frameSize),
                                             total_num_frames):
            bitstream = nvenc.Encode(input_gpu_frame)  # encode frame one by one
            bitstream = bytearray(bitstream)
            encFile.write(bitstream)

        bitstream = nvenc.EndEncode()  # flush encoder queue
        bitstream = bytearray(bitstream)
        encFile.write(bitstream)

五.视频解码

1.解码cli命令

cpp 复制代码

video) (base) root@autodl-container-2b344a8a9f-e5fa6b67:~/autodl-tmp/deepstream_libraries/decode_video# python3 decode.py --gpu_id 0 --encoded_file_path ../assets/videos/pexels-chiel-slotman-4423925-1920x1080-25fps.mp4 --raw_file_path output.yuv --use_device_memory 1
[INFO ][00:13:25] Media format: QuickTime / MOV (mov,mp4,m4a,3gp,3g2,mj2)
[NULL @ 0x562dad250d40] No codec provided to avcodec_open2()
[WARN ][00:13:25] ChromaFormat not recognized. Assuming 420
FPS =  25.0
Session Initialization Time: 21 ms 
Session Deinitialization Time: 226 ms

2.依赖包

cpp 复制代码

import sys
import argparse
import numpy as np

sys.path.append('../')
from pathlib import Path
import pycuda.driver as cuda
import PyNvVideoCodec as nvc
from common.nvc_utils import cast_address_to_1d_bytearray

3.创建解码器和缓冲空间

cpp 复制代码

    nv_dmx = nvc.CreateDemuxer(filename=enc_file_path)
    nv_dec = nvc.CreateDecoder(gpuid=0,
                               codec=nv_dmx.GetNvCodecId(),
                               cudacontext=0,
                               cudastream=0,
                               usedevicememory=use_device_memory)

4.pipline

cpp 复制代码

# 以写入模式打开输出文件
    with open(dec_file_path, "wb") as decFile:
        # 遍历解复用器，逐个获取数据包（packet）
        for packet in nv_dmx:
            # 送入解码器进行解码，解码器返回多个解码后的帧
            for decoded_frame in nv_dec.Decode(packet):
                # 对于 NV12 格式，解码器会返回两个平面（Y 平面和 UV 平面）
                if not seq_triggered:
                    decoded_frame_size = nv_dec.GetFrameSize()  # 获取解码后帧的字节大小
                    raw_frame = np.ndarray(shape=decoded_frame_size, dtype=np.uint8)  # 申请内存存储解码帧
                    seq_triggered = True
                
                # 获取 Y 平面的设备地址
                luma_base_addr = decoded_frame.GetPtrToPlane(0)
                
                if use_device_memory:
                    # 如果使用 GPU 设备内存，需要手动从 GPU 复制到 CPU
                    cuda.memcpy_dtoh(raw_frame, luma_base_addr)  # 设备到主机拷贝
                    bits = bytearray(raw_frame)  # 转换为字节流
                    decFile.write(bits)  # 写入文件
                else:
                    # 如果使用的是主机内存，则直接转换地址为 1D 数组并写入文件
                    new_array = cast_address_to_1d_bytearray(
                        base_address=luma_base_addr,
                        size=decoded_frame.framesize()
                    )
                    decFile.write(bytearray(new_array))