ffmpeg 使用滤镜实现播放倍速

FFmpeg 的滤镜可以实现，对原始音视频流进行裁剪、缩放、变速、降噪等基础处理；添加特效（如水印、字幕、转场、滤镜效果）；修复音视频缺陷（如去除杂音、校正色彩）；适配不同场景的需求（如调整分辨率以适应设备、混音以匹配多声道输出）等功能。下面本文将使用滤镜来实现播放倍速功能。

cpp 复制代码

    AVFilterGraph* m_filterGraph = nullptr;
    AVFilterContext* m_videoSrcFilterCtx = nullptr;
    AVFilterContext* m_videoSinkFilterCtx = nullptr;
    AVFilterContext* m_audioSrcFilterCtx = nullptr;
    AVFilterContext* m_audioSinkFilterCtx = nullptr;

这个有几个成员变量，其中 m_filterGraph 是用来管理滤镜的，我们创建滤镜，配置滤镜图的时候都需要用到它。简单来说，我们创建这样一个对象，然后在接下来跟滤镜有关的操作中，参数为 AVFilterGraph 的，都把它传进去就行了。

下面几个分别是视频源滤镜上下文，视频输出滤镜上下文，音频源滤镜上下文，音频输出滤镜上下文。我们会将 AVFrame 传入源滤镜上下文，然后从输出滤镜上下文获取新的 AVFrame.

cpp 复制代码

    const AVFilter* buffersrc = avfilter_get_by_name("buffer");
    const AVFilter* buffersink = avfilter_get_by_name("buffersink");
    const AVFilter* setpts = avfilter_get_by_name("setpts");
    const AVFilter* abuffersrc = avfilter_get_by_name("abuffer");
    const AVFilter* abuffersink = avfilter_get_by_name("abuffersink");
    const AVFilter* atempo = avfilter_get_by_name("atempo");

这里有几个 AVFilter 对象，其中 buffer 对应视频源滤镜，buffersink 对应视频输出滤镜，abuffer 和 abuffersink 则是音频输入和输出滤镜。setpts 滤镜是用来改变视频帧的 pts 的，而 atempo 滤镜是用来改变音频播放速度的。

cpp 复制代码

    int ret = 0;
    const AVFilter* buffersrc = avfilter_get_by_name("buffer");
    const AVFilter* buffersink = avfilter_get_by_name("buffersink");
    AVRational time_base = m_videoCodecContext->time_base;
    char args[512];
    snprintf(args, sizeof(args),
             "video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
             m_videoCodecContext->width, m_videoCodecContext->height, m_videoCodecContext->pix_fmt,
             time_base.num, time_base.den,
             m_videoCodecContext->sample_aspect_ratio.num, m_videoCodecContext->sample_aspect_ratio.den);

    ret = avfilter_graph_create_filter(&m_videoSrcFilterCtx, buffersrc, "in", args, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 buffer 滤镜" << std::endl;
        return false;
    }

    ret = avfilter_graph_create_filter(&m_videoSinkFilterCtx, buffersink, "out", NULL, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 buffersink 滤镜" << std::endl;
        return false;
    }
    AVFilterContext* videoSpeedCtx = nullptr;
    const AVFilter* setpts = avfilter_get_by_name("setpts");
    if ((ret = avfilter_graph_create_filter(&videoSpeedCtx,
                                            setpts, "speed",
                                            "0.5*PTS", nullptr, m_filterGraph)) < 0) {
        std::cout << "无法创建视频滤镜 " << std::endl;
        return -1;
    }

avfilter_graph_create_filter 用于创建和初始化滤镜，第一个参数是滤镜上下文，第二个参数是 AVFilter，第三个参数是名称，没啥用，随便设置即可，第四个参数是一个字符串，用于描述滤镜的属性。第五个参数没用，第六个参数则是滤镜图。

重点说明下第四个参数，输入滤镜的第四个参数是视频帧/音频帧的一些属性

视频参数

必须设置

video_size (视频尺寸)
格式：widthxheight
示例：1280x720
pix_fmt (像素格式)
格式：整数
示例：0(AV_PIX_FMT_YUV420P)
time_base (时间基)
格式：num/den
示例：1/1000
推荐
pixel_aspect (像素宽高比)
格式：num/den
示例：1/1
描述像素形状，影响显示比例
可选（根据滤镜链需求）
frame_rate (帧率)
格式：num/den或小数
示例：30000/1001(29.97fps)
说明：当滤镜链需要知道帧率时设置
sar (样本宽高比)
格式：num/den
示例：1/1
说明：与pixel_aspect类似，有时需要同时设置
sws_param (缩放参数)
格式：字符串
示例：flags=bicubic
说明：指定缩放算法，当需要调整大小时使用

音频参数

time_base (时间基)
格式：num/den
示例：1/44100
sample_rate (采样率)
格式：整数
示例：44100、48000
sample_fmt (采样格式)
格式：字符串
示例：s16、fltp
channel_layout (声道布局)
格式：十六进制或字符串
示例：0x3、stereo、mono

输出滤镜的参数放空即可，setpts 和 atempo 则跟滤镜有关。

cpp 复制代码

		if (avfilter_link(m_videoSrcFilterCtx, 0, videoSpeedCtx, 0) < 0 ||
        avfilter_link(videoSpeedCtx, 0, m_videoSinkFilterCtx, 0) < 0) {
        std::cerr << "连接视频滤镜失败" << std::endl;
        return false;
    }

avfilter_link 用于链接滤镜，上面的代码表示数据从 m_videoSrcFilterCtx 流向 videoSpeedCtx，再从 videoSpeedCtx 流向 m_videoSinkFilterCtx，很好理解，数据从输入滤镜开始，流向中间的滤镜，有中间的滤镜进行处理，再流向下一个滤镜，最终流入到输出滤镜。

0 表示滤镜输入，输出端口的索引，有的滤镜有多个输入或输出，比如我们如果想把两段音频合并到一起，第一段音频就传入到融合滤镜的第一个端口，第二段音频传入到第二个端口。

cpp 复制代码

		if ((ret = avfilter_graph_config(m_filterGraph, nullptr)) < 0) {
        std::cerr << "滤镜图配置失败 " << std::endl;
        return -1;
    }

这段代码用于检查滤镜图中所有链接和格式的有效性并进行配置。

至此，滤镜的相关设置就结束了。接下去只需将 AVFrame 传入输入滤镜上下文，再从输出滤镜上下文接收新的 AVFrame 即可

cpp 复制代码

	av_buffersrc_add_frame(m_videoSrcFilterCtx, frame);
	while (av_buffersink_get_frame(m_videoSinkFilterCtx, filterFrame) >= 0) {
		// do something
	}

需要注意的是，虽然 setpts 的功能只是用于改变视频帧的 pts，但是实际上 filterFrame 的 format 和 frame 不同，因此格式转化的参数需要从 filterFrame 获取。

下面介绍滤镜连接的另外一种方式，这种方式与上一种的区别在于不需要手动连接滤镜，只需要创建 AVFilterInOut 对象，然后将其和输入滤镜和输出滤镜绑定，通过 avfilter_graph_parse_ptr 函数，自动地去链接滤镜。

cpp 复制代码

    m_filterGraph = avfilter_graph_alloc();
    if (!m_filterGraph) {
        return false;
    }

    // 创建视频滤镜
    int ret = 0;
    const AVFilter* buffersrc = avfilter_get_by_name("buffer");
    const AVFilter* buffersink = avfilter_get_by_name("buffersink");
    AVRational time_base = m_videoCodecContext->time_base;

    char args[512];
    snprintf(args, sizeof(args),
             "video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
             m_videoCodecContext->width, m_videoCodecContext->height, m_videoCodecContext->pix_fmt,
             time_base.num, time_base.den,
             m_videoCodecContext->sample_aspect_ratio.num, m_videoCodecContext->sample_aspect_ratio.den);

    ret = avfilter_graph_create_filter(&m_videoSrcFilterCtx, buffersrc, "in",
                                       args, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 buffer 滤镜" << std::endl;
        return false;
    }

    // 设置buffersink参数
    ret = avfilter_graph_create_filter(&m_videoSinkFilterCtx, buffersink, "out",
                                       NULL, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 buffersink 滤镜" << std::endl;
        return false;
    }

前面的部分也是差不多的，创建滤镜图，创建源滤镜和输出滤镜。

cpp 复制代码

    AVFilterInOut* outputs = avfilter_inout_alloc();
    AVFilterInOut* inputs = avfilter_inout_alloc();
    if (!outputs || !inputs) {
        return false;
    }
    outputs->name = av_strdup("in");
    outputs->filter_ctx = m_videoSrcFilterCtx;
    outputs->pad_idx = 0;
    outputs->next = NULL;

    inputs->name = av_strdup("out");
    inputs->filter_ctx = m_videoSinkFilterCtx;
    inputs->pad_idx = 0;
    inputs->next = NULL;

    if ((ret = avfilter_graph_parse_ptr(m_filterGraph, "setpts=PTS/2",
                                        &inputs, &outputs, NULL)) < 0) {
        std::cout << "无法解析滤镜描述" << std::endl;
        return false;
    }

    avfilter_inout_free(&inputs);
    avfilter_inout_free(&outputs);

这里创建了两个 AVFilterInOut 对象，并设置参数，需要注意的是 outputs 对应的是输入滤镜，inputs 对应的是输出滤镜，可以理解为数据从输入滤镜输出到 AVFilterInOut ，所以是 output。

那么为什么不把 inputs 取名成 outputs 这个就跟 avfilter_graph_parse_ptr 有关了。

avfilter_graph_parse_ptr 函数声明如下：

cpp 复制代码

int avfilter_graph_parse_ptr(AVFilterGraph *graph, const char *filters,
                             AVFilterInOut **inputs, AVFilterInOut **outputs,
                             void *log_ctx);

AVFilterInOut **inputs 参数就对应上面代码中的 inputs 对象，另外一个同理。

cpp 复制代码

avfilter_graph_parse_ptr(m_filterGraph, "setpts=PTS/2", &inputs, &outputs, NULL))

avfilter_graph_parse_ptr 将根据传入的 filters 自动地去创建滤镜链

cpp 复制代码

		if ((ret = avfilter_graph_config(m_filterGraph, nullptr)) < 0) {
        std::cerr << "滤镜图配置失败 " << std::endl;
        return -1;
    }

最后同样需要调用 avfilter_graph_config 配置滤镜图

不过倍速后播放，音频和视频会有些偏差，暂时不确定是哪方面的问题

VideoDecoder.h 代码如下

cpp 复制代码

#ifndef VIDEODECODER_H
#define VIDEODECODER_H
#include <atomic>
#include <functional>
#include <string>
#include <vector>
#include <mutex>
#include <condition_variable>
extern "C" {
#include "libavcodec/avcodec.h"
#include "libavformat/avformat.h"
#include "libavutil/avutil.h"
#include "libswscale/swscale.h"
#include "libswresample/swresample.h"
#include "libavutil/channel_layout.h"
#include "libswresample/swresample.h"
#include "libavutil/opt.h"
#include "libavutil/time.h"

#include "libavfilter/avfilter.h"      // 滤镜系统
#include "libavfilter/buffersrc.h"      // 缓冲源滤镜
#include "libavfilter/buffersink.h"     // 缓冲接收滤镜
#include "libavutil/pixdesc.h"

}
#include "Timer.h"
#include "VideoStructs.h"

using namespace std;

class VideoDecoder
{
public:
    VideoDecoder();
    ~VideoDecoder();
    bool open(const string& filePath);
    VideoParams getVideoParam() { return m_videoParam;}
    AudioParams getAudioParam() { return m_audioParam;}
    bool isStop() { return m_stop; }

    void addVideoCallback(std::function<bool(AVFrame*)> callback) {
        m_videoCallbacks.emplace_back(callback);
    }

    void addAudioCallback(std::function<bool(AVFrame*)> callback) {
        m_audioCallbacks.emplace_back(callback);
    }

    void play();
    void pause();
    void stop();
    void close();

private:
    void showError(int ret);
    void readPacket();
    void decodeVideo();
    void decodeAudio();

    // 格式转换
    bool initSwrContext(const AVFrame* frame);
    bool initSwsContext(const AVFrame* frame);
    // 滤镜
    bool initFilter();
    bool initFilterByDescribe();
    // 硬解码
    AVHWDeviceType findSupportedHWDeviceType(const AVCodec* codec);
    bool initHWDevice();
    bool initHWFramesContext();
    bool hwFrameToSwFrame(AVFrame* hwFrame, AVFrame* swFrame);

private:
    VideoParams m_videoParam;
    AudioParams m_audioParam;

    // 播放控制及音视频同步
    bool m_play = false;
    bool m_stop = true;
    Timer* m_timer;

    std::vector<AVPacket> m_videoPackets;
    std::vector<AVPacket> m_audioPackets;
    std::mutex m_videoMutex;
    std::mutex m_audioMutex;

    std::condition_variable m_videoAvailable;
    std::condition_variable m_audioAvailable;

    std::mutex m_fullMutex;
    std::condition_variable m_queueFull;

    std::mutex m_playMutex;
    std::condition_variable m_playCondition;
    std::atomic<int> m_threadCount;

    // 回调
    std::vector<std::function<bool(AVFrame*)>> m_videoCallbacks;
    std::vector<std::function<bool(AVFrame*)>> m_audioCallbacks;

    // 格式转化
    SwsContext* m_swsContext = nullptr;
    SwrContext* m_swrContext = nullptr;                 // 音频格式转换实例

    // 解码
    AVFormatContext* m_formatContext = nullptr;
    int m_videoIndex = -1;                              // 视频流所在索引
    int m_audioIndex = -1;                              // 音频流所在索引
    AVCodecContext* m_videoCodecContext = nullptr;      // 视频解码器实例
    AVCodecContext* m_audioCodecContext = nullptr;      // 音频解码器实例
    // 硬解码
    AVBufferRef* m_hwDeviceCtx = nullptr;   // 硬件设备上下文
    AVBufferRef* m_hwFramesCtx = nullptr;   // 硬件帧上下文
    AVHWDeviceType m_hwDeviceType = AV_HWDEVICE_TYPE_NONE; // 硬件设备类型（如NVDEC）
    enum AVPixelFormat m_hwPixFmt = AV_PIX_FMT_NONE; // 硬件像素格式（如AV_PIX_FMT_CUDA）
    bool m_bHardWare = false;

    // 滤镜
    AVFilterGraph* m_filterGraph = nullptr;
    AVFilterContext* m_videoSrcFilterCtx = nullptr;
    AVFilterContext* m_videoSinkFilterCtx = nullptr;
    AVFilterContext* m_audioSrcFilterCtx = nullptr;
    AVFilterContext* m_audioSinkFilterCtx = nullptr;
};

#endif // VIDEODECODER_H

VideoDecoder.cpp 代码如下

cpp 复制代码

#include "VideoDecoder.h"
#include <iostream>
#include <thread>

VideoDecoder::VideoDecoder() {
    m_timer = new Timer;
}

VideoDecoder::~VideoDecoder()
{
    delete m_timer;
}

bool VideoDecoder::open(const string &filePath)
{
    close();
    int ret = avformat_open_input(&m_formatContext, filePath.c_str(), NULL, NULL);
    if (ret < 0) {
        showError(ret);
        close();
        return false;
    }

    // 查找流信息
    ret = avformat_find_stream_info(m_formatContext, NULL);
    if (ret < 0) {
        close();
        return false;
    }

    // 查找视频流
    m_videoIndex = av_find_best_stream(m_formatContext, AVMEDIA_TYPE_VIDEO, -1, -1, nullptr, 0);
    if (m_videoIndex == -1) {
        close();
        return false;
    }

    // 查找视频解码器
    AVStream *videoStream = m_formatContext->streams[m_videoIndex];
    const AVCodec *videoCodec = avcodec_find_decoder(videoStream->codecpar->codec_id);
    if (videoCodec == nullptr) {
        close();
        return false;
    }

    // 创建视频解码器
    m_videoCodecContext = avcodec_alloc_context3(videoCodec);
    if (m_videoCodecContext == nullptr) {
        close();
        return false;
    }
    // 把视频流中的编解码参数复制给解码器的实例
    avcodec_parameters_to_context(m_videoCodecContext, videoStream->codecpar);
    // 打开视频解码器实例
    ret = avcodec_open2(m_videoCodecContext, NULL, NULL);
    if (ret < 0) {
        close();
        return ret;
    }

    if (m_videoCodecContext->time_base.num <= 0 || m_videoCodecContext->time_base.den <= 0) {
        // 强制同步为流的 time_base
        m_videoCodecContext->time_base = videoStream->time_base;
    }

    m_videoParam.bitRate = m_videoCodecContext->bit_rate;
    m_videoParam.width = m_videoCodecContext->width;
    m_videoParam.height = m_videoCodecContext->height;
    m_videoParam.fps = m_videoCodecContext->framerate.num / m_videoCodecContext->framerate.den ;
    m_videoParam.gopSize = m_videoCodecContext->gop_size;
    m_videoParam.maxBFrames = m_videoCodecContext->max_b_frames;

    /*
    m_hwDeviceType = findSupportedHWDeviceType(videoCodec); // 查找支持的硬件类型
    if (m_hwDeviceType != AV_HWDEVICE_TYPE_NONE) {
        if (!initHWDevice()) { // 初始化硬件设备
            m_bHardWare = false;
            m_hwDeviceType = AV_HWDEVICE_TYPE_NONE;
            m_hwPixFmt = AV_PIX_FMT_NONE;
            // 硬解码失败，仍尝试软解码（释放已绑定的硬件上下文）
            if (m_videoCodecContext->hw_device_ctx) {
                av_buffer_unref(&m_videoCodecContext->hw_device_ctx);
                m_videoCodecContext->hw_device_ctx = nullptr;
            }
        }
    }
    */
    // 打开音频解码器实例
    m_audioIndex = av_find_best_stream(m_formatContext, AVMEDIA_TYPE_AUDIO, -1, -1, NULL, 0);
    if (m_audioIndex >= 0) {
        AVStream *audioStream = m_formatContext->streams[m_audioIndex];
        const AVCodec *audioCodec = avcodec_find_decoder(audioStream->codecpar->codec_id);
        if (audioCodec == nullptr) {
            close();
            return false;
        }
        m_audioCodecContext = avcodec_alloc_context3(audioCodec);
        if (m_audioCodecContext == nullptr) {
            close();
            return false;
        }
        avcodec_parameters_to_context(m_audioCodecContext, audioStream->codecpar);
        ret = avcodec_open2(m_audioCodecContext, audioCodec, NULL);
        if (ret < 0) {
            close();
            return false;
        }

        m_audioParam.sampleRate = m_audioCodecContext->sample_rate;
        m_audioParam.sampleFmt = m_audioCodecContext->sample_fmt;
        m_audioParam.bitRate = m_audioCodecContext->bit_rate;
        m_audioParam.channels = m_audioCodecContext->ch_layout.nb_channels;
        m_audioParam.samples = m_audioCodecContext->frame_size;
    }

    initFilter();
    //initFilterByDescribe();
    return true;
}

void VideoDecoder::play()
{
    if (m_stop) {
        m_stop = false;
        m_play = true;
        m_timer->start();
        std::thread([this](){
            readPacket();
            m_threadCount--;
        }).detach();

        std::thread([this](){
            decodeVideo();
            m_threadCount--;
        }).detach();

        std::thread([this](){
            decodeAudio();
            m_threadCount--;
        }).detach();
        m_threadCount = 3;

    } else {
        m_timer->start();
        m_play = true;
        m_playCondition.notify_all();
    }

}

void VideoDecoder::pause()
{
    m_play = false;
    m_timer->pause();
}

void VideoDecoder::stop()
{
    m_stop = true;
    m_timer->stop();
    // 唤醒所有条件变量，避免线程无法退出
    m_queueFull.notify_all();
    m_videoAvailable.notify_all();
    m_audioAvailable.notify_all();
    while (m_threadCount > 0) {
        std::this_thread::sleep_for(std::chrono::milliseconds(100));
    }
}

void VideoDecoder::close() {
    if (m_swsContext != nullptr) {
        sws_free_context(&m_swsContext);
    }

    if (m_swrContext != nullptr) {
        swr_free(&m_swrContext);
    }

    if (m_formatContext != nullptr) {
        avformat_close_input(&m_formatContext);
    }

    if (m_videoCodecContext != nullptr) {
        avcodec_free_context(&m_videoCodecContext);
    }

    if (m_audioCodecContext != nullptr) {
        avcodec_free_context(&m_audioCodecContext);
    }
}

void VideoDecoder::showError(int ret)
{
    char errorBuf[1024];
    av_strerror(ret, errorBuf, sizeof(errorBuf));
    std::cerr << errorBuf << std::endl;
}

void VideoDecoder::readPacket()
{
    AVPacket* packet = av_packet_alloc();
    while (!m_stop && av_read_frame(m_formatContext, packet) >= 0) { // 轮询数据包
        if (packet->stream_index == m_videoIndex) {
            m_videoMutex.lock();
            m_videoPackets.emplace_back(*packet);
            m_videoMutex.unlock();
            m_videoAvailable.notify_all();
        } else if (packet->stream_index == m_audioIndex) {
            m_audioMutex.lock();
            m_audioPackets.emplace_back(*packet);
            m_audioMutex.unlock();
            m_audioAvailable.notify_all();
        }

        {
            std::unique_lock<std::mutex> lock(m_fullMutex);
            m_queueFull.wait(lock, [this]{
                return m_videoPackets.size() <= 10 && m_audioPackets.size() <= 10 || m_stop;
            });
        }
    }
    av_packet_free(&packet);
    stop();
}

void VideoDecoder::decodeVideo()
{
    AVFrame* frame = av_frame_alloc();
    AVFrame* filterFrame = av_frame_alloc();
    AVFrame* convertFrame = av_frame_alloc();
    AVFrame* swFrame = nullptr;
    if (m_bHardWare) {
        swFrame = av_frame_alloc();
    }
    while (!m_stop) {
        {
            std::unique_lock<std::mutex> lock(m_videoMutex);
            m_videoAvailable.wait(lock, [this]{
                return !m_videoPackets.empty() || m_stop;
            });
        }

        if (m_stop) {
            break;
        }

        while (!m_play) {
            std::this_thread::sleep_for(std::chrono::milliseconds(5));
        }

        m_videoMutex.lock();
        AVPacket packet = m_videoPackets.front();
        m_videoPackets.erase(m_videoPackets.begin());
        m_videoMutex.unlock();
        m_queueFull.notify_all();
        int ret = avcodec_send_packet(m_videoCodecContext, &packet);
        av_packet_unref(&packet);
        while ( ret >= 0) {
            ret = avcodec_receive_frame(m_videoCodecContext, frame);
            if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
                break;
            } else if (ret < 0) {
                return;
            }
            if (m_bHardWare) {
                hwFrameToSwFrame(frame, swFrame);
                int ret = av_buffersrc_add_frame(m_videoSrcFilterCtx, swFrame);
                if (ret < 0) {
                    std::cerr << "推送视频帧到源滤镜失败" << std::endl;
                    showError(ret);
                    continue;
                }
            } else {
                int ret = av_buffersrc_add_frame(m_videoSrcFilterCtx, frame);
                if (ret < 0) {
                    std::cerr << "推送视频帧到源滤镜失败" << std::endl;
                    showError(ret);
                    continue;
                }
            }
            while (av_buffersink_get_frame(m_videoSinkFilterCtx, filterFrame) >= 0) {
                if (!m_swsContext) {
                    initSwsContext(filterFrame);
                }

                convertFrame->format = AV_PIX_FMT_RGB24;
                convertFrame->width = m_videoCodecContext->width;
                convertFrame->height = m_videoCodecContext->height;
                convertFrame->pts = filterFrame->pts;
                int ret = av_frame_get_buffer(convertFrame, 0);
                if (ret < 0) {
                    showError(ret);
                    return;
                }

                ret = sws_scale(
                    m_swsContext,
                    filterFrame->data, filterFrame->linesize,
                    0, filterFrame->height,
                    convertFrame->data, convertFrame->linesize);
                av_frame_copy_props(convertFrame, filterFrame);
                double pts_in_seconds = av_q2d(m_videoCodecContext->time_base) * convertFrame->pts;
                double diff = pts_in_seconds - m_timer->elapsedtime();
                //while (diff > 0.001 && !m_stop) {
                    //std::this_thread::sleep_for(std::chrono::milliseconds(5));
                    //diff = pts_in_seconds - m_timer->elapsedtime();
                //}

                for (const auto& callback : m_videoCallbacks) {
                    callback(convertFrame);
                }
                av_frame_unref(filterFrame);
                av_frame_unref(convertFrame);
            }
            av_frame_unref(frame);
        }
    }
    av_frame_free(&frame);
    av_frame_free(&convertFrame);
    av_frame_free(&filterFrame);
    if (m_bHardWare) {
        av_frame_free(&swFrame);
    }
}

void VideoDecoder::decodeAudio()
{
    AVFrame *frame = av_frame_alloc();
    AVFrame *convertFrame = av_frame_alloc();
    AVFrame *filterFrame = av_frame_alloc();
    while (!m_stop) {
        {
            std::unique_lock<std::mutex> lock(m_audioMutex);
            m_audioAvailable.wait(lock, [this]{
                return !m_audioPackets.empty() || m_stop;
            });
        }

        if (m_stop) {
            break;
        }

        while (!m_play) {
            std::this_thread::sleep_for(std::chrono::milliseconds(5));
        }

        m_audioMutex.lock();
        AVPacket packet = m_audioPackets.front();
        m_audioPackets.erase(m_audioPackets.begin());
        m_audioMutex.unlock();
        m_queueFull.notify_all();
        int ret = avcodec_send_packet(m_audioCodecContext, &packet);
        av_packet_unref(&packet);
        while (ret >= 0) {
            ret = avcodec_receive_frame(m_audioCodecContext, frame);
            if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
                break;
            } else if (ret < 0) {
                return;
            }
            ret = av_buffersrc_add_frame(m_audioSrcFilterCtx, frame);
            if (ret < 0) {
                std::cerr << "推送音频帧到源滤镜失败" << std::endl;
                showError(ret);
                continue;
            }

            while (av_buffersink_get_frame(m_audioSinkFilterCtx, filterFrame) >= 0) {
                if (!m_swrContext) {
                    initSwrContext(filterFrame);
                }

                int samples = swr_get_out_samples(m_swrContext, filterFrame->nb_samples);
                if (samples < 0) {
                    av_frame_free(&convertFrame);
                    return;
                }
                convertFrame->format = AV_SAMPLE_FMT_S16;
                int ret = av_channel_layout_copy(&convertFrame->ch_layout, &filterFrame->ch_layout);
                if (ret < 0) {
                   showError(ret);
                    return;
                }
                convertFrame->nb_samples = samples;
                ret = av_frame_get_buffer(convertFrame, 0);
                if (ret < 0) {
                    showError(ret);
                    av_frame_free(&convertFrame);
                    return;
                }
                convertFrame->pts = filterFrame->pts;
                samples = swr_convert(m_swrContext,
                                      convertFrame->data, samples,
                                      (const uint8_t **)filterFrame->data, filterFrame->nb_samples);
                if (samples < 0) {
                    av_frame_free(&convertFrame);
                    showError(samples);
                    return;
                }
                double pts_in_seconds = av_q2d(m_audioCodecContext->time_base) * convertFrame->pts;
                double diff = pts_in_seconds - m_timer->elapsedtime();
                while (diff > 0.001 && !m_stop) {
                    std::this_thread::sleep_for(std::chrono::milliseconds(5));
                    diff = pts_in_seconds - m_timer->elapsedtime();
                }
                convertFrame->nb_samples = samples;
                //av_frame_copy_props(convertFrame, filterFrame);
                //convertFrame->sample_rate = filterFrame->sample_rate;

                for (const auto& callback : m_audioCallbacks) {
                    callback(convertFrame);
                }
                av_frame_unref(filterFrame);
                av_frame_unref(convertFrame);
            }
            av_frame_unref(frame);
        }
    }
    av_frame_free(&frame);
    av_frame_free(&filterFrame);
    av_frame_free(&convertFrame);
}

bool VideoDecoder::initSwrContext(const AVFrame* frame)
{
    if (!m_audioCodecContext) {
        return false;
    }
    int ret = swr_alloc_set_opts2(&m_swrContext,
                                  &m_audioCodecContext->ch_layout,
                                  AV_SAMPLE_FMT_S16,
                                  m_audioCodecContext->sample_rate,
                                  &frame->ch_layout,
                                  (AVSampleFormat)frame->format,
                                  frame->sample_rate,
                                  0, NULL);
    if (ret < 0) {
        showError(ret);
        return false;
    }

    ret = swr_init(m_swrContext);
    if (ret < 0) {
        showError(ret);
        swr_free(&m_swrContext);
        m_swrContext = nullptr;
        return false;
    }
    return true;
}

bool VideoDecoder::initSwsContext(const AVFrame* frame)
{
    // 创建转换上下文并指定颜色空间
    m_swsContext = sws_getContext(
        frame->width, frame->height, (AVPixelFormat)frame->format,
        m_videoCodecContext->width, m_videoCodecContext->height, AV_PIX_FMT_RGB24,
        SWS_BILINEAR, nullptr, nullptr, nullptr);
    return m_swsContext != nullptr;
}

bool VideoDecoder::initFilter()
{
    m_filterGraph = avfilter_graph_alloc();
    if (!m_filterGraph) {
        return false;
    }

    // 创建视频滤镜
    int ret = 0;
    const AVFilter* buffersrc = avfilter_get_by_name("buffer");
    const AVFilter* buffersink = avfilter_get_by_name("buffersink");
    AVRational time_base = m_videoCodecContext->time_base;
    char args[512];
    snprintf(args, sizeof(args),
             "video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
             m_videoCodecContext->width, m_videoCodecContext->height, m_videoCodecContext->pix_fmt,
             time_base.num, time_base.den,
             m_videoCodecContext->sample_aspect_ratio.num, m_videoCodecContext->sample_aspect_ratio.den);

    ret = avfilter_graph_create_filter(&m_videoSrcFilterCtx, buffersrc, "in", args, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 buffer 滤镜" << std::endl;
        return false;
    }

    ret = avfilter_graph_create_filter(&m_videoSinkFilterCtx, buffersink, "out", NULL, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 buffersink 滤镜" << std::endl;
        return false;
    }

    AVFilterContext* videoSpeedCtx = nullptr;
    const AVFilter* setpts = avfilter_get_by_name("setpts");
    if ((ret = avfilter_graph_create_filter(&videoSpeedCtx,
                                            setpts, "speed",
                                            "0.5*PTS", nullptr, m_filterGraph)) < 0) {
        std::cout << "无法创建视频滤镜 " << std::endl;
        return -1;
    }

    if (avfilter_link(m_videoSrcFilterCtx, 0, videoSpeedCtx, 0) < 0 ||
        avfilter_link(videoSpeedCtx, 0, m_videoSinkFilterCtx, 0) < 0) {
        std::cerr << "连接视频滤镜失败" << std::endl;
        return false;
    }

    const AVFilter* abuffersrc = avfilter_get_by_name("abuffer");
    const AVFilter* abuffersink = avfilter_get_by_name("abuffersink");

    char ch_layout_str[256] = {0};
    size_t buf_size = sizeof(ch_layout_str);
    av_channel_layout_describe(&m_audioCodecContext->ch_layout, ch_layout_str, buf_size);

    char aargs[512];
    ret = snprintf(aargs, sizeof(aargs),
                   "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=%s",
                   m_audioCodecContext->time_base.num, m_audioCodecContext->time_base.den, m_audioCodecContext->sample_rate,
                   av_get_sample_fmt_name(m_audioCodecContext->sample_fmt), ch_layout_str);

    ret = avfilter_graph_create_filter(&m_audioSrcFilterCtx, abuffersrc, "in", aargs, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 buffer 滤镜" << std::endl;
        return false;
    }

    ret = avfilter_graph_create_filter(&m_audioSinkFilterCtx, abuffersink, "out", NULL, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 abuffersink 滤镜" << std::endl;
        showError(ret);
        return false;
    }

    // atempo 无法处理超过两倍的加速，但是可以通过叠加滤镜进行处理
    AVFilterContext* audioSpeedCtx = nullptr;
    const AVFilter* atempo = avfilter_get_by_name("atempo");
    if (avfilter_graph_create_filter(&audioSpeedCtx, atempo, "atempo", "tempo=2.0", nullptr, m_filterGraph) < 0) {
        std::cerr << "创建音频倍速滤镜失败 " << std::endl;
        return false;
    }

    if (avfilter_link(m_audioSrcFilterCtx, 0, audioSpeedCtx, 0) < 0 ||
        avfilter_link(audioSpeedCtx, 0, m_audioSinkFilterCtx, 0) < 0) {
        std::cerr << "连接音频滤镜失败" << std::endl;
        return false;
    }

    if ((ret = avfilter_graph_config(m_filterGraph, nullptr)) < 0) {
        std::cerr << "滤镜图配置失败 " << std::endl;
        return -1;
    }

    return true;
}

bool VideoDecoder::initFilterByDescribe()
{
    m_filterGraph = avfilter_graph_alloc();
    if (!m_filterGraph) {
        return false;
    }

    // 创建视频滤镜
    int ret = 0;
    const AVFilter* buffersrc = avfilter_get_by_name("buffer");
    const AVFilter* buffersink = avfilter_get_by_name("buffersink");
    AVRational time_base = m_videoCodecContext->time_base;

    char args[512];
    snprintf(args, sizeof(args),
             "video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
             m_videoCodecContext->width, m_videoCodecContext->height, m_videoCodecContext->pix_fmt,
             time_base.num, time_base.den,
             m_videoCodecContext->sample_aspect_ratio.num, m_videoCodecContext->sample_aspect_ratio.den);

    ret = avfilter_graph_create_filter(&m_videoSrcFilterCtx, buffersrc, "in",
                                       args, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 buffer 滤镜" << std::endl;
        return false;
    }

    // 设置buffersink参数
    ret = avfilter_graph_create_filter(&m_videoSinkFilterCtx, buffersink, "out",
                                       NULL, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 buffersink 滤镜" << std::endl;
        return false;
    }

    // 这里需要注意的是 outputs 对应的是输入滤镜，inputs 对应的是输出滤镜
    // 可以理解为数据从输入滤镜输出到 AVFilterInOut ，所以是 output,
    AVFilterInOut* outputs = avfilter_inout_alloc();
    AVFilterInOut* inputs = avfilter_inout_alloc();
    if (!outputs || !inputs) {
        return false;
    }
    outputs->name = av_strdup("in");
    outputs->filter_ctx = m_videoSrcFilterCtx;
    outputs->pad_idx = 0;
    outputs->next = NULL;

    inputs->name = av_strdup("out");
    inputs->filter_ctx = m_videoSinkFilterCtx;
    inputs->pad_idx = 0;
    inputs->next = NULL;

    if ((ret = avfilter_graph_parse_ptr(m_filterGraph, "setpts=PTS/2",
                                        &inputs, &outputs, NULL)) < 0) {
        std::cout << "无法解析滤镜描述" << std::endl;
        return false;
    }

    avfilter_inout_free(&inputs);
    avfilter_inout_free(&outputs);

    const AVFilter* abuffersrc = avfilter_get_by_name("abuffer");
    const AVFilter* abuffersink = avfilter_get_by_name("abuffersink");

    char ch_layout_str[256] = {0};
    size_t buf_size = sizeof(ch_layout_str);
    av_channel_layout_describe(&m_audioCodecContext->ch_layout, ch_layout_str, buf_size);

    char aargs[512];
    ret = snprintf(aargs, sizeof(aargs),
                   "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=%s",
                   m_audioCodecContext->time_base.num, m_audioCodecContext->time_base.den, m_audioCodecContext->sample_rate,
                   av_get_sample_fmt_name(m_audioCodecContext->sample_fmt), ch_layout_str);

    ret = avfilter_graph_create_filter(&m_audioSrcFilterCtx, abuffersrc, "in",
                                       aargs, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 buffer 滤镜" << std::endl;
        return false;
    }

    ret = avfilter_graph_create_filter(&m_audioSinkFilterCtx, abuffersink, "out",
                                       NULL, NULL, m_filterGraph);
    if (ret < 0) {
        std::cout << "无法创建 abuffersink 滤镜" << std::endl;
        showError(ret);
        return false;
    }

    AVFilterInOut* ainputs = avfilter_inout_alloc();
    AVFilterInOut* aoutputs = avfilter_inout_alloc();
    if (!aoutputs || !ainputs) {
        return false;
    }
    aoutputs->name = av_strdup("in");
    aoutputs->filter_ctx = m_audioSrcFilterCtx;
    aoutputs->pad_idx = 0;
    aoutputs->next = NULL;

    ainputs->name = av_strdup("out");
    ainputs->filter_ctx = m_audioSinkFilterCtx;
    ainputs->pad_idx = 0;
    ainputs->next = NULL;

    ret = avfilter_graph_parse_ptr(m_filterGraph, "atempo=2", &ainputs, &aoutputs, NULL);
    if (ret < 0) {
        av_log(NULL, AV_LOG_ERROR, "Cannot parse audio filter string\n");
        showError(ret);
        return false;
    }

    if ((ret = avfilter_graph_config(m_filterGraph, NULL)) < 0) {
        std::cout << "滤镜图配置失败" << std::endl;
        return false;
    }

    avfilter_inout_free(&ainputs);
    avfilter_inout_free(&aoutputs);
    return true;
}

AVHWDeviceType VideoDecoder::findSupportedHWDeviceType(const AVCodec* codec) {
    for (int i = 0;; i++) {
        const AVCodecHWConfig* hwConfig = avcodec_get_hw_config(codec, i);
        if (hwConfig != nullptr) {
            m_hwPixFmt = hwConfig->pix_fmt; // 记录硬件像素格式
            return hwConfig->device_type;
        } else {
            break;
        }
    }
    return AV_HWDEVICE_TYPE_NONE;
}

bool VideoDecoder::initHWDevice()
{
    if (m_hwDeviceType == AV_HWDEVICE_TYPE_NONE || !m_videoCodecContext) {
        return false;
    }

    // 创建硬件设备上下文（NULL表示自动选择设备，如默认GPU）
    int ret = av_hwdevice_ctx_create(&m_hwDeviceCtx, m_hwDeviceType,
                                     NULL, NULL, 0);
    if (ret < 0) {
        showError(ret);
        std::cerr << "创建硬件设备上下文失败（如GPU未识别）" << std::endl;
        return false;
    }

    // 将硬件设备上下文绑定到解码器
    m_videoCodecContext->hw_device_ctx = av_buffer_ref(m_hwDeviceCtx);
    if (!m_videoCodecContext->hw_device_ctx) {
        showError(AVERROR(ENOMEM));
        av_buffer_unref(&m_hwDeviceCtx);
        m_hwDeviceCtx = nullptr;
        return false;
    }

    // 初始化硬件帧上下文（管理硬件帧池）
    if (!initHWFramesContext()) {
        av_buffer_unref(&m_hwDeviceCtx);
        m_hwDeviceCtx = nullptr;
        return false;
    }

    m_bHardWare = true;
    return true;
}

bool VideoDecoder::initHWFramesContext()
{
    if (!m_hwDeviceCtx || !m_videoCodecContext) return false;

    // 分配硬件帧上下文
    m_hwFramesCtx = av_hwframe_ctx_alloc(m_hwDeviceCtx);
    if (!m_hwFramesCtx) {
        std::cerr << "分配硬件帧上下文失败" << std::endl;
        return false;
    }

    // 设置硬件帧参数
    AVHWFramesContext* hwFramesCtx = (AVHWFramesContext*)m_hwFramesCtx->data;
    hwFramesCtx->format = m_hwPixFmt;                  // 硬件像素格式
    if (m_hwPixFmt == AV_PIX_FMT_CUDA || m_hwPixFmt == AV_PIX_FMT_VULKAN) {
        hwFramesCtx->sw_format = AV_PIX_FMT_NV12;
    } else if (m_hwPixFmt == AV_PIX_FMT_VAAPI) {
        hwFramesCtx->sw_format = AV_PIX_FMT_YUV420P;
    }
    hwFramesCtx->width = m_videoCodecContext->width;   // 视频宽度
    hwFramesCtx->height = m_videoCodecContext->height; // 视频高度
    hwFramesCtx->initial_pool_size = 16;               // 帧池大小（建议 8-32）

    // 3. 初始化硬件帧上下文
    int ret = av_hwframe_ctx_init(m_hwFramesCtx);
    if (ret < 0) {
        showError(ret);
        av_buffer_unref(&m_hwFramesCtx);
        std::cerr << "初始化硬件帧上下文失败" << std::endl;
        return false;
    }

    // 4. 将硬件帧上下文绑定到解码器
    m_videoCodecContext->hw_frames_ctx = av_buffer_ref(m_hwFramesCtx);
    if (!m_videoCodecContext->hw_frames_ctx) {
        std::cerr << "绑定硬件帧上下文到解码器失败" << std::endl;
        av_buffer_unref(&m_hwFramesCtx);
        return false;
    }

    return true;
}

bool VideoDecoder::hwFrameToSwFrame(AVFrame *hwFrame, AVFrame *swFrame)
{
    if (!hwFrame || !swFrame) {
        return false;
    }

    AVHWFramesContext* hwFramesCtx = (AVHWFramesContext*)m_hwFramesCtx->data;
    av_frame_unref(swFrame);
    swFrame->format = hwFramesCtx->sw_format;
    swFrame->width = hwFrame->width;
    swFrame->height = hwFrame->height;

    int ret = av_frame_get_buffer(swFrame, 0);
    if (ret < 0) {
        showError(ret);
        std::cerr << "分配软件帧内存失败" << std::endl;
        return false;
    }

    // 硬件帧数据传输到软件帧（GPU -> CPU）
    ret = av_hwframe_transfer_data(swFrame, hwFrame, 0);
    if (ret < 0) {
        showError(ret);
        std::cerr << "硬件帧转软件帧失败（可能是GPU资源不足）" << std::endl;
        return false;
    }
    // 将 时间戳（pts） 音频的采样率（sample rate） 视频的像素宽高比（sample aspect ratio）等字段从源帧（src）复制到目标帧（dst）
    av_frame_copy_props(swFrame, hwFrame);

    return true;
}

AudioPlayer.h 代码如下

cpp 复制代码

#ifndef AUDIOPLAYER_H
#define AUDIOPLAYER_H
#include <SDL.h>
#include <vector>
#include <mutex>
#include <condition_variable>
#include "VideoStructs.h"
struct AVFrame;
using namespace std;
class AudioPlayer
{
public:
    AudioPlayer();
    ~AudioPlayer();
    bool initSdl(const AudioParams& param);
    void audioCallbackImpl(Uint8* stream, int len);
    static void audioCallback(void* userdata, Uint8* stream, int len);
    bool displayAudio(AVFrame* frame);

private:
    void closeSdl();
private:
    vector<AVFrame*> m_frames;
    std::mutex m_mutex;
    std::condition_variable m_available;
    std::condition_variable m_full;
    SDL_AudioDeviceID m_deviceId;
    bool m_stop = false;
    AVFrame* m_currentFrame = nullptr;
    int m_offset;

};

#endif // AUDIOPLAYER_H

AudioPlayer.cpp 代码如下

cpp 复制代码

#include "AudioPlayer.h"
#include <iostream>
#include <thread>
extern "C" {
#include "libavcodec/avcodec.h"
}
AudioPlayer::AudioPlayer() {
}

AudioPlayer::~AudioPlayer()
{
    m_stop = true;
    m_available.notify_all();
    closeSdl();
}

bool AudioPlayer::initSdl(const AudioParams &param)
{
    // 初始化SDL音频
    if (SDL_Init(SDL_INIT_AUDIO) < 0) {
        std::cerr << "SDL初始化失败: " << SDL_GetError() << std::endl;
        return false;
    }

    SDL_AudioSpec desired_spec, obtained_spec;
    desired_spec.freq = param.sampleRate;               // 采样率
    desired_spec.format = AUDIO_S16SYS;                 // 16位有符号整数
    desired_spec.channels = 2;
    desired_spec.silence = 0;                           // 静音值
    desired_spec.samples = param.samples;                         // 缓冲区大小
    if (desired_spec.samples <= 0) {
        desired_spec.samples = 1024;
    }
    desired_spec.callback = audioCallback;              // 回调函数
    desired_spec.userdata = this;                       // 传递this指针

    m_deviceId = SDL_OpenAudioDevice(nullptr, 0, &desired_spec, &obtained_spec, 0);
    if (m_deviceId == 0) {
        std::cerr << "无法打开音频设备: " << SDL_GetError() << std::endl;
        return false;
    }
    SDL_PauseAudioDevice(m_deviceId, 0);
    return true;
}

void AudioPlayer::audioCallbackImpl(Uint8 *stream, int len)
{
    SDL_memset(stream, 0, len);
    {
        std::unique_lock<std::mutex> lock(m_mutex);
        m_available.wait(lock, [this] {
            return m_frames.size() > 4 || m_deviceId == 0 || m_stop;
        });
    }
    if (m_currentFrame == nullptr) {
        m_currentFrame= m_frames.front();
        m_frames.erase(m_frames.begin());
        m_offset = 0;
    }
    int offset = 0;
    while (len > 0 && !m_stop) {
        if (m_deviceId == 0 || m_stop) {
            return;
        }

        // 计算本次回调需要复制的数据量
        int bytes_to_copy = std::min(len, m_currentFrame->linesize[0] - m_offset);
        // 复制数据到音频缓冲区
        memcpy(stream + offset, m_currentFrame->data[0] + m_offset, bytes_to_copy);
        // 更新位置
        offset += bytes_to_copy;
        m_offset += bytes_to_copy;
        len -= bytes_to_copy;
        if (m_currentFrame->linesize[0] == m_offset) {
            std::unique_lock<std::mutex> lock(m_mutex);
            av_frame_free(&m_currentFrame);
            m_currentFrame = m_frames.front();
            m_frames.erase(m_frames.begin());
            m_offset = 0;
        }
    }
}

void AudioPlayer::audioCallback(void *userdata, Uint8 *stream, int len)
{
    AudioPlayer* player = static_cast<AudioPlayer*>(userdata);
    player->audioCallbackImpl(stream, len);
}

bool AudioPlayer::displayAudio(AVFrame *frame)
{
    {
        std::lock_guard<std::mutex> lock(m_mutex);
        AVFrame* cloneFrame = av_frame_alloc();
        if (av_frame_ref(cloneFrame, frame) != 0) {
            av_frame_free(&cloneFrame);
            return false;
        }
        m_frames.emplace_back(cloneFrame);
    }
    m_available.notify_one();
    return true;
}

void AudioPlayer::closeSdl()
{
    SDL_AudioStatus status = SDL_GetAudioDeviceStatus(m_deviceId);
    if (status == SDL_AUDIO_PLAYING) {
        SDL_PauseAudioDevice(m_deviceId, 1);
    }

    if (m_deviceId != 0) {
        SDL_CloseAudioDevice(m_deviceId);
        m_deviceId = 0;
    }
    SDL_QuitSubSystem(SDL_INIT_AUDIO);
}

VideoWidget.h 代码如下

cpp 复制代码

#include <QOpenGLWidget>
#include <QOpenGLFunctions>
#include <QOpenGLShaderProgram>
#include <QOpenGLTexture>
#include <QMutex>
#include <SDL.h>

struct AVFrame;
class VideoWidget : public QOpenGLWidget, protected QOpenGLFunctions {
    Q_OBJECT
public:
    explicit VideoWidget(QWidget *parent = nullptr);
    ~VideoWidget() override;
    // 接收解码后的帧并更新显示
    bool displayVideo(AVFrame* frame);
    // 设置是否保持宽高比
    void setKeepAspectRatio(bool keep);
    // 清除显示内容
    void clear();

protected:
    void initializeGL() override;
    void resizeGL(int w, int h) override;
    void paintGL() override;

private:
    bool initShader();
    void createTextures(int width, int height);
    void updateTextures(const AVFrame* frame);
    // 重置顶点坐标（处理宽高比）
    void resetVertices(int windowWidth, int windowHeight);


private:
    QOpenGLShaderProgram *m_program = nullptr;  // 着色器程序
    QOpenGLTexture* m_texture = nullptr;
    QMutex m_mutex;                             // 多线程同步锁
    AVFrame* m_currentFrame = nullptr;          // 当前待渲染的帧
    int m_frameWidth = 0;                       // 帧宽度
    int m_frameHeight = 0;                      // 帧高度
    bool m_hasNewFrame = false;                 // 是否有新帧待渲染
    bool m_keepAspectRatio = true;              // 是否保持宽高比
    GLfloat m_vertices[8] = {0};                // 顶点坐标数组
    bool m_isPlaying = false;                   // 播放状态标记
    QString m_errorMsg;                         // 错误信息
};

VideoWidget.cpp 代码如下

cpp 复制代码

#include "VideoWidget.h"
#include <QDebug>
#include <QPainter>
#include <QEvent>
extern "C" {
#include "libavcodec/avcodec.h"
}

VideoWidget::VideoWidget(QWidget *parent)
    : QOpenGLWidget(parent) {
    setAutoFillBackground(false);
    setAttribute(Qt::WA_OpaquePaintEvent);
    setAcceptDrops(true);
}

VideoWidget::~VideoWidget() {
    makeCurrent();
    // 释放纹理资源
    if (m_texture) {
        delete m_texture;
        m_texture = nullptr;
    }
    delete m_program;
    // 释放缓存的帧
    if (m_currentFrame) {
        av_frame_free(&m_currentFrame);
        m_currentFrame = nullptr;
    }
    doneCurrent();
}

void VideoWidget::setKeepAspectRatio(bool keep) {
    if (m_keepAspectRatio != keep) {
        m_keepAspectRatio = keep;
        resetVertices(width(), height());
        update();
    }
}

void VideoWidget::clear() {
    QMutexLocker locker(&m_mutex);
    if (m_currentFrame) {
        av_frame_free(&m_currentFrame);
        m_currentFrame = nullptr;
    }
    m_hasNewFrame = false;
    m_isPlaying = false;
    m_errorMsg.clear();
    update();
}

bool VideoWidget::displayVideo(AVFrame *frame) {

    QMutexLocker locker(&m_mutex);  // 加锁保护共享资源
    // 释放旧帧
    if (m_currentFrame) {
        av_frame_free(&m_currentFrame);
    }

    // 复制新帧（避免原帧被解码器释放）
    m_currentFrame = av_frame_alloc();
    if (av_frame_ref(m_currentFrame, frame) != 0) {
        qWarning() << "无法复制视频帧";
        av_frame_free(&m_currentFrame);
        return false;
    }

    // 更新帧尺寸（如果有变化）
    m_frameWidth = m_currentFrame->width;
    m_frameHeight = m_currentFrame->height;
    resetVertices(m_frameWidth, m_frameHeight);
    m_hasNewFrame = true;
    m_isPlaying = true;
    m_errorMsg.clear();
    // 触发重绘
    update();

    return true;
}

void VideoWidget::initializeGL() {
    initializeOpenGLFunctions();
    glClearColor(0.0f, 0.0f, 0.0f, 1.0f); // 黑色背景
    glEnable(GL_TEXTURE_2D);

    // 初始化着色器（核心修改4：RGB着色器替换YUV着色器）
    if (!initShader()) {
        m_errorMsg = "着色器初始化失败";
        qFatal("%s", m_errorMsg.toUtf8().constData());
    }

    // 初始化顶点坐标
    resetVertices(width(), height());

    // 启用顶点属性（0：顶点坐标，1：纹理坐标）
    glEnableVertexAttribArray(0);
    glEnableVertexAttribArray(1);
}

void VideoWidget::resizeGL(int w, int h) {
    glViewport(0, 0, w, h);  // 设置视口为窗口大小
    resetVertices(w, h);     // 重新计算顶点坐标
}

void VideoWidget::paintGL() {
    glClear(GL_COLOR_BUFFER_BIT); // 清除颜色缓冲区
    QMutexLocker locker(&m_mutex);

    if (!m_isPlaying && !m_hasNewFrame) {
        return;
    }

    if (m_hasNewFrame && m_currentFrame) {
        // 首次渲染或帧尺寸变化时，创建RGB纹理
        if (!m_texture || m_frameWidth != m_currentFrame->width || m_frameHeight != m_currentFrame->height) {
            createTextures(m_frameWidth, m_frameHeight);
        }
        // 更新RGB纹理数据
        updateTextures(m_currentFrame);
        m_hasNewFrame = false;
    }

    // 绑定着色器并绘制（仅绑定单个RGB纹理）
    if (m_program && m_texture) {
        m_program->bind();
        m_texture->bind(0);
        m_program->setUniformValue("u_texture", 0);

        static const GLfloat texCoords[] = {
            0.0f, 1.0f,  // 左下
            1.0f, 1.0f,  // 右下
            0.0f, 0.0f,  // 左上
            1.0f, 0.0f   // 右上
        };

        // 传递顶点坐标和纹理坐标
        glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 0, m_vertices);
        glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, 0, texCoords);

        // 绘制四边形（GL_TRIANGLE_STRIP 高效绘制4个顶点）
        glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
        m_program->release();
        m_texture->release();
    }
}

bool VideoWidget::initShader() {
    m_program = new QOpenGLShaderProgram;

    // 顶点着色器：传递顶点坐标和纹理坐标
    const char* vertexShader = R"(
        #version 120
        attribute vec2 vertexIn;       // 顶点坐标（location 0）
        attribute vec2 textureIn;      // 纹理坐标（location 1）
        varying vec2 textureOut;       // 传递给片段着色器的纹理坐标
        void main() {
            gl_Position = vec4(vertexIn, 0.0, 1.0);
            textureOut = textureIn;
        }
    )";

    // 片段着色器：实现YUV420P转RGB（BT.601标准）
    const char* fragmentShader = R"(
#version 120
        #ifdef GL_ES
        precision mediump float; // 移动设备精度兼容
        #endif
        varying vec2 textureOut;       // 从顶点着色器接收的纹理坐标
        uniform sampler2D u_texture;   // RGB纹理（仅1个）

        void main() {
            // 直接采样RGB纹理（无需YUV转RGB计算）
            vec3 rgbColor = texture2D(u_texture, textureOut).rgb;
            // 输出RGB颜色（Alpha=1.0，不透明）
            gl_FragColor = vec4(rgbColor, 1.0);
        }
    )";

    // 编译并链接着色器
    if (!m_program->addShaderFromSourceCode(QOpenGLShader::Vertex, vertexShader)) {
        qWarning() << "顶点着色器错误：" << m_program->log();
        return false;
    }
    if (!m_program->addShaderFromSourceCode(QOpenGLShader::Fragment, fragmentShader)) {
        qWarning() << "片段着色器错误：" << m_program->log();
        return false;
    }
    if (!m_program->link()) {
        qWarning() << "着色器链接错误：" << m_program->log();
        return false;
    }
    return m_program->bind();
}

void VideoWidget::createTextures(int width, int height) {
    // 释放旧纹理（避免内存泄漏）
    if (m_texture) {
        delete m_texture;
        m_texture = nullptr;
    }

    // 创建2D纹理（格式为 RGB888，对应 RGB24 每通道8位）
    m_texture = new QOpenGLTexture(QOpenGLTexture::Target2D);
    m_texture->create();
    m_texture->setSize(width, height); // 纹理尺寸 = 帧尺寸
    m_texture->setFormat(QOpenGLTexture::RGBAFormat); // RGB24 对应格式
    // 过滤模式：线性插值（缩放时平滑）
    m_texture->setMinMagFilters(QOpenGLTexture::Linear, QOpenGLTexture::Linear);
    // 环绕模式：边缘像素延伸（避免纹理边缘失真）
    m_texture->setWrapMode(QOpenGLTexture::ClampToEdge);
    m_texture->allocateStorage(); // 分配纹理内存
}

void VideoWidget::updateTextures(const AVFrame* frame) {
    if (!frame || frame->format != AV_PIX_FMT_RGB24) {
        m_errorMsg = "不支持的帧格式（仅支持 RGB24）";
        qWarning() << m_errorMsg;
        return;
    }

    GLint prevRowLength;
    glGetIntegerv(GL_UNPACK_ROW_LENGTH, &prevRowLength);
    glBindTexture(GL_TEXTURE_2D, m_texture->textureId());

    int pixelPerRow = frame->linesize[0] / 3;
    glPixelStorei(GL_UNPACK_ROW_LENGTH, pixelPerRow); // 设置每行像素数
    glTexSubImage2D(
        GL_TEXTURE_2D, 0,          // 纹理目标、多级纹理层级
        0, 0,                      // 纹理偏移（x,y）
        frame->width, frame->height,// 纹理更新区域尺寸
        GL_RGB,                    // 像素数据格式（RGB顺序）
        GL_UNSIGNED_BYTE,          // 像素数据类型（无符号字节）
        frame->data[0]             // RGB24 数据指针
        );

    // 恢复原像素存储模式
    glPixelStorei(GL_UNPACK_ROW_LENGTH, prevRowLength);
    glBindTexture(GL_TEXTURE_2D, 0); // 解绑纹理
}

void VideoWidget::resetVertices(int windowWidth, int windowHeight) {
    if (windowWidth <= 0 || windowHeight <= 0 || !m_keepAspectRatio ||
        m_frameWidth <= 0 || m_frameHeight <= 0) {
        // 铺满模式
        const GLfloat fullScreenVertices[] = {
            -1.0f, -1.0f,
            1.0f, -1.0f,
            -1.0f,  1.0f,
            1.0f,  1.0f
        };
        memcpy(m_vertices, fullScreenVertices, sizeof(fullScreenVertices));
        return;
    }

    // 保持宽高比计算
    float videoAspect = static_cast<float>(m_frameWidth) / m_frameHeight;
    float windowAspect = static_cast<float>(windowWidth) / windowHeight;

    float scaleX = 1.0f;
    float scaleY = 1.0f;

    if (videoAspect > windowAspect) {
        // 视频更宽，按宽度缩放
        scaleX = windowAspect / videoAspect;
    } else {
        // 视频更高，按高度缩放
        scaleY = videoAspect / windowAspect;
    }

    // 计算顶点坐标
    const GLfloat vertices[] = {
        -scaleX, -scaleY,
        scaleX, -scaleY,
        -scaleX,  scaleY,
        scaleX,  scaleY
    };
    memcpy(m_vertices, vertices, sizeof(vertices));
}

Timer.h 代码如下

cpp 复制代码

#ifndef TIMER_H
#define TIMER_H
#include <mutex>
class Timer
{
public:
    Timer();
    void start();
    void pause();
    void stop();
    double elapsedtime();

private:
    double currentTimestamp();

private:
    std::mutex m_timeMutex;
    double m_lastclock;                                 // 记录上次更新的时间戳
    double m_elapsedtime;
};

#endif // TIMER_H

Timer.cpp 代码如下

cpp 复制代码

#include "Timer.h"
#include <chrono>

Timer::Timer() {
    m_elapsedtime = 0;
}

void Timer::start()
{
    m_lastclock = currentTimestamp();
}

void Timer::pause()
{
    m_timeMutex.lock();
    m_elapsedtime += currentTimestamp() - m_lastclock;
    m_lastclock = currentTimestamp();
    m_timeMutex.unlock();
}

void Timer::stop()
{
    m_timeMutex.lock();
    m_elapsedtime = 0;
    m_timeMutex.unlock();
}

double Timer::elapsedtime()
{
    m_timeMutex.lock();
    m_elapsedtime += currentTimestamp() - m_lastclock;
    m_lastclock = currentTimestamp();
    m_timeMutex.unlock();
    return m_elapsedtime;
}

double Timer::currentTimestamp()
{
    auto now = std::chrono::system_clock::now();
    auto duration = now.time_since_epoch();
    return std::chrono::duration<double>(duration).count();
}