文章目录
-
-
- [FFmpeg 中 read_thread 的引用计数管理机制详解](#FFmpeg 中 read_thread 的引用计数管理机制详解)
-
- 背景：为什么需要引用计数？
- [核心数据结构（来自 libavutil/buffer.h）](#核心数据结构（来自 libavutil/buffer.h）)
- [read_thread 中的引用计数流程](#read_thread 中的引用计数流程)
-
- [读取 Packet（av_read_frame）](#读取 Packet（av_read_frame）)
- 分发到队列（packet_queue_put）
- [read_thread 结束本次循环](#read_thread 结束本次循环)
- [解码线程消费 Packet](#解码线程消费 Packet)
- 特殊情形处理
-
- 丢弃无关流
- [EOF Flush（Null Packet）](#EOF Flush（Null Packet）)
- 引用计数操作函数总结
- 引用计数和C++的智能指针有什么区别?
- code
-
FFmpeg 中 read_thread 的引用计数管理机制详解
背景：为什么需要引用计数？
在多线程音视频播放器（如 ffplay）中：
- read_thread 负责从文件/网络读取压缩数据包（AVPacket）
- 解码线程（video_thread/audio_thread）消费这些 packet 进行解码
- 两者通过队列（PacketQueue）传递数据
若直接传递原始指针，极易出现：
- 内存提前释放（生产者释放后消费者仍在使用）
- 内存泄漏（无人释放）
- double-free（多次释放同一内存）
FFmpeg 通过基于 AVBufferRef 的引用计数机制，实现安全、高效、零拷贝的数据共享。
核心数据结构（来自 libavutil/buffer.h）
AVBuffer：真正的"所有者"
c
struct AVBuffer {
uint8_t *data; // 压缩数据起始地址
int size; // 数据大小
void (*free)(void *opaque, uint8_t *data); // 自定义释放回调
void *opaque; // 回调上下文
int refcount; // 【关键】引用计数（原子操作）
volatile int flags;
};- refcount 记录当前有多少个 AVBufferRef 指向此对象
- 当 refcount == 0 时，自动调用 free 回调释放 data 和自身
AVBufferRef：轻量级引用句柄
c
typedef struct AVBufferRef {
AVBuffer *buffer; // 指向 AVBuffer（所有者）
uint8_t *data; // 缓存 buffer->data（避免间接访问）
int size; // 缓存 buffer->size
} AVBufferRef;- AVBufferRef 不包含 refcount 字段！引用计数仅存在于 AVBuffer 中
AVPacket：封装媒体包
c
typedef struct AVPacket {
AVBufferRef *buf; // 指向带引用计数的数据缓冲区
int64_t pts, dts;
uint8_t *data; // 通常等于 buf->data
int size;
int stream_index;
// ...其他元数据...
} AVPacket;- 实际压缩数据由 buf->buffer->data 管理
- 元数据（pts/dts 等）属于 AVPacket 本身，每次复制都会深拷贝
read_thread 中的引用计数流程
读取 Packet（av_read_frame）
c
AVPacket *pkt = av_packet_alloc(); // 初始化空 packet
ret = av_read_frame(ic, pkt);- Demuxer 读取原始压缩数据（如 10KB H.264 NALU）
- 调用 av_new_packet(pkt, size)：
- 分配 AVBuffer（含数据内存）
- 设置 pkt->buf = av_buffer_create(...) → refcount = 1
- pkt->data = pkt->buf->data
- 填充 pts/dts/stream_index 等元数据
- pkt 是唯一持有者（refcount = 1）
分发到队列（packet_queue_put）
c
if (pkt->stream_index == is->video_stream) {
packet_queue_put(&is->videoq, pkt);
}- packet_queue_put 内部调用 av_packet_ref(&pkt1->pkt, pkt)
- av_packet_ref(&dst, &src) 行为：
- 元数据（pts/dts/stream_index）→ 深拷贝
- 数据缓冲区 → 调用 av_buffer_ref(src->buf)：
- dst->buf = src->buf
- src->buf->buffer->refcount++（原子操作）
- 引用计数从 1 → 2：
- read_thread 的 pkt 贡献 1
- 队列节点 pkt1->pkt 贡献 1
read_thread 结束本次循环
- 不调用 av_packet_unref(pkt)
- 下次循环调用 av_read_frame(ic, pkt) 时：
- 自动对 pkt 调用 av_packet_unref（清理旧数据）
- refcount--（从 2 → 1）
- 填入新 packet（新 AVBuffer，refcount = 1）
解码线程消费 Packet
c
while (1) {
ret = packet_queue_get(&is->videoq, &local_pkt, 1, &serial);
if (ret <= 0) break;
avcodec_send_packet(is->video_dec_ctx, &local_pkt);
av_packet_unref(&local_pkt); // ← 必须！
}- packet_queue_get 行为：
- 从队列取出节点 pkt1
- 执行 local_pkt = pkt1->pkt（结构体赋值）
- 释放节点 av_free(pkt1)
- 所有权转移给 local_pkt
- av_packet_unref(&local_pkt) 行为：
- 调用 av_buffer_unref(local_pkt.buf)
- refcount--（从 1 → 0）
- 触发 AVBuffer->free 回调 → 释放压缩数据和 AVBuffer
特殊情形处理
丢弃无关流
c
} else {
av_packet_unref(pkt); // 无关流（如第二音轨）
}- pkt 是唯一持有者（refcount = 1）
- 显式 unref → 立即释放内存
EOF Flush（Null Packet）
c
packet_queue_put_nullpacket(&is->videoq, is->video_stream);- 构造 pkt.data = NULL, pkt.size = 0
- pkt.buf == NULL → av_packet_ref 仅复制元数据
- 解码线程收到后调用 avcodec_send_packet(NULL) flush 解码器
- av_packet_unref 无副作用
引用计数操作函数总结
| 函数 | 作用 | 对 refcount 影响 |
|---|---|---|
| av_read_frame() | 读取新 packet | 自动 unref 旧 packet（-1），新建 packet（=1） |
| av_packet_ref(dst, src) | 安全复制 packet | refcount++ |
| av_packet_unref(pkt) | 释放 packet 引用 | refcount--，若为 0 则释放内存 |
| av_packet_move_ref(dst, src) | 转移所有权 | dst 接管引用，src 清空（refcount 不变） |
引用计数和C++的智能指针有什么区别?
实现层面的差异
| 维度 | FFmpeg (AVBufferRef) | C++ (std::shared_ptr) |
|---|---|---|
| 语言 | 纯 C（无类、无模板、无析构函数） | C++（依赖构造/析构、模板、运算符重载） |
| 数据结构 | 手动管理的结构体 + 原子操作 | 模板类 + 编译器自动生成代码 |
| 引用计数存储位置 | 在 AVBuffer 内部（与数据紧邻） | 控制块（control block），通常堆分配 |
| 线程安全 | 是（使用原子操作增减 refcount） | C++11 起：是（标准要求原子操作） |
| 自定义删除器 | 通过 av_buffer_create(data, size, free_cb, opaque, flags) 支持 |
shared_ptr<T>(ptr, deleter) 支持 |
| 循环引用处理 | ❌ 不支持（需手动避免） | ❌ shared_ptr 也不支持（需 weak_ptr） |
使用方式对比 :
FFmpeg（C 风格）
c
// 创建带引用计数的缓冲区
AVBufferRef *buf = av_buffer_alloc(1024); // refcount = 1
// 增加引用
AVBufferRef *buf2 = av_buffer_ref(buf); // refcount = 2
// 减少引用（可能触发释放）
av_buffer_unref(&buf); // refcount = 1
av_buffer_unref(&buf2); // refcount = 0 → 调用 free 回调释放内存 显式调用 ref / unref；
无作用域自动管理（必须手动配对）；
错误容易导致泄漏或 double-free。
C++（RAII 风格）
cpp
{
std::shared_ptr<MyData> ptr1 = std::make_shared<MyData>(...); // refcount = 1
{
std::shared_ptr<MyData> ptr2 = ptr1; // refcount = 2
// ptr2 离开作用域 → 自动析构 → refcount = 1
}
// ptr1 离开作用域 → refcount = 0 → 自动 delete
} 自动管理生命周期（离开作用域自动析构）；
无需手动调用 unref；
更安全，符合现代 C++ 哲学。
code
c
#include "config.h"
#include <inttypes.h>
#include <math.h>
#include <limits.h>
#include <signal.h>
#include <stdint.h>
#include "libavutil/avstring.h"
#include "libavutil/eval.h"
#include "libavutil/mathematics.h"
#include "libavutil/pixdesc.h"
#include "libavutil/imgutils.h"
#include "libavutil/dict.h"
#include "libavutil/parseutils.h"
#include "libavutil/samplefmt.h"
#include "libavutil/avassert.h"
#include "libavutil/time.h"
#include "libavformat/avformat.h"
#include "libavdevice/avdevice.h"
#include "libswscale/swscale.h"
#include "libavutil/opt.h"
#include "libavcodec/avfft.h"
#include "libswresample/swresample.h"
#if CONFIG_AVFILTER
# include "libavfilter/avfilter.h"
# include "libavfilter/buffersink.h"
# include "libavfilter/buffersrc.h"
#endif
#include <SDL.h>
#include <SDL_thread.h>
#include "cmdutils.h"
#include <assert.h>
const char program_name[] = "ffplay";
const int program_birth_year = 2003;
#define MAX_QUEUE_SIZE (15 * 1024 * 1024)
#define MIN_FRAMES 25
#define EXTERNAL_CLOCK_MIN_FRAMES 2
#define EXTERNAL_CLOCK_MAX_FRAMES 10
/* Minimum SDL audio buffer size, in samples. */
#define SDL_AUDIO_MIN_BUFFER_SIZE 512
/* Calculate actual buffer size keeping in mind not cause too frequent audio callbacks */
#define SDL_AUDIO_MAX_CALLBACKS_PER_SEC 30
/* Step size for volume control in dB */
#define SDL_VOLUME_STEP (0.75)
/* no AV sync correction is done if below the minimum AV sync threshold */
#define AV_SYNC_THRESHOLD_MIN 0.04
/* AV sync correction is done if above the maximum AV sync threshold */
#define AV_SYNC_THRESHOLD_MAX 0.1
/* If a frame duration is longer than this, it will not be duplicated to compensate AV sync */
#define AV_SYNC_FRAMEDUP_THRESHOLD 0.1
/* no AV correction is done if too big error */
#define AV_NOSYNC_THRESHOLD 10.0
/* maximum audio speed change to get correct sync */
#define SAMPLE_CORRECTION_PERCENT_MAX 10
/* external clock speed adjustment constants for realtime sources based on buffer fullness */
#define EXTERNAL_CLOCK_SPEED_MIN 0.900
#define EXTERNAL_CLOCK_SPEED_MAX 1.010
#define EXTERNAL_CLOCK_SPEED_STEP 0.001
/* we use about AUDIO_DIFF_AVG_NB A-V differences to make the average */
#define AUDIO_DIFF_AVG_NB 20
/* polls for possible required screen refresh at least this often, should be less than 1/fps */
#define REFRESH_RATE 0.01
/* NOTE: the size must be big enough to compensate the hardware audio buffersize size */
/* TODO: We assume that a decoded and resampled frame fits into this buffer */
#define SAMPLE_ARRAY_SIZE (8 * 65536)
#define CURSOR_HIDE_DELAY 1000000
#define USE_ONEPASS_SUBTITLE_RENDER 1
static unsigned sws_flags = SWS_BICUBIC;
typedef struct MyAVPacketList {
AVPacket pkt; //解封装后的数据
struct MyAVPacketList *next; //下一个节点
int serial; //播放序列
} MyAVPacketList;
typedef struct PacketQueue {
MyAVPacketList *first_pkt, *last_pkt; // 队首，队尾指针
int nb_packets; // 包数量，也就是队列元素数量
int size; // 队列所有元素的数据大小总和
int64_t duration; // 队列所有元素的数据播放持续时间
int abort_request; // 用户退出请求标志
int serial; // 播放序列号，和MyAVPacketList的serial作用相同，但改变的时序稍微有点不同
SDL_mutex *mutex; // 用于维持PacketQueue的多线程安全(SDL_mutex可以按pthread_mutex_t理解）
SDL_cond *cond; // 用于读、写线程相互通知(SDL_cond可以按pthread_cond_t理解)
} PacketQueue;
#define VIDEO_PICTURE_QUEUE_SIZE 3 // 图像帧缓存数量
#define SUBPICTURE_QUEUE_SIZE 16 // 字幕帧缓存数量
#define SAMPLE_QUEUE_SIZE 9 // 采样帧缓存数量
#define FRAME_QUEUE_SIZE FFMAX(SAMPLE_QUEUE_SIZE, FFMAX(VIDEO_PICTURE_QUEUE_SIZE, SUBPICTURE_QUEUE_SIZE))
typedef struct AudioParams {
int freq; // 采样率
int channels; // 通道数
int64_t channel_layout; // 通道布局，比如2.1声道，5.1声道等
enum AVSampleFormat fmt; // 音频采样格式，比如AV_SAMPLE_FMT_S16表示为有符号16bit深度，交错排列模式。
int frame_size; // 一个采样单元占用的字节数（比如2通道时，则左右通道各采样一次合成一个采样单元）
int bytes_per_sec; // 一秒时间的字节数，比如采样率48Khz，2 channel，16bit，则一秒48000*2*16/8=192000
} AudioParams;
// 这里讲的系统时钟 是通过av_gettime_relative()获取到的时钟，单位为微妙
typedef struct Clock {
double pts; // 时钟基础, 当前帧(待播放)显示时间戳，播放后，当前帧变成上一帧
// 当前pts与当前系统时钟的差值, audio、video对于该值是独立的
double pts_drift; // clock base minus time at which we updated the clock
// 当前时钟(如视频时钟)最后一次更新时间，也可称当前时钟时间
double last_updated; // 最后一次更新的系统时钟
double speed; // 时钟速度控制，用于控制播放速度
// 播放序列，所谓播放序列就是一段连续的播放动作，一个seek操作会启动一段新的播放序列
int serial; // clock is based on a packet with this serial
int paused; // = 1 说明是暂停状态
// 指向packet_serial
int *queue_serial; /* pointer to the current packet queue serial, used for obsolete clock detection */
} Clock;
/* Common struct for handling all types of decoded data and allocated render buffers. */
// 用于缓存解码后的数据
typedef struct Frame {
AVFrame *frame; // 指向数据帧
AVSubtitle sub; // 用于字幕
int serial; // 帧序列，在seek的操作时serial会变化
double pts; // 时间戳，单位为秒
double duration; // 该帧持续时间，单位为秒
int64_t pos; // 该帧在输入文件中的字节位置
int width; // 图像宽度
int height; // 图像高读
int format; // 对于图像为(enum AVPixelFormat)，
// 对于声音则为(enum AVSampleFormat)
AVRational sar; // 图像的宽高比（16:9，4:3...），如果未知或未指定则为0/1
int uploaded; // 用来记录该帧是否已经显示过？
int flip_v; // =1则旋转180， = 0则正常播放
} Frame;
/* 这是一个循环队列，windex是指其中的首元素，rindex是指其中的尾部元素. */
typedef struct FrameQueue {
Frame queue[FRAME_QUEUE_SIZE]; // FRAME_QUEUE_SIZE 最大size, 数字太大时会占用大量的内存，需要注意该值的设置
int rindex; // 读索引。待播放时读取此帧进行播放，播放后此帧成为上一帧
int windex; // 写索引
int size; // 当前总帧数
int max_size; // 可存储最大帧数
int keep_last; // = 1说明要在队列里面保持最后一帧的数据不释放，只在销毁队列的时候才将其真正释放
int rindex_shown; // 初始化为0，配合keep_last=1使用
SDL_mutex *mutex; // 互斥量
SDL_cond *cond; // 条件变量
PacketQueue *pktq; // 数据包缓冲队列
} FrameQueue;
/**
*音视频同步方式，缺省以音频为基准
*/
enum {
AV_SYNC_AUDIO_MASTER, // 以音频为基准
AV_SYNC_VIDEO_MASTER, // 以视频为基准
AV_SYNC_EXTERNAL_CLOCK, // 以外部时钟为基准，synchronize to an external clock */
};
/**
* 解码器封装
*/
typedef struct Decoder {
AVPacket pkt;
PacketQueue *queue; // 数据包队列
AVCodecContext *avctx; // 解码器上下文
int pkt_serial; // 包序列
int finished; // =0，解码器处于工作状态；=非0，解码器处于空闲状态
int packet_pending; // =0，解码器处于异常状态，需要考虑重置解码器；=1，解码器处于正常状态
SDL_cond *empty_queue_cond; // 检查到packet队列空时发送 signal缓存read_thread读取数据
int64_t start_pts; // 初始化时是stream的start time
AVRational start_pts_tb; // 初始化时是stream的time_base
int64_t next_pts; // 记录最近一次解码后的frame的pts，当解出来的部分帧没有有效的pts时则使用next_pts进行推算
AVRational next_pts_tb; // next_pts的单位
SDL_Thread *decoder_tid; // 线程句柄
} Decoder;
typedef struct VideoState {
SDL_Thread *read_tid; // 读线程句柄
AVInputFormat *iformat; // 指向demuxer
int abort_request; // =1时请求退出播放
int force_refresh; // =1时需要刷新画面，请求立即刷新画面的意思
int paused; // =1时暂停，=0时播放
int last_paused; // 暂存"暂停"/"播放"状态
int queue_attachments_req;
int seek_req; // 标识一次seek请求
int seek_flags; // seek标志，诸如AVSEEK_FLAG_BYTE等
int64_t seek_pos; // 请求seek的目标位置(当前位置+增量)
int64_t seek_rel; // 本次seek的位置增量
int read_pause_return;
AVFormatContext *ic; // iformat的上下文
int realtime; // =1为实时流
Clock audclk; // 音频时钟
Clock vidclk; // 视频时钟
Clock extclk; // 外部时钟
FrameQueue pictq; // 视频Frame队列
FrameQueue subpq; // 字幕Frame队列
FrameQueue sampq; // 采样Frame队列
Decoder auddec; // 音频解码器
Decoder viddec; // 视频解码器
Decoder subdec; // 字幕解码器
int audio_stream ; // 音频流索引
int av_sync_type; // 音视频同步类型, 默认audio master
double audio_clock; // 当前音频帧的PTS+当前帧Duration
int audio_clock_serial; // 播放序列，seek可改变此值
// 以下4个参数 非audio master同步方式使用
double audio_diff_cum; // used for AV difference average computation
double audio_diff_avg_coef;
double audio_diff_threshold;
int audio_diff_avg_count;
// end
AVStream *audio_st; // 音频流
PacketQueue audioq; // 音频packet队列
int audio_hw_buf_size; // SDL音频缓冲区的大小(字节为单位)
// 指向待播放的一帧音频数据，指向的数据区将被拷入SDL音频缓冲区。若经过重采样则指向audio_buf1，
// 否则指向frame中的音频
uint8_t *audio_buf; // 指向需要重采样的数据
uint8_t *audio_buf1; // 指向重采样后的数据
unsigned int audio_buf_size; // 待播放的一帧音频数据(audio_buf指向)的大小
unsigned int audio_buf1_size; // 申请到的音频缓冲区audio_buf1的实际尺寸
int audio_buf_index; // 更新拷贝位置 当前音频帧中已拷入SDL音频缓冲区
// 的位置索引(指向第一个待拷贝字节)
// 当前音频帧中尚未拷入SDL音频缓冲区的数据量:
// audio_buf_size = audio_buf_index + audio_write_buf_size
int audio_write_buf_size;
int audio_volume; // 音量
int muted; // =1静音，=0则正常
struct AudioParams audio_src; // 音频frame的参数
#if CONFIG_AVFILTER
struct AudioParams audio_filter_src;
#endif
struct AudioParams audio_tgt; // SDL支持的音频参数，重采样转换：audio_src->audio_tgt
struct SwrContext *swr_ctx; // 音频重采样context
int frame_drops_early; // 丢弃视频packet计数
int frame_drops_late; // 丢弃视频frame计数
enum ShowMode {
SHOW_MODE_NONE = -1, SHOW_MODE_VIDEO = 0, SHOW_MODE_WAVES, SHOW_MODE_RDFT, SHOW_MODE_NB
} show_mode;
// 音频波形显示使用
int16_t sample_array[SAMPLE_ARRAY_SIZE];
int sample_array_index;
int last_i_start;
RDFTContext *rdft;
int rdft_bits;
FFTSample *rdft_data;
int xpos;
double last_vis_time;
SDL_Texture *vis_texture;
SDL_Texture *sub_texture; // 字幕显示
SDL_Texture *vid_texture; // 视频显示
int subtitle_stream; // 字幕流索引
AVStream *subtitle_st; // 字幕流
PacketQueue subtitleq; // 字幕packet队列
double frame_timer; // 记录最后一帧播放的时刻
double frame_last_returned_time;
double frame_last_filter_delay;
int video_stream; // 视频流索引
AVStream *video_st; // 视频流
PacketQueue videoq; // 视频队列
double max_frame_duration; // 一帧最大间隔. above this, we consider the jump a timestamp discontinuity
struct SwsContext *img_convert_ctx; // 视频尺寸格式变换
struct SwsContext *sub_convert_ctx; // 字幕尺寸格式变换
int eof; // 是否读取结束
char *filename; // 文件名
int width, height, xleft, ytop; // 宽、高，x起始坐标，y起始坐标
int step; // =1 步进播放模式, =0 其他模式
#if CONFIG_AVFILTER
int vfilter_idx;
AVFilterContext *in_video_filter; // the first filter in the video chain
AVFilterContext *out_video_filter; // the last filter in the video chain
AVFilterContext *in_audio_filter; // the first filter in the audio chain
AVFilterContext *out_audio_filter; // the last filter in the audio chain
AVFilterGraph *agraph; // audio filter graph
#endif
// 保留最近的相应audio、video、subtitle流的steam index
int last_video_stream, last_audio_stream, last_subtitle_stream;
SDL_cond *continue_read_thread; // 当读取数据队列满了后进入休眠时，可以通过该condition唤醒读线程
} VideoState;
/* options specified by the user */
static AVInputFormat *file_iformat; //
static const char *input_filename;
static const char *window_title;
static int default_width = 640;
static int default_height = 480;
static int screen_width = 0;
static int screen_height = 0;
static int screen_left = SDL_WINDOWPOS_CENTERED; // 显示视频窗口的x坐标，默认在居中
static int screen_top = SDL_WINDOWPOS_CENTERED; // 显示视频窗口的y坐标，默认居中
static int audio_disable;
static int video_disable;
static int subtitle_disable;
static const char* wanted_stream_spec[AVMEDIA_TYPE_NB] = {0};
static int seek_by_bytes = -1;
static float seek_interval = 10; // 可以指定seek的间隔
static int display_disable;
static int borderless;
static int alwaysontop; // 是否顶置
static int startup_volume = 100; // 起始音量
static int show_status = 1;
static int av_sync_type = AV_SYNC_AUDIO_MASTER;
static int64_t start_time = AV_NOPTS_VALUE;
static int64_t duration = AV_NOPTS_VALUE;
static int fast = 0;
static int genpts = 0;
static int lowres = 0;
static int decoder_reorder_pts = -1;
static int autoexit;
static int exit_on_keydown;
static int exit_on_mousedown;
static int loop = 1;
static int framedrop = -1;
static int infinite_buffer = -1;
static enum ShowMode show_mode = SHOW_MODE_NONE;
static const char *audio_codec_name;
static const char *subtitle_codec_name;
static const char *video_codec_name;
double rdftspeed = 0.02;
static int64_t cursor_last_shown;
static int cursor_hidden = 0;
#if CONFIG_AVFILTER
static const char **vfilters_list = NULL;
static int nb_vfilters = 0;
static char *afilters = NULL;
#endif
static int autorotate = 1;
static int find_stream_info = 1;
static int filter_nbthreads = 0; // filter线程数量
/* current context */
static int is_full_screen;
static int64_t audio_callback_time;
static AVPacket flush_pkt;
#define FF_QUIT_EVENT (SDL_USEREVENT + 2)
static SDL_Window *window;
static SDL_Renderer *renderer;
static SDL_RendererInfo renderer_info = {0};
static SDL_AudioDeviceID audio_dev;
static const struct TextureFormatEntry {
enum AVPixelFormat format;
int texture_fmt;
} sdl_texture_format_map[] = {
{ AV_PIX_FMT_RGB8, SDL_PIXELFORMAT_RGB332 },
{ AV_PIX_FMT_RGB444, SDL_PIXELFORMAT_RGB444 },
{ AV_PIX_FMT_RGB555, SDL_PIXELFORMAT_RGB555 },
{ AV_PIX_FMT_BGR555, SDL_PIXELFORMAT_BGR555 },
{ AV_PIX_FMT_RGB565, SDL_PIXELFORMAT_RGB565 },
{ AV_PIX_FMT_BGR565, SDL_PIXELFORMAT_BGR565 },
{ AV_PIX_FMT_RGB24, SDL_PIXELFORMAT_RGB24 },
{ AV_PIX_FMT_BGR24, SDL_PIXELFORMAT_BGR24 },
{ AV_PIX_FMT_0RGB32, SDL_PIXELFORMAT_RGB888 },
{ AV_PIX_FMT_0BGR32, SDL_PIXELFORMAT_BGR888 },
{ AV_PIX_FMT_NE(RGB0, 0BGR), SDL_PIXELFORMAT_RGBX8888 },
{ AV_PIX_FMT_NE(BGR0, 0RGB), SDL_PIXELFORMAT_BGRX8888 },
{ AV_PIX_FMT_RGB32, SDL_PIXELFORMAT_ARGB8888 },
{ AV_PIX_FMT_RGB32_1, SDL_PIXELFORMAT_RGBA8888 },
{ AV_PIX_FMT_BGR32, SDL_PIXELFORMAT_ABGR8888 },
{ AV_PIX_FMT_BGR32_1, SDL_PIXELFORMAT_BGRA8888 },
{ AV_PIX_FMT_YUV420P, SDL_PIXELFORMAT_IYUV },
{ AV_PIX_FMT_YUYV422, SDL_PIXELFORMAT_YUY2 },
{ AV_PIX_FMT_UYVY422, SDL_PIXELFORMAT_UYVY },
{ AV_PIX_FMT_NONE, SDL_PIXELFORMAT_UNKNOWN },
};
#if CONFIG_AVFILTER
static int opt_add_vfilter(void *optctx, const char *opt, const char *arg)
{
GROW_ARRAY(vfilters_list, nb_vfilters);
vfilters_list[nb_vfilters - 1] = arg;
return 0;
}
#endif
static inline
int cmp_audio_fmts(enum AVSampleFormat fmt1, int64_t channel_count1,
enum AVSampleFormat fmt2, int64_t channel_count2)
{
/* If channel count == 1, planar and non-planar formats are the same */
if (channel_count1 == 1 && channel_count2 == 1)
return av_get_packed_sample_fmt(fmt1) != av_get_packed_sample_fmt(fmt2);
else
return channel_count1 != channel_count2 || fmt1 != fmt2;
}
static inline
int64_t get_valid_channel_layout(int64_t channel_layout, int channels)
{
if (channel_layout && av_get_channel_layout_nb_channels(channel_layout) == channels)
return channel_layout;
else
return 0;
}
static int packet_queue_put_private(PacketQueue *q, AVPacket *pkt)
{
MyAVPacketList *pkt1;
if (q->abort_request) //如果已中止，则放入失败
return -1;
pkt1 = av_malloc(sizeof(MyAVPacketList)); //分配节点内存
if (!pkt1) //内存不足，则放入失败
return -1;
// 没有做引用计数，那这里也说明av_read_frame不会释放替用户释放buffer。
pkt1->pkt = *pkt; //拷贝AVPacket(浅拷贝，AVPacket.data等内存并没有拷贝)
pkt1->next = NULL;
if (pkt == &flush_pkt)//如果放入的是flush_pkt，需要增加队列的播放序列号，以区分不连续的两段数据
{
q->serial++;
printf("q->serial = %d\n", q->serial++);
}
pkt1->serial = q->serial; //用队列序列号标记节点
/* 队列操作：如果last_pkt为空，说明队列是空的，新增节点为队头；
* 否则，队列有数据，则让原队尾的next为新增节点。 最后将队尾指向新增节点
*/
if (!q->last_pkt)
q->first_pkt = pkt1;
else
q->last_pkt->next = pkt1;
q->last_pkt = pkt1;
//队列属性操作：增加节点数、cache大小、cache总时长, 用来控制队列的大小
q->nb_packets++;
q->size += pkt1->pkt.size + sizeof(*pkt1);
q->duration += pkt1->pkt.duration;
/* XXX: should duplicate packet data in DV case */
//发出信号，表明当前队列中有数据了，通知等待中的读线程可以取数据了
SDL_CondSignal(q->cond);
return 0;
}
static int packet_queue_put(PacketQueue *q, AVPacket *pkt)
{
int ret;
SDL_LockMutex(q->mutex);
ret = packet_queue_put_private(q, pkt);//主要实现
SDL_UnlockMutex(q->mutex);
if (pkt != &flush_pkt && ret < 0)
av_packet_unref(pkt); //放入失败，释放AVPacket
return ret;
}
static int packet_queue_put_nullpacket(PacketQueue *q, int stream_index)
{
AVPacket pkt1, *pkt = &pkt1;
av_init_packet(pkt);
pkt->data = NULL;
pkt->size = 0;
pkt->stream_index = stream_index;
return packet_queue_put(q, pkt);
}
/* packet queue handling */
static int packet_queue_init(PacketQueue *q)
{
memset(q, 0, sizeof(PacketQueue));
q->mutex = SDL_CreateMutex();
if (!q->mutex) {
av_log(NULL, AV_LOG_FATAL, "SDL_CreateMutex(): %s\n", SDL_GetError());
return AVERROR(ENOMEM);
}
q->cond = SDL_CreateCond();
if (!q->cond) {
av_log(NULL, AV_LOG_FATAL, "SDL_CreateCond(): %s\n", SDL_GetError());
return AVERROR(ENOMEM);
}
q->abort_request = 1;
return 0;
}
static void packet_queue_flush(PacketQueue *q)
{
MyAVPacketList *pkt, *pkt1;
SDL_LockMutex(q->mutex);
for (pkt = q->first_pkt; pkt; pkt = pkt1) {
pkt1 = pkt->next;
av_packet_unref(&pkt->pkt);
av_freep(&pkt);
}
q->last_pkt = NULL;
q->first_pkt = NULL;
q->nb_packets = 0;
q->size = 0;
q->duration = 0;
SDL_UnlockMutex(q->mutex);
}
static void packet_queue_destroy(PacketQueue *q)
{
packet_queue_flush(q); //先清除所有的节点
SDL_DestroyMutex(q->mutex);
SDL_DestroyCond(q->cond);
}
static void packet_queue_abort(PacketQueue *q)
{
SDL_LockMutex(q->mutex);
q->abort_request = 1; // 请求退出
SDL_CondSignal(q->cond); //释放一个条件信号
SDL_UnlockMutex(q->mutex);
}
static void packet_queue_start(PacketQueue *q)
{
SDL_LockMutex(q->mutex);
q->abort_request = 0;
packet_queue_put_private(q, &flush_pkt); //这里放入了一个flush_pkt
SDL_UnlockMutex(q->mutex);
}
/* return < 0 if aborted, 0 if no packet and > 0 if packet. */
/**
* @brief packet_queue_get
* @param q 队列
* @param pkt 输出参数，即MyAVPacketList.pkt
* @param block 调用者是否需要在没节点可取的情况下阻塞等待
* @param serial 输出参数，即MyAVPacketList.serial
* @return <0: aborted; =0: no packet; >0: has packet
*/
static int packet_queue_get(PacketQueue *q, AVPacket *pkt, int block, int *serial)
{
MyAVPacketList *pkt1;
int ret;
SDL_LockMutex(q->mutex); // 加锁
for (;;) {
if (q->abort_request) {
ret = -1;
break;
}
pkt1 = q->first_pkt; //MyAVPacketList *pkt1; 从队头拿数据
if (pkt1) { //队列中有数据
q->first_pkt = pkt1->next; //队头移到第二个节点
if (!q->first_pkt)
q->last_pkt = NULL;
q->nb_packets--; //节点数减1
q->size -= pkt1->pkt.size + sizeof(*pkt1); //cache大小扣除一个节点
q->duration -= pkt1->pkt.duration; //总时长扣除一个节点
//返回AVPacket，这里发生一次AVPacket结构体拷贝，AVPacket的data只拷贝了指针
*pkt = pkt1->pkt;
if (serial) //如果需要输出serial，把serial输出
*serial = pkt1->serial;
av_free(pkt1); //释放节点内存,只是释放节点，而不是释放AVPacket
ret = 1;
break;
} else if (!block) { //队列中没有数据，且非阻塞调用
ret = 0;
break;
} else { //队列中没有数据，且阻塞调用
//这里没有break。for循环的另一个作用是在条件变量满足后重复上述代码取出节点
SDL_CondWait(q->cond, q->mutex);
}
}
SDL_UnlockMutex(q->mutex); // 释放锁
return ret;
}
static void decoder_init(Decoder *d, AVCodecContext *avctx, PacketQueue *queue, SDL_cond *empty_queue_cond) {
memset(d, 0, sizeof(Decoder));
d->avctx = avctx;
d->queue = queue;
d->empty_queue_cond = empty_queue_cond;
d->start_pts = AV_NOPTS_VALUE;
d->pkt_serial = -1;
}
static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) {
int ret = AVERROR(EAGAIN);
for (;;) {
AVPacket pkt;
if (d->queue->serial == d->pkt_serial) {
do {
if (d->queue->abort_request)
return -1;
switch (d->avctx->codec_type) {
case AVMEDIA_TYPE_VIDEO:
ret = avcodec_receive_frame(d->avctx, frame);
//printf("frame pts:%ld, dts:%ld\n", frame->pts, frame->pkt_dts);
if (ret >= 0) {
if (decoder_reorder_pts == -1) {
frame->pts = frame->best_effort_timestamp;
} else if (!decoder_reorder_pts) {
frame->pts = frame->pkt_dts;
}
}
break;
case AVMEDIA_TYPE_AUDIO:
ret = avcodec_receive_frame(d->avctx, frame);
if (ret >= 0) {
AVRational tb = (AVRational){1, frame->sample_rate};
if (frame->pts != AV_NOPTS_VALUE)
frame->pts = av_rescale_q(frame->pts, d->avctx->pkt_timebase, tb);
else if (d->next_pts != AV_NOPTS_VALUE)
frame->pts = av_rescale_q(d->next_pts, d->next_pts_tb, tb);
if (frame->pts != AV_NOPTS_VALUE) {
d->next_pts = frame->pts + frame->nb_samples;
d->next_pts_tb = tb;
}
}
break;
}
if (ret == AVERROR_EOF) {
d->finished = d->pkt_serial;
printf("avcodec_flush_buffers %s(%d)\n", __FUNCTION__, __LINE__);
avcodec_flush_buffers(d->avctx);
return 0;
}
if (ret >= 0)
return 1;
} while (ret != AVERROR(EAGAIN));
}
do {
if (d->queue->nb_packets == 0) // 没有数据可读
SDL_CondSignal(d->empty_queue_cond);
if (d->packet_pending) {
av_packet_move_ref(&pkt, &d->pkt);
d->packet_pending = 0;
} else {
if (packet_queue_get(d->queue, &pkt, 1, &d->pkt_serial) < 0)
return -1;
}
} while (d->queue->serial != d->pkt_serial);
if (pkt.data == flush_pkt.data) {
// when seeking or when switching to a different stream
avcodec_flush_buffers(d->avctx); //清空里面的缓存帧
d->finished = 0;
d->next_pts = d->start_pts;
d->next_pts_tb = d->start_pts_tb;
} else {
if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
int got_frame = 0;
ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, &pkt);
if (ret < 0) {
ret = AVERROR(EAGAIN);
} else {
if (got_frame && !pkt.data) {
d->packet_pending = 1;
av_packet_move_ref(&d->pkt, &pkt);
}
ret = got_frame ? 0 : (pkt.data ? AVERROR(EAGAIN) : AVERROR_EOF);
}
} else {
if (avcodec_send_packet(d->avctx, &pkt) == AVERROR(EAGAIN)) {
av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both returned EAGAIN, which is an API violation.\n");
d->packet_pending = 1;
av_packet_move_ref(&d->pkt, &pkt);
}
}
av_packet_unref(&pkt); // 一定要自己去释放音视频数据
}
}
}
static void decoder_destroy(Decoder *d) {
av_packet_unref(&d->pkt);
avcodec_free_context(&d->avctx);
}
static void frame_queue_unref_item(Frame *vp)
{
av_frame_unref(vp->frame); /* 释放数据 */
avsubtitle_free(&vp->sub);
}
/* 初始化FrameQueue，视频和音频keep_last设置为1，字幕设置为0 */
static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size, int keep_last)
{
int i;
memset(f, 0, sizeof(FrameQueue));
if (!(f->mutex = SDL_CreateMutex())) {
av_log(NULL, AV_LOG_FATAL, "SDL_CreateMutex(): %s\n", SDL_GetError());
return AVERROR(ENOMEM);
}
if (!(f->cond = SDL_CreateCond())) {
av_log(NULL, AV_LOG_FATAL, "SDL_CreateCond(): %s\n", SDL_GetError());
return AVERROR(ENOMEM);
}
f->pktq = pktq;
f->max_size = FFMIN(max_size, FRAME_QUEUE_SIZE);
f->keep_last = !!keep_last;
for (i = 0; i < f->max_size; i++)
if (!(f->queue[i].frame = av_frame_alloc())) // 分配AVFrame结构体
return AVERROR(ENOMEM);
return 0;
}
static void frame_queue_destory(FrameQueue *f)
{
int i;
for (i = 0; i < f->max_size; i++) {
Frame *vp = &f->queue[i];
// 释放对vp->frame中的数据缓冲区的引用，注意不是释放frame对象本身
frame_queue_unref_item(vp);
// 释放vp->frame对象
av_frame_free(&vp->frame);
}
SDL_DestroyMutex(f->mutex);
SDL_DestroyCond(f->cond);
}
static void frame_queue_signal(FrameQueue *f)
{
SDL_LockMutex(f->mutex);
SDL_CondSignal(f->cond);
SDL_UnlockMutex(f->mutex);
}
/* 获取队列当前Frame, 在调用该函数前先调用frame_queue_nb_remaining确保有frame可读 */
static Frame *frame_queue_peek(FrameQueue *f)
{
return &f->queue[(f->rindex + f->rindex_shown) % f->max_size];
}
/* 获取当前Frame的下一Frame, 此时要确保queue里面至少有2个Frame */
// 不管你什么时候调用，返回来肯定不是 NULL
static Frame *frame_queue_peek_next(FrameQueue *f)
{
return &f->queue[(f->rindex + f->rindex_shown + 1) % f->max_size];
}
/* 获取last Frame：
* 当rindex_shown=0时，和frame_queue_peek效果一样
* 当rindex_shown=1时，读取的是已经显示过的frame
*/
static Frame *frame_queue_peek_last(FrameQueue *f)
{
return &f->queue[f->rindex]; // 这时候才有意义
}
// 获取可写指针
static Frame *frame_queue_peek_writable(FrameQueue *f)
{
/* wait until we have space to put a new frame */
SDL_LockMutex(f->mutex);
while (f->size >= f->max_size &&
!f->pktq->abort_request) { /* 检查是否需要退出 */
SDL_CondWait(f->cond, f->mutex);
}
SDL_UnlockMutex(f->mutex);
if (f->pktq->abort_request) /* 检查是不是要退出 */
return NULL;
return &f->queue[f->windex];
}
static Frame *frame_queue_peek_readable(FrameQueue *f)
{
/* wait until we have a readable a new frame */
SDL_LockMutex(f->mutex);
while (f->size - f->rindex_shown <= 0 &&
!f->pktq->abort_request) {
SDL_CondWait(f->cond, f->mutex);
}
SDL_UnlockMutex(f->mutex);
if (f->pktq->abort_request)
return NULL;
return &f->queue[(f->rindex + f->rindex_shown) % f->max_size];
}
// 更新写指针
static void frame_queue_push(FrameQueue *f)
{
if (++f->windex == f->max_size)
f->windex = 0;
SDL_LockMutex(f->mutex);
f->size++;
SDL_CondSignal(f->cond); // 当_readable在等待时则可以唤醒
SDL_UnlockMutex(f->mutex);
}
/* 释放当前frame，并更新读索引rindex，
* 当keep_last为1, rindex_show为0时不去更新rindex,也不释放当前frame */
static void frame_queue_next(FrameQueue *f)
{
if (f->keep_last && !f->rindex_shown) {
f->rindex_shown = 1; // 第一次进来没有更新，对应的frame就没有释放
return;
}
frame_queue_unref_item(&f->queue[f->rindex]);
if (++f->rindex == f->max_size)
f->rindex = 0;
SDL_LockMutex(f->mutex);
f->size--;
SDL_CondSignal(f->cond);
SDL_UnlockMutex(f->mutex);
}
/* return the number of undisplayed frames in the queue */
static int frame_queue_nb_remaining(FrameQueue *f)
{
return f->size - f->rindex_shown; // 注意这里为什么要减去f->rindex_shown
}
/* return last shown position */
static int64_t frame_queue_last_pos(FrameQueue *f)
{
Frame *fp = &f->queue[f->rindex];
if (f->rindex_shown && fp->serial == f->pktq->serial)
return fp->pos;
else
return -1;
}
static void decoder_abort(Decoder *d, FrameQueue *fq)
{
packet_queue_abort(d->queue);
frame_queue_signal(fq);
SDL_WaitThread(d->decoder_tid, NULL);
d->decoder_tid = NULL;
packet_queue_flush(d->queue);
}
static inline void fill_rectangle(int x, int y, int w, int h)
{
SDL_Rect rect;
rect.x = x;
rect.y = y;
rect.w = w;
rect.h = h;
if (w && h)
SDL_RenderFillRect(renderer, &rect);
}
static int realloc_texture(SDL_Texture **texture, Uint32 new_format, int new_width, int new_height, SDL_BlendMode blendmode, int init_texture)
{
Uint32 format;
int access, w, h;
if (!*texture || SDL_QueryTexture(*texture, &format, &access, &w, &h) < 0 || new_width != w || new_height != h || new_format != format) {
void *pixels;
int pitch;
if (*texture)
SDL_DestroyTexture(*texture);
if (!(*texture = SDL_CreateTexture(renderer, new_format, SDL_TEXTUREACCESS_STREAMING, new_width, new_height)))
return -1;
if (SDL_SetTextureBlendMode(*texture, blendmode) < 0)
return -1;
if (init_texture) {
if (SDL_LockTexture(*texture, NULL, &pixels, &pitch) < 0)
return -1;
memset(pixels, 0, pitch * new_height);
SDL_UnlockTexture(*texture);
}
av_log(NULL, AV_LOG_VERBOSE, "Created %dx%d texture with %s.\n", new_width, new_height, SDL_GetPixelFormatName(new_format));
}
return 0;
}
// 4.2.0 和4.0有区别
/**
* @brief 将帧宽高按照sar最大适配到窗口
* @param rect
* @param scr_xleft
* @param scr_ytop
* @param scr_width
* @param scr_height
* @param pic_width
* @param pic_height
* @param pic_sar
*/
static void calculate_display_rect(SDL_Rect *rect,
int scr_xleft, int scr_ytop, int scr_width, int scr_height,
int pic_width, int pic_height, AVRational pic_sar)
{
AVRational aspect_ratio = pic_sar;
int64_t width, height, x, y;
if (av_cmp_q(aspect_ratio, av_make_q(0, 1)) <= 0)
aspect_ratio = av_make_q(1, 1);
aspect_ratio = av_mul_q(aspect_ratio, av_make_q(pic_width, pic_height));
/* XXX: we suppose the screen has a 1.0 pixel ratio */
height = scr_height;
width = av_rescale(height, aspect_ratio.num, aspect_ratio.den) & ~1;
if (width > scr_width) {
width = scr_width;
height = av_rescale(width, aspect_ratio.den, aspect_ratio.num) & ~1;
}
x = (scr_width - width) / 2;
y = (scr_height - height) / 2;
rect->x = scr_xleft + x;
rect->y = scr_ytop + y;
rect->w = FFMAX((int)width, 1);
rect->h = FFMAX((int)height, 1);
}
static void get_sdl_pix_fmt_and_blendmode(int format, Uint32 *sdl_pix_fmt, SDL_BlendMode *sdl_blendmode)
{
int i;
*sdl_blendmode = SDL_BLENDMODE_NONE;
*sdl_pix_fmt = SDL_PIXELFORMAT_UNKNOWN;
if (format == AV_PIX_FMT_RGB32 ||
format == AV_PIX_FMT_RGB32_1 ||
format == AV_PIX_FMT_BGR32 ||
format == AV_PIX_FMT_BGR32_1)
*sdl_blendmode = SDL_BLENDMODE_BLEND;
for (i = 0; i < FF_ARRAY_ELEMS(sdl_texture_format_map) - 1; i++) {
if (format == sdl_texture_format_map[i].format) {
*sdl_pix_fmt = sdl_texture_format_map[i].texture_fmt;
return;
}
}
}
static int upload_texture(SDL_Texture **tex, AVFrame *frame, struct SwsContext **img_convert_ctx) {
int ret = 0;
Uint32 sdl_pix_fmt;
SDL_BlendMode sdl_blendmode;
get_sdl_pix_fmt_and_blendmode(frame->format, &sdl_pix_fmt, &sdl_blendmode);
if (realloc_texture(tex, sdl_pix_fmt == SDL_PIXELFORMAT_UNKNOWN ? SDL_PIXELFORMAT_ARGB8888 : sdl_pix_fmt, frame->width, frame->height, sdl_blendmode, 0) < 0)
return -1;
switch (sdl_pix_fmt) {
case SDL_PIXELFORMAT_UNKNOWN:
/* This should only happen if we are not using avfilter... */
*img_convert_ctx = sws_getCachedContext(*img_convert_ctx,
frame->width, frame->height, frame->format, frame->width, frame->height,
AV_PIX_FMT_BGRA, sws_flags, NULL, NULL, NULL);
if (*img_convert_ctx != NULL) {
uint8_t *pixels[4];
int pitch[4];
if (!SDL_LockTexture(*tex, NULL, (void **)pixels, pitch)) {
sws_scale(*img_convert_ctx, (const uint8_t * const *)frame->data, frame->linesize,
0, frame->height, pixels, pitch);
SDL_UnlockTexture(*tex);
}
} else {
av_log(NULL, AV_LOG_FATAL, "Cannot initialize the conversion context\n");
ret = -1;
}
break;
case SDL_PIXELFORMAT_IYUV:
if (frame->linesize[0] > 0 && frame->linesize[1] > 0 && frame->linesize[2] > 0) {
ret = SDL_UpdateYUVTexture(*tex, NULL, frame->data[0], frame->linesize[0],
frame->data[1], frame->linesize[1],
frame->data[2], frame->linesize[2]);
} else if (frame->linesize[0] < 0 && frame->linesize[1] < 0 && frame->linesize[2] < 0) {
ret = SDL_UpdateYUVTexture(*tex, NULL, frame->data[0] + frame->linesize[0] * (frame->height - 1), -frame->linesize[0],
frame->data[1] + frame->linesize[1] * (AV_CEIL_RSHIFT(frame->height, 1) - 1), -frame->linesize[1],
frame->data[2] + frame->linesize[2] * (AV_CEIL_RSHIFT(frame->height, 1) - 1), -frame->linesize[2]);
} else {
av_log(NULL, AV_LOG_ERROR, "Mixed negative and positive linesizes are not supported.\n");
return -1;
}
break;
default:
if (frame->linesize[0] < 0) {
ret = SDL_UpdateTexture(*tex, NULL, frame->data[0] + frame->linesize[0] * (frame->height - 1), -frame->linesize[0]);
} else {
ret = SDL_UpdateTexture(*tex, NULL, frame->data[0], frame->linesize[0]);
}
break;
}
return ret;
}
static void set_sdl_yuv_conversion_mode(AVFrame *frame)
{
#if SDL_VERSION_ATLEAST(2,0,8)
SDL_YUV_CONVERSION_MODE mode = SDL_YUV_CONVERSION_AUTOMATIC;
if (frame && (frame->format == AV_PIX_FMT_YUV420P || frame->format == AV_PIX_FMT_YUYV422 || frame->format == AV_PIX_FMT_UYVY422)) {
if (frame->color_range == AVCOL_RANGE_JPEG)
mode = SDL_YUV_CONVERSION_JPEG;
else if (frame->colorspace == AVCOL_SPC_BT709)
mode = SDL_YUV_CONVERSION_BT709;
else if (frame->colorspace == AVCOL_SPC_BT470BG || frame->colorspace == AVCOL_SPC_SMPTE170M || frame->colorspace == AVCOL_SPC_SMPTE240M)
mode = SDL_YUV_CONVERSION_BT601;
}
SDL_SetYUVConversionMode(mode);
#endif
}
static void video_image_display(VideoState *is)
{
Frame *vp;
Frame *sp = NULL;
SDL_Rect rect;
// keep_last的作用就出来了,我们是有调用frame_queue_next, 但最近出队列的帧并没有真正销毁
// 所以这里可以读取出来显示
vp = frame_queue_peek_last(&is->pictq);
if (is->subtitle_st) {
if (frame_queue_nb_remaining(&is->subpq) > 0) {
sp = frame_queue_peek(&is->subpq);
if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time / 1000)) {
if (!sp->uploaded) {
uint8_t* pixels[4];
int pitch[4];
int i;
if (!sp->width || !sp->height) {
sp->width = vp->width;
sp->height = vp->height;
}
if (realloc_texture(&is->sub_texture, SDL_PIXELFORMAT_ARGB8888, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0)
return;
for (i = 0; i < sp->sub.num_rects; i++) {
AVSubtitleRect *sub_rect = sp->sub.rects[i];
sub_rect->x = av_clip(sub_rect->x, 0, sp->width );
sub_rect->y = av_clip(sub_rect->y, 0, sp->height);
sub_rect->w = av_clip(sub_rect->w, 0, sp->width - sub_rect->x);
sub_rect->h = av_clip(sub_rect->h, 0, sp->height - sub_rect->y);
is->sub_convert_ctx = sws_getCachedContext(is->sub_convert_ctx,
sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8,
sub_rect->w, sub_rect->h, AV_PIX_FMT_BGRA,
0, NULL, NULL, NULL);
if (!is->sub_convert_ctx) {
av_log(NULL, AV_LOG_FATAL, "Cannot initialize the conversion context\n");
return;
}
if (!SDL_LockTexture(is->sub_texture, (SDL_Rect *)sub_rect, (void **)pixels, pitch)) {
sws_scale(is->sub_convert_ctx, (const uint8_t * const *)sub_rect->data, sub_rect->linesize,
0, sub_rect->h, pixels, pitch);
SDL_UnlockTexture(is->sub_texture);
}
}
sp->uploaded = 1;
}
} else
sp = NULL;
}
}
//将帧宽高按照sar最大适配到窗口
calculate_display_rect(&rect, is->xleft, is->ytop, is->width, is->height, vp->width, vp->height, vp->sar);
if (!vp->uploaded) {
if (upload_texture(&is->vid_texture, vp->frame, &is->img_convert_ctx) < 0)
return;
vp->uploaded = 1;
vp->flip_v = vp->frame->linesize[0] < 0;
}
set_sdl_yuv_conversion_mode(vp->frame);
SDL_RenderCopyEx(renderer, is->vid_texture, NULL, &rect, 0, NULL, vp->flip_v ? SDL_FLIP_VERTICAL : 0);
set_sdl_yuv_conversion_mode(NULL);
if (sp) {
#if USE_ONEPASS_SUBTITLE_RENDER
SDL_RenderCopy(renderer, is->sub_texture, NULL, &rect);
#else
int i;
double xratio = (double)rect.w / (double)sp->width;
double yratio = (double)rect.h / (double)sp->height;
for (i = 0; i < sp->sub.num_rects; i++) {
SDL_Rect *sub_rect = (SDL_Rect*)sp->sub.rects[i];
SDL_Rect target = {.x = rect.x + sub_rect->x * xratio,
.y = rect.y + sub_rect->y * yratio,
.w = sub_rect->w * xratio,
.h = sub_rect->h * yratio};
SDL_RenderCopy(renderer, is->sub_texture, sub_rect, &target);
}
#endif
}
}
static inline int compute_mod(int a, int b)
{
return a < 0 ? a%b + b : a%b;
}
static void video_audio_display(VideoState *s)
{
int i, i_start, x, y1, y, ys, delay, n, nb_display_channels;
int ch, channels, h, h2;
int64_t time_diff;
int rdft_bits, nb_freq;
for (rdft_bits = 1; (1 << rdft_bits) < 2 * s->height; rdft_bits++)
;
nb_freq = 1 << (rdft_bits - 1);
/* compute display index : center on currently output samples */
channels = s->audio_tgt.channels;
nb_display_channels = channels;
if (!s->paused) {
int data_used= s->show_mode == SHOW_MODE_WAVES ? s->width : (2*nb_freq);
n = 2 * channels;
delay = s->audio_write_buf_size;
delay /= n;
/* to be more precise, we take into account the time spent since
the last buffer computation */
if (audio_callback_time) {
time_diff = av_gettime_relative() - audio_callback_time;
delay -= (time_diff * s->audio_tgt.freq) / 1000000;
}
delay += 2 * data_used;
if (delay < data_used)
delay = data_used;
i_start= x = compute_mod(s->sample_array_index - delay * channels, SAMPLE_ARRAY_SIZE);
if (s->show_mode == SHOW_MODE_WAVES) {
h = INT_MIN;
for (i = 0; i < 1000; i += channels) {
int idx = (SAMPLE_ARRAY_SIZE + x - i) % SAMPLE_ARRAY_SIZE;
int a = s->sample_array[idx];
int b = s->sample_array[(idx + 4 * channels) % SAMPLE_ARRAY_SIZE];
int c = s->sample_array[(idx + 5 * channels) % SAMPLE_ARRAY_SIZE];
int d = s->sample_array[(idx + 9 * channels) % SAMPLE_ARRAY_SIZE];
int score = a - d;
if (h < score && (b ^ c) < 0) {
h = score;
i_start = idx;
}
}
}
s->last_i_start = i_start;
} else {
i_start = s->last_i_start;
}
if (s->show_mode == SHOW_MODE_WAVES) {
SDL_SetRenderDrawColor(renderer, 255, 255, 255, 255);
/* total height for one channel */
h = s->height / nb_display_channels;
/* graph height / 2 */
h2 = (h * 9) / 20;
for (ch = 0; ch < nb_display_channels; ch++) {
i = i_start + ch;
y1 = s->ytop + ch * h + (h / 2); /* position of center line */
for (x = 0; x < s->width; x++) {
y = (s->sample_array[i] * h2) >> 15;
if (y < 0) {
y = -y;
ys = y1 - y;
} else {
ys = y1;
}
fill_rectangle(s->xleft + x, ys, 1, y);
i += channels;
if (i >= SAMPLE_ARRAY_SIZE)
i -= SAMPLE_ARRAY_SIZE;
}
}
SDL_SetRenderDrawColor(renderer, 0, 0, 255, 255);
for (ch = 1; ch < nb_display_channels; ch++) {
y = s->ytop + ch * h;
fill_rectangle(s->xleft, y, s->width, 1);
}
} else {
if (realloc_texture(&s->vis_texture, SDL_PIXELFORMAT_ARGB8888, s->width, s->height, SDL_BLENDMODE_NONE, 1) < 0)
return;
nb_display_channels= FFMIN(nb_display_channels, 2);
if (rdft_bits != s->rdft_bits) {
av_rdft_end(s->rdft);
av_free(s->rdft_data);
s->rdft = av_rdft_init(rdft_bits, DFT_R2C);
s->rdft_bits = rdft_bits;
s->rdft_data = av_malloc_array(nb_freq, 4 *sizeof(*s->rdft_data));
}
if (!s->rdft || !s->rdft_data){
av_log(NULL, AV_LOG_ERROR, "Failed to allocate buffers for RDFT, switching to waves display\n");
s->show_mode = SHOW_MODE_WAVES;
} else {
FFTSample *data[2];
SDL_Rect rect = {.x = s->xpos, .y = 0, .w = 1, .h = s->height};
uint32_t *pixels;
int pitch;
for (ch = 0; ch < nb_display_channels; ch++) {
data[ch] = s->rdft_data + 2 * nb_freq * ch;
i = i_start + ch;
for (x = 0; x < 2 * nb_freq; x++) {
double w = (x-nb_freq) * (1.0 / nb_freq);
data[ch][x] = s->sample_array[i] * (1.0 - w * w);
i += channels;
if (i >= SAMPLE_ARRAY_SIZE)
i -= SAMPLE_ARRAY_SIZE;
}
av_rdft_calc(s->rdft, data[ch]);
}
/* Least efficient way to do this, we should of course
* directly access it but it is more than fast enough. */
if (!SDL_LockTexture(s->vis_texture, &rect, (void **)&pixels, &pitch)) {
pitch >>= 2;
pixels += pitch * s->height;
for (y = 0; y < s->height; y++) {
double w = 1 / sqrt(nb_freq);
int a = sqrt(w * sqrt(data[0][2 * y + 0] * data[0][2 * y + 0] + data[0][2 * y + 1] * data[0][2 * y + 1]));
int b = (nb_display_channels == 2 ) ? sqrt(w * hypot(data[1][2 * y + 0], data[1][2 * y + 1]))
: a;
a = FFMIN(a, 255);
b = FFMIN(b, 255);
pixels -= pitch;
*pixels = (a << 16) + (b << 8) + ((a+b) >> 1);
}
SDL_UnlockTexture(s->vis_texture);
}
SDL_RenderCopy(renderer, s->vis_texture, NULL, NULL);
}
if (!s->paused)
s->xpos++;
if (s->xpos >= s->width)
s->xpos= s->xleft;
}
}
static void stream_component_close(VideoState *is, int stream_index)
{
AVFormatContext *ic = is->ic;
AVCodecParameters *codecpar;
if (stream_index < 0 || stream_index >= ic->nb_streams)
return;
codecpar = ic->streams[stream_index]->codecpar;
switch (codecpar->codec_type) {
case AVMEDIA_TYPE_AUDIO:
decoder_abort(&is->auddec, &is->sampq);
SDL_CloseAudioDevice(audio_dev);
decoder_destroy(&is->auddec);
swr_free(&is->swr_ctx);
av_freep(&is->audio_buf1);
is->audio_buf1_size = 0;
is->audio_buf = NULL;
if (is->rdft) {
av_rdft_end(is->rdft);
av_freep(&is->rdft_data);
is->rdft = NULL;
is->rdft_bits = 0;
}
break;
case AVMEDIA_TYPE_VIDEO:
decoder_abort(&is->viddec, &is->pictq);
decoder_destroy(&is->viddec);
break;
case AVMEDIA_TYPE_SUBTITLE:
decoder_abort(&is->subdec, &is->subpq);
decoder_destroy(&is->subdec);
break;
default:
break;
}
ic->streams[stream_index]->discard = AVDISCARD_ALL;
switch (codecpar->codec_type) {
case AVMEDIA_TYPE_AUDIO:
is->audio_st = NULL;
is->audio_stream = -1;
break;
case AVMEDIA_TYPE_VIDEO:
is->video_st = NULL;
is->video_stream = -1;
break;
case AVMEDIA_TYPE_SUBTITLE:
is->subtitle_st = NULL;
is->subtitle_stream = -1;
break;
default:
break;
}
}
static void stream_close(VideoState *is)
{
/* XXX: use a special url_shutdown call to abort parse cleanly */
is->abort_request = 1; // 请求退出
SDL_WaitThread(is->read_tid, NULL); // 等待数据读取线程退出
/* close each stream */
if (is->audio_stream >= 0)
stream_component_close(is, is->audio_stream);
if (is->video_stream >= 0)
stream_component_close(is, is->video_stream);
if (is->subtitle_stream >= 0)
stream_component_close(is, is->subtitle_stream);
avformat_close_input(&is->ic);
packet_queue_destroy(&is->videoq);
packet_queue_destroy(&is->audioq);
packet_queue_destroy(&is->subtitleq);
/* free all pictures */
frame_queue_destory(&is->pictq);
frame_queue_destory(&is->sampq);
frame_queue_destory(&is->subpq);
SDL_DestroyCond(is->continue_read_thread);
sws_freeContext(is->img_convert_ctx);
sws_freeContext(is->sub_convert_ctx);
av_free(is->filename);
if (is->vis_texture)
SDL_DestroyTexture(is->vis_texture);
if (is->vid_texture)
SDL_DestroyTexture(is->vid_texture);
if (is->sub_texture)
SDL_DestroyTexture(is->sub_texture);
av_free(is);
}
static void do_exit(VideoState *is)
{
if (is) {
stream_close(is);
}
if (renderer)
SDL_DestroyRenderer(renderer);
if (window)
SDL_DestroyWindow(window);
uninit_opts();
#if CONFIG_AVFILTER
av_freep(&vfilters_list);
#endif
avformat_network_deinit();
if (show_status)
printf("\n");
SDL_Quit();
av_log(NULL, AV_LOG_QUIET, "%s", "");
exit(0);
}
static void sigterm_handler(int sig)
{
exit(123);
}
static void set_default_window_size(int width, int height, AVRational sar)
{
SDL_Rect rect;
int max_width = screen_width ? screen_width : INT_MAX;
int max_height = screen_height ? screen_height : INT_MAX;
if (max_width == INT_MAX && max_height == INT_MAX)
max_height = height;
calculate_display_rect(&rect, 0, 0, max_width, max_height, width, height, sar);
default_width = rect.w;
default_height = rect.h;
}
static int video_open(VideoState *is)
{
int w,h;
w = screen_width ? screen_width : default_width;
h = screen_height ? screen_height : default_height;
if (!window_title)
window_title = input_filename;
SDL_SetWindowTitle(window, window_title);
SDL_SetWindowSize(window, w, h);
SDL_SetWindowPosition(window, screen_left, screen_top);
if (is_full_screen)
SDL_SetWindowFullscreen(window, SDL_WINDOW_FULLSCREEN_DESKTOP);
SDL_ShowWindow(window);
is->width = w;
is->height = h;
return 0;
}
/* display the current picture, if any */
static void video_display(VideoState *is)
{
if (!is->width)
video_open(is);
SDL_SetRenderDrawColor(renderer, 0, 0, 0, 255);
SDL_RenderClear(renderer);
if (is->audio_st && is->show_mode != SHOW_MODE_VIDEO)
video_audio_display(is);
else if (is->video_st)
video_image_display(is); //显示一帧视频画面
SDL_RenderPresent(renderer);
}
/**
* 获取到的实际上是:最后一帧的pts 加上 从处理最后一帧开始到现在的时间,具体参考set_clock_at 和get_clock的代码
* c->pts_drift=最后一帧的pts-从处理最后一帧时间
* clock=c->pts_drift+现在的时候
* get_clock(&is->vidclk) ==is->vidclk.pts, av_gettime_relative() / 1000000.0 -is->vidclk.last_updated +is->vidclk.pts
*/
static double get_clock(Clock *c)
{
if (*c->queue_serial != c->serial)
return NAN;
if (c->paused) {
return c->pts;
} else {
double time = av_gettime_relative() / 1000000.0;
return c->pts_drift + time - (time - c->last_updated) * (1.0 - c->speed);
}
}
static void set_clock_at(Clock *c, double pts, int serial, double time)
{
c->pts = pts; /* 当前帧的pts */
c->last_updated = time; /* 最后更新的时间，实际上是当前的一个系统时间 */
c->pts_drift = c->pts - time; /* 当前帧pts和系统时间的差值，正常播放情况下两者的差值应该是比较固定的，因为两者都是以时间为基准进行线性增长 */
c->serial = serial;
}
static void set_clock(Clock *c, double pts, int serial)
{
double time = av_gettime_relative() / 1000000.0;
set_clock_at(c, pts, serial, time);
}
static void set_clock_speed(Clock *c, double speed)
{
set_clock(c, get_clock(c), c->serial);
c->speed = speed;
}
static void init_clock(Clock *c, int *queue_serial)
{
c->speed = 1.0;
c->paused = 0;
c->queue_serial = queue_serial;
set_clock(c, NAN, -1);
}
static void sync_clock_to_slave(Clock *c, Clock *slave)
{
double clock = get_clock(c);
double slave_clock = get_clock(slave);
if (!isnan(slave_clock) && (isnan(clock) || fabs(clock - slave_clock) > AV_NOSYNC_THRESHOLD))
set_clock(c, slave_clock, slave->serial);
}
static int get_master_sync_type(VideoState *is) {
if (is->av_sync_type == AV_SYNC_VIDEO_MASTER) {
if (is->video_st)
return AV_SYNC_VIDEO_MASTER;
else
return AV_SYNC_AUDIO_MASTER; /* 如果没有视频成分则使用 audio master */
} else if (is->av_sync_type == AV_SYNC_AUDIO_MASTER) {
if (is->audio_st)
return AV_SYNC_AUDIO_MASTER;
else
return AV_SYNC_EXTERNAL_CLOCK; /* 没有音频的时候那就用外部时钟 */
} else {
return AV_SYNC_EXTERNAL_CLOCK;
}
}
/* get the current master clock value */
static double get_master_clock(VideoState *is)
{
double val;
switch (get_master_sync_type(is)) {
case AV_SYNC_VIDEO_MASTER:
val = get_clock(&is->vidclk);
break;
case AV_SYNC_AUDIO_MASTER:
val = get_clock(&is->audclk);
break;
default:
val = get_clock(&is->extclk);
break;
}
return val;
}
static void check_external_clock_speed(VideoState *is) {
if (is->video_stream >= 0 && is->videoq.nb_packets <= EXTERNAL_CLOCK_MIN_FRAMES ||
is->audio_stream >= 0 && is->audioq.nb_packets <= EXTERNAL_CLOCK_MIN_FRAMES) {
set_clock_speed(&is->extclk, FFMAX(EXTERNAL_CLOCK_SPEED_MIN, is->extclk.speed - EXTERNAL_CLOCK_SPEED_STEP));
} else if ((is->video_stream < 0 || is->videoq.nb_packets > EXTERNAL_CLOCK_MAX_FRAMES) &&
(is->audio_stream < 0 || is->audioq.nb_packets > EXTERNAL_CLOCK_MAX_FRAMES)) {
set_clock_speed(&is->extclk, FFMIN(EXTERNAL_CLOCK_SPEED_MAX, is->extclk.speed + EXTERNAL_CLOCK_SPEED_STEP));
} else {
double speed = is->extclk.speed;
if (speed != 1.0)
set_clock_speed(&is->extclk, speed + EXTERNAL_CLOCK_SPEED_STEP * (1.0 - speed) / fabs(1.0 - speed));
}
}
/* seek in the stream */
static void stream_seek(VideoState *is, int64_t pos, int64_t rel, int seek_by_bytes)
{
if (!is->seek_req) {
is->seek_pos = pos; // 微秒 1727,118,162
is->seek_rel = rel;
is->seek_flags &= ~AVSEEK_FLAG_BYTE;
if (seek_by_bytes)
is->seek_flags |= AVSEEK_FLAG_BYTE;
is->seek_req = 1; // 请求seek
SDL_CondSignal(is->continue_read_thread);
}
}
/* pause or resume the video */
static void stream_toggle_pause(VideoState *is)
{
if (is->paused) {
/* 恢复暂停状态时也需要恢复时钟，需要更新vidclk */
is->frame_timer += av_gettime_relative() / 1000000.0 - is->vidclk.last_updated;
if (is->read_pause_return != AVERROR(ENOSYS)) {
is->vidclk.paused = 0;
}
set_clock(&is->vidclk, get_clock(&is->vidclk), is->vidclk.serial);
}
set_clock(&is->extclk, get_clock(&is->extclk), is->extclk.serial);
// 切换 pause/resume 两种状态
is->paused = is->audclk.paused = is->vidclk.paused = is->extclk.paused = !is->paused;
printf("is->step = %d; stream_toggle_pause\n", is->step);
}
static void toggle_pause(VideoState *is)
{
stream_toggle_pause(is);
is->step = 0;
printf("is->step = 0; toggle_pause\n");
}
static void toggle_mute(VideoState *is)
{
is->muted = !is->muted;
}
static void update_volume(VideoState *is, int sign, double step)
{
double volume_level = is->audio_volume ? (20 * log(is->audio_volume / (double)SDL_MIX_MAXVOLUME) / log(10)) : -1000.0;
int new_volume = lrint(SDL_MIX_MAXVOLUME * pow(10.0, (volume_level + sign * step) / 20.0));
is->audio_volume = av_clip(is->audio_volume == new_volume ? (is->audio_volume + sign) : new_volume, 0, SDL_MIX_MAXVOLUME);
}
static void step_to_next_frame(VideoState *is)
{
/* if the stream is paused unpause it, then step */
if (is->paused)
stream_toggle_pause(is);
is->step = 1;
printf("is->step = 1; step_to_next_frame\n");
}
/**
* @brief 计算正在显示帧需要持续播放的时间。
* @param delay 该参数实际传递的是当前显示帧和待播放帧的间隔。
* @param is
* @return 返回当前显示帧要持续播放的时间。为什么要调整返回的delay？为什么不支持使用相邻间隔帧时间？
*/
static double compute_target_delay(double delay, VideoState *is)
{
double sync_threshold, diff = 0;
/* update delay to follow master synchronisation source */
/* 如果发现当前主Clock源不是video，则计算当前视频时钟与主时钟的差值 */
if (get_master_sync_type(is) != AV_SYNC_VIDEO_MASTER) {
/* if video is slave, we try to correct big delays by
duplicating or deleting a frame */
diff = get_clock(&is->vidclk) - get_master_clock(is);
/* skip or repeat frame. We take into account the
delay to compute the threshold. I still don't know
if it is the best guess */
sync_threshold = FFMAX(AV_SYNC_THRESHOLD_MIN,
FFMIN(AV_SYNC_THRESHOLD_MAX, delay));
if (!isnan(diff) && fabs(diff) < is->max_frame_duration) { // diff在最大帧duration内
if (diff <= -sync_threshold) { // 视频已经落后了
delay = FFMAX(0, delay + diff); //
}
else if (diff >= sync_threshold && delay > AV_SYNC_FRAMEDUP_THRESHOLD) {
// 视频超前
//AV_SYNC_FRAMEDUP_THRESHOLD是0.1，此时如果delay>0.1, 如果2*delay时间就有点久
delay = delay + diff;
}
else if (diff >= sync_threshold) {
delay = 2 * delay; // 保持在 2 * AV_SYNC_FRAMEDUP_THRESHOLD内, 即是2*0.1 = 0.2秒内
} else {
// 其他条件就是delay = delay; 维持原来的delay, 依靠frame_timer+duration和当前时间进行对比
}
}
} else {
// 如果是以video为同步，则直接返回last_duration
}
av_log(NULL, AV_LOG_TRACE, "video: delay=%0.3f A-V=%f\n",
delay, -diff);
return delay;
}
// 计算上一帧需要持续的duration，这里有校正算法
static double vp_duration(VideoState *is, Frame *vp, Frame *nextvp) {
if (vp->serial == nextvp->serial) { // 同一播放序列
double duration = nextvp->pts - vp->pts;
if (isnan(duration) // duration 数值异常
|| duration <= 0 // pts值没有递增时
|| duration > is->max_frame_duration // 超过了最大帧范围
) {
return vp->duration; /* 异常时以帧时间为基准(1秒/帧率) */
}
else {
return duration;
}
} else { // 不同播放序列
return 0.0;
}
}
static void update_video_pts(VideoState *is, double pts, int64_t pos, int serial) {
/* update current video pts */
set_clock(&is->vidclk, pts, serial);
sync_clock_to_slave(&is->extclk, &is->vidclk);
}
/* called to display each frame */
/* 非暂停或强制刷新的时候，循环调用video_refresh */
static void video_refresh(void *opaque, double *remaining_time)
{
VideoState *is = opaque;
double time;
Frame *sp, *sp2;
if (!is->paused && get_master_sync_type(is) == AV_SYNC_EXTERNAL_CLOCK && is->realtime)
check_external_clock_speed(is);
if (!display_disable && is->show_mode != SHOW_MODE_VIDEO && is->audio_st) {
time = av_gettime_relative() / 1000000.0;
if (is->force_refresh || is->last_vis_time + rdftspeed < time) {
video_display(is);
is->last_vis_time = time;
}
*remaining_time = FFMIN(*remaining_time, is->last_vis_time + rdftspeed - time);
}
if (is->video_st) {
retry:
if (frame_queue_nb_remaining(&is->pictq) == 0) {// 所有帧已显示
// nothing to do, no picture to display in the queue
// 什么都不做，队列中没有图像可显示
} else {
double last_duration, duration, delay;
Frame *vp, *lastvp;
/* dequeue the picture */
// 从队列取出上一个Frame
lastvp = frame_queue_peek_last(&is->pictq);// 上一帧：上次在显示的帧
vp = frame_queue_peek(&is->pictq); // 读取待显示的帧
// lastvp 上一帧(正在显示的帧)
// vp 等待显示的帧
if (vp->serial != is->videoq.serial) {
// 如果不是最新的播放序列，则将其从队列，以尽快读取最新的播放序列的Frame
frame_queue_next(&is->pictq);
goto retry;
}
if (lastvp->serial != vp->serial) {
// 新的播放序列重置当前时间
is->frame_timer = av_gettime_relative() / 1000000.0;
}
if (is->paused)
{
goto display;
printf("视频暂停is->paused");
}
/* compute nominal last_duration */
/*
* last_duration 计算上一帧应显示的时长
*/
last_duration = vp_duration(is, lastvp, vp);
/* 经过compute_target_delay方法，计算出待显示帧vp需要等待的时间 */
// 如果以video同步，则delay直接等于last_duration。
// 如果以audio或外部时钟同步，则需要比对主时钟调整待显示帧vp要等待的时间。
delay = compute_target_delay(last_duration, is);
time= av_gettime_relative()/1000000.0;
/* is->frame_timer 实际上就是上一帧lastvp的播放时间,
* is->frame_timer + delay 是待显示帧vp该播放的时间
* */
if (time < is->frame_timer + delay) { //判断是否继续显示上一帧
// 当前系统时刻还未到达上一帧的结束时刻，那么还应该继续显示上一帧。
// 计算出最小等待时间
*remaining_time = FFMIN(is->frame_timer + delay - time, *remaining_time);
goto display;
}
// 走到这一步，说明已经到了或过了该显示的时间，待显示帧vp的状态变更为当前要显示的帧
is->frame_timer += delay; // 更新当前帧播放的时间
if (delay > 0 && time - is->frame_timer > AV_SYNC_THRESHOLD_MAX) {
is->frame_timer = time; //如果和系统时间差距太大，就纠正为系统时间
}
SDL_LockMutex(is->pictq.mutex);
if (!isnan(vp->pts))
update_video_pts(is, vp->pts, vp->pos, vp->serial); // 更新video时钟
SDL_UnlockMutex(is->pictq.mutex);
if (frame_queue_nb_remaining(&is->pictq) > 1) {//只有有nextvp才会分析是否有必要检测是否该丢帧
Frame *nextvp = frame_queue_peek_next(&is->pictq);
duration = vp_duration(is, vp, nextvp);
if(!is->step // 非逐帧模式才检测是否需要丢帧 is->step==1 为逐帧播放
&& (framedrop>0 || // cpu解帧过慢
(framedrop && get_master_sync_type(is) != AV_SYNC_VIDEO_MASTER)) // 非视频同步方式
&& time > is->frame_timer + duration // 确实落后了一帧数据
) {
is->frame_drops_late++; // 统计丢帧情况
frame_queue_next(&is->pictq); // 帧出队列真正丢掉
goto retry;
}
}
if (is->subtitle_st) {
while (frame_queue_nb_remaining(&is->subpq) > 0) {
sp = frame_queue_peek(&is->subpq);
if (frame_queue_nb_remaining(&is->subpq) > 1)
sp2 = frame_queue_peek_next(&is->subpq);
else
sp2 = NULL;
if (sp->serial != is->subtitleq.serial
|| (is->vidclk.pts > (sp->pts + ((float) sp->sub.end_display_time / 1000)))
|| (sp2 && is->vidclk.pts > (sp2->pts + ((float) sp2->sub.start_display_time / 1000))))
{
if (sp->uploaded) {
int i;
for (i = 0; i < sp->sub.num_rects; i++) {
AVSubtitleRect *sub_rect = sp->sub.rects[i];
uint8_t *pixels;
int pitch, j;
if (!SDL_LockTexture(is->sub_texture, (SDL_Rect *)sub_rect, (void **)&pixels, &pitch)) {
for (j = 0; j < sub_rect->h; j++, pixels += pitch)
memset(pixels, 0, sub_rect->w << 2);
SDL_UnlockTexture(is->sub_texture);
}
}
}
frame_queue_next(&is->subpq);
} else {
break;
}
}
}
frame_queue_next(&is->pictq); // 当前vp帧出队列
is->force_refresh = 1; /* 说明需要刷新视频帧 */
if (is->step && !is->paused)
stream_toggle_pause(is);
}
display:
/* display picture */
if (!display_disable && is->force_refresh && is->show_mode == SHOW_MODE_VIDEO && is->pictq.rindex_shown)
video_display(is);
}
is->force_refresh = 0;
if (show_status) {
static int64_t last_time;
int64_t cur_time;
int aqsize, vqsize, sqsize;
double av_diff;
cur_time = av_gettime_relative();
if (!last_time || (cur_time - last_time) >= 30000) {
aqsize = 0;
vqsize = 0;
sqsize = 0;
if (is->audio_st)
aqsize = is->audioq.size;
if (is->video_st)
vqsize = is->videoq.size;
if (is->subtitle_st)
sqsize = is->subtitleq.size;
av_diff = 0;
if (is->audio_st && is->video_st)
av_diff = get_clock(&is->audclk) - get_clock(&is->vidclk);
else if (is->video_st)
av_diff = get_master_clock(is) - get_clock(&is->vidclk);
else if (is->audio_st)
av_diff = get_master_clock(is) - get_clock(&is->audclk);
av_log(NULL, AV_LOG_INFO,
"%7.2f %s:%7.3f fd=%4d aq=%5dKB vq=%5dKB sq=%5dB f=%"PRId64"/%"PRId64" \r",
get_master_clock(is),
(is->audio_st && is->video_st) ? "A-V" : (is->video_st ? "M-V" : (is->audio_st ? "M-A" : " ")),
av_diff,
is->frame_drops_early + is->frame_drops_late,
aqsize / 1024,
vqsize / 1024,
sqsize,
is->video_st ? is->viddec.avctx->pts_correction_num_faulty_dts : 0,
is->video_st ? is->viddec.avctx->pts_correction_num_faulty_pts : 0);
fflush(stdout);
last_time = cur_time;
}
}
}
static int queue_picture(VideoState *is, AVFrame *src_frame, double pts, double duration, int64_t pos, int serial)
{
Frame *vp;
#if defined(DEBUG_SYNC)
printf("frame_type=%c pts=%0.3f\n",
av_get_picture_type_char(src_frame->pict_type), pts);
#endif
if (!(vp = frame_queue_peek_writable(&is->pictq))) // 检测队列是否有可写空间
return -1; // Frame队列满了则返回-1
// 执行到这步说已经获取到了可写入的Frame
vp->sar = src_frame->sample_aspect_ratio;
vp->uploaded = 0;
vp->width = src_frame->width;
vp->height = src_frame->height;
vp->format = src_frame->format;
vp->pts = pts;
vp->duration = duration;
vp->pos = pos;
vp->serial = serial;
set_default_window_size(vp->width, vp->height, vp->sar);
av_frame_move_ref(vp->frame, src_frame); // 将src中所有数据拷贝到dst中，并复位src。
frame_queue_push(&is->pictq); // 更新写索引位置
return 0;
}
/**
* @brief 获取视频帧
* @param is
* @param frame 指向获取的视频帧
* @return
*/
static int get_video_frame(VideoState *is, AVFrame *frame)
{
int got_picture;
if ((got_picture = decoder_decode_frame(&is->viddec, frame, NULL)) < 0)
return -1;
if (got_picture) {
double dpts = NAN;
if (frame->pts != AV_NOPTS_VALUE)
dpts = av_q2d(is->video_st->time_base) * frame->pts;
frame->sample_aspect_ratio = av_guess_sample_aspect_ratio(is->ic, is->video_st, frame);
if (framedrop>0 || (framedrop && get_master_sync_type(is) != AV_SYNC_VIDEO_MASTER)) {
if (frame->pts != AV_NOPTS_VALUE) {
double diff = dpts - get_master_clock(is);
if (!isnan(diff) && fabs(diff) < AV_NOSYNC_THRESHOLD &&
diff - is->frame_last_filter_delay < 0 &&
is->viddec.pkt_serial == is->vidclk.serial &&
is->videoq.nb_packets) {
is->frame_drops_early++;
av_frame_unref(frame);
got_picture = 0;
}
}
}
}
return got_picture;
}
#if CONFIG_AVFILTER
static int configure_filtergraph(AVFilterGraph *graph, const char *filtergraph,
AVFilterContext *source_ctx, AVFilterContext *sink_ctx)
{
int ret, i;
int nb_filters = graph->nb_filters;
AVFilterInOut *outputs = NULL, *inputs = NULL;
if (filtergraph) {
outputs = avfilter_inout_alloc();
inputs = avfilter_inout_alloc();
if (!outputs || !inputs) {
ret = AVERROR(ENOMEM);
goto fail;
}
outputs->name = av_strdup("in");
outputs->filter_ctx = source_ctx;
outputs->pad_idx = 0;
outputs->next = NULL;
inputs->name = av_strdup("out");
inputs->filter_ctx = sink_ctx;
inputs->pad_idx = 0;
inputs->next = NULL;
if ((ret = avfilter_graph_parse_ptr(graph, filtergraph, &inputs, &outputs, NULL)) < 0)
goto fail;
} else {
if ((ret = avfilter_link(source_ctx, 0, sink_ctx, 0)) < 0)
goto fail;
}
/* Reorder the filters to ensure that inputs of the custom filters are merged first */
for (i = 0; i < graph->nb_filters - nb_filters; i++)
FFSWAP(AVFilterContext*, graph->filters[i], graph->filters[i + nb_filters]);
ret = avfilter_graph_config(graph, NULL);
fail:
avfilter_inout_free(&outputs);
avfilter_inout_free(&inputs);
return ret;
}
static int configure_video_filters(AVFilterGraph *graph, VideoState *is, const char *vfilters, AVFrame *frame)
{
enum AVPixelFormat pix_fmts[FF_ARRAY_ELEMS(sdl_texture_format_map)];
char sws_flags_str[512] = "";
char buffersrc_args[256];
int ret;
AVFilterContext *filt_src = NULL, *filt_out = NULL, *last_filter = NULL;
AVCodecParameters *codecpar = is->video_st->codecpar;
AVRational fr = av_guess_frame_rate(is->ic, is->video_st, NULL);
AVDictionaryEntry *e = NULL;
int nb_pix_fmts = 0;
int i, j;
for (i = 0; i < renderer_info.num_texture_formats; i++) {
for (j = 0; j < FF_ARRAY_ELEMS(sdl_texture_format_map) - 1; j++) {
if (renderer_info.texture_formats[i] == sdl_texture_format_map[j].texture_fmt) {
pix_fmts[nb_pix_fmts++] = sdl_texture_format_map[j].format;
break;
}
}
}
pix_fmts[nb_pix_fmts] = AV_PIX_FMT_NONE;
while ((e = av_dict_get(sws_dict, "", e, AV_DICT_IGNORE_SUFFIX))) {
if (!strcmp(e->key, "sws_flags")) {
av_strlcatf(sws_flags_str, sizeof(sws_flags_str), "%s=%s:", "flags", e->value);
} else
av_strlcatf(sws_flags_str, sizeof(sws_flags_str), "%s=%s:", e->key, e->value);
}
if (strlen(sws_flags_str))
sws_flags_str[strlen(sws_flags_str)-1] = '\0';
graph->scale_sws_opts = av_strdup(sws_flags_str);
snprintf(buffersrc_args, sizeof(buffersrc_args),
"video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
frame->width, frame->height, frame->format,
is->video_st->time_base.num, is->video_st->time_base.den,
codecpar->sample_aspect_ratio.num, FFMAX(codecpar->sample_aspect_ratio.den, 1));
if (fr.num && fr.den)
av_strlcatf(buffersrc_args, sizeof(buffersrc_args), ":frame_rate=%d/%d", fr.num, fr.den);
if ((ret = avfilter_graph_create_filter(&filt_src,
avfilter_get_by_name("buffer"),
"ffplay_buffer", buffersrc_args, NULL,
graph)) < 0)
goto fail;
ret = avfilter_graph_create_filter(&filt_out,
avfilter_get_by_name("buffersink"),
"ffplay_buffersink", NULL, NULL, graph);
if (ret < 0)
goto fail;
if ((ret = av_opt_set_int_list(filt_out, "pix_fmts", pix_fmts, AV_PIX_FMT_NONE, AV_OPT_SEARCH_CHILDREN)) < 0)
goto fail;
last_filter = filt_out;
/* Note: this macro adds a filter before the lastly added filter, so the
* processing order of the filters is in reverse */
#define INSERT_FILT(name, arg) do { \
AVFilterContext *filt_ctx; \
\
ret = avfilter_graph_create_filter(&filt_ctx, \
avfilter_get_by_name(name), \
"ffplay_" name, arg, NULL, graph); \
if (ret < 0) \
goto fail; \
\
ret = avfilter_link(filt_ctx, 0, last_filter, 0); \
if (ret < 0) \
goto fail; \
\
last_filter = filt_ctx; \
} while (0)
if (autorotate) {
double theta = get_rotation(is->video_st);
if (fabs(theta - 90) < 1.0) {
INSERT_FILT("transpose", "clock");
} else if (fabs(theta - 180) < 1.0) {
INSERT_FILT("hflip", NULL);
INSERT_FILT("vflip", NULL);
} else if (fabs(theta - 270) < 1.0) {
INSERT_FILT("transpose", "cclock");
} else if (fabs(theta) > 1.0) {
char rotate_buf[64];
snprintf(rotate_buf, sizeof(rotate_buf), "%f*PI/180", theta);
INSERT_FILT("rotate", rotate_buf);
}
}
if ((ret = configure_filtergraph(graph, vfilters, filt_src, last_filter)) < 0)
goto fail;
is->in_video_filter = filt_src;
is->out_video_filter = filt_out;
fail:
return ret;
}
static int configure_audio_filters(VideoState *is, const char *afilters, int force_output_format)
{
static const enum AVSampleFormat sample_fmts[] = { AV_SAMPLE_FMT_S16, AV_SAMPLE_FMT_NONE };
int sample_rates[2] = { 0, -1 };
int64_t channel_layouts[2] = { 0, -1 };
int channels[2] = { 0, -1 };
AVFilterContext *filt_asrc = NULL, *filt_asink = NULL;
char aresample_swr_opts[512] = "";
AVDictionaryEntry *e = NULL;
char asrc_args[256];
int ret;
avfilter_graph_free(&is->agraph);
if (!(is->agraph = avfilter_graph_alloc()))
return AVERROR(ENOMEM);
is->agraph->nb_threads = filter_nbthreads;
while ((e = av_dict_get(swr_opts, "", e, AV_DICT_IGNORE_SUFFIX)))
av_strlcatf(aresample_swr_opts, sizeof(aresample_swr_opts), "%s=%s:", e->key, e->value);
if (strlen(aresample_swr_opts))
aresample_swr_opts[strlen(aresample_swr_opts)-1] = '\0';
av_opt_set(is->agraph, "aresample_swr_opts", aresample_swr_opts, 0);
ret = snprintf(asrc_args, sizeof(asrc_args),
"sample_rate=%d:sample_fmt=%s:channels=%d:time_base=%d/%d",
is->audio_filter_src.freq, av_get_sample_fmt_name(is->audio_filter_src.fmt),
is->audio_filter_src.channels,
1, is->audio_filter_src.freq);
if (is->audio_filter_src.channel_layout)
snprintf(asrc_args + ret, sizeof(asrc_args) - ret,
":channel_layout=0x%"PRIx64, is->audio_filter_src.channel_layout);
ret = avfilter_graph_create_filter(&filt_asrc,
avfilter_get_by_name("abuffer"), "ffplay_abuffer",
asrc_args, NULL, is->agraph);
if (ret < 0)
goto end;
ret = avfilter_graph_create_filter(&filt_asink,
avfilter_get_by_name("abuffersink"), "ffplay_abuffersink",
NULL, NULL, is->agraph);
if (ret < 0)
goto end;
if ((ret = av_opt_set_int_list(filt_asink, "sample_fmts", sample_fmts, AV_SAMPLE_FMT_NONE, AV_OPT_SEARCH_CHILDREN)) < 0)
goto end;
if ((ret = av_opt_set_int(filt_asink, "all_channel_counts", 1, AV_OPT_SEARCH_CHILDREN)) < 0)
goto end;
if (force_output_format) {
channel_layouts[0] = is->audio_tgt.channel_layout;
channels [0] = is->audio_tgt.channels;
sample_rates [0] = is->audio_tgt.freq;
if ((ret = av_opt_set_int(filt_asink, "all_channel_counts", 0, AV_OPT_SEARCH_CHILDREN)) < 0)
goto end;
if ((ret = av_opt_set_int_list(filt_asink, "channel_layouts", channel_layouts, -1, AV_OPT_SEARCH_CHILDREN)) < 0)
goto end;
if ((ret = av_opt_set_int_list(filt_asink, "channel_counts" , channels , -1, AV_OPT_SEARCH_CHILDREN)) < 0)
goto end;
if ((ret = av_opt_set_int_list(filt_asink, "sample_rates" , sample_rates , -1, AV_OPT_SEARCH_CHILDREN)) < 0)
goto end;
}
if ((ret = configure_filtergraph(is->agraph, afilters, filt_asrc, filt_asink)) < 0)
goto end;
is->in_audio_filter = filt_asrc;
is->out_audio_filter = filt_asink;
end:
if (ret < 0)
avfilter_graph_free(&is->agraph);
return ret;
}
#endif /* CONFIG_AVFILTER */
static int audio_thread(void *arg)
{
VideoState *is = arg;
AVFrame *frame = av_frame_alloc();
Frame *af;
#if CONFIG_AVFILTER
int last_serial = -1;
int64_t dec_channel_layout;
int reconfigure;
#endif
int got_frame = 0;
AVRational tb;
int ret = 0;
if (!frame)
return AVERROR(ENOMEM);
do {
if ((got_frame = decoder_decode_frame(&is->auddec, frame, NULL)) < 0)
goto the_end;
if (got_frame) {
tb = (AVRational){1, frame->sample_rate};
#if CONFIG_AVFILTER
dec_channel_layout = get_valid_channel_layout(frame->channel_layout, frame->channels);
reconfigure =
cmp_audio_fmts(is->audio_filter_src.fmt, is->audio_filter_src.channels,
frame->format, frame->channels) ||
is->audio_filter_src.channel_layout != dec_channel_layout ||
is->audio_filter_src.freq != frame->sample_rate ||
is->auddec.pkt_serial != last_serial;
if (reconfigure) {
char buf1[1024], buf2[1024];
av_get_channel_layout_string(buf1, sizeof(buf1), -1, is->audio_filter_src.channel_layout);
av_get_channel_layout_string(buf2, sizeof(buf2), -1, dec_channel_layout);
av_log(NULL, AV_LOG_DEBUG,
"Audio frame changed from rate:%d ch:%d fmt:%s layout:%s serial:%d to rate:%d ch:%d fmt:%s layout:%s serial:%d\n",
is->audio_filter_src.freq, is->audio_filter_src.channels, av_get_sample_fmt_name(is->audio_filter_src.fmt), buf1, last_serial,
frame->sample_rate, frame->channels, av_get_sample_fmt_name(frame->format), buf2, is->auddec.pkt_serial);
is->audio_filter_src.fmt = frame->format;
is->audio_filter_src.channels = frame->channels;
is->audio_filter_src.channel_layout = dec_channel_layout;
is->audio_filter_src.freq = frame->sample_rate;
last_serial = is->auddec.pkt_serial;
if ((ret = configure_audio_filters(is, afilters, 1)) < 0)
goto the_end;
}
if ((ret = av_buffersrc_add_frame(is->in_audio_filter, frame)) < 0)
goto the_end;
while ((ret = av_buffersink_get_frame_flags(is->out_audio_filter, frame, 0)) >= 0) {
tb = av_buffersink_get_time_base(is->out_audio_filter);
#endif
if (!(af = frame_queue_peek_writable(&is->sampq)))
goto the_end;
af->pts = (frame->pts == AV_NOPTS_VALUE) ? NAN : frame->pts * av_q2d(tb);
af->pos = frame->pkt_pos;
af->serial = is->auddec.pkt_serial;
af->duration = av_q2d((AVRational){frame->nb_samples, frame->sample_rate});
av_frame_move_ref(af->frame, frame);
frame_queue_push(&is->sampq);
#if CONFIG_AVFILTER
if (is->audioq.serial != is->auddec.pkt_serial)
break;
}
if (ret == AVERROR_EOF)
is->auddec.finished = is->auddec.pkt_serial;
#endif
}
} while (ret >= 0 || ret == AVERROR(EAGAIN) || ret == AVERROR_EOF);
the_end:
#if CONFIG_AVFILTER
avfilter_graph_free(&is->agraph);
#endif
av_frame_free(&frame);
return ret;
}
/**
* 创建解码线程, audio/video有独立的线程
*/
static int decoder_start(Decoder *d, int (*fn)(void *), const char *thread_name, void* arg)
{
packet_queue_start(d->queue);
d->decoder_tid = SDL_CreateThread(fn, thread_name, arg);
if (!d->decoder_tid) {
av_log(NULL, AV_LOG_ERROR, "SDL_CreateThread(): %s\n", SDL_GetError());
return AVERROR(ENOMEM);
}
return 0;
}
static int video_thread(void *arg)
{
VideoState *is = arg;
AVFrame *frame = av_frame_alloc();
double pts;
double duration;
int ret;
AVRational tb = is->video_st->time_base; // 获取stream timebase
// 获取帧率，以便计算每帧picture 的duration
AVRational frame_rate = av_guess_frame_rate(is->ic, is->video_st, NULL);
#if CONFIG_AVFILTER
AVFilterGraph *graph = NULL;
AVFilterContext *filt_out = NULL, *filt_in = NULL;
int last_w = 0;
int last_h = 0;
enum AVPixelFormat last_format = -2;
int last_serial = -1;
int last_vfilter_idx = 0;
#endif
if (!frame)
return AVERROR(ENOMEM);
for (;;) { // 循环取出视频解码的帧数据
ret = get_video_frame(is, frame);
if (ret < 0)
goto the_end;
if (!ret)
continue;
#if CONFIG_AVFILTER
if ( last_w != frame->width
|| last_h != frame->height
|| last_format != frame->format
|| last_serial != is->viddec.pkt_serial
|| last_vfilter_idx != is->vfilter_idx) {
av_log(NULL, AV_LOG_DEBUG,
"Video frame changed from size:%dx%d format:%s serial:%d to size:%dx%d format:%s serial:%d\n",
last_w, last_h,
(const char *)av_x_if_null(av_get_pix_fmt_name(last_format), "none"), last_serial,
frame->width, frame->height,
(const char *)av_x_if_null(av_get_pix_fmt_name(frame->format), "none"), is->viddec.pkt_serial);
avfilter_graph_free(&graph);
graph = avfilter_graph_alloc();
if (!graph) {
ret = AVERROR(ENOMEM);
goto the_end;
}
graph->nb_threads = filter_nbthreads;
if ((ret = configure_video_filters(graph, is, vfilters_list ? vfilters_list[is->vfilter_idx] : NULL, frame)) < 0) {
SDL_Event event;
event.type = FF_QUIT_EVENT;
event.user.data1 = is;
SDL_PushEvent(&event);
goto the_end;
}
filt_in = is->in_video_filter;
filt_out = is->out_video_filter;
last_w = frame->width;
last_h = frame->height;
last_format = frame->format;
last_serial = is->viddec.pkt_serial;
last_vfilter_idx = is->vfilter_idx;
frame_rate = av_buffersink_get_frame_rate(filt_out);
}
ret = av_buffersrc_add_frame(filt_in, frame);
if (ret < 0)
goto the_end;
while (ret >= 0) {
is->frame_last_returned_time = av_gettime_relative() / 1000000.0;
ret = av_buffersink_get_frame_flags(filt_out, frame, 0);
if (ret < 0) {
if (ret == AVERROR_EOF)
is->viddec.finished = is->viddec.pkt_serial;
ret = 0;
break;
}
is->frame_last_filter_delay = av_gettime_relative() / 1000000.0 - is->frame_last_returned_time;
if (fabs(is->frame_last_filter_delay) > AV_NOSYNC_THRESHOLD / 10.0)
is->frame_last_filter_delay = 0;
tb = av_buffersink_get_time_base(filt_out);
#endif
// 1/帧率 = duration 单位秒, 没有帧率时则设置为0
duration = (frame_rate.num && frame_rate.den ? av_q2d((AVRational){frame_rate.den, frame_rate.num}) : 0);
pts = (frame->pts == AV_NOPTS_VALUE) ? NAN : frame->pts * av_q2d(tb);
// 插入视频帧队列
ret = queue_picture(is, frame, pts, duration, frame->pkt_pos, is->viddec.pkt_serial);
av_frame_unref(frame);
#if CONFIG_AVFILTER
if (is->videoq.serial != is->viddec.pkt_serial)
break;
}
#endif
if (ret < 0)
goto the_end;
}
the_end:
#if CONFIG_AVFILTER
avfilter_graph_free(&graph);
#endif
av_frame_free(&frame);
return 0;
}
static int subtitle_thread(void *arg)
{
VideoState *is = arg;
Frame *sp;
int got_subtitle;
double pts;
for (;;) {
if (!(sp = frame_queue_peek_writable(&is->subpq)))
return 0;
if ((got_subtitle = decoder_decode_frame(&is->subdec, NULL, &sp->sub)) < 0)
break;
pts = 0;
if (got_subtitle && sp->sub.format == 0) {
if (sp->sub.pts != AV_NOPTS_VALUE)
pts = sp->sub.pts / (double)AV_TIME_BASE;
sp->pts = pts;
sp->serial = is->subdec.pkt_serial;
sp->width = is->subdec.avctx->width;
sp->height = is->subdec.avctx->height;
sp->uploaded = 0;
/* now we can update the picture count */
frame_queue_push(&is->subpq);
} else if (got_subtitle) {
avsubtitle_free(&sp->sub);
}
}
return 0;
}
/* copy samples for viewing in editor window */
static void update_sample_display(VideoState *is, short *samples, int samples_size)
{
int size, len;
size = samples_size / sizeof(short);
while (size > 0) {
len = SAMPLE_ARRAY_SIZE - is->sample_array_index;
if (len > size)
len = size;
memcpy(is->sample_array + is->sample_array_index, samples, len * sizeof(short));
samples += len;
is->sample_array_index += len;
if (is->sample_array_index >= SAMPLE_ARRAY_SIZE)
is->sample_array_index = 0;
size -= len;
}
}
/* return the wanted number of samples to get better sync if sync_type is video
* or external master clock */
/**
* @brief 如果sync_type是视频或外部主时钟，则返回所需样本数以获得更好的同步
* @param is
* @param nb_samples 正常播放的采样数量
* @return
*/
static int synchronize_audio(VideoState *is, int nb_samples)
{
int wanted_nb_samples = nb_samples;
/* if not master, then we try to remove or add samples to correct the clock */
if (get_master_sync_type(is) != AV_SYNC_AUDIO_MASTER) {
double diff, avg_diff;
int min_nb_samples, max_nb_samples;
diff = get_clock(&is->audclk) - get_master_clock(is);
if (!isnan(diff) && fabs(diff) < AV_NOSYNC_THRESHOLD) {
is->audio_diff_cum = diff + is->audio_diff_avg_coef * is->audio_diff_cum;
if (is->audio_diff_avg_count < AUDIO_DIFF_AVG_NB) {
/* not enough measures to have a correct estimate */
is->audio_diff_avg_count++;
} else {
/* estimate the A-V difference */
avg_diff = is->audio_diff_cum * (1.0 - is->audio_diff_avg_coef);
if (fabs(avg_diff) >= is->audio_diff_threshold) {
wanted_nb_samples = nb_samples + (int)(diff * is->audio_src.freq);
min_nb_samples = ((nb_samples * (100 - SAMPLE_CORRECTION_PERCENT_MAX) / 100));
max_nb_samples = ((nb_samples * (100 + SAMPLE_CORRECTION_PERCENT_MAX) / 100));
wanted_nb_samples = av_clip(wanted_nb_samples, min_nb_samples, max_nb_samples);
}
av_log(NULL, AV_LOG_TRACE, "diff=%f adiff=%f sample_diff=%d apts=%0.3f %f\n",
diff, avg_diff, wanted_nb_samples - nb_samples,
is->audio_clock, is->audio_diff_threshold);
}
} else {
/* too big difference : may be initial PTS errors, so
reset A-V filter */
is->audio_diff_avg_count = 0;
is->audio_diff_cum = 0;
}
}
return wanted_nb_samples;
}
/**
* Decode one audio frame and return its uncompressed size.
*
* The processed audio frame is decoded, converted if required, and
* stored in is->audio_buf, with size in bytes given by the return
* value.
*/
static int audio_decode_frame(VideoState *is)
{
int data_size, resampled_data_size;
int64_t dec_channel_layout;
av_unused double audio_clock0;
int wanted_nb_samples;
Frame *af;
if (is->paused)
return -1;
do {
#if defined(_WIN32)
while (frame_queue_nb_remaining(&is->sampq) == 0) {
if ((av_gettime_relative() - audio_callback_time) > 1000000LL * is->audio_hw_buf_size / is->audio_tgt.bytes_per_sec / 2)
return -1;
av_usleep (1000);
}
#endif
if (!(af = frame_queue_peek_readable(&is->sampq)))
return -1;
frame_queue_next(&is->sampq);
} while (af->serial != is->audioq.serial);
data_size = av_samples_get_buffer_size(NULL, af->frame->channels,
af->frame->nb_samples,
af->frame->format, 1);
dec_channel_layout =
(af->frame->channel_layout && af->frame->channels == av_get_channel_layout_nb_channels(af->frame->channel_layout)) ?
af->frame->channel_layout : av_get_default_channel_layout(af->frame->channels);
wanted_nb_samples = synchronize_audio(is, af->frame->nb_samples);
if (af->frame->format != is->audio_src.fmt ||
dec_channel_layout != is->audio_src.channel_layout ||
af->frame->sample_rate != is->audio_src.freq ||
(wanted_nb_samples != af->frame->nb_samples && !is->swr_ctx)) {
swr_free(&is->swr_ctx);
is->swr_ctx = swr_alloc_set_opts(NULL,
is->audio_tgt.channel_layout, is->audio_tgt.fmt, is->audio_tgt.freq,
dec_channel_layout, af->frame->format, af->frame->sample_rate,
0, NULL);
if (!is->swr_ctx || swr_init(is->swr_ctx) < 0) {
av_log(NULL, AV_LOG_ERROR,
"Cannot create sample rate converter for conversion of %d Hz %s %d channels to %d Hz %s %d channels!\n",
af->frame->sample_rate, av_get_sample_fmt_name(af->frame->format), af->frame->channels,
is->audio_tgt.freq, av_get_sample_fmt_name(is->audio_tgt.fmt), is->audio_tgt.channels);
swr_free(&is->swr_ctx);
return -1;
}
is->audio_src.channel_layout = dec_channel_layout;
is->audio_src.channels = af->frame->channels;
is->audio_src.freq = af->frame->sample_rate;
is->audio_src.fmt = af->frame->format;
}
if (is->swr_ctx) {
const uint8_t **in = (const uint8_t **)af->frame->extended_data;
uint8_t **out = &is->audio_buf1;
int out_count = (int64_t)wanted_nb_samples * is->audio_tgt.freq / af->frame->sample_rate + 256;
int out_size = av_samples_get_buffer_size(NULL, is->audio_tgt.channels, out_count, is->audio_tgt.fmt, 0);
int len2;
if (out_size < 0) {
av_log(NULL, AV_LOG_ERROR, "av_samples_get_buffer_size() failed\n");
return -1;
}
if (wanted_nb_samples != af->frame->nb_samples) {
if (swr_set_compensation(is->swr_ctx, (wanted_nb_samples - af->frame->nb_samples) * is->audio_tgt.freq / af->frame->sample_rate,
wanted_nb_samples * is->audio_tgt.freq / af->frame->sample_rate) < 0) {
av_log(NULL, AV_LOG_ERROR, "swr_set_compensation() failed\n");
return -1;
}
}
av_fast_malloc(&is->audio_buf1, &is->audio_buf1_size, out_size);
if (!is->audio_buf1)
return AVERROR(ENOMEM);
len2 = swr_convert(is->swr_ctx, out, out_count, in, af->frame->nb_samples);
if (len2 < 0) {
av_log(NULL, AV_LOG_ERROR, "swr_convert() failed\n");
return -1;
}
if (len2 == out_count) {
av_log(NULL, AV_LOG_WARNING, "audio buffer is probably too small\n");
if (swr_init(is->swr_ctx) < 0)
swr_free(&is->swr_ctx);
}
is->audio_buf = is->audio_buf1;
resampled_data_size = len2 * is->audio_tgt.channels * av_get_bytes_per_sample(is->audio_tgt.fmt);
} else {
is->audio_buf = af->frame->data[0];
resampled_data_size = data_size;
}
audio_clock0 = is->audio_clock;
/* update the audio clock with the pts */
if (!isnan(af->pts))
is->audio_clock = af->pts + (double) af->frame->nb_samples / af->frame->sample_rate;
else
is->audio_clock = NAN;
is->audio_clock_serial = af->serial;
#ifdef DEBUG
{
static double last_clock;
printf("audio: delay=%0.3f clock=%0.3f clock0=%0.3f\n",
is->audio_clock - last_clock,
is->audio_clock, audio_clock0);
last_clock = is->audio_clock;
}
#endif
return resampled_data_size;
}
/* prepare a new audio buffer */
/**
* @brief sdl_audio_callback
* @param opaque 指向user的数据
* @param stream 拷贝码流的地址
* @param len 需要拷贝的长度
*/
static void sdl_audio_callback(void *opaque, Uint8 *stream, int len)
{
VideoState *is = opaque;
int audio_size, len1;
audio_callback_time = av_gettime_relative();
while (len > 0) {
/* 如果is->audio_buf_index < is->audio_buf_size则说明上次拷贝还剩余一些数据，先拷贝到stream再调用audio_decode_frame */
if (is->audio_buf_index >= is->audio_buf_size) {
audio_size = audio_decode_frame(is);
if (audio_size < 0) {
/* if error, just output silence */
is->audio_buf = NULL;
is->audio_buf_size = SDL_AUDIO_MIN_BUFFER_SIZE / is->audio_tgt.frame_size * is->audio_tgt.frame_size;
} else {
if (is->show_mode != SHOW_MODE_VIDEO)
update_sample_display(is, (int16_t *)is->audio_buf, audio_size);
is->audio_buf_size = audio_size;
}
is->audio_buf_index = 0;
}
len1 = is->audio_buf_size - is->audio_buf_index;
if (len1 > len)
len1 = len;
/* 判断是否为静音，以及当前音量的大小，如果音量不是最大则需要处理pcm数据 */
if (!is->muted && is->audio_buf && is->audio_volume == SDL_MIX_MAXVOLUME)
memcpy(stream, (uint8_t *)is->audio_buf + is->audio_buf_index, len1);
else {
memset(stream, 0, len1);
// 3.调整音量
/* 如果处于mute状态则直接使用stream填0数据, 暂停时is->audio_buf = NULL */
if (!is->muted && is->audio_buf)
SDL_MixAudioFormat(stream, (uint8_t *)is->audio_buf + is->audio_buf_index,
AUDIO_S16SYS, len1, is->audio_volume);
}
len -= len1;
stream += len1;
/* 更新is->audio_buf_index，指向audio_buf中未被拷贝到stream的数据（剩余数据）的起始位置 */
is->audio_buf_index += len1;
}
is->audio_write_buf_size = is->audio_buf_size - is->audio_buf_index;
/* Let's assume the audio driver that is used by SDL has two periods. */
if (!isnan(is->audio_clock)) {
set_clock_at(&is->audclk, is->audio_clock - (double)(2 * is->audio_hw_buf_size + is->audio_write_buf_size) / is->audio_tgt.bytes_per_sec, is->audio_clock_serial, audio_callback_time / 1000000.0);
sync_clock_to_slave(&is->extclk, &is->audclk);
}
}
static int audio_open(void *opaque, int64_t wanted_channel_layout, int wanted_nb_channels, int wanted_sample_rate, struct AudioParams *audio_hw_params)
{
SDL_AudioSpec wanted_spec, spec;
const char *env;
static const int next_nb_channels[] = {0, 0, 1, 6, 2, 6, 4, 6};
static const int next_sample_rates[] = {0, 44100, 48000, 96000, 192000};
int next_sample_rate_idx = FF_ARRAY_ELEMS(next_sample_rates) - 1;
env = SDL_getenv("SDL_AUDIO_CHANNELS");
if (env) {
wanted_nb_channels = atoi(env);
wanted_channel_layout = av_get_default_channel_layout(wanted_nb_channels);
}
if (!wanted_channel_layout || wanted_nb_channels != av_get_channel_layout_nb_channels(wanted_channel_layout)) {
wanted_channel_layout = av_get_default_channel_layout(wanted_nb_channels);
wanted_channel_layout &= ~AV_CH_LAYOUT_STEREO_DOWNMIX;
}
wanted_nb_channels = av_get_channel_layout_nb_channels(wanted_channel_layout);
wanted_spec.channels = wanted_nb_channels;
wanted_spec.freq = wanted_sample_rate;
if (wanted_spec.freq <= 0 || wanted_spec.channels <= 0) {
av_log(NULL, AV_LOG_ERROR, "Invalid sample rate or channel count!\n");
return -1;
}
while (next_sample_rate_idx && next_sample_rates[next_sample_rate_idx] >= wanted_spec.freq)
next_sample_rate_idx--;
wanted_spec.format = AUDIO_S16SYS;
wanted_spec.silence = 0;
/*
* 一次读取多长的数据
* SDL_AUDIO_MAX_CALLBACKS_PER_SEC一秒最多回调次数，避免频繁的回调
* Audio buffer size in samples (power of 2)
*/
wanted_spec.samples = FFMAX(SDL_AUDIO_MIN_BUFFER_SIZE, 2 << av_log2(wanted_spec.freq / SDL_AUDIO_MAX_CALLBACKS_PER_SEC));
wanted_spec.callback = sdl_audio_callback;
wanted_spec.userdata = opaque;
while (!(audio_dev = SDL_OpenAudioDevice(NULL, 0, &wanted_spec, &spec, SDL_AUDIO_ALLOW_FREQUENCY_CHANGE | SDL_AUDIO_ALLOW_CHANNELS_CHANGE))) {
av_log(NULL, AV_LOG_WARNING, "SDL_OpenAudio (%d channels, %d Hz): %s\n",
wanted_spec.channels, wanted_spec.freq, SDL_GetError());
wanted_spec.channels = next_nb_channels[FFMIN(7, wanted_spec.channels)];
if (!wanted_spec.channels) {
wanted_spec.freq = next_sample_rates[next_sample_rate_idx--];
wanted_spec.channels = wanted_nb_channels;
if (!wanted_spec.freq) {
av_log(NULL, AV_LOG_ERROR,
"No more combinations to try, audio open failed\n");
return -1;
}
}
wanted_channel_layout = av_get_default_channel_layout(wanted_spec.channels);
}
if (spec.format != AUDIO_S16SYS) {
av_log(NULL, AV_LOG_ERROR,
"SDL advised audio format %d is not supported!\n", spec.format);
return -1;
}
if (spec.channels != wanted_spec.channels) {
wanted_channel_layout = av_get_default_channel_layout(spec.channels);
if (!wanted_channel_layout) {
av_log(NULL, AV_LOG_ERROR,
"SDL advised channel count %d is not supported!\n", spec.channels);
return -1;
}
}
// audio_hw_params保存的参数，就是在做重采样的时候要转成的格式。
audio_hw_params->fmt = AV_SAMPLE_FMT_S16;
audio_hw_params->freq = spec.freq;
audio_hw_params->channel_layout = wanted_channel_layout;
audio_hw_params->channels = spec.channels;
/* audio_hw_params->frame_size这里只是计算一个采样点占用的字节数 */
audio_hw_params->frame_size = av_samples_get_buffer_size(NULL, audio_hw_params->channels, 1, audio_hw_params->fmt, 1);
audio_hw_params->bytes_per_sec = av_samples_get_buffer_size(NULL, audio_hw_params->channels, audio_hw_params->freq, audio_hw_params->fmt, 1);
if (audio_hw_params->bytes_per_sec <= 0 || audio_hw_params->frame_size <= 0) {
av_log(NULL, AV_LOG_ERROR, "av_samples_get_buffer_size failed\n");
return -1;
}
// 比如2帧数据，一帧就是1024个采样点， 1024*2*2 * 2 = 8096字节
return spec.size; /* 硬件内部缓存的数据字节, samples * channels *byte_per_sample */
}
/* open a given stream. Return 0 if OK */
/**
* @brief stream_component_open
* @param is
* @param stream_index 流索引
* @return Return 0 if OK
*/
static int stream_component_open(VideoState *is, int stream_index)
{
AVFormatContext *ic = is->ic;
AVCodecContext *avctx;
AVCodec *codec;
const char *forced_codec_name = NULL;
AVDictionary *opts = NULL;
AVDictionaryEntry *t = NULL;
int sample_rate, nb_channels;
int64_t channel_layout;
int ret = 0;
int stream_lowres = lowres;
if (stream_index < 0 || stream_index >= ic->nb_streams)
return -1;
/* 为解码器分配一个编解码器上下文结构体 */
avctx = avcodec_alloc_context3(NULL);
if (!avctx)
return AVERROR(ENOMEM);
/* 将码流中的编解码器信息拷贝到新分配的编解码器上下文结构体 */
ret = avcodec_parameters_to_context(avctx, ic->streams[stream_index]->codecpar);
if (ret < 0)
goto fail;
// 设置pkt_timebase
avctx->pkt_timebase = ic->streams[stream_index]->time_base;
/* 根据codec_id查找解码器 */
codec = avcodec_find_decoder(avctx->codec_id);
switch(avctx->codec_type){
case AVMEDIA_TYPE_AUDIO : is->last_audio_stream = stream_index;
forced_codec_name = audio_codec_name; break;
case AVMEDIA_TYPE_SUBTITLE: is->last_subtitle_stream = stream_index;
forced_codec_name = subtitle_codec_name; break;
case AVMEDIA_TYPE_VIDEO : is->last_video_stream = stream_index;
forced_codec_name = video_codec_name; break;
}
if (forced_codec_name)
codec = avcodec_find_decoder_by_name(forced_codec_name);
if (!codec) {
if (forced_codec_name) av_log(NULL, AV_LOG_WARNING,
"No codec could be found with name '%s'\n", forced_codec_name);
else av_log(NULL, AV_LOG_WARNING,
"No decoder could be found for codec %s\n", avcodec_get_name(avctx->codec_id));
ret = AVERROR(EINVAL);
goto fail;
}
avctx->codec_id = codec->id;
if (stream_lowres > codec->max_lowres) {
av_log(avctx, AV_LOG_WARNING, "The maximum value for lowres supported by the decoder is %d\n",
codec->max_lowres);
stream_lowres = codec->max_lowres;
}
avctx->lowres = stream_lowres;
if (fast)
avctx->flags2 |= AV_CODEC_FLAG2_FAST;
opts = filter_codec_opts(codec_opts, avctx->codec_id, ic, ic->streams[stream_index], codec);
if (!av_dict_get(opts, "threads", NULL, 0))
av_dict_set(&opts, "threads", "auto", 0);
if (stream_lowres)
av_dict_set_int(&opts, "lowres", stream_lowres, 0);
if (avctx->codec_type == AVMEDIA_TYPE_VIDEO || avctx->codec_type == AVMEDIA_TYPE_AUDIO)
av_dict_set(&opts, "refcounted_frames", "1", 0);
if ((ret = avcodec_open2(avctx, codec, &opts)) < 0) {
goto fail;
}
if ((t = av_dict_get(opts, "", NULL, AV_DICT_IGNORE_SUFFIX))) {
av_log(NULL, AV_LOG_ERROR, "Option %s not found.\n", t->key);
ret = AVERROR_OPTION_NOT_FOUND;
goto fail;
}
is->eof = 0;
ic->streams[stream_index]->discard = AVDISCARD_DEFAULT;
switch (avctx->codec_type) {
case AVMEDIA_TYPE_AUDIO:
#if CONFIG_AVFILTER
{
AVFilterContext *sink;
is->audio_filter_src.freq = avctx->sample_rate;
is->audio_filter_src.channels = avctx->channels;
is->audio_filter_src.channel_layout = get_valid_channel_layout(avctx->channel_layout, avctx->channels);
is->audio_filter_src.fmt = avctx->sample_fmt;
if ((ret = configure_audio_filters(is, afilters, 0)) < 0)
goto fail;
sink = is->out_audio_filter;
sample_rate = av_buffersink_get_sample_rate(sink);
nb_channels = av_buffersink_get_channels(sink);
channel_layout = av_buffersink_get_channel_layout(sink);
}
#else
sample_rate = avctx->sample_rate;
nb_channels = avctx->channels;
channel_layout = avctx->channel_layout;
#endif
/* prepare audio output 准备音频输出*/
if ((ret = audio_open(is, channel_layout, nb_channels, sample_rate, &is->audio_tgt)) < 0)
goto fail;
is->audio_hw_buf_size = ret;
is->audio_src = is->audio_tgt;
is->audio_buf_size = 0;
is->audio_buf_index = 0;
/* init averaging filter 初始化averaging滤镜, 非audio master时使用 */
is->audio_diff_avg_coef = exp(log(0.01) / AUDIO_DIFF_AVG_NB);
is->audio_diff_avg_count = 0;
/* 由于我们没有精确的音频数据填充FIFO,故只有在大于该阈值时才进行校正音频同步*/
is->audio_diff_threshold = (double)(is->audio_hw_buf_size) / is->audio_tgt.bytes_per_sec;
is->audio_stream = stream_index;
is->audio_st = ic->streams[stream_index];
decoder_init(&is->auddec, avctx, &is->audioq, is->continue_read_thread);
if ((is->ic->iformat->flags & (AVFMT_NOBINSEARCH | AVFMT_NOGENSEARCH | AVFMT_NO_BYTE_SEEK)) && !is->ic->iformat->read_seek) {
is->auddec.start_pts = is->audio_st->start_time;
is->auddec.start_pts_tb = is->audio_st->time_base;
}
if ((ret = decoder_start(&is->auddec, audio_thread, "audio_decoder", is)) < 0)
goto out;
SDL_PauseAudioDevice(audio_dev, 0);
break;
case AVMEDIA_TYPE_VIDEO:
is->video_stream = stream_index;
is->video_st = ic->streams[stream_index];
decoder_init(&is->viddec, avctx, &is->videoq, is->continue_read_thread);
if ((ret = decoder_start(&is->viddec, video_thread, "video_decoder", is)) < 0)
goto out;
is->queue_attachments_req = 1;
break;
case AVMEDIA_TYPE_SUBTITLE:
is->subtitle_stream = stream_index;
is->subtitle_st = ic->streams[stream_index];
decoder_init(&is->subdec, avctx, &is->subtitleq, is->continue_read_thread);
if ((ret = decoder_start(&is->subdec, subtitle_thread, "subtitle_decoder", is)) < 0)
goto out;
break;
default:
break;
}
goto out;
fail:
avcodec_free_context(&avctx);
out:
av_dict_free(&opts);
return ret;
}
/**
* @brief 这里是设置给ffmpeg内部，当read出现阻塞的时候就会调用该函数
* @param ctx
* @return 若直接退出阻塞则返回1，等待读取则返回0
*/
static int decode_interrupt_cb(void *ctx)
{
static int64_t s_pre_time = 0;
int64_t cur_time = av_gettime_relative() / 1000;
printf("decode_interrupt_cb interval:%lldms\n", cur_time - s_pre_time);
s_pre_time = cur_time;
VideoState *is = ctx;
return is->abort_request;
}
static int stream_has_enough_packets(AVStream *st, int stream_id, PacketQueue *queue) {
return stream_id < 0 ||
queue->abort_request ||
(st->disposition & AV_DISPOSITION_ATTACHED_PIC) ||
queue->nb_packets > MIN_FRAMES && (!queue->duration || av_q2d(st->time_base) * queue->duration > 1.0);
}
static int is_realtime(AVFormatContext *s)
{
if( !strcmp(s->iformat->name, "rtp")
|| !strcmp(s->iformat->name, "rtsp")
|| !strcmp(s->iformat->name, "sdp")
)
return 1;
if(s->pb && ( !strncmp(s->url, "rtp:", 4)
|| !strncmp(s->url, "udp:", 4)
)
)
return 1;
return 0;
}
/* this thread gets the stream from the disk or the network */
/*
* 数据都由这里读取
* 主要功能是做解复用，从码流中分离音视频packet，并插入缓存队列
*/
static int read_thread(void *arg)
{
VideoState *is = arg;
AVFormatContext *ic = NULL;
int err, i, ret;
int st_index[AVMEDIA_TYPE_NB];
AVPacket pkt1, *pkt = &pkt1;
int64_t stream_start_time;
int pkt_in_play_range = 0;
AVDictionaryEntry *t;
SDL_mutex *wait_mutex = SDL_CreateMutex();
int scan_all_pmts_set = 0;
int64_t pkt_ts;
if (!wait_mutex) {
av_log(NULL, AV_LOG_FATAL, "SDL_CreateMutex(): %s\n", SDL_GetError());
ret = AVERROR(ENOMEM);
goto fail;
}
memset(st_index, -1, sizeof(st_index));
// 初始化为-1,如果一直为-1说明没相应steam
is->last_video_stream = is->video_stream = -1;
is->last_audio_stream = is->audio_stream = -1;
is->last_subtitle_stream = is->subtitle_stream = -1;
is->eof = 0; // =1是表明数据读取完毕
/* 1. 创建上下文结构体，这个结构体是最上层的结构体，表示输入上下文 */
ic = avformat_alloc_context();
if (!ic) {
av_log(NULL, AV_LOG_FATAL, "Could not allocate context.\n");
ret = AVERROR(ENOMEM);
goto fail;
}
/* 2.设置中断回调函数，如果出错或者退出，就根据目前程序设置的状态选择继续check或者直接退出 */
/* 当open出现阻塞的时候时候，会调用interrupt_callback.callback
* 回调函数中返回1则代表ffmpeg结束阻塞可以将操纵权交给用户线程并返回错误码
* 回调函数中返回0则代表ffmpeg继续阻塞直到ffmpeg正常工作为止，所以要退出死等则需要返回1
*/
ic->interrupt_callback.callback = decode_interrupt_cb;
ic->interrupt_callback.opaque = is;
//特定选项处理
if (!av_dict_get(format_opts, "scan_all_pmts", NULL, AV_DICT_MATCH_CASE)) {
av_dict_set(&format_opts, "scan_all_pmts", "1", AV_DICT_DONT_OVERWRITE);
scan_all_pmts_set = 1;
}
/* 3.打开文件，主要是探测协议类型，如果是网络文件则创建网络链接等 */
err = avformat_open_input(&ic, is->filename, is->iformat, &format_opts);
if (err < 0) {
print_error(is->filename, err);
ret = -1;
goto fail;
}
if (scan_all_pmts_set)
av_dict_set(&format_opts, "scan_all_pmts", NULL, AV_DICT_MATCH_CASE);
if ((t = av_dict_get(format_opts, "", NULL, AV_DICT_IGNORE_SUFFIX))) {
av_log(NULL, AV_LOG_ERROR, "Option %s not found.\n", t->key);
ret = AVERROR_OPTION_NOT_FOUND;
goto fail;
}
is->ic = ic;
if (genpts)
ic->flags |= AVFMT_FLAG_GENPTS;
av_format_inject_global_side_data(ic);
if (find_stream_info) {
AVDictionary **opts = setup_find_stream_info_opts(ic, codec_opts);
int orig_nb_streams = ic->nb_streams;
/*
* 4.探测媒体类型，可得到当前文件的封装格式，音视频编码参数等信息
* 调用该函数后得多的参数信息会比只调用avformat_open_input更为详细，其本质上是去做了decdoe packet获取信息的工作
* codecpar, filled by libavformat on stream creation or in avformat_find_stream_info()
*/
err = avformat_find_stream_info(ic, opts);
for (i = 0; i < orig_nb_streams; i++)
av_dict_free(&opts[i]);
av_freep(&opts);
if (err < 0) {
av_log(NULL, AV_LOG_WARNING,
"%s: could not find codec parameters\n", is->filename);
ret = -1;
goto fail;
}
}
if (ic->pb)
ic->pb->eof_reached = 0; // FIXME hack, ffplay maybe should not use avio_feof() to test for the end
if (seek_by_bytes < 0)
seek_by_bytes = !!(ic->iformat->flags & AVFMT_TS_DISCONT) && strcmp("ogg", ic->iformat->name);
is->max_frame_duration = (ic->iformat->flags & AVFMT_TS_DISCONT) ? 10.0 : 3600.0;
if (!window_title && (t = av_dict_get(ic->metadata, "title", NULL, 0)))
window_title = av_asprintf("%s - %s", t->value, input_filename);
/* if seeking requested, we execute it */
/* 如果需要指定起始位置 */
if (start_time != AV_NOPTS_VALUE) {
int64_t timestamp;
timestamp = start_time;
/* add the stream start time */
if (ic->start_time != AV_NOPTS_VALUE)
timestamp += ic->start_time;
ret = avformat_seek_file(ic, -1, INT64_MIN, timestamp, INT64_MAX, 0);
if (ret < 0) {
av_log(NULL, AV_LOG_WARNING, "%s: could not seek to position %0.3f\n",
is->filename, (double)timestamp / AV_TIME_BASE);
}
}
/* 是否为实时流媒体 */
is->realtime = is_realtime(ic);
if (show_status)
av_dump_format(ic, 0, is->filename, 0);
//根据用户指定来查找流
for (i = 0; i < ic->nb_streams; i++) {
AVStream *st = ic->streams[i];
enum AVMediaType type = st->codecpar->codec_type;
st->discard = AVDISCARD_ALL;
if (type >= 0 && wanted_stream_spec[type] && st_index[type] == -1)
if (avformat_match_stream_specifier(ic, st, wanted_stream_spec[type]) > 0)
st_index[type] = i;
}
for (i = 0; i < AVMEDIA_TYPE_NB; i++) {
if (wanted_stream_spec[i] && st_index[i] == -1) {
av_log(NULL, AV_LOG_ERROR, "Stream specifier %s does not match any %s stream\n", wanted_stream_spec[i], av_get_media_type_string(i));
st_index[i] = INT_MAX;
}
}
//利用av_find_best_stream选择流，
if (!video_disable)
st_index[AVMEDIA_TYPE_VIDEO] =
av_find_best_stream(ic, AVMEDIA_TYPE_VIDEO,
st_index[AVMEDIA_TYPE_VIDEO], -1, NULL, 0);
if (!audio_disable)
st_index[AVMEDIA_TYPE_AUDIO] =
av_find_best_stream(ic, AVMEDIA_TYPE_AUDIO,
st_index[AVMEDIA_TYPE_AUDIO],
st_index[AVMEDIA_TYPE_VIDEO],
NULL, 0);
if (!video_disable && !subtitle_disable)
st_index[AVMEDIA_TYPE_SUBTITLE] =
av_find_best_stream(ic, AVMEDIA_TYPE_SUBTITLE,
st_index[AVMEDIA_TYPE_SUBTITLE],
(st_index[AVMEDIA_TYPE_AUDIO] >= 0 ?
st_index[AVMEDIA_TYPE_AUDIO] :
st_index[AVMEDIA_TYPE_VIDEO]),
NULL, 0);
is->show_mode = show_mode;
if (st_index[AVMEDIA_TYPE_VIDEO] >= 0) {
AVStream *st = ic->streams[st_index[AVMEDIA_TYPE_VIDEO]];
AVCodecParameters *codecpar = st->codecpar;
AVRational sar = av_guess_sample_aspect_ratio(ic, st, NULL);
if (codecpar->width)
set_default_window_size(codecpar->width, codecpar->height, sar);
}
/* open the streams */
/* 5.打开视频、音频解码器。在此会打开相应解码器，并创建相应的解码线程。 */
if (st_index[AVMEDIA_TYPE_AUDIO] >= 0) {// 如果有音频流则打开音频流
stream_component_open(is, st_index[AVMEDIA_TYPE_AUDIO]);
}
ret = -1;
if (st_index[AVMEDIA_TYPE_VIDEO] >= 0) { // 如果有视频流则打开视频流
ret = stream_component_open(is, st_index[AVMEDIA_TYPE_VIDEO]);
}
if (is->show_mode == SHOW_MODE_NONE) {
//选择怎么显示，如果视频打开成功，就显示视频画面，否则，显示音频对应的频谱图
is->show_mode = ret >= 0 ? SHOW_MODE_VIDEO : SHOW_MODE_RDFT;
}
if (st_index[AVMEDIA_TYPE_SUBTITLE] >= 0) { // 如果有字幕流则打开字幕流
stream_component_open(is, st_index[AVMEDIA_TYPE_SUBTITLE]);
}
if (is->video_stream < 0 && is->audio_stream < 0) {
av_log(NULL, AV_LOG_FATAL, "Failed to open file '%s' or configure filtergraph\n",
is->filename);
ret = -1;
goto fail;
}
if (infinite_buffer < 0 && is->realtime)
infinite_buffer = 1;
for (;;) {
if (is->abort_request)
break;
if (is->paused != is->last_paused) {
is->last_paused = is->paused;
if (is->paused)
is->read_pause_return = av_read_pause(ic);
else
av_read_play(ic);
}
#if CONFIG_RTSP_DEMUXER || CONFIG_MMSH_PROTOCOL
if (is->paused &&
(!strcmp(ic->iformat->name, "rtsp") ||
(ic->pb && !strncmp(input_filename, "mmsh:", 5)))) {
/* wait 10 ms to avoid trying to get another packet */
/* XXX: horrible */
// 等待10ms，避免立马尝试下一个Packet
SDL_Delay(10);
continue;
}
#endif
if (is->seek_req) { // 是否有seek请求
int64_t seek_target = is->seek_pos;
int64_t seek_min = is->seek_rel > 0 ? seek_target - is->seek_rel + 2: INT64_MIN;
int64_t seek_max = is->seek_rel < 0 ? seek_target - is->seek_rel - 2: INT64_MAX;
// FIXME the +-2 is due to rounding being not done in the correct direction in generation
// of the seek_pos/seek_rel variables
// 修复由于四舍五入，没有再seek_pos/seek_rel变量的正确方向上进行
ret = avformat_seek_file(is->ic, -1, seek_min, seek_target, seek_max, is->seek_flags);
if (ret < 0) {
av_log(NULL, AV_LOG_ERROR,
"%s: error while seeking\n", is->ic->url);
} else {
/* seek的时候，要把原先的数据情况，并重启解码器，put flush_pkt的目的是告知解码线程需要
* reset decoder
*/
if (is->audio_stream >= 0) { // 如果有音频流
packet_queue_flush(&is->audioq); // 清空packet队列数据
// 放入flush pkt, 用来开起新的一个播放序列, 解码器读取到flush_pkt也清空解码器
packet_queue_put(&is->audioq, &flush_pkt);
}
if (is->subtitle_stream >= 0) { // 如果有字幕流
packet_queue_flush(&is->subtitleq); // 和上同理
packet_queue_put(&is->subtitleq, &flush_pkt);
}
if (is->video_stream >= 0) { // 如果有视频流
packet_queue_flush(&is->videoq); // 和上同理
packet_queue_put(&is->videoq, &flush_pkt);
}
if (is->seek_flags & AVSEEK_FLAG_BYTE) {
set_clock(&is->extclk, NAN, 0);
} else {
set_clock(&is->extclk, seek_target / (double)AV_TIME_BASE, 0);
}
}
is->seek_req = 0;
is->queue_attachments_req = 1;
is->eof = 0;
if (is->paused)
step_to_next_frame(is);
}
if (is->queue_attachments_req) {
// attached_pic 附带的图片。比如说一些MP3，AAC音频文件附带的专辑封面，所以需要注意的是音频文件不一定只存在音频流本身
if (is->video_st && is->video_st->disposition & AV_DISPOSITION_ATTACHED_PIC) {
AVPacket copy = { 0 };
if ((ret = av_packet_ref(©, &is->video_st->attached_pic)) < 0)
goto fail;
packet_queue_put(&is->videoq, ©);
packet_queue_put_nullpacket(&is->videoq, is->video_stream);
}
is->queue_attachments_req = 0;
}
/* if the queue are full, no need to read more */
/* 缓存队列有足够的包，不需要继续读取数据 */
if (infinite_buffer<1 &&
(is->audioq.size + is->videoq.size + is->subtitleq.size > MAX_QUEUE_SIZE
|| (stream_has_enough_packets(is->audio_st, is->audio_stream, &is->audioq) &&
stream_has_enough_packets(is->video_st, is->video_stream, &is->videoq) &&
stream_has_enough_packets(is->subtitle_st, is->subtitle_stream, &is->subtitleq)))) {
/* wait 10 ms */
SDL_LockMutex(wait_mutex);
// 如果没有唤醒则超时10ms退出，比如在seek操作时这里会被唤醒
SDL_CondWaitTimeout(is->continue_read_thread, wait_mutex, 10);
SDL_UnlockMutex(wait_mutex);
continue;
}
if (!is->paused && // 非暂停
// 这里的执行是因为码流读取完毕后 插入空包所致
(!is->audio_st || (is->auddec.finished == is->audioq.serial && frame_queue_nb_remaining(&is->sampq) == 0)) &&
(!is->video_st || (is->viddec.finished == is->videoq.serial && frame_queue_nb_remaining(&is->pictq) == 0))) {
if (loop != 1 && (!loop || --loop)) {
stream_seek(is, start_time != AV_NOPTS_VALUE ? start_time : 0, 0, 0);
} else if (autoexit) {
ret = AVERROR_EOF;
goto fail;
}
}
/* 6.读取媒体数据，得到的是音视频分离后、解码前的数据 */
ret = av_read_frame(ic, pkt); // 调用不会释放pkt的数据，都是自己重新申请
if (ret < 0) {
if ((ret == AVERROR_EOF || avio_feof(ic->pb)) && !is->eof) {
// 插入空包说明码流数据读取完毕了，之前讲解码的时候说过刷空包是为了从解码器把所有帧都读出来
if (is->video_stream >= 0)
packet_queue_put_nullpacket(&is->videoq, is->video_stream);
if (is->audio_stream >= 0)
packet_queue_put_nullpacket(&is->audioq, is->audio_stream);
if (is->subtitle_stream >= 0)
packet_queue_put_nullpacket(&is->subtitleq, is->subtitle_stream);
is->eof = 1; // 文件读取完毕
}
if (ic->pb && ic->pb->error)
break;
SDL_LockMutex(wait_mutex);
SDL_CondWaitTimeout(is->continue_read_thread, wait_mutex, 10);
SDL_UnlockMutex(wait_mutex);
continue; // 继续循环
} else {
is->eof = 0;
}
/* check if packet is in play range specified by user, then queue, otherwise discard */
stream_start_time = ic->streams[pkt->stream_index]->start_time;
pkt_ts = pkt->pts == AV_NOPTS_VALUE ? pkt->dts : pkt->pts;
// 这里的duration是在命令行时用来指定播放长度
pkt_in_play_range = duration == AV_NOPTS_VALUE ||
(pkt_ts - (stream_start_time != AV_NOPTS_VALUE ? stream_start_time : 0)) *
av_q2d(ic->streams[pkt->stream_index]->time_base) -
(double)(start_time != AV_NOPTS_VALUE ? start_time : 0) / 1000000
<= ((double)duration / 1000000);
/*
* 7.将音视频数据分别送入相应的queue中
* 重复6、7步，即可不断获取待播放的数据。
*/
if (pkt->stream_index == is->audio_stream && pkt_in_play_range) {
packet_queue_put(&is->audioq, pkt);
} else if (pkt->stream_index == is->video_stream && pkt_in_play_range
&& !(is->video_st->disposition & AV_DISPOSITION_ATTACHED_PIC)) {
//printf("pkt pts:%ld, dts:%ld\n", pkt->pts, pkt->dts);
packet_queue_put(&is->videoq, pkt);
} else if (pkt->stream_index == is->subtitle_stream && pkt_in_play_range) {
packet_queue_put(&is->subtitleq, pkt);
} else {
av_packet_unref(pkt);// C++ share_ptr
}
}
ret = 0;
fail:
if (ic && !is->ic)
avformat_close_input(&ic);
if (ret != 0) {
SDL_Event event;
event.type = FF_QUIT_EVENT;
event.user.data1 = is;
SDL_PushEvent(&event);
}
SDL_DestroyMutex(wait_mutex);
return 0;
}
static VideoState *stream_open(const char *filename, AVInputFormat *iformat)
{
VideoState *is;
is = av_mallocz(sizeof(VideoState)); /* 分配结构体并初始化 */
if (!is)
return NULL;
is->filename = av_strdup(filename);
if (!is->filename)
goto fail;
is->iformat = iformat;
is->ytop = 0;
is->xleft = 0;
/* 初始化帧队列 */
if (frame_queue_init(&is->pictq, &is->videoq, VIDEO_PICTURE_QUEUE_SIZE, 1) < 0)
goto fail;
if (frame_queue_init(&is->subpq, &is->subtitleq, SUBPICTURE_QUEUE_SIZE, 0) < 0)
goto fail;
if (frame_queue_init(&is->sampq, &is->audioq, SAMPLE_QUEUE_SIZE, 1) < 0)
goto fail;
if (packet_queue_init(&is->videoq) < 0 ||
packet_queue_init(&is->audioq) < 0 ||
packet_queue_init(&is->subtitleq) < 0)
goto fail;
/* 创建 */
if (!(is->continue_read_thread = SDL_CreateCond())) {
av_log(NULL, AV_LOG_FATAL, "SDL_CreateCond(): %s\n", SDL_GetError());
goto fail;
}
/*
* 初始化时钟
* 时钟序列->queue_serial，实际上指向的是is->videoq.serial
*/
init_clock(&is->vidclk, &is->videoq.serial);
init_clock(&is->audclk, &is->audioq.serial);
init_clock(&is->extclk, &is->extclk.serial);
is->audio_clock_serial = -1;
/* 初始化音量 */
if (startup_volume < 0)
av_log(NULL, AV_LOG_WARNING, "-volume=%d < 0, setting to 0\n", startup_volume);
if (startup_volume > 100)
av_log(NULL, AV_LOG_WARNING, "-volume=%d > 100, setting to 100\n", startup_volume);
startup_volume = av_clip(startup_volume, 0, 100);
startup_volume = av_clip(SDL_MIX_MAXVOLUME * startup_volume / 100, 0, SDL_MIX_MAXVOLUME);
is->audio_volume = startup_volume;
is->muted = 0;
is->av_sync_type = av_sync_type;
/* 创建读线程 */
is->read_tid = SDL_CreateThread(read_thread, "read_thread", is);
if (!is->read_tid) {
av_log(NULL, AV_LOG_FATAL, "SDL_CreateThread(): %s\n", SDL_GetError());
fail:
stream_close(is);
return NULL;
}
return is;
}
static void stream_cycle_channel(VideoState *is, int codec_type)
{
AVFormatContext *ic = is->ic;
int start_index, stream_index;
int old_index;
AVStream *st;
AVProgram *p = NULL;
int nb_streams = is->ic->nb_streams;
if (codec_type == AVMEDIA_TYPE_VIDEO) {
start_index = is->last_video_stream;
old_index = is->video_stream;
} else if (codec_type == AVMEDIA_TYPE_AUDIO) {
start_index = is->last_audio_stream;
old_index = is->audio_stream;
} else {
start_index = is->last_subtitle_stream;
old_index = is->subtitle_stream;
}
stream_index = start_index;
if (codec_type != AVMEDIA_TYPE_VIDEO && is->video_stream != -1) {
p = av_find_program_from_stream(ic, NULL, is->video_stream);
if (p) {
nb_streams = p->nb_stream_indexes;
for (start_index = 0; start_index < nb_streams; start_index++)
if (p->stream_index[start_index] == stream_index)
break;
if (start_index == nb_streams)
start_index = -1;
stream_index = start_index;
}
}
for (;;) {
if (++stream_index >= nb_streams)
{
if (codec_type == AVMEDIA_TYPE_SUBTITLE)
{
stream_index = -1;
is->last_subtitle_stream = -1;
goto the_end;
}
if (start_index == -1)
return;
stream_index = 0;
}
if (stream_index == start_index)
return;
st = is->ic->streams[p ? p->stream_index[stream_index] : stream_index];
if (st->codecpar->codec_type == codec_type) {
/* check that parameters are OK */
switch (codec_type) {
case AVMEDIA_TYPE_AUDIO:
if (st->codecpar->sample_rate != 0 &&
st->codecpar->channels != 0)
goto the_end;
break;
case AVMEDIA_TYPE_VIDEO:
case AVMEDIA_TYPE_SUBTITLE:
goto the_end;
default:
break;
}
}
}
the_end:
if (p && stream_index != -1)
stream_index = p->stream_index[stream_index];
av_log(NULL, AV_LOG_INFO, "Switch %s stream from #%d to #%d\n",
av_get_media_type_string(codec_type),
old_index,
stream_index);
stream_component_close(is, old_index);
stream_component_open(is, stream_index);
}
static void toggle_full_screen(VideoState *is)
{
is_full_screen = !is_full_screen;
SDL_SetWindowFullscreen(window, is_full_screen ? SDL_WINDOW_FULLSCREEN_DESKTOP : 0);
}
static void toggle_audio_display(VideoState *is)
{
int next = is->show_mode;
do {
next = (next + 1) % SHOW_MODE_NB;
} while (next != is->show_mode && (next == SHOW_MODE_VIDEO && !is->video_st || next != SHOW_MODE_VIDEO && !is->audio_st));
if (is->show_mode != next) {
is->force_refresh = 1;
is->show_mode = next;
}
}
static void refresh_loop_wait_event(VideoState *is, SDL_Event *event) {
double remaining_time = 0.0; /* 休眠等待，remaining_time的计算在video_refresh中 */
/* 调用SDL_PeepEvents前先调用SDL_PumpEvents，将输入设备的事件抽到事件队列中 */
SDL_PumpEvents();
/*
* SDL_PeepEvents check是否事件，比如鼠标移入显示区等
* 从事件队列中拿一个事件，放到event中，如果没有事件，则进入循环中
*/
while (!SDL_PeepEvents(event, 1, SDL_GETEVENT, SDL_FIRSTEVENT, SDL_LASTEVENT)) {
if (!cursor_hidden && av_gettime_relative() - cursor_last_shown > CURSOR_HIDE_DELAY) {
SDL_ShowCursor(0);
cursor_hidden = 1;
}
/*
* remaining_time就是用来进行音视频同步的。
* 在video_refresh函数中，根据当前帧显示时刻(display time)和实际时刻(actual time)计算需要sleep的时间，保证帧按时显示
*/
if (remaining_time > 0.0)
av_usleep((int64_t)(remaining_time * 1000000.0));
remaining_time = REFRESH_RATE;
if (is->show_mode != SHOW_MODE_NONE && (!is->paused || is->force_refresh))
video_refresh(is, &remaining_time);
SDL_PumpEvents();
}
}
static void seek_chapter(VideoState *is, int incr)
{
int64_t pos = get_master_clock(is) * AV_TIME_BASE;
int i;
if (!is->ic->nb_chapters)
return;
/* find the current chapter */
for (i = 0; i < is->ic->nb_chapters; i++) {
AVChapter *ch = is->ic->chapters[i];
if (av_compare_ts(pos, AV_TIME_BASE_Q, ch->start, ch->time_base) < 0) {
i--;
break;
}
}
i += incr;
i = FFMAX(i, 0);
if (i >= is->ic->nb_chapters)
return;
av_log(NULL, AV_LOG_VERBOSE, "Seeking to chapter %d.\n", i);
stream_seek(is, av_rescale_q(is->ic->chapters[i]->start, is->ic->chapters[i]->time_base,
AV_TIME_BASE_Q), 0, 0);
}
/* handle an event sent by the GUI */
static void event_loop(VideoState *cur_stream)
{
SDL_Event event;
double incr, pos, frac;
for (;;) {
double x;
refresh_loop_wait_event(cur_stream, &event);
switch (event.type) {
case SDL_KEYDOWN: /* 键盘事件 */
if (exit_on_keydown || event.key.keysym.sym == SDLK_ESCAPE || event.key.keysym.sym == SDLK_q) {
do_exit(cur_stream);
break;
}
if (!cur_stream->width)
continue;
switch (event.key.keysym.sym) {
case SDLK_f:
toggle_full_screen(cur_stream);
cur_stream->force_refresh = 1;
break;
case SDLK_p:
case SDLK_SPACE:
toggle_pause(cur_stream);
break;
case SDLK_m:
toggle_mute(cur_stream);
break;
case SDLK_KP_MULTIPLY:
case SDLK_0:
update_volume(cur_stream, 1, SDL_VOLUME_STEP);
break;
case SDLK_KP_DIVIDE:
case SDLK_9:
update_volume(cur_stream, -1, SDL_VOLUME_STEP);
break;
case SDLK_s: // S: Step to next frame
step_to_next_frame(cur_stream);
break;
case SDLK_a:
stream_cycle_channel(cur_stream, AVMEDIA_TYPE_AUDIO);
break;
case SDLK_v:
stream_cycle_channel(cur_stream, AVMEDIA_TYPE_VIDEO);
break;
case SDLK_c:
stream_cycle_channel(cur_stream, AVMEDIA_TYPE_VIDEO);
stream_cycle_channel(cur_stream, AVMEDIA_TYPE_AUDIO);
stream_cycle_channel(cur_stream, AVMEDIA_TYPE_SUBTITLE);
break;
case SDLK_t:
stream_cycle_channel(cur_stream, AVMEDIA_TYPE_SUBTITLE);
break;
case SDLK_w:
#if CONFIG_AVFILTER
if (cur_stream->show_mode == SHOW_MODE_VIDEO && cur_stream->vfilter_idx < nb_vfilters - 1) {
if (++cur_stream->vfilter_idx >= nb_vfilters)
cur_stream->vfilter_idx = 0;
} else {
cur_stream->vfilter_idx = 0;
toggle_audio_display(cur_stream);
}
#else
toggle_audio_display(cur_stream);
#endif
break;
case SDLK_PAGEUP:
if (cur_stream->ic->nb_chapters <= 1) {
incr = 600.0;
goto do_seek;
}
seek_chapter(cur_stream, 1);
break;
case SDLK_PAGEDOWN:
if (cur_stream->ic->nb_chapters <= 1) {
incr = -600.0;
goto do_seek;
}
seek_chapter(cur_stream, -1);
break;
case SDLK_LEFT:
incr = seek_interval ? -seek_interval : -10.0;
goto do_seek;
case SDLK_RIGHT:
incr = seek_interval ? seek_interval : 10.0;
goto do_seek;
case SDLK_UP:
incr = 60.0;
goto do_seek;
case SDLK_DOWN:
incr = -60.0;
do_seek:
if (seek_by_bytes) {
pos = -1;
if (pos < 0 && cur_stream->video_stream >= 0)
pos = frame_queue_last_pos(&cur_stream->pictq);
if (pos < 0 && cur_stream->audio_stream >= 0)
pos = frame_queue_last_pos(&cur_stream->sampq);
if (pos < 0)
pos = avio_tell(cur_stream->ic->pb);
if (cur_stream->ic->bit_rate)
incr *= cur_stream->ic->bit_rate / 8.0;
else
incr *= 180000.0;
pos += incr;
stream_seek(cur_stream, pos, incr, 1);
} else {
pos = get_master_clock(cur_stream);
if (isnan(pos))
pos = (double)cur_stream->seek_pos / AV_TIME_BASE;
pos += incr;
if (cur_stream->ic->start_time != AV_NOPTS_VALUE && pos < cur_stream->ic->start_time / (double)AV_TIME_BASE)
pos = cur_stream->ic->start_time / (double)AV_TIME_BASE;
stream_seek(cur_stream, (int64_t)(pos * AV_TIME_BASE), (int64_t)(incr * AV_TIME_BASE), 0);
}
break;
default:
break;
}
break;
case SDL_MOUSEBUTTONDOWN: /* 鼠标按下事件 */
if (exit_on_mousedown) {
do_exit(cur_stream);
break;
}
if (event.button.button == SDL_BUTTON_LEFT) {
static int64_t last_mouse_left_click = 0;
if (av_gettime_relative() - last_mouse_left_click <= 500000) {
toggle_full_screen(cur_stream);
cur_stream->force_refresh = 1;
last_mouse_left_click = 0;
} else {
last_mouse_left_click = av_gettime_relative();
}
}
case SDL_MOUSEMOTION: /* 鼠标移动事件 */
if (cursor_hidden) {
SDL_ShowCursor(1);
cursor_hidden = 0;
}
cursor_last_shown = av_gettime_relative();
if (event.type == SDL_MOUSEBUTTONDOWN) {
if (event.button.button != SDL_BUTTON_RIGHT)
break;
x = event.button.x;
} else {
if (!(event.motion.state & SDL_BUTTON_RMASK))
break;
x = event.motion.x;
}
if (seek_by_bytes || cur_stream->ic->duration <= 0) {
uint64_t size = avio_size(cur_stream->ic->pb);
stream_seek(cur_stream, size*x/cur_stream->width, 0, 1);
} else {
int64_t ts;
int ns, hh, mm, ss;
int tns, thh, tmm, tss;
tns = cur_stream->ic->duration / 1000000LL;
thh = tns / 3600;
tmm = (tns % 3600) / 60;
tss = (tns % 60);
frac = x / cur_stream->width;
ns = frac * tns;
hh = ns / 3600;
mm = (ns % 3600) / 60;
ss = (ns % 60);
av_log(NULL, AV_LOG_INFO,
"Seek to %2.0f%% (%2d:%02d:%02d) of total duration (%2d:%02d:%02d) \n", frac*100,
hh, mm, ss, thh, tmm, tss);
ts = frac * cur_stream->ic->duration;
if (cur_stream->ic->start_time != AV_NOPTS_VALUE)
ts += cur_stream->ic->start_time;
stream_seek(cur_stream, ts, 0, 0);
}
break;
case SDL_WINDOWEVENT: /* 窗口事件 */
switch (event.window.event) {
case SDL_WINDOWEVENT_SIZE_CHANGED:
screen_width = cur_stream->width = event.window.data1;
screen_height = cur_stream->height = event.window.data2;
if (cur_stream->vis_texture) {
SDL_DestroyTexture(cur_stream->vis_texture);
cur_stream->vis_texture = NULL;
}
case SDL_WINDOWEVENT_EXPOSED:
cur_stream->force_refresh = 1;
}
break;
case SDL_QUIT:
case FF_QUIT_EVENT: /* ffplay自定义事件 */
do_exit(cur_stream);
break;
default:
break;
}
}
}
static int opt_frame_size(void *optctx, const char *opt, const char *arg)
{
av_log(NULL, AV_LOG_WARNING, "Option -s is deprecated, use -video_size.\n");
return opt_default(NULL, "video_size", arg);
}
static int opt_width(void *optctx, const char *opt, const char *arg)
{
screen_width = parse_number_or_die(opt, arg, OPT_INT64, 1, INT_MAX);
return 0;
}
static int opt_height(void *optctx, const char *opt, const char *arg)
{
screen_height = parse_number_or_die(opt, arg, OPT_INT64, 1, INT_MAX);
return 0;
}
static int opt_format(void *optctx, const char *opt, const char *arg)
{
file_iformat = av_find_input_format(arg);
if (!file_iformat) {
av_log(NULL, AV_LOG_FATAL, "Unknown input format: %s\n", arg);
return AVERROR(EINVAL);
}
return 0;
}
static int opt_frame_pix_fmt(void *optctx, const char *opt, const char *arg)
{
av_log(NULL, AV_LOG_WARNING, "Option -pix_fmt is deprecated, use -pixel_format.\n");
return opt_default(NULL, "pixel_format", arg);
}
static int opt_sync(void *optctx, const char *opt, const char *arg)
{
if (!strcmp(arg, "audio"))
av_sync_type = AV_SYNC_AUDIO_MASTER;
else if (!strcmp(arg, "video"))
av_sync_type = AV_SYNC_VIDEO_MASTER;
else if (!strcmp(arg, "ext"))
av_sync_type = AV_SYNC_EXTERNAL_CLOCK;
else {
av_log(NULL, AV_LOG_ERROR, "Unknown value for %s: %s\n", opt, arg);
exit(1);
}
return 0;
}
static int opt_seek(void *optctx, const char *opt, const char *arg)
{
start_time = parse_time_or_die(opt, arg, 1);
return 0;
}
static int opt_duration(void *optctx, const char *opt, const char *arg)
{
duration = parse_time_or_die(opt, arg, 1);
return 0;
}
static int opt_show_mode(void *optctx, const char *opt, const char *arg)
{
show_mode = !strcmp(arg, "video") ? SHOW_MODE_VIDEO :
!strcmp(arg, "waves") ? SHOW_MODE_WAVES :
!strcmp(arg, "rdft" ) ? SHOW_MODE_RDFT :
parse_number_or_die(opt, arg, OPT_INT, 0, SHOW_MODE_NB-1);
return 0;
}
static void opt_input_file(void *optctx, const char *filename)
{
if (input_filename) {
av_log(NULL, AV_LOG_FATAL,
"Argument '%s' provided as input filename, but '%s' was already specified.\n",
filename, input_filename);
exit(1);
}
if (!strcmp(filename, "-"))
filename = "pipe:";
input_filename = filename;
}
static int opt_codec(void *optctx, const char *opt, const char *arg)
{
const char *spec = strchr(opt, ':');
if (!spec) {
av_log(NULL, AV_LOG_ERROR,
"No media specifier was specified in '%s' in option '%s'\n",
arg, opt);
return AVERROR(EINVAL);
}
spec++;
switch (spec[0]) {
case 'a' : audio_codec_name = arg; break;
case 's' : subtitle_codec_name = arg; break;
case 'v' : video_codec_name = arg; break;
default:
av_log(NULL, AV_LOG_ERROR,
"Invalid media specifier '%s' in option '%s'\n", spec, opt);
return AVERROR(EINVAL);
}
return 0;
}
static int dummy;
static const OptionDef options[] = {
CMDUTILS_COMMON_OPTIONS
{ "x", HAS_ARG, { .func_arg = opt_width }, "force displayed width", "width" },
{ "y", HAS_ARG, { .func_arg = opt_height }, "force displayed height", "height" },
{ "s", HAS_ARG | OPT_VIDEO, { .func_arg = opt_frame_size }, "set frame size (WxH or abbreviation)", "size" },
{ "fs", OPT_BOOL, { &is_full_screen }, "force full screen" },
{ "an", OPT_BOOL, { &audio_disable }, "disable audio" },
{ "vn", OPT_BOOL, { &video_disable }, "disable video" },
{ "sn", OPT_BOOL, { &subtitle_disable }, "disable subtitling" },
{ "ast", OPT_STRING | HAS_ARG | OPT_EXPERT, { &wanted_stream_spec[AVMEDIA_TYPE_AUDIO] }, "select desired audio stream", "stream_specifier" },
{ "vst", OPT_STRING | HAS_ARG | OPT_EXPERT, { &wanted_stream_spec[AVMEDIA_TYPE_VIDEO] }, "select desired video stream", "stream_specifier" },
{ "sst", OPT_STRING | HAS_ARG | OPT_EXPERT, { &wanted_stream_spec[AVMEDIA_TYPE_SUBTITLE] }, "select desired subtitle stream", "stream_specifier" },
{ "ss", HAS_ARG, { .func_arg = opt_seek }, "seek to a given position in seconds", "pos" },
{ "t", HAS_ARG, { .func_arg = opt_duration }, "play \"duration\" seconds of audio/video", "duration" },
{ "bytes", OPT_INT | HAS_ARG, { &seek_by_bytes }, "seek by bytes 0=off 1=on -1=auto", "val" },
{ "seek_interval", OPT_FLOAT | HAS_ARG, { &seek_interval }, "set seek interval for left/right keys, in seconds", "seconds" },
{ "nodisp", OPT_BOOL, { &display_disable }, "disable graphical display" },
{ "noborder", OPT_BOOL, { &borderless }, "borderless window" },
{ "alwaysontop", OPT_BOOL, { &alwaysontop }, "window always on top" },
{ "volume", OPT_INT | HAS_ARG, { &startup_volume}, "set startup volume 0=min 100=max", "volume" },
{ "f", HAS_ARG, { .func_arg = opt_format }, "force format", "fmt" },
{ "pix_fmt", HAS_ARG | OPT_EXPERT | OPT_VIDEO, { .func_arg = opt_frame_pix_fmt }, "set pixel format", "format" },
{ "stats", OPT_BOOL | OPT_EXPERT, { &show_status }, "show status", "" },
{ "fast", OPT_BOOL | OPT_EXPERT, { &fast }, "non spec compliant optimizations", "" },
{ "genpts", OPT_BOOL | OPT_EXPERT, { &genpts }, "generate pts", "" },
{ "drp", OPT_INT | HAS_ARG | OPT_EXPERT, { &decoder_reorder_pts }, "let decoder reorder pts 0=off 1=on -1=auto", ""},
{ "lowres", OPT_INT | HAS_ARG | OPT_EXPERT, { &lowres }, "", "" },
{ "sync", HAS_ARG | OPT_EXPERT, { .func_arg = opt_sync }, "set audio-video sync. type (type=audio/video/ext)", "type" },
{ "autoexit", OPT_BOOL | OPT_EXPERT, { &autoexit }, "exit at the end", "" },
{ "exitonkeydown", OPT_BOOL | OPT_EXPERT, { &exit_on_keydown }, "exit on key down", "" },
{ "exitonmousedown", OPT_BOOL | OPT_EXPERT, { &exit_on_mousedown }, "exit on mouse down", "" },
{ "loop", OPT_INT | HAS_ARG | OPT_EXPERT, { &loop }, "set number of times the playback shall be looped", "loop count" },
{ "framedrop", OPT_BOOL | OPT_EXPERT, { &framedrop }, "drop frames when cpu is too slow", "" },
{ "infbuf", OPT_BOOL | OPT_EXPERT, { &infinite_buffer }, "don't limit the input buffer size (useful with realtime streams)", "" },
{ "window_title", OPT_STRING | HAS_ARG, { &window_title }, "set window title", "window title" },
{ "left", OPT_INT | HAS_ARG | OPT_EXPERT, { &screen_left }, "set the x position for the left of the window", "x pos" },
{ "top", OPT_INT | HAS_ARG | OPT_EXPERT, { &screen_top }, "set the y position for the top of the window", "y pos" },
#if CONFIG_AVFILTER
{ "vf", OPT_EXPERT | HAS_ARG, { .func_arg = opt_add_vfilter }, "set video filters", "filter_graph" },
{ "af", OPT_STRING | HAS_ARG, { &afilters }, "set audio filters", "filter_graph" },
#endif
{ "rdftspeed", OPT_INT | HAS_ARG| OPT_AUDIO | OPT_EXPERT, { &rdftspeed }, "rdft speed", "msecs" },
{ "showmode", HAS_ARG, { .func_arg = opt_show_mode}, "select show mode (0 = video, 1 = waves, 2 = RDFT)", "mode" },
{ "default", HAS_ARG | OPT_AUDIO | OPT_VIDEO | OPT_EXPERT, { .func_arg = opt_default }, "generic catch all option", "" },
{ "i", OPT_BOOL, { &dummy}, "read specified file", "input_file"},
{ "codec", HAS_ARG, { .func_arg = opt_codec}, "force decoder", "decoder_name" },
{ "acodec", HAS_ARG | OPT_STRING | OPT_EXPERT, { &audio_codec_name }, "force audio decoder", "decoder_name" },
{ "scodec", HAS_ARG | OPT_STRING | OPT_EXPERT, { &subtitle_codec_name }, "force subtitle decoder", "decoder_name" },
{ "vcodec", HAS_ARG | OPT_STRING | OPT_EXPERT, { &video_codec_name }, "force video decoder", "decoder_name" },
{ "autorotate", OPT_BOOL, { &autorotate }, "automatically rotate video", "" },
{ "find_stream_info", OPT_BOOL | OPT_INPUT | OPT_EXPERT, { &find_stream_info },
"read and decode the streams to fill missing information with heuristics" },
{ "filter_threads", HAS_ARG | OPT_INT | OPT_EXPERT, { &filter_nbthreads }, "number of filter threads per graph" },
{ NULL, },
};
static void show_usage(void)
{
av_log(NULL, AV_LOG_INFO, "Simple media player\n");
av_log(NULL, AV_LOG_INFO, "usage: %s [options] input_file\n", program_name);
av_log(NULL, AV_LOG_INFO, "\n");
}
void show_help_default(const char *opt, const char *arg)
{
av_log_set_callback(log_callback_help);
show_usage();
show_help_options(options, "Main options:", 0, OPT_EXPERT, 0);
show_help_options(options, "Advanced options:", OPT_EXPERT, 0, 0);
printf("\n");
show_help_children(avcodec_get_class(), AV_OPT_FLAG_DECODING_PARAM);
show_help_children(avformat_get_class(), AV_OPT_FLAG_DECODING_PARAM);
#if !CONFIG_AVFILTER
show_help_children(sws_get_class(), AV_OPT_FLAG_ENCODING_PARAM);
#else
show_help_children(avfilter_get_class(), AV_OPT_FLAG_FILTERING_PARAM);
#endif
printf("\nWhile playing:\n"
"q, ESC quit\n"
"f toggle full screen\n"
"p, SPC pause\n"
"m toggle mute\n"
"9, 0 decrease and increase volume respectively\n"
"/, * decrease and increase volume respectively\n"
"a cycle audio channel in the current program\n"
"v cycle video channel\n"
"t cycle subtitle channel in the current program\n"
"c cycle program\n"
"w cycle video filters or show modes\n"
"s activate frame-step mode\n"
"left/right seek backward/forward 10 seconds or to custom interval if -seek_interval is set\n"
"down/up seek backward/forward 1 minute\n"
"page down/page up seek backward/forward 10 minutes\n"
"right mouse click seek to percentage in file corresponding to fraction of width\n"
"left double-click toggle full screen\n"
);
}
/* Called from the main */
int main(int argc, char **argv)
{
int flags;
VideoState *is;
// av_log_set_level(AV_LOG_TRACE);
init_dynload();
av_log_set_flags(AV_LOG_SKIP_REPEATED);
parse_loglevel(argc, argv, options);
/// av_log_set_level(AV_LOG_DEBUG);
/* register all codecs, demux and protocols */
#if CONFIG_AVDEVICE
avdevice_register_all();
#endif
avformat_network_init();
init_opts();
signal(SIGINT , sigterm_handler); /* Interrupt (ANSI). */
signal(SIGTERM, sigterm_handler); /* Termination (ANSI). */
show_banner(argc, argv, options);
parse_options(NULL, argc, argv, options, opt_input_file);
if (!input_filename) {
show_usage();
av_log(NULL, AV_LOG_FATAL, "An input file must be specified\n");
av_log(NULL, AV_LOG_FATAL,
"Use -h to get full help or, even better, run 'man %s'\n", program_name);
exit(1);
}
/* 是否显示视频 */
if (display_disable) {
video_disable = 1;
}
flags = SDL_INIT_VIDEO | SDL_INIT_AUDIO | SDL_INIT_TIMER;
/* 是否运行音频 */
if (audio_disable)
flags &= ~SDL_INIT_AUDIO;
else {
/* Try to work around an occasional ALSA buffer underflow issue when the
* period size is NPOT due to ALSA resampling by forcing the buffer size. */
if (!SDL_getenv("SDL_AUDIO_ALSA_SET_BUFFER_SIZE"))
SDL_setenv("SDL_AUDIO_ALSA_SET_BUFFER_SIZE","1", 1);
}
if (display_disable)
flags &= ~SDL_INIT_VIDEO;
if (SDL_Init (flags)) {
av_log(NULL, AV_LOG_FATAL, "Could not initialize SDL - %s\n", SDL_GetError());
av_log(NULL, AV_LOG_FATAL, "(Did you set the DISPLAY variable?)\n");
exit(1);
}
SDL_EventState(SDL_SYSWMEVENT, SDL_IGNORE);
SDL_EventState(SDL_USEREVENT, SDL_IGNORE);
av_init_packet(&flush_pkt); // 初始化flush_packet
flush_pkt.data = (uint8_t *)&flush_pkt; // 初始化为数据指向自己本身
if (!display_disable) {
int flags = SDL_WINDOW_HIDDEN;
if (alwaysontop)
#if SDL_VERSION_ATLEAST(2,0,5)
flags |= SDL_WINDOW_ALWAYS_ON_TOP;
#else
av_log(NULL, AV_LOG_WARNING, "Your SDL version doesn't support SDL_WINDOW_ALWAYS_ON_TOP. Feature will be inactive.\n");
#endif
if (borderless)
flags |= SDL_WINDOW_BORDERLESS;
else
flags |= SDL_WINDOW_RESIZABLE;
window = SDL_CreateWindow(program_name, SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, default_width, default_height, flags);
SDL_SetHint(SDL_HINT_RENDER_SCALE_QUALITY, "linear");
if (window) {
renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED | SDL_RENDERER_PRESENTVSYNC);
if (!renderer) {
av_log(NULL, AV_LOG_WARNING, "Failed to initialize a hardware accelerated renderer: %s\n", SDL_GetError());
renderer = SDL_CreateRenderer(window, -1, 0);
}
if (renderer) {
if (!SDL_GetRendererInfo(renderer, &renderer_info))
av_log(NULL, AV_LOG_VERBOSE, "Initialized %s renderer.\n", renderer_info.name);
}
}
if (!window || !renderer || !renderer_info.num_texture_formats) {
av_log(NULL, AV_LOG_FATAL, "Failed to create window or renderer: %s", SDL_GetError());
do_exit(NULL);
}
}
is = stream_open(input_filename, file_iformat);
if (!is) {
av_log(NULL, AV_LOG_FATAL, "Failed to initialize VideoState!\n");
do_exit(NULL);
}
/* 事件响应 */
event_loop(is);
/* never returns */
return 0;
}