ffmpeg-api记录

1. Frame

基于引用计数的frame

av_frame_ref: ref
av_frame_unref: unref
av_frame_move_ref:所有权转移
av_frame_clone: copy一份，引用计数=1

2. Packet

3. AVAudioFifo

音频重采样时，用来缓存aac的帧(1024才能送去编码)

av_audio_fifo_alloc
av_audio_fifo_size: 大小查询
av_audio_fifo_read: 消费fifo
av_audio_fifo_write: 写入fifo

4. mux

目标: 根据配置, 给定图片、音频，转码(如果需要)并mux

graph LR init_muxer_using_config[根据配置初始化mux] init_muxer_using_origin_context[根据原视频初始化mux] interact_input_image_audio_saple[交替输入图像/音频] --> convert_if_need[如果需要则进行转码] --> mux convert_if_need --> video_convert[视频转码逻辑] video_convert --> init_sws_getContext_using_format_width_height[使用src/dst像素格式、宽、高初始化] init_sws_getContext_using_format_width_height --> call_sws_scale[调用sws_scale接口得到转码过后的frame] call_sws_scale --> call_send_frame_and_mux[正常调用send_frame接口进行mux] convert_if_need --> audio_convert[音频转码设置] audio_convert --> init_swr_using_swr_alloc_set_opts2[使用src/dst布局、编码格式、采样率初始化 swr_alloc_set_opts2 转换器] audio_convert --> init_audio_fifo[使用av_audio_fifo_alloc+dst编码格式、通道布局初始化音频fifo队列] init_swr_using_swr_alloc_set_opts2 --> call_swr_convert[调用swr_convert得到转码过后的frame] call_swr_convert --> push_fifo[将结果push到audio fifo中] push_fifo --> consume_fifo[消费fifo如果samples大于1024] consume_fifo --> call_send_frame_and_mux

4.1 mux.Stream

cpp 复制代码

enum class stream_type {none, video, audio};
struct Stream {
	stream_type type;
	int index{-1};
	AVCodecContext *codec_ctx{nullptr};
	// for video
	SwsContext *vid_sws_ctx{nullptr};
	int64_t vid_next_pts{0};
	// for audio
	SwrContext *aud_swr_ctx{nullptr};
	int64_t handled_samples{0};
	AVAudioFifo *fifo{nullptr};

	// common frame
	AVFrame *frame{nullptr};
};

5. 硬件编解码

需要处理的问题

硬件上下文初始化
将AVFrame转换到硬件能够接受的输入
host/device的数据传输