拉取RTSP流后的几个去向探讨(播放|转RTMP|轻量级RTSP服务|本地录制|GB28181)

​RTSP流的几个去处

写了很多关于RTSP播放和转发的blog了,今天我们做个简单的汇总,以大牛直播SDK的Android平台为例,拉取到RTSP流,除了本地播放,还有几个流向:

功能实现:

**1. 拉流:**通过RTSP直播播放SDK的数据回调接口,拿到音视频数据;

**2. 转推:**通过RTMP直播推送SDK的编码后数据输入接口,把回调上来的数据,传给RTMP直播推送模块,实现RTSP数据流到RTMP服务器的转发,同时也可以转发到轻量级RTSP服务和GB28181平台;

**3. 录像:**如果需要录像,借助RTSP直播播放SDK,拉到音视频数据后,直接存储MP4文件即可;

**4. 快照:**如果需要实时快照,拉流后,解码调用播放端快照接口,生成快照,因为快照涉及到video数据解码,如无必要,可不必开启,不然会额外消耗性能。

**5. 拉流预览:**如需预览拉流数据,只要调用播放端的播放接口,即可实现拉流数据预览;

**6. 数据转AAC后转发:**考虑到好多监控设备出来的音频可能是PCMA/PCMU的,如需要更通用的音频格式,可以转AAC后,在通过RTMP推送;

**7. 转推实时静音:**只需要在传audio数据的地方,加个判断即可;

**8. 拉流速度反馈:**通过RTSP播放端的实时码率反馈event,拿到实时带宽占用即可;

**9. 整体网络状态反馈:**考虑到有些摄像头可能会临时或异常关闭,RTMP服务器亦是,可以通过推拉流的event回调状态,查看那整体网络情况,如此界定:是拉不到流,还是推不到RTMP服务器。

先说拉取数据,拉取RTSP流的时候,设置音视频数据回调。

ini 复制代码
    /*
     * SmartRelayDemo.java
     * Author: daniusdk.com
     */
    private boolean StartPull()
	{
		if ( isPulling )
			return false;

		if(!isPlaying)
		{
			if (!OpenPullHandle())
				return false;
		}

		libPlayer.SmartPlayerSetAudioDataCallback(player_handle_, new PlayerAudioDataCallback(stream_publisher_));
		libPlayer.SmartPlayerSetVideoDataCallback(player_handle_, new PlayerVideoDataCallback(stream_publisher_));

		int is_pull_trans_code  = 1;
		libPlayer.SmartPlayerSetPullStreamAudioTranscodeAAC(player_handle_, is_pull_trans_code);

		int startRet = libPlayer.SmartPlayerStartPullStream(player_handle_);

		if (startRet != 0) {
			Log.e(TAG, "Failed to start pull stream!");

			if(!isPlaying)
			{
				releasePlayerHandle();
			}

			return false;
		}

		isPulling = true;
		return true;
	}

对应的OpenPullHandle()实现如下:

ini 复制代码
	private boolean OpenPullHandle()
	{
		//playbackUrl可自定义
		//playbackUrl = "rtsp://admin:daniulive12345@192.168.0.120:554/h264/ch1/main/av_stream";

		if (playbackUrl == null) {
			Log.e(TAG, "playback URL is null...");
			return false;
		}

		player_handle_ = libPlayer.SmartPlayerOpen(context_);

		if (player_handle_ == 0) {
			Log.e(TAG, "playerHandle is null..");
			return false;
		}

		libPlayer.SetSmartPlayerEventCallbackV2(player_handle_,
				new EventHandlePlayerV2());

		libPlayer.SmartPlayerSetBuffer(player_handle_, playBuffer);

		// set report download speed
		libPlayer.SmartPlayerSetReportDownloadSpeed(player_handle_, 1, 3);

		//设置RTSP超时时间
		int rtsp_timeout = 10;
		libPlayer.SmartPlayerSetRTSPTimeout(player_handle_, rtsp_timeout);

		//设置RTSP TCP/UDP模式自动切换
		int is_auto_switch_tcp_udp = 1;
		libPlayer.SmartPlayerSetRTSPAutoSwitchTcpUdp(player_handle_, is_auto_switch_tcp_udp);

		// It only used when playback RTSP stream..
		//libPlayer.SmartPlayerSetRTSPTcpMode(playerHandle, 1);

		libPlayer.SmartPlayerSetUrl(player_handle_, playbackUrl);

		return true;
	}

音频处理如下:

ini 复制代码
	class PlayerAudioDataCallback implements NTAudioDataCallback
	{
		private WeakReference<LibPublisherWrapper> publisher_;
		private int audio_buffer_size = 0;
		private int param_info_size = 0;

		private ByteBuffer audio_buffer_ = null;
		private ByteBuffer parameter_info_ = null;

		public PlayerAudioDataCallback(LibPublisherWrapper publisher) {
			if (publisher != null)
				publisher_ = new WeakReference<>(publisher);
		}

		@Override
		public ByteBuffer getAudioByteBuffer(int size)
		{
			//Log.i("getAudioByteBuffer", "size: " + size);

			if( size < 1 )
			{
				return null;
			}

			if ( size <= audio_buffer_size && audio_buffer_ != null )
			{
				return audio_buffer_;
			}

			audio_buffer_size = size + 512;
			audio_buffer_size = (audio_buffer_size+0xf) & (~0xf);

			audio_buffer_ = ByteBuffer.allocateDirect(audio_buffer_size);

			// Log.i("getAudioByteBuffer", "size: " + size + " buffer_size:" + audio_buffer_size);

			return audio_buffer_;
		}

		@Override
		public ByteBuffer getAudioParameterInfo(int size)
		{
			//Log.i("getAudioParameterInfo", "size: " + size);

			if(size < 1)
			{
				return null;
			}

			if ( size <= param_info_size &&  parameter_info_ != null )
			{
				return  parameter_info_;
			}

			param_info_size = size + 32;
			param_info_size = (param_info_size+0xf) & (~0xf);

			parameter_info_ = ByteBuffer.allocateDirect(param_info_size);

			//Log.i("getAudioParameterInfo", "size: " + size + " buffer_size:" + param_info_size);

			return parameter_info_;
		}

		public void onAudioDataCallback(int ret, int audio_codec_id, int sample_size, int is_key_frame, long timestamp, int sample_rate, int channel, int parameter_info_size, long reserve)
		{
			//Log.i("onAudioDataCallback", "ret: " + ret + ", audio_codec_id: " + audio_codec_id + ", sample_size: " + sample_size + ", timestamp: " + timestamp +
			//		",sample_rate:" + sample_rate);

			if ( audio_buffer_ == null)
				return;

			LibPublisherWrapper publisher = publisher_.get();
			if (null == publisher)
				return;

			if (!publisher.is_publishing())
				return;

			audio_buffer_.rewind();

			publisher.PostAudioEncodedData(audio_codec_id, audio_buffer_, sample_size, is_key_frame, timestamp, parameter_info_, parameter_info_size);
		}
	}

视频处理如下:

java 复制代码
	class PlayerVideoDataCallback implements NTVideoDataCallback
	{
		private WeakReference<LibPublisherWrapper> publisher_;
		private int video_buffer_size = 0;

		private ByteBuffer video_buffer_ = null;

		public PlayerVideoDataCallback(LibPublisherWrapper publisher) {
			if (publisher != null)
				publisher_ = new WeakReference<>(publisher);
		}

		@Override
		public ByteBuffer getVideoByteBuffer(int size)
		{
			//Log.i("getVideoByteBuffer", "size: " + size);

			if( size < 1 )
			{
				return null;
			}

			if ( size <= video_buffer_size &&  video_buffer_ != null )
			{
				return  video_buffer_;
			}

			video_buffer_size = size + 1024;
			video_buffer_size = (video_buffer_size+0xf) & (~0xf);

			video_buffer_ = ByteBuffer.allocateDirect(video_buffer_size);

			// Log.i("getVideoByteBuffer", "size: " + size + " buffer_size:" + video_buffer_size);

			return video_buffer_;
		}

		public void onVideoDataCallback(int ret, int video_codec_id, int sample_size, int is_key_frame, long timestamp, int width, int height, long presentation_timestamp)
		{
			//Log.i("onVideoDataCallback", "ret: " + ret + ", video_codec_id: " + video_codec_id + ", sample_size: " + sample_size + ", is_key_frame: "+ is_key_frame +  ", timestamp: " + timestamp +
			//		",presentation_timestamp:" + presentation_timestamp);

			if ( video_buffer_ == null)
				return;

			LibPublisherWrapper publisher = publisher_.get();
			if (null == publisher)
				return;

			if (!publisher.is_publishing())
				return;

			video_buffer_.rewind();

			publisher.PostVideoEncodedData(video_codec_id, video_buffer_, sample_size, is_key_frame, timestamp, presentation_timestamp);
		}
	}

本地播放

ini 复制代码
​
	private boolean StartPlay()
	{
		if(isPlaying)
			return false;

		if(!isPulling)
		{
			if (!OpenPullHandle())
				return false;
		}

		// 如果第二个参数设置为null,则播放纯音频
		libPlayer.SmartPlayerSetSurface(player_handle_, sSurfaceView);
		//libPlayer.SmartPlayerSetSurface(player_handle_, null);

		libPlayer.SmartPlayerSetRenderScaleMode(player_handle_, 1);

		libPlayer.SmartPlayerSetFastStartup(player_handle_, isFastStartup ? 1 : 0);

		libPlayer.SmartPlayerSetAudioOutputType(player_handle_, 1);

		if (isMute) {
			libPlayer.SmartPlayerSetMute(player_handle_, isMute ? 1	: 0);
		}

		if (isHardwareDecoder)
		{
			int isSupportH264HwDecoder = libPlayer.SetSmartPlayerVideoHWDecoder(player_handle_, 1);

			int isSupportHevcHwDecoder = libPlayer.SetSmartPlayerVideoHevcHWDecoder(player_handle_, 1);

			Log.i(TAG, "isSupportH264HwDecoder: " + isSupportH264HwDecoder + ", isSupportHevcHwDecoder: " + isSupportHevcHwDecoder);
		}

		libPlayer.SmartPlayerSetLowLatencyMode(player_handle_, isLowLatency ? 1	: 0);

		libPlayer.SmartPlayerSetRotation(player_handle_, rotate_degrees);

		int iPlaybackRet = libPlayer.SmartPlayerStartPlay(player_handle_);

		if (iPlaybackRet != 0 && !isPulling) {
			Log.e(TAG, "StartPlay failed!");

			releasePlayerHandle();
			return false;
		}

		isPlaying = true;

		return true;
	}

	private void StopPlay()
	{
		if ( !isPlaying )
			return;

		isPlaying = false;

		if (null == libPlayer || 0 == player_handle_)
			return;

		libPlayer.SmartPlayerStopPlay(player_handle_);
	}

​

转推RTMP

scss 复制代码
		btnRTMPPusher.setOnClickListener(new Button.OnClickListener() {

			// @Override
			public void onClick(View v) {

				if (stream_publisher_.is_rtmp_publishing()) {
					stopPush();

					btnRTMPPusher.setText("推送RTMP");
					return;
				}

				Log.i(TAG, "onClick start push rtmp..");
				InitAndSetConfig();

				//relayStreamUrl = "rtmp://192.168.0.108:1935/hls/stream1";

				if (!stream_publisher_.SetURL(relayStreamUrl))
					Log.e(TAG, "Failed to set publish stream URL..");

				boolean start_ret = stream_publisher_.StartPublisher();
				if (!start_ret) {
					stream_publisher_.try_release();
					Log.e(TAG, "Failed to start push stream..");
					return;
				}

				btnRTMPPusher.setText("停止推送");
			}
		});

转推轻量级RTSP服务

ini 复制代码
	//启动/停止RTSP服务
	class ButtonRtspServiceListener implements View.OnClickListener {
		public void onClick(View v) {
			if (isRTSPServiceRunning) {
				stopRtspService();

				btnRtspService.setText("启动RTSP服务");
				btnRtspPublisher.setEnabled(false);

				isRTSPServiceRunning = false;
				return;
			}

			Log.i(TAG, "onClick start rtsp service..");

			rtsp_handle_ = libPublisher.OpenRtspServer(0);

			if (rtsp_handle_ == 0) {
				Log.e(TAG, "创建rtsp server实例失败! 请检查SDK有效性");
			} else {
				int port = 28554;
				if (libPublisher.SetRtspServerPort(rtsp_handle_, port) != 0) {
					libPublisher.CloseRtspServer(rtsp_handle_);
					rtsp_handle_ = 0;
					Log.e(TAG, "创建rtsp server端口失败! 请检查端口是否重复或者端口不在范围内!");
				}

				if (libPublisher.StartRtspServer(rtsp_handle_, 0) == 0) {
					Log.i(TAG, "启动rtsp server 成功!");
				} else {
					libPublisher.CloseRtspServer(rtsp_handle_);
					rtsp_handle_ = 0;
					Log.e(TAG, "启动rtsp server失败! 请检查设置的端口是否被占用!");
				}

				btnRtspService.setText("停止RTSP服务");
				btnRtspPublisher.setEnabled(true);

				isRTSPServiceRunning = true;
			}
		}
	}

	//发布/停止RTSP流
	class ButtonRtspPublisherListener implements View.OnClickListener {
		public void onClick(View v) {
			if (stream_publisher_.is_rtsp_publishing()) {
				stopRtspPublisher();

				btnRtspPublisher.setText("发布RTSP流");
				btnGetRtspSessionNumbers.setEnabled(false);
				btnRtspService.setEnabled(true);
				return;
			}

			Log.i(TAG, "onClick start rtsp publisher..");

			InitAndSetConfig();

			String rtsp_stream_name = "stream1";
			stream_publisher_.SetRtspStreamName(rtsp_stream_name);
			stream_publisher_.ClearRtspStreamServer();

			stream_publisher_.AddRtspStreamServer(rtsp_handle_);

			if (!stream_publisher_.StartRtspStream()) {
				stream_publisher_.try_release();
				Log.e(TAG, "调用发布rtsp流接口失败!");
				return;
			}

			btnRtspPublisher.setText("停止RTSP流");
			btnGetRtspSessionNumbers.setEnabled(true);
			btnRtspService.setEnabled(false);
		}
	}

	//当前RTSP会话数弹出框
	private void PopRtspSessionNumberDialog(int session_numbers) {
		final EditText inputUrlTxt = new EditText(this);
		inputUrlTxt.setFocusable(true);
		inputUrlTxt.setEnabled(false);

		String session_numbers_tag = "RTSP服务当前客户会话数: " + session_numbers;
		inputUrlTxt.setText(session_numbers_tag);

		AlertDialog.Builder builderUrl = new AlertDialog.Builder(this);
		builderUrl
				.setTitle("内置RTSP服务")
				.setView(inputUrlTxt).setNegativeButton("确定", null);
		builderUrl.show();
	}

	//获取RTSP会话数
	class ButtonGetRtspSessionNumbersListener implements OnClickListener {
		public void onClick(View v) {
			if (libPublisher != null && rtsp_handle_ != 0) {
				int session_numbers = libPublisher.GetRtspServerClientSessionNumbers(rtsp_handle_);

				Log.i(TAG, "GetRtspSessionNumbers: " + session_numbers);

				PopRtspSessionNumberDialog(session_numbers);
			}
		}
	};

拉取RTSP后录像

scss 复制代码
		btnStartStopRecorder.setOnClickListener(new Button.OnClickListener() {

			// @Override
			public void onClick(View v) {

				if (isRecording)
				{
					StopRecorder();
					btnStartStopRecorder.setText(" 开始录像");
				} else {
					Log.i(TAG, "onClick StartRecorder..");

					boolean startRet = StartRecorder();

					if (!startRet) {
						Log.e(TAG, "Failed to call StartRecorder().");
						return;
					}

					btnStartStopRecorder.setText("停止录像");
				}
			}
		});

转推GB28181平台

scss 复制代码
​
	class ButtonGB28181AgentListener implements OnClickListener {
		public void onClick(View v) {
			stopGB28181Stream();
			destoryRTPSender();

			if (null == gb28181_agent_ ) {
				if( !initGB28181Agent() )
					return;
			}

			if (gb28181_agent_.isRunning()) {
				gb28181_agent_.terminateAllPlays(true);// 目前测试下来,发送BYE之后,有些服务器会立即发送INVITE,是否发送BYE根据实际情况看
				gb28181_agent_.stop();
				btnGB28181Agent.setText("启动GB28181");
			}
			else {
				if ( gb28181_agent_.start() ) {
					btnGB28181Agent.setText("停止GB28181");
				}
			}
		}
	}

	//停止GB28181 媒体流
	private void stopGB28181Stream() {
		stream_publisher_.StopGB28181MediaStream();
		stream_publisher_.try_release();
	}

​

总结

一个好的转发模块,首先要低延迟!其次足够稳定、灵活、有状态反馈机制、资源占用低,如果可以跨平台,还能以SDK形式提供,会给开发者提供更大的便利!Android平台拉取RTSP流后,有了数据源,开发者可以在一个推送实例中,转推到不同的业务场景,实现高效率低延迟的数据转发。

相关推荐
关键帧Keyframe2 天前
音视频面试题集锦第 7 期
音视频开发·视频编码·客户端
关键帧Keyframe2 天前
音视频面试题集锦第 8 期
ios·音视频开发·客户端
蚝油菜花7 天前
MimicTalk:字节跳动和浙江大学联合推出 15 分钟生成 3D 说话人脸视频的生成模型
人工智能·开源·音视频开发
音视频牛哥9 天前
Android平台RTSP|RTMP播放器高效率如何回调YUV或RGB数据?
音视频开发·视频编码·直播
<Sunny>11 天前
MPP音视频总结
音视频开发·1024程序员节·海思mpp
GoFly开发者11 天前
GoFly快速开发框架已集成了RTSP流媒体服务器(直播、录播)代码插件-开发者集成流媒体服务器更加方便
go·音视频开发
aqi0013 天前
FFmpeg开发笔记(六十)使用国产的ijkplayer播放器观看网络视频
android·ffmpeg·音视频·直播·流媒体
linkedbyte13 天前
常用录屏软件功能比较
音视频·直播·录屏
aqi0016 天前
FFmpeg开发笔记(五十九)Linux编译ijkplayer的Android平台so库
android·ffmpeg·音视频·直播·流媒体