Android平台GB28181设备接入模块开发填坑指南

音视频牛哥2023-11-27 23:01

​技术背景

为什么要开发Android平台GB28181设备接入模块？这个问题不再赘述，在做Android平台GB28181客户端的时候，媒体数据这块，我们已经有了很好的积累，因为在此之前，我们就开发了非常成熟的RTMP推送、轻量级RTSP服务、录像模块、针对音视频的对接处理单元。这让我们在做Android平台GB28181设备接入模块的时候，可以有更多的精力在信令交互和国标平台对接。

好多开发者会觉得，GB28181设备接入模块有啥好做的？不就找个开源的SIP信令，视频编码ps打包下投递到国标平台就好了吗？

事实上，当回头看看开发的功能时，就会觉得，一两个月的东西，仅就可以作为项目交付或demo使用，并不会有多大的商业价值，因为需要解决的问题实在太多了。

  • [视频格式]H.264/H.265(Android H.265硬编码)；
  • [音频格式]G.711 A律、AAC；
  • [音量调节]Android平台采集端支持实时音量调节；
  • [H.264硬编码]支持H.264特定机型硬编码；
  • [H.265硬编码]支持H.265特定机型硬编码；
  • [软硬编码参数配置]支持gop间隔、帧率、bit-rate设置；
  • [软编码参数配置]支持软编码profile、软编码速度、可变码率设置；
  • 支持纯视频、音视频PS打包传输；
  • 支持RTP OVER UDP和RTP OVER TCP被动模式（TCP媒体流传输客户端）；
  • 支持信令通道网络传输协议TCP/UDP设置；
  • 支持注册、注销，支持注册刷新及注册有效期设置；
  • 支持设备目录查询应答；
  • 支持心跳机制，支持心跳间隔、心跳检测次数设置；
  • 支持移动设备位置(MobilePosition)订阅和通知；
  • 支持语音广播；
  • 支持语音对讲；
  • 支持历史视音频文件检索；
  • 支持历史视音频文件下载；
  • 支持历史视音频文件回放；
  • 支持云台控制和预置位查询；
  • [实时水印]支持动态文字水印、png水印；
  • [镜像]Android平台支持前置摄像头实时镜像功能；
  • [实时静音]支持实时静音/取消静音；
  • [实时快照]支持实时快照；
  • [降噪]支持环境音、手机干扰等引起的噪音降噪处理、自动增益、VAD检测；
  • [外部编码前视频数据对接]支持YUV数据对接；
  • [外部编码前音频数据对接]支持PCM对接；
  • [外部编码后视频数据对接]支持外部H.264数据对接；
  • [外部编码后音频数据对接]外部AAC数据对接；
  • [扩展录像功能]支持和录像模块组合使用，录像相关功能。

技术实现

​

[视频格式]H.264/H.265(Android H.265硬编码)

目前GB28181-2022已经明确表示支持H.265，GB28181设备接入这块，如果需要有好的画质，编码算法这块，一定需要做好，Android端除了低分辨率软编外，超过1280*720，一般建议硬编码。

java 复制代码 
	 /**
	  * Set Video H.264 HW Encoder, if support HW encoder, it will return 0(设置H.264硬编码)
	  * 
	  * @param kbps: the kbps of different resolution.
	  * 
	  * @return {0} if successful
	  */
   public native int SetSmartPublisherVideoHWEncoder(long handle, int kbps);

	/**
	 * Set Video H.265(hevc) hardware encoder, if support H.265(hevc) hardware encoder, it will return 0(设置H.265硬编码)
	 *
	 * @param kbps: the kbps of different resolution.
	 *
	 * @return {0} if successful
	 */
	public native int SetSmartPublisherVideoHevcHWEncoder(long handle, int kbps);

硬编码参数设置

java 复制代码 
	/*
	* 设置视频硬编码码率控制模式
	* @param hw_bitrate_mode: -1表示使用默认值, 不设置也会使用默认值, 0:CQ, 1:VBR, 2:CBR, 3:CBR_FD, 请参考:android.media.MediaCodecInfo.EncoderCapabilities
	* 注意硬编码和手机硬件有关，多数手机只支持部分码率模式, 另外硬编码设备差异很大，不同设备同一码率控制模式效果可能不一样
	* @return {0} if successful
	*/
	public native int SetVideoHWEncoderBitrateMode(long handle, int hw_bitrate_mode);


	/*
	 * 设置视频硬编码复杂度, 安卓5.0及以上支持
	 * @param hw_complexity: -1表示不设置, 请参考:android.media.MediaCodecInfo.EncoderCapabilities.getComplexityRange() 和 android.media.MediaFormat.KEY_COMPLEXITY
	 * 注意硬编码和手机硬件有关，部分手机可能不支持此设置
	 * @return {0} if successful
	 */
	public native int SetVideoHWEncoderComplexity(long handle, int hw_complexity);

	/*
	 * 设置视频硬编码质量, 安卓9及以上支持, 仅当硬编码器码率控制模式(BitrateMode)是CQ(constant-quality mode)时才有效
	 * @param hw_quality: -1表示不设置, 请参考:android.media.MediaCodecInfo.EncoderCapabilities.getQualityRange() 和 android.media.MediaFormat.KEY_QUALITY
	 * 注意硬编码和手机硬件有关，部分手机可能不支持此设置
	 * @return {0} if successful
	 */
	public native int SetVideoHWEncoderQuality(long handle, int hw_quality);

	/*
	 * 设置H.264硬编码Profile, 安卓7及以上支持
	 * @param hw_avc_profile: 0表示使用默认值, 0x01: Baseline, 0x02: Main, 0x08: High, 0x10000: ConstrainedBaseline, 0x80000: ConstrainedHigh;
	 * 注意: ConstrainedBaseline 和 ConstrainedHigh 可能多数设备不支持,
	 * H.264推荐使用 High 或者 ConstrainedHigh, 如果您使用的手机硬解码解不了，那还是设置Baseline
	 * 如果设置的Profile硬编码器不支持，应编码器会使用默认值
	 * 具体参考:android.media.MediaCodecInfo.CodecProfileLevel
	 * @return {0} if successful
	 */
	public native int SetAVCHWEncoderProfile(long handle, int hw_avc_profile);

	/*
	 * 设置H.264硬编码Level, 这个只有在设置了Profile的情况下才有效, 安卓7及以上支持
	 * @param hw_avc_level: 0表示使用默认值, 0x100: Level3, 0x200: Level3.1, 0x400: Level3.2,
	 * 0x800: Level4, 0x1000: Level4.1, 0x2000: Level4.2,
	 * 0x4000: Level5, 0x8000: Level5.1,  0x10000: Level5.2,
	 * 0x20000: Level6, 0x40000: Level6.1,  0x80000: Level6.2,
	 * 如果设置的level太高硬编码器不支持，SDK内部会做相应调整
	 * 注意: 640*480@25fps最小支持的是Level3, 720p最小支持的是Level3.1, 1080p最小支持的是Level4
	 * 具体参考:android.media.MediaCodecInfo.CodecProfileLevel
	 * @return {0} if successful
	 */
	public native int SetAVCHWEncoderLevel(long handle, int hw_avc_level);

	/*
	 * 设置视频硬编码最大码率, 安卓没有相关文档说明, 所以不建议设置,
	 * @param hw_max_bitrate: 每秒最大码率, 单位bps
	 * @return {0} if successful
	 */
	public native int SetVideoHWEncoderMaxBitrate(long handle, long hw_max_bitrate);

[音频格式]G.711 A律、AAC

java 复制代码 
    /**
     * Set audio encoder type(设置音频编码类型)
     * 
     * @param type: if with 1:AAC, if with 2: SPEEX, if with 3: PCMA
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetAudioCodecType(long handle, int type);


	/**
	 * Set audio encoder bit-rate(设置音频编码码率), 当前只对AAC编码有效
	 *
	 * @param kbit_rate: 码率(单位是kbps), 如果是0的话将使用默认码率, 必须大于等于0
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetAudioBitRate(long handle, int kbit_rate);

[音量调节]Android平台采集端支持实时音量调节

java 复制代码 
	/**
	 * 设置输入音量, 这个接口一般不建议调用, 在一些特殊情况下可能会用, 一般不建议放大音量
	 *
	 * @param index: 一般是0和1, 如果没有混音的只用0, 有混音的话, 0,1分别设置音量
	 *
	 * @param volume: 音量，默认是1.0，范围是[0.0, 5.0], 设置成0静音, 1音量不变
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetInputAudioVolume(long handle, int index, float volume);

[软硬编码参数配置]支持gop间隔、帧率、bit-rate设置，支持软编码profile、软编码速度、可变码率设置

java 复制代码 
	/**
	 * Set software encode vbr mode(软编码可变码率).
	 *
	 * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
	 *
	 * is_enable_vbr: if 0: NOT enable vbr mode, 1: enable vbr
	 *
	 * video_quality: vbr video quality, range with (1,50), default 23
	 *
	 * vbr_max_kbitrate: vbr max encode bit-rate(kbps)
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetSwVBRMode(long handle, int is_enable_vbr, int video_quality, int vbr_max_kbitrate);

    /**
     * Set gop interval(设置I帧间隔)
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * gopInterval: encode I frame interval, the value always > 0
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetGopInterval(long handle, int gopInterval);
    
    /**
     * Set software encode video bit-rate(设置视频软编码bit-rate)
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * avgBitRate: average encode bit-rate(kbps)
     * 
     * maxBitRate: max encode bit-rate(kbps)
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetSWVideoBitRate(long handle, int avgBitRate, int maxBitRate);
    
    /**
     * Set fps(设置帧率)
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * fps: the fps of video, range with (1,25).
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetFPS(long handle, int fps);
    
	/**
     * Set software video encoder profile(设置视频编码profile).
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * profile: the software video encoder profile, range with (1,3).
     * 
     * 1: baseline profile
     * 2: main profile
     * 3: high profile
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetSWVideoEncoderProfile(long handle, int profile);
    
    
    /**
     * Set software video encoder speed(设置视频软编码编码速度)
     * 
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     * 
     * @param speed: range with(1, 6), the default speed is 6. 
     * 
     * if with 1, CPU is lowest.
     * if with 6, CPU is highest.
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetSWVideoEncoderSpeed(long handle, int speed);

信令通道网络传输协议TCP/UDP设置

ini 复制代码 
gb28181_agent_.setTransportProtocol(gb28181_sip_trans_protocol_==0?"UDP":"TCP");

支持注册、注销，支持注册刷新及注册有效期设置

typescript 复制代码 
private int gb28181_reg_expired_           = 3600; // 注册有效期时间最小3600秒

// GB28181配置
gb28181_agent_.config(gb28181_reg_expired_, gb28181_heartbeat_interval_, gb28181_heartbeat_count_);

@Override
public void ntsRegisterOK(String dateString) {
  Log.i(TAG, "ntsRegisterOK Date: " + (dateString!= null? dateString : ""));
}

@Override
public void ntsRegisterTimeout() {
  Log.e(TAG, "ntsRegisterTimeout");
}

@Override
public void ntsRegisterTransportError(String errorInfo) {
  Log.e(TAG, "ntsRegisterTransportError error:" + (errorInfo != null?errorInfo :""));
}

支持心跳机制，支持心跳间隔、心跳检测次数设置

scss 复制代码 
private int gb28181_heartbeat_interval_    = 20; // 心跳间隔GB28181默认是60, 目前调整到20秒
private int gb28181_heartbeat_count_       = 3; // 心跳间隔3次失败，表示和服务器断开了

@Override
public void ntsOnHeartBeatException(int exceptionCount,  String lastExceptionInfo) {
  Log.e(TAG, "ntsOnHeartBeatException heart beat timeout count reached, count:" + exceptionCount+
        ", exception info:" + (lastExceptionInfo!=null?lastExceptionInfo:""));

  // 停止信令, 然后重启
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "gb28281_heart_beart_timeout");

      record_executor_.cancel_tasks();

      stopRecordDownloads(true);
      stopPlaybacks(true);

      stopAudioPlayer();
      destoryRTPReceiver();

      gb_broadcast_source_id_ = null;
      gb_broadcast_target_id_ = null;
      btnGB28181AudioBroadcast.setText("GB28181语音广播");
      btnGB28181AudioBroadcast.setEnabled(false);

      stopGB28181Stream();
      destoryRTPSender();

      if (gb28181_agent_ != null) {
        gb28181_agent_.terminateAllAudioBroadcasts(true);
        gb28181_agent_.terminateAllPlays(true);

        Log.i(TAG, "gb28281_heart_beart_timeout sip stop");
        gb28181_agent_.stop();

        String local_ip_addr = IPAddrUtils.getIpAddress(context_);
        if (local_ip_addr != null && !local_ip_addr.isEmpty() ) {
          Log.i(TAG, "gb28281_heart_beart_timeout get local ip addr: " + local_ip_addr);
          gb28181_agent_.setLocalAddress(local_ip_addr);
        }

        record_executor_.cancel_tasks();

        initPlaybacks(null);
        initRecordDownloads(null);

        Log.i(TAG, "gb28281_heart_beart_timeout sip start");
        gb28181_agent_.start();
      }
    }

  },0);
}

支持移动设备位置(MobilePosition)订阅和通知

java 复制代码 
com.gb.ntsignalling.Device gb_device = new com.gb.ntsignalling.Device("34020000001380000001", "安卓测试设备", Build.MANUFACTURER, Build.MODEL,
                                                                      "宇宙","火星1","火星", true);

if (mLongitude != null && mLatitude != null) {
  com.gb.ntsignalling.DevicePosition device_pos = new com.gb.ntsignalling.DevicePosition();

  device_pos.setTime(mLocationTime);
  device_pos.setLongitude(mLongitude);
  device_pos.setLatitude(mLatitude);
  gb_device.setPosition(device_pos);

  gb_device.setSupportMobilePosition(true); // 设置支持移动位置上报
}

@Override
public void ntsOnDevicePositionRequest(String deviceId, int interval) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      getLocation(context_);

      Log.v(TAG, "ntsOnDevicePositionRequest, deviceId:" + this.device_id_ + ", Longitude:" + mLongitude
            + ", Latitude:" + mLatitude + ", Time:" + mLocationTime);


      if (mLongitude != null && mLatitude != null) {
        com.gb.ntsignalling.DevicePosition device_pos = new com.gb.ntsignalling.DevicePosition();

        device_pos.setTime(mLocationTime);
        device_pos.setLongitude(mLongitude);
        device_pos.setLatitude(mLatitude);

        if (gb28181_agent_ != null ) {
          gb28181_agent_.updateDevicePosition(device_id_, device_pos);
        }
      }
    }

    private String device_id_;
    private int interval_;

    public Runnable set(String device_id, int interval) {
      this.device_id_ = device_id;
      this.interval_ = interval;
      return this;
    }

  }.set(deviceId, interval),0);
}

支持语音广播和语音对讲

typescript 复制代码 
@Override
public void ntsOnAudioBroadcast(String commandFromUserName, String commandFromUserNameAtDomain, String sourceID, String targetID) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "ntsOnAudioBroadcastPlay, fromFromUserName:" + command_from_user_name_
            + " FromUserNameAtDomain:" + command_from_user_name_at_domain_
            + " sourceID:" + source_id_ + ", targetID:" + target_id_);

      stopAudioPlayer();
      destoryRTPReceiver();

      if (gb28181_agent_ != null ) {
        String local_ip_addr = IPAddrUtils.getIpAddress(context_);

        boolean is_tcp = true; // 考虑到跨网段, 默认用TCP传输rtp包
        rtp_receiver_handle_ = lib_player_.CreateRTPReceiver(0);
        if (rtp_receiver_handle_ != 0 ) {
          lib_player_.SetRTPReceiverTransportProtocol(rtp_receiver_handle_, is_tcp?1:0);
          lib_player_.SetRTPReceiverIPAddressType(rtp_receiver_handle_, 0);

          if (0 == lib_player_.CreateRTPReceiverSession(rtp_receiver_handle_, 0) ) {
            int local_port = lib_player_.GetRTPReceiverLocalPort(rtp_receiver_handle_);
            boolean ret = gb28181_agent_.inviteAudioBroadcast(command_from_user_name_,command_from_user_name_at_domain_,
                                                              source_id_, target_id_, "IP4", local_ip_addr, local_port, is_tcp?"TCP/RTP/AVP":"RTP/AVP");

            if (!ret ) {
              destoryRTPReceiver();
              btnGB28181AudioBroadcast.setText("GB28181语音广播");
            }
            else {
              btnGB28181AudioBroadcast.setText("GB28181语音广播呼叫中");
            }
          } else {
            destoryRTPReceiver();
            btnGB28181AudioBroadcast.setText("GB28181语音广播");
          }
        }
      }
    }

    private String command_from_user_name_;
    private String command_from_user_name_at_domain_;
    private String source_id_;
    private String target_id_;

    public Runnable set(String command_from_user_name, String command_from_user_name_at_domain, String source_id, String target_id) {
      this.command_from_user_name_ = command_from_user_name;
      this.command_from_user_name_at_domain_ = command_from_user_name_at_domain;
      this.source_id_ = source_id;
      this.target_id_ = target_id;
      return this;
    }

  }.set(commandFromUserName, commandFromUserNameAtDomain, sourceID, targetID),0);
}

@Override
public void ntsOnInviteAudioBroadcastException(String sourceID, String targetID, String errorInfo) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "ntsOnInviteAudioBroadcastException, sourceID:" + source_id_ + ", targetID:" + target_id_);

      destoryRTPReceiver();
      btnGB28181AudioBroadcast.setText("GB28181语音广播");
    }

    private String source_id_;
    private String target_id_;

    public Runnable set(String source_id, String target_id) {
      this.source_id_ = source_id;
      this.target_id_ = target_id;
      return this;
    }

  }.set(sourceID, targetID),0);
}

@Override
public void ntsOnInviteAudioBroadcastTimeout(String sourceID, String targetID) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "ntsOnInviteAudioBroadcastTimeout, sourceID:" + source_id_ + ", targetID:" + target_id_);

      destoryRTPReceiver();
      btnGB28181AudioBroadcast.setText("GB28181语音广播");
    }

    private String source_id_;
    private String target_id_;

    public Runnable set(String source_id, String target_id) {
      this.source_id_ = source_id;
      this.target_id_ = target_id;
      return this;
    }

  }.set(sourceID, targetID),0);
}

支持历史视音频文件检索、支持历史视音频文件下载和回放

java 复制代码 
/**
 * Author: daniusdk.com
 */
package com.gb.ntsignalling;
 
public interface GBSIPAgent {
    void addDownloadListener(GBSIPAgentDownloadListener downloadListener);
 
    void removeDownloadListener(GBSIPAgentDownloadListener removeListener);
 
    /*
    *响应Invite Download 200 OK
    */
    boolean respondDownloadInviteOK(long id, String deviceId, String startTime, String stopTime, MediaSessionDescription localMediaDescription);
 
    /*
    *响应Invite Download 其他状态码
    */
    boolean respondDownloadInvite(int statusCode, long id, String deviceId, String startTime, String stopTime);
 
    /*
    * 媒体流发送者在文件下载结束后发Message消息通知SIP服务器回文件已发送完成
    * notifyType 必须是"121"
     */
    boolean notifyDownloadMediaStatus(long id, String deviceId, String startTime, String stopTime, String notifyType);
 
    /*
     *终止Download会话
     */
    void terminateDownload(long id, String deviceId, String startTime, String stopTime, boolean isSendBYE);
 
    /*
     *终止所有Download会话
     */
    void terminateAllDownloads(boolean isSendBYE);
 
}

package com.gb.ntsignalling;
 
public interface GBSIPAgent {
    void addPlaybackListener(GBSIPAgentPlaybackListener playbackListener);
 
    void removePlaybackListener(GBSIPAgentPlaybackListener playbackListener);
 
    /*
     *响应Invite Playback 200 OK
     */
    boolean respondPlaybackInviteOK(long id, String deviceId, String startTime, String stopTime, MediaSessionDescription localMediaDescription);
 
    /*
     *响应Invite Playback 其他状态码
     */
    boolean respondPlaybackInvite(int statusCode, long id, String deviceId);
 
    /*
     * 媒体流发送者在回放结束后发Message消息通知SIP服务器回放文件已发送完成
     * notifyType 必须是"121"
     */
    boolean notifyPlaybackMediaStatus(long id, String deviceId, String notifyType);
 
    /*
     *终止Playback会话
     */
    void terminatePlayback(long id, String deviceId, boolean isSendBYE);
 
    /*
     *终止所有Playback会话
     */
    void terminateAllPlaybacks(boolean isSendBYE);
}
 
 
/**
* 信令Playback Listener
*/
package com.gb.ntsignalling;
 
public interface GBSIPAgentPlaybackListener {
    /*
     *收到s=Playback的历史回放Invite
     */
    void ntsOnInvitePlayback(long id, String deviceId, SessionDescription sessionDescription);
 
    /*
     *发送Playback invite response 异常
     */
    void ntsOnPlaybackInviteResponseException(long id, String deviceId, int statusCode, String errorInfo);
 
    /*
     * 收到CANCEL Playback INVITE请求
     */
    void ntsOnCancelPlayback(long id, String deviceId);
 
    /*
     * 收到Ack
     */
    void ntsOnAckPlayback(long id, String deviceId);
 
    /*
    * 播放命令
     */
    void ntsOnPlaybackMANSRTSPPlayCommand(long id, String deviceId);
 
    /*
     * 暂停命令
     */
    void ntsOnPlaybackMANSRTSPPauseCommand(long id, String deviceId);
 
    /*
     * 快进/慢进命令
     */
    void ntsOnPlaybackMANSRTSPScaleCommand(long id, String deviceId, double scale);
 
    /*
     * 随机拖动命令
     */
    void ntsOnPlaybackMANSRTSPSeekCommand(long id, String deviceId, double position_sec);
 
    /*
     * 停止命令
     */
    void ntsOnPlaybackMANSRTSPTeardownCommand(long id, String deviceId);
 
    /*
     * 收到Bye
     */
    void ntsOnByePlayback(long id, String deviceId);
 
    /*
     * 不是在收到BYE Message情况下， 终止Playback
     */
    void ntsOnTerminatePlayback(long id, String deviceId);
 
    /*
     * Playback会话对应的对话终止, 一般不会触发这个回调,目前只有在响应了200K, 但在64*T1时间后还没收到ACK,才可能会出发
    收到这个, 请做相关清理处理
    */
    void ntsOnPlaybackDialogTerminated(long id, String deviceId);
}
 
 
/**
* 部分JNI接口, rtp ps 打包发送等代码C++实现
*/
 
public class SmartPublisherJniV2 {
 
     /**
	 * Open publisher(启动推送实例)
	 *
	 * @param ctx: get by this.getApplicationContext()
	 * 
	 * @param audio_opt:
	 * if 0: 不推送音频
	 * if 1: 推送编码前音频(PCM)
	 * if 2: 推送编码后音频(aac/pcma/pcmu/speex).
	 * 
	 * @param video_opt:
	 * if 0: 不推送视频
	 * if 1: 推送编码前视频(NV12/I420/RGBA8888等格式)
	 * if 2: 推送编码后视频(AVC/HEVC)
	 * if 3: 层叠加模式
	 *
	 * <pre>This function must be called firstly.</pre>
	 *
	 * @return the handle of publisher instance
	 */
    public native long SmartPublisherOpen(Object ctx, int audio_opt, int video_opt,  int width, int height);
 
    
     /**
	 * 设置流类型
	 * @param type: 0:表示 live 流, 1:表示 on-demand 流, SDK默认为0(live流)
	 * 注意: 流类型设置当前仅对GB28181媒体流有效
	 * @return {0} if successful
	 */
    public native int SetStreamType(long handle, int type);
 
 
    /**
	 * 投递视频 on demand包, 当前只用于GB28181推送, 注意ByteBuffer对象必须是DirectBuffer
	 *
	 * @param codec_id: 编码id, 当前支持H264和H265, 1:H264, 2:H265
	 *
	 * @param packet: 视频数据, 包格式请参考H264/H265 Annex B Byte stream format, 例如:
	 *                0x00000001 nal_unit 0x00000001 ...
	 *                H264 IDR: 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....
	 *                H265 IDR: 0x00000001 vps 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....
	 *
	 * @param offset: 偏移量
	 * @param size: packet size
	 * @param pts_us: 时间戳, 单位微秒
	 * @param is_pts_discontinuity: 是否时间戳间断，0:未间断，1:间断
	 * @param is_key: 是否是关键帧, 0:非关键帧, 1:关键帧
	 * @param codec_specific_data: 可选参数，可传null, 对于H264关键帧包, 如果packet不含sps和pps, 可传0x00000001 sps 0x00000001 pps
	 *                    ,对于H265关键帧包, 如果packet不含vps,sps和pps, 可传0x00000001 vps 0x00000001 sps 0x00000001 pps
	 * @param codec_specific_data_size: codec_specific_data size
	 * @param width: 图像宽, 可传0
	 * @param height: 图像高, 可传0
	 *
	 * @return {0} if successful
	 */
	public native int PostVideoOnDemandPacketByteBuffer(long handle, int codec_id,
														ByteBuffer packet, int offset, int size, long pts_us, int is_pts_discontinuity, int is_key,
														byte[] codec_specific_data, int codec_specific_data_size,
									  					int width, int height);
 
	
     /**
	 * 投递音频on demand包, 当前只用于GB28181推送, 注意ByteBuffer对象必须是DirectBuffer
	 *
	 * @param codec_id: 编码id, 当前支持PCMA和AAC, 65536:PCMA, 65538:AAC
	 * @param packet: 音频数据
	 * @param offset：packet偏移量
	 * @param size: packet size
	 * @param pts_us: 时间戳, 单位微秒
	 * @param is_pts_discontinuity: 是否时间戳间断，0:未间断，1:间断
	 * @param codec_specific_data: 如果是AAC的话，需要传 Audio Specific Configuration
	 * @param codec_specific_data_size: codec_specific_data size
	 * @param sample_rate: 采样率
	 * @param channels: 通道数
	 *
	 * @return {0} if successful
	 */
	public native int PostAudioOnDemandPacketByteBuffer(long handle, int codec_id,
														ByteBuffer packet, int offset, int size, long pts_us, int is_pts_discontinuity,
														byte[] codec_specific_data, int codec_specific_data_size,
														int sample_rate, int channels);
																											
	/**
	 * on demand source完成seek后, 请调用
	 * @return {0} if successful
	 */
	public native int OnSeekProcessed(long handle);
 
	/**
	 * 启动 GB28181 媒体流
	 *
	 * @return {0} if successful
	 */
	public native int StartGB28181MediaStream(long handle);
 
 
    /**
	 * 停止 GB28181 媒体流
	 *
	 * @return {0} if successful
	 */
	public native int StopGB28181MediaStream(long handle);
 
    
	/**
     * 关闭推送实例，结束时必须调用close接口释放资源
	 *
	 * @return {0} if successful
	 */
    public native int SmartPublisherClose(long handle);
 
}
 
 
/**
* Listener部分实现代码
*/
 
public class PlaybackListenerImpl implements com.gb.ntsignalling.GBSIPAgentPlaybackListener {
 
    /*
     *收到s=Playback的文件下载Invite
     */
    @Override
    public void ntsOnInvitePlayback(long id, String deviceId, SessionDescription sdp) {
        if (!post_task(new PlaybackListenerImpl.OnInviteTask(this.context_, this.is_exit_, this.senders_map_, deviceId, sdp, id))) {
            Log.e(TAG, "ntsOnInvitePlayback post_task failed, " + RecordSender.make_print_tuple(id, deviceId, sdp.getTime().getStartTime(),  sdp.getTime().getStopTime()));
 
            // 这里不发488, 等待事务超时也可以的
            GBSIPAgent agent = this.context_.get_agent();
            if (agent != null)
                agent.respondPlaybackInvite(488, id, deviceId);
        }
    }
 
    /*
     *发送Playback invite response 异常
     */
    @Override
    public void ntsOnPlaybackInviteResponseException(long id, String deviceId, int statusCode, String errorInfo) {
        Log.i(TAG, "ntsOnPlaybackInviteResponseException, status_code:" + statusCode + ", "
                + RecordSender.make_print_tuple(id, deviceId) + ",  error_info:" + errorInfo);
 
        RecordSender sender = senders_map_.remove(id);
        if (null == sender)
            return;
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * 收到CANCEL Playback INVITE请求
     */
    @Override
    public void ntsOnCancelPlayback(long id, String deviceId) {
        Log.i(TAG, "ntsOnCancelPlayback, " + RecordSender.make_print_tuple(id, deviceId));
 
        RecordSender sender = senders_map_.remove(id);
        if (null == sender)
            return;
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * 收到Ack
     */
    @Override
    public void ntsOnAckPlayback(long id, String deviceId) {
        Log.i(TAG, "ntsOnAckPlayback, "+ RecordSender.make_print_tuple(id, deviceId));
 
        RecordSender sender = senders_map_.get(id);
        if (null == sender) {
            Log.e(TAG, "ntsOnAckPlayback get sender is null, " + RecordSender.make_print_tuple(id, deviceId));
 
            GBSIPAgent agent = this.context_.get_agent();
            if (agent != null)
                agent.terminatePlayback(id, deviceId, false);
 
            return;
        }
 
        PlaybackListenerImpl.StartTask task = new PlaybackListenerImpl.StartTask(sender, this.senders_map_);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * 收到Bye
     */
    @Override
    public void ntsOnByePlayback(long id, String deviceId) {
        Log.i(TAG, "ntsOnByePlayback, "+ RecordSender.make_print_tuple(id, deviceId));
 
        RecordSender sender = this.senders_map_.remove(id);
        if (null == sender)
            return;
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * 播放命令
     */
    @Override
    public void ntsOnPlaybackMANSRTSPPlayCommand(long id, String deviceId) {
        RecordSender sender = this.senders_map_.get(id);
        if (null == sender) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPPlayCommand can not get sender " + RecordSender.make_print_tuple(id, deviceId));
            return;
        }
 
        sender.post_play_command();
 
        Log.i(TAG, "ntsOnPlaybackMANSRTSPPlayCommand " + RecordSender.make_print_tuple(id, deviceId));
    }
 
    /*
     * 暂停命令
     */
    @Override
    public void ntsOnPlaybackMANSRTSPPauseCommand(long id, String deviceId) {
        RecordSender sender = this.senders_map_.get(id);
        if (null == sender) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPPauseCommand can not get sender " + RecordSender.make_print_tuple(id, deviceId));
            return;
        }
 
        sender.post_pause_command();
 
        Log.i(TAG, "ntsOnPlaybackMANSRTSPPauseCommand " + RecordSender.make_print_tuple(id, deviceId));
    }
 
    /*
     * 快进/慢进命令
     */
    @Override
    public void ntsOnPlaybackMANSRTSPScaleCommand(long id, String deviceId, double scale) {
        if (scale < 0.01) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPScaleCommand invalid scale:" + scale  + " " + RecordSender.make_print_tuple(id, deviceId));
            return;
        }
 
        RecordSender sender = this.senders_map_.get(id);
        if (null == sender) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPScaleCommand can not get sender, scale:" + scale  + " " + RecordSender.make_print_tuple(id, deviceId));
            return;
        }
 
        sender.post_scale_command(scale);
 
        Log.i(TAG, "ntsOnPlaybackMANSRTSPScaleCommand, scale:" + scale + " " + RecordSender.make_print_tuple(id, deviceId));
    }
 
    /*
     * 随机拖动命令
     */
    @Override
    public void ntsOnPlaybackMANSRTSPSeekCommand(long id, String device_id, double position_sec) {
        if (position_sec < 0.0) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPSeekCommand invalid seek pos:" + position_sec  + ", " + RecordSender.make_print_tuple(id, device_id));
            return;
        }
 
        RecordSender sender = this.senders_map_.get(id);
        if (null == sender) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPSeekCommand can not get sender " + RecordSender.make_print_tuple(id, device_id));
            return;
        }
 
        long offset_ms = sender.get_file_start_time_offset_ms();
        position_sec += (offset_ms/1000.0);
 
        sender.post_seek_command(position_sec);
 
        Log.i(TAG, "ntsOnPlaybackMANSRTSPSeekCommand seek pos:" + RecordSender.out_point_3(position_sec) + "s, " + RecordSender.make_print_tuple(id, device_id));
    }
 
    /*
     * 停止命令
     */
    @Override
    public void ntsOnPlaybackMANSRTSPTeardownCommand(long id, String device_id) {
        CallTerminatePlaybackTask call_terminate_task =  new CallTerminatePlaybackTask(this.context_, id, device_id, true);
        post_task(call_terminate_task);
 
        RecordSender sender = this.senders_map_.remove(id);
        if (null == sender) {
            Log.w(TAG, "ntsOnPlaybackMANSRTSPTeardownCommand can not remove sender " + RecordSender.make_print_tuple(id, device_id));
            return;
        }
 
        Log.i(TAG, "ntsOnPlaybackMANSRTSPTeardownCommand " + RecordSender.make_print_tuple(id, device_id));
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * 不是在收到BYE Message情况下， 终止Playback
     */
    @Override
    public void ntsOnTerminatePlayback(long id, String deviceId) {
        Log.i(TAG, "ntsOnTerminatePlayback, "+ RecordSender.make_print_tuple(id, deviceId));
 
        RecordSender sender = this.senders_map_.remove(id);
        if (null == sender)
            return;
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * Playback会话对应的对话终止, 一般不会触发这个回调,目前只有在响应了200K, 但在64*T1时间后还没收到ACK,才可能会出发
    收到这个, 请做相关清理处理
    */
    @Override
    public void ntsOnPlaybackDialogTerminated(long id, String deviceId) {
        Log.i(TAG, "ntsOnPlaybackDialogTerminated, "+ RecordSender.make_print_tuple(id, deviceId));
 
        RecordSender sender = this.senders_map_.remove(id);
        if (null == sender)
            return;
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
}

支持云台控制和预置位查询

typescript 复制代码 
@Override
public void ntsOnDevicePresetQueryCommand(String fromUserName, String fromUserNameAtDomain, String sn, String deviceId) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "DaniuSDK ntsOnDevicePresetQueryCommand from_user_name:" + from_user_name_ + ", sn:" + sn_ + ", device_id:" + device_id_);

      List<com.gb.ntsignalling.PresetItem> preset_list = new LinkedList<>();


      preset_list.add(new com.gb.ntsignalling.PresetItem("1", "Android PreSet1"));
      preset_list.add(new com.gb.ntsignalling.PresetItem("2", "Android PreSet2"));

      if (gb28181_agent_ != null )
        gb28181_agent_.respondDevicePresetQueryCommand(this.from_user_name_, this.from_user_name_at_domain_, this.sn_, this.device_id_, preset_list);
    }

    private String from_user_name_;
    private String from_user_name_at_domain_;
    private String sn_;
    private String device_id_;

    public Runnable set(String from_user_name, String from_user_name_at_domain,String sn, String device_id) {
      this.from_user_name_ = from_user_name;
      this.from_user_name_at_domain_ = from_user_name_at_domain;
      this.sn_ = sn;
      this.device_id_ = device_id;
      return this;
    }

  }.set(fromUserName, fromUserNameAtDomain, sn, deviceId),0);
}

@Override
public void ntsOnDeviceControlPTZCmd(String deviceId, String typeValue) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "DaniuSDK ntsOnDeviceControlPTZCmd device_id:" + device_id_ + " PTZType:" + ptz_type_);

      if (null == ptz_type_)
        return;

      ptz_type_ = ptz_type_.trim();
      if (ptz_type_.length() != 16)
        return;

      int instruction = hexStringToInt(ptz_type_.substring(6, 8));
      int combination_code2 = hexStringToInt(ptz_type_.substring(12, 14));

      //Android平台GB28181设备接入端，针对性的解析处理即可，这里不再赘述

      private String device_id_;
      private String ptz_type_;

      public Runnable set(String device_id, String ptz_type) {
        this.device_id_ = device_id;
        this.ptz_type_ = ptz_type;
        return this;
      }

    }.set(deviceId, typeValue),0);
  }

[实时水印]支持动态文字水印、png水印

ini 复制代码 
watermarkSelctor = (Spinner) findViewById(R.id.watermarkSelctor);

final String[] watermarks = new String[]{"图片水印", "全部水印", "文字水印", "不加水印"};

ArrayAdapter<String> adapterWatermark = new ArrayAdapter<String>(this,
                                                                 android.R.layout.simple_spinner_item, watermarks);

adapterWatermark.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);

watermarkSelctor.setAdapter(adapterWatermark);

watermarkSelctor.setSelection(3,true);
watemarkType = 3;   //默认不加水印

watermarkSelctor.setOnItemSelectedListener(new AdapterView.OnItemSelectedListener() {

  @Override
  public void onItemSelected(AdapterView<?> parent, View view,
                             int position, long id) {

    watemarkType = position;

    Log.i(TAG, "[水印类型]Currently choosing: " + watermarks[position] + ", watemarkType: " + watemarkType);

    if (layer_post_thread_ != null) {
      layer_post_thread_.enableText(isHasTextWatermark());
      layer_post_thread_.enablePicture(isHasPictureWatermark());
    }
  }

  @Override
  public void onNothingSelected(AdapterView<?> parent) {

  }
});

[实时快照]支持实时快照

typescript 复制代码 
class ButtonCaptureImageListener implements View.OnClickListener {
  @SuppressLint("SimpleDateFormat")
  public void onClick(View v) {
    if(isPushingRtmp || isRecording || isRTSPPublisherRunning || isGB28181StreamRunning)
    {
      String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
      String imageFileName = "dn_" + timeStamp;    //创建以时间命名的文件名称

      String imagePath = imageSavePath + "/" + imageFileName + ".png";

      Log.i(TAG, "imagePath:" + imagePath);

      libPublisher.SmartPublisherSaveCurImage(publisherHandle, imagePath);
    }
    else
    {
      Log.e(TAG, "快照失败，请确保在推送、录像或内置RTSP服务发布状态..");
    }
  }
}

[降噪]支持环境音、手机干扰等引起的噪音降噪处理、自动增益、VAD检测

ini 复制代码 
boolean is_noise_suppression = true;
libPublisher.SmartPublisherSetNoiseSuppression(publisherHandle, is_noise_suppression ? 1 : 0);

boolean is_agc = false;
libPublisher.SmartPublisherSetAGC(publisherHandle, is_agc ? 1 : 0);

int echo_cancel_delay = 0;
libPublisher.SmartPublisherSetEchoCancellation(publisherHandle, 1, echo_cancel_delay);

外部编码前后视频数据对接

  • 编码前数据（目前支持的有YV12/NV21/NV12/I420/RGB24/RGBA32/RGB565等数据类型），其中，Android平台前后摄像头数据，或者屏幕数据，或者Unity拿到的数据，均属编码前数据；
  • 编码后数据（如无人机等264/HEVC数据，或者本地解析的MP4音视频数据）；

[扩展录像功能]支持和录像SDK组合使用，录像相关功能

ini 复制代码 
class ButtonStartRecorderListener implements View.OnClickListener {
  public void onClick(View v) {
    if (isRecording) {
      stopRecorder();

      if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {
        ConfigControlEnable(true);
      }

      btnStartRecorder.setText("实时录像");

      btnPauseRecorder.setText("暂停录像");
      btnPauseRecorder.setEnabled(false);
      isPauseRecording = true;

      return;
    }

    Log.i(TAG, "onClick start recorder..");

    if (libPublisher == null)
      return;

    if (!isPushingRtmp && !isRTSPPublisherRunning&& !isGB28181StreamRunning) {
      InitAndSetConfig();
    }

    ConfigRecorderParam();

    int startRet = libPublisher.SmartPublisherStartRecorder(publisherHandle);
    if (startRet != 0) {
      if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {
        if (publisherHandle != 0) {
          long handle = publisherHandle;
          publisherHandle = 0;
          libPublisher.SmartPublisherClose(handle);
        }
      }

      Log.e(TAG, "Failed to start recorder.");
      return;
    }

    if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {
      CheckInitAudioRecorder();
      ConfigControlEnable(false);
    }

    startLayerPostThread();

    btnStartRecorder.setText("停止录像");
    isRecording = true;

    btnPauseRecorder.setEnabled(true);
    isPauseRecording = true;
  }
}

class ButtonPauseRecorderListener implements View.OnClickListener {
  public void onClick(View v) {
    if (isRecording) {

      if(isPauseRecording)
      {
        int ret = libPublisher.SmartPublisherPauseRecorder(publisherHandle, 1);

        if (ret == 0)
        {
          isPauseRecording = false;
          btnPauseRecorder.setText("恢复录像");
        }
        else if(ret == 3)
        {
          Log.e(TAG, "Pause recorder failed, please re-try again..");
        }
        else
        {
          Log.e(TAG, "Pause recorder failed..");
        }
      }
      else
      {
        int ret = libPublisher.SmartPublisherPauseRecorder(publisherHandle, 0);

        if (ret == 0)
        {
          isPauseRecording = true;
          btnPauseRecorder.setText("暂停录像");
        }
        else if(ret == 3)
        {
          Log.e(TAG, "Resume recorder failed, please re-try again..");
        }
        else
        {
          Log.e(TAG, "Resume recorder failed..");
        }
      }
    }
  }
}

总结

Android平台GB28181设备接入侧模块，如果需要做的更好，上述提到的技术层面的问题解决了还不够，还需要针对各类国标平台适配对接，只有这样，才能更好的为执法记录仪、智能安全帽、智能监控、智慧零售、智慧教育、远程办公、明厨亮灶、智慧交通、智慧工地、雪亮工程、平安乡村、生产运输、车载终端等场景服务。

​

相关推荐
dvlinker19 小时前
【音视频开发】使用支持硬件加速的D3D11绘图遇到的绘图失败与绘图崩溃问题的记录与总结
音视频开发·c/c++·视频播放·d3d11·d3d11绘图模式
aqi004 天前
FFmpeg开发笔记（五十八）把32位采样的MP3转换为16位的PCM音频
ffmpeg·音视频·直播·流媒体
音视频牛哥6 天前
Android平台GB28181实时回传流程和技术实现
音视频开发·视频编码·直播
音视频牛哥8 天前
RTMP、RTSP直播播放器的低延迟设计探讨
音视频开发·视频编码·直播
音视频牛哥12 天前
电脑共享同屏的几种方法分享
音视频开发·视频编码·直播
aqi0014 天前
FFmpeg开发笔记（五十四）使用EasyPusher实现移动端的RTSP直播
android·ffmpeg·音视频·直播·流媒体
aqi0015 天前
FFmpeg开发笔记（五十三）移动端的国产直播录制工具EasyPusher
android·ffmpeg·音视频·直播·流媒体
加油吧x青年16 天前
Web端开启直播技术方案分享
前端·webrtc·直播
aqi001 个月前
FFmpeg开发笔记（五十二）移动端的国产视频播放器GSYVideoPlayer
android·ffmpeg·音视频·直播·流媒体
热门推荐
01组基轨迹建模 GBTM的介绍与实现（Stata 或 R）02【经验分享】Ubuntu22.04安装微信(linux官方版)032024年高教社杯数学建模国赛C题超详细解题思路分析04分享几个惊艳的 Three.js 个人站点05安卓系列机型永久去除data分区加密 详细步骤解析06【2024数模国赛赛题思路公开】国赛B题思路丨附可运行代码丨无偿自提07springer 在线投稿编译踩坑08Ubuntu24.04安装中文输入法09【2024高教社杯全国大学生数学建模竞赛】B题 生产过程中的决策问题——解题思路 代码 论文10CTF网络安全大赛简单的web抓包题目：HEADache