Android平台GB28181设备接入模块开发填坑指南

音视频牛哥2023-11-27 23:01

​技术背景

为什么要开发Android平台GB28181设备接入模块？这个问题不再赘述，在做Android平台GB28181客户端的时候，媒体数据这块，我们已经有了很好的积累，因为在此之前，我们就开发了非常成熟的RTMP推送、轻量级RTSP服务、录像模块、针对音视频的对接处理单元。这让我们在做Android平台GB28181设备接入模块的时候，可以有更多的精力在信令交互和国标平台对接。

好多开发者会觉得，GB28181设备接入模块有啥好做的？不就找个开源的SIP信令，视频编码ps打包下投递到国标平台就好了吗？

事实上，当回头看看开发的功能时，就会觉得，一两个月的东西，仅就可以作为项目交付或demo使用，并不会有多大的商业价值，因为需要解决的问题实在太多了。

  • [视频格式]H.264/H.265(Android H.265硬编码)；
  • [音频格式]G.711 A律、AAC；
  • [音量调节]Android平台采集端支持实时音量调节；
  • [H.264硬编码]支持H.264特定机型硬编码；
  • [H.265硬编码]支持H.265特定机型硬编码；
  • [软硬编码参数配置]支持gop间隔、帧率、bit-rate设置；
  • [软编码参数配置]支持软编码profile、软编码速度、可变码率设置；
  • 支持纯视频、音视频PS打包传输；
  • 支持RTP OVER UDP和RTP OVER TCP被动模式（TCP媒体流传输客户端）；
  • 支持信令通道网络传输协议TCP/UDP设置；
  • 支持注册、注销，支持注册刷新及注册有效期设置；
  • 支持设备目录查询应答；
  • 支持心跳机制，支持心跳间隔、心跳检测次数设置；
  • 支持移动设备位置(MobilePosition)订阅和通知；
  • 支持语音广播；
  • 支持语音对讲；
  • 支持历史视音频文件检索；
  • 支持历史视音频文件下载；
  • 支持历史视音频文件回放；
  • 支持云台控制和预置位查询；
  • [实时水印]支持动态文字水印、png水印；
  • [镜像]Android平台支持前置摄像头实时镜像功能；
  • [实时静音]支持实时静音/取消静音；
  • [实时快照]支持实时快照；
  • [降噪]支持环境音、手机干扰等引起的噪音降噪处理、自动增益、VAD检测；
  • [外部编码前视频数据对接]支持YUV数据对接；
  • [外部编码前音频数据对接]支持PCM对接；
  • [外部编码后视频数据对接]支持外部H.264数据对接；
  • [外部编码后音频数据对接]外部AAC数据对接；
  • [扩展录像功能]支持和录像模块组合使用，录像相关功能。

技术实现

​

[视频格式]H.264/H.265(Android H.265硬编码)

目前GB28181-2022已经明确表示支持H.265，GB28181设备接入这块，如果需要有好的画质，编码算法这块，一定需要做好，Android端除了低分辨率软编外，超过1280*720，一般建议硬编码。

java 复制代码 
	 /**
	  * Set Video H.264 HW Encoder, if support HW encoder, it will return 0(设置H.264硬编码)
	  * 
	  * @param kbps: the kbps of different resolution.
	  * 
	  * @return {0} if successful
	  */
   public native int SetSmartPublisherVideoHWEncoder(long handle, int kbps);

	/**
	 * Set Video H.265(hevc) hardware encoder, if support H.265(hevc) hardware encoder, it will return 0(设置H.265硬编码)
	 *
	 * @param kbps: the kbps of different resolution.
	 *
	 * @return {0} if successful
	 */
	public native int SetSmartPublisherVideoHevcHWEncoder(long handle, int kbps);

硬编码参数设置

java 复制代码 
	/*
	* 设置视频硬编码码率控制模式
	* @param hw_bitrate_mode: -1表示使用默认值, 不设置也会使用默认值, 0:CQ, 1:VBR, 2:CBR, 3:CBR_FD, 请参考:android.media.MediaCodecInfo.EncoderCapabilities
	* 注意硬编码和手机硬件有关，多数手机只支持部分码率模式, 另外硬编码设备差异很大，不同设备同一码率控制模式效果可能不一样
	* @return {0} if successful
	*/
	public native int SetVideoHWEncoderBitrateMode(long handle, int hw_bitrate_mode);


	/*
	 * 设置视频硬编码复杂度, 安卓5.0及以上支持
	 * @param hw_complexity: -1表示不设置, 请参考:android.media.MediaCodecInfo.EncoderCapabilities.getComplexityRange() 和 android.media.MediaFormat.KEY_COMPLEXITY
	 * 注意硬编码和手机硬件有关，部分手机可能不支持此设置
	 * @return {0} if successful
	 */
	public native int SetVideoHWEncoderComplexity(long handle, int hw_complexity);

	/*
	 * 设置视频硬编码质量, 安卓9及以上支持, 仅当硬编码器码率控制模式(BitrateMode)是CQ(constant-quality mode)时才有效
	 * @param hw_quality: -1表示不设置, 请参考:android.media.MediaCodecInfo.EncoderCapabilities.getQualityRange() 和 android.media.MediaFormat.KEY_QUALITY
	 * 注意硬编码和手机硬件有关，部分手机可能不支持此设置
	 * @return {0} if successful
	 */
	public native int SetVideoHWEncoderQuality(long handle, int hw_quality);

	/*
	 * 设置H.264硬编码Profile, 安卓7及以上支持
	 * @param hw_avc_profile: 0表示使用默认值, 0x01: Baseline, 0x02: Main, 0x08: High, 0x10000: ConstrainedBaseline, 0x80000: ConstrainedHigh;
	 * 注意: ConstrainedBaseline 和 ConstrainedHigh 可能多数设备不支持,
	 * H.264推荐使用 High 或者 ConstrainedHigh, 如果您使用的手机硬解码解不了，那还是设置Baseline
	 * 如果设置的Profile硬编码器不支持，应编码器会使用默认值
	 * 具体参考:android.media.MediaCodecInfo.CodecProfileLevel
	 * @return {0} if successful
	 */
	public native int SetAVCHWEncoderProfile(long handle, int hw_avc_profile);

	/*
	 * 设置H.264硬编码Level, 这个只有在设置了Profile的情况下才有效, 安卓7及以上支持
	 * @param hw_avc_level: 0表示使用默认值, 0x100: Level3, 0x200: Level3.1, 0x400: Level3.2,
	 * 0x800: Level4, 0x1000: Level4.1, 0x2000: Level4.2,
	 * 0x4000: Level5, 0x8000: Level5.1,  0x10000: Level5.2,
	 * 0x20000: Level6, 0x40000: Level6.1,  0x80000: Level6.2,
	 * 如果设置的level太高硬编码器不支持，SDK内部会做相应调整
	 * 注意: 640*480@25fps最小支持的是Level3, 720p最小支持的是Level3.1, 1080p最小支持的是Level4
	 * 具体参考:android.media.MediaCodecInfo.CodecProfileLevel
	 * @return {0} if successful
	 */
	public native int SetAVCHWEncoderLevel(long handle, int hw_avc_level);

	/*
	 * 设置视频硬编码最大码率, 安卓没有相关文档说明, 所以不建议设置,
	 * @param hw_max_bitrate: 每秒最大码率, 单位bps
	 * @return {0} if successful
	 */
	public native int SetVideoHWEncoderMaxBitrate(long handle, long hw_max_bitrate);

[音频格式]G.711 A律、AAC

java 复制代码 
    /**
     * Set audio encoder type(设置音频编码类型)
     * 
     * @param type: if with 1:AAC, if with 2: SPEEX, if with 3: PCMA
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetAudioCodecType(long handle, int type);


	/**
	 * Set audio encoder bit-rate(设置音频编码码率), 当前只对AAC编码有效
	 *
	 * @param kbit_rate: 码率(单位是kbps), 如果是0的话将使用默认码率, 必须大于等于0
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetAudioBitRate(long handle, int kbit_rate);

[音量调节]Android平台采集端支持实时音量调节

java 复制代码 
	/**
	 * 设置输入音量, 这个接口一般不建议调用, 在一些特殊情况下可能会用, 一般不建议放大音量
	 *
	 * @param index: 一般是0和1, 如果没有混音的只用0, 有混音的话, 0,1分别设置音量
	 *
	 * @param volume: 音量，默认是1.0，范围是[0.0, 5.0], 设置成0静音, 1音量不变
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetInputAudioVolume(long handle, int index, float volume);

[软硬编码参数配置]支持gop间隔、帧率、bit-rate设置，支持软编码profile、软编码速度、可变码率设置

java 复制代码 
	/**
	 * Set software encode vbr mode(软编码可变码率).
	 *
	 * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
	 *
	 * is_enable_vbr: if 0: NOT enable vbr mode, 1: enable vbr
	 *
	 * video_quality: vbr video quality, range with (1,50), default 23
	 *
	 * vbr_max_kbitrate: vbr max encode bit-rate(kbps)
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherSetSwVBRMode(long handle, int is_enable_vbr, int video_quality, int vbr_max_kbitrate);

    /**
     * Set gop interval(设置I帧间隔)
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * gopInterval: encode I frame interval, the value always > 0
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetGopInterval(long handle, int gopInterval);
    
    /**
     * Set software encode video bit-rate(设置视频软编码bit-rate)
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * avgBitRate: average encode bit-rate(kbps)
     * 
     * maxBitRate: max encode bit-rate(kbps)
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetSWVideoBitRate(long handle, int avgBitRate, int maxBitRate);
    
    /**
     * Set fps(设置帧率)
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * fps: the fps of video, range with (1,25).
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetFPS(long handle, int fps);
    
	/**
     * Set software video encoder profile(设置视频编码profile).
     *
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     *
     * profile: the software video encoder profile, range with (1,3).
     * 
     * 1: baseline profile
     * 2: main profile
     * 3: high profile
     *
     * @return {0} if successful
     */
    public native int SmartPublisherSetSWVideoEncoderProfile(long handle, int profile);
    
    
    /**
     * Set software video encoder speed(设置视频软编码编码速度)
     * 
     * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>
     * 
     * @param speed: range with(1, 6), the default speed is 6. 
     * 
     * if with 1, CPU is lowest.
     * if with 6, CPU is highest.
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetSWVideoEncoderSpeed(long handle, int speed);

信令通道网络传输协议TCP/UDP设置

ini 复制代码 
gb28181_agent_.setTransportProtocol(gb28181_sip_trans_protocol_==0?"UDP":"TCP");

支持注册、注销，支持注册刷新及注册有效期设置

typescript 复制代码 
private int gb28181_reg_expired_           = 3600; // 注册有效期时间最小3600秒

// GB28181配置
gb28181_agent_.config(gb28181_reg_expired_, gb28181_heartbeat_interval_, gb28181_heartbeat_count_);

@Override
public void ntsRegisterOK(String dateString) {
  Log.i(TAG, "ntsRegisterOK Date: " + (dateString!= null? dateString : ""));
}

@Override
public void ntsRegisterTimeout() {
  Log.e(TAG, "ntsRegisterTimeout");
}

@Override
public void ntsRegisterTransportError(String errorInfo) {
  Log.e(TAG, "ntsRegisterTransportError error:" + (errorInfo != null?errorInfo :""));
}

支持心跳机制，支持心跳间隔、心跳检测次数设置

scss 复制代码 
private int gb28181_heartbeat_interval_    = 20; // 心跳间隔GB28181默认是60, 目前调整到20秒
private int gb28181_heartbeat_count_       = 3; // 心跳间隔3次失败，表示和服务器断开了

@Override
public void ntsOnHeartBeatException(int exceptionCount,  String lastExceptionInfo) {
  Log.e(TAG, "ntsOnHeartBeatException heart beat timeout count reached, count:" + exceptionCount+
        ", exception info:" + (lastExceptionInfo!=null?lastExceptionInfo:""));

  // 停止信令, 然后重启
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "gb28281_heart_beart_timeout");

      record_executor_.cancel_tasks();

      stopRecordDownloads(true);
      stopPlaybacks(true);

      stopAudioPlayer();
      destoryRTPReceiver();

      gb_broadcast_source_id_ = null;
      gb_broadcast_target_id_ = null;
      btnGB28181AudioBroadcast.setText("GB28181语音广播");
      btnGB28181AudioBroadcast.setEnabled(false);

      stopGB28181Stream();
      destoryRTPSender();

      if (gb28181_agent_ != null) {
        gb28181_agent_.terminateAllAudioBroadcasts(true);
        gb28181_agent_.terminateAllPlays(true);

        Log.i(TAG, "gb28281_heart_beart_timeout sip stop");
        gb28181_agent_.stop();

        String local_ip_addr = IPAddrUtils.getIpAddress(context_);
        if (local_ip_addr != null && !local_ip_addr.isEmpty() ) {
          Log.i(TAG, "gb28281_heart_beart_timeout get local ip addr: " + local_ip_addr);
          gb28181_agent_.setLocalAddress(local_ip_addr);
        }

        record_executor_.cancel_tasks();

        initPlaybacks(null);
        initRecordDownloads(null);

        Log.i(TAG, "gb28281_heart_beart_timeout sip start");
        gb28181_agent_.start();
      }
    }

  },0);
}

支持移动设备位置(MobilePosition)订阅和通知

java 复制代码 
com.gb.ntsignalling.Device gb_device = new com.gb.ntsignalling.Device("34020000001380000001", "安卓测试设备", Build.MANUFACTURER, Build.MODEL,
                                                                      "宇宙","火星1","火星", true);

if (mLongitude != null && mLatitude != null) {
  com.gb.ntsignalling.DevicePosition device_pos = new com.gb.ntsignalling.DevicePosition();

  device_pos.setTime(mLocationTime);
  device_pos.setLongitude(mLongitude);
  device_pos.setLatitude(mLatitude);
  gb_device.setPosition(device_pos);

  gb_device.setSupportMobilePosition(true); // 设置支持移动位置上报
}

@Override
public void ntsOnDevicePositionRequest(String deviceId, int interval) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      getLocation(context_);

      Log.v(TAG, "ntsOnDevicePositionRequest, deviceId:" + this.device_id_ + ", Longitude:" + mLongitude
            + ", Latitude:" + mLatitude + ", Time:" + mLocationTime);


      if (mLongitude != null && mLatitude != null) {
        com.gb.ntsignalling.DevicePosition device_pos = new com.gb.ntsignalling.DevicePosition();

        device_pos.setTime(mLocationTime);
        device_pos.setLongitude(mLongitude);
        device_pos.setLatitude(mLatitude);

        if (gb28181_agent_ != null ) {
          gb28181_agent_.updateDevicePosition(device_id_, device_pos);
        }
      }
    }

    private String device_id_;
    private int interval_;

    public Runnable set(String device_id, int interval) {
      this.device_id_ = device_id;
      this.interval_ = interval;
      return this;
    }

  }.set(deviceId, interval),0);
}

支持语音广播和语音对讲

typescript 复制代码 
@Override
public void ntsOnAudioBroadcast(String commandFromUserName, String commandFromUserNameAtDomain, String sourceID, String targetID) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "ntsOnAudioBroadcastPlay, fromFromUserName:" + command_from_user_name_
            + " FromUserNameAtDomain:" + command_from_user_name_at_domain_
            + " sourceID:" + source_id_ + ", targetID:" + target_id_);

      stopAudioPlayer();
      destoryRTPReceiver();

      if (gb28181_agent_ != null ) {
        String local_ip_addr = IPAddrUtils.getIpAddress(context_);

        boolean is_tcp = true; // 考虑到跨网段, 默认用TCP传输rtp包
        rtp_receiver_handle_ = lib_player_.CreateRTPReceiver(0);
        if (rtp_receiver_handle_ != 0 ) {
          lib_player_.SetRTPReceiverTransportProtocol(rtp_receiver_handle_, is_tcp?1:0);
          lib_player_.SetRTPReceiverIPAddressType(rtp_receiver_handle_, 0);

          if (0 == lib_player_.CreateRTPReceiverSession(rtp_receiver_handle_, 0) ) {
            int local_port = lib_player_.GetRTPReceiverLocalPort(rtp_receiver_handle_);
            boolean ret = gb28181_agent_.inviteAudioBroadcast(command_from_user_name_,command_from_user_name_at_domain_,
                                                              source_id_, target_id_, "IP4", local_ip_addr, local_port, is_tcp?"TCP/RTP/AVP":"RTP/AVP");

            if (!ret ) {
              destoryRTPReceiver();
              btnGB28181AudioBroadcast.setText("GB28181语音广播");
            }
            else {
              btnGB28181AudioBroadcast.setText("GB28181语音广播呼叫中");
            }
          } else {
            destoryRTPReceiver();
            btnGB28181AudioBroadcast.setText("GB28181语音广播");
          }
        }
      }
    }

    private String command_from_user_name_;
    private String command_from_user_name_at_domain_;
    private String source_id_;
    private String target_id_;

    public Runnable set(String command_from_user_name, String command_from_user_name_at_domain, String source_id, String target_id) {
      this.command_from_user_name_ = command_from_user_name;
      this.command_from_user_name_at_domain_ = command_from_user_name_at_domain;
      this.source_id_ = source_id;
      this.target_id_ = target_id;
      return this;
    }

  }.set(commandFromUserName, commandFromUserNameAtDomain, sourceID, targetID),0);
}

@Override
public void ntsOnInviteAudioBroadcastException(String sourceID, String targetID, String errorInfo) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "ntsOnInviteAudioBroadcastException, sourceID:" + source_id_ + ", targetID:" + target_id_);

      destoryRTPReceiver();
      btnGB28181AudioBroadcast.setText("GB28181语音广播");
    }

    private String source_id_;
    private String target_id_;

    public Runnable set(String source_id, String target_id) {
      this.source_id_ = source_id;
      this.target_id_ = target_id;
      return this;
    }

  }.set(sourceID, targetID),0);
}

@Override
public void ntsOnInviteAudioBroadcastTimeout(String sourceID, String targetID) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "ntsOnInviteAudioBroadcastTimeout, sourceID:" + source_id_ + ", targetID:" + target_id_);

      destoryRTPReceiver();
      btnGB28181AudioBroadcast.setText("GB28181语音广播");
    }

    private String source_id_;
    private String target_id_;

    public Runnable set(String source_id, String target_id) {
      this.source_id_ = source_id;
      this.target_id_ = target_id;
      return this;
    }

  }.set(sourceID, targetID),0);
}

支持历史视音频文件检索、支持历史视音频文件下载和回放

java 复制代码 
/**
 * Author: daniusdk.com
 */
package com.gb.ntsignalling;
 
public interface GBSIPAgent {
    void addDownloadListener(GBSIPAgentDownloadListener downloadListener);
 
    void removeDownloadListener(GBSIPAgentDownloadListener removeListener);
 
    /*
    *响应Invite Download 200 OK
    */
    boolean respondDownloadInviteOK(long id, String deviceId, String startTime, String stopTime, MediaSessionDescription localMediaDescription);
 
    /*
    *响应Invite Download 其他状态码
    */
    boolean respondDownloadInvite(int statusCode, long id, String deviceId, String startTime, String stopTime);
 
    /*
    * 媒体流发送者在文件下载结束后发Message消息通知SIP服务器回文件已发送完成
    * notifyType 必须是"121"
     */
    boolean notifyDownloadMediaStatus(long id, String deviceId, String startTime, String stopTime, String notifyType);
 
    /*
     *终止Download会话
     */
    void terminateDownload(long id, String deviceId, String startTime, String stopTime, boolean isSendBYE);
 
    /*
     *终止所有Download会话
     */
    void terminateAllDownloads(boolean isSendBYE);
 
}

package com.gb.ntsignalling;
 
public interface GBSIPAgent {
    void addPlaybackListener(GBSIPAgentPlaybackListener playbackListener);
 
    void removePlaybackListener(GBSIPAgentPlaybackListener playbackListener);
 
    /*
     *响应Invite Playback 200 OK
     */
    boolean respondPlaybackInviteOK(long id, String deviceId, String startTime, String stopTime, MediaSessionDescription localMediaDescription);
 
    /*
     *响应Invite Playback 其他状态码
     */
    boolean respondPlaybackInvite(int statusCode, long id, String deviceId);
 
    /*
     * 媒体流发送者在回放结束后发Message消息通知SIP服务器回放文件已发送完成
     * notifyType 必须是"121"
     */
    boolean notifyPlaybackMediaStatus(long id, String deviceId, String notifyType);
 
    /*
     *终止Playback会话
     */
    void terminatePlayback(long id, String deviceId, boolean isSendBYE);
 
    /*
     *终止所有Playback会话
     */
    void terminateAllPlaybacks(boolean isSendBYE);
}
 
 
/**
* 信令Playback Listener
*/
package com.gb.ntsignalling;
 
public interface GBSIPAgentPlaybackListener {
    /*
     *收到s=Playback的历史回放Invite
     */
    void ntsOnInvitePlayback(long id, String deviceId, SessionDescription sessionDescription);
 
    /*
     *发送Playback invite response 异常
     */
    void ntsOnPlaybackInviteResponseException(long id, String deviceId, int statusCode, String errorInfo);
 
    /*
     * 收到CANCEL Playback INVITE请求
     */
    void ntsOnCancelPlayback(long id, String deviceId);
 
    /*
     * 收到Ack
     */
    void ntsOnAckPlayback(long id, String deviceId);
 
    /*
    * 播放命令
     */
    void ntsOnPlaybackMANSRTSPPlayCommand(long id, String deviceId);
 
    /*
     * 暂停命令
     */
    void ntsOnPlaybackMANSRTSPPauseCommand(long id, String deviceId);
 
    /*
     * 快进/慢进命令
     */
    void ntsOnPlaybackMANSRTSPScaleCommand(long id, String deviceId, double scale);
 
    /*
     * 随机拖动命令
     */
    void ntsOnPlaybackMANSRTSPSeekCommand(long id, String deviceId, double position_sec);
 
    /*
     * 停止命令
     */
    void ntsOnPlaybackMANSRTSPTeardownCommand(long id, String deviceId);
 
    /*
     * 收到Bye
     */
    void ntsOnByePlayback(long id, String deviceId);
 
    /*
     * 不是在收到BYE Message情况下， 终止Playback
     */
    void ntsOnTerminatePlayback(long id, String deviceId);
 
    /*
     * Playback会话对应的对话终止, 一般不会触发这个回调,目前只有在响应了200K, 但在64*T1时间后还没收到ACK,才可能会出发
    收到这个, 请做相关清理处理
    */
    void ntsOnPlaybackDialogTerminated(long id, String deviceId);
}
 
 
/**
* 部分JNI接口, rtp ps 打包发送等代码C++实现
*/
 
public class SmartPublisherJniV2 {
 
     /**
	 * Open publisher(启动推送实例)
	 *
	 * @param ctx: get by this.getApplicationContext()
	 * 
	 * @param audio_opt:
	 * if 0: 不推送音频
	 * if 1: 推送编码前音频(PCM)
	 * if 2: 推送编码后音频(aac/pcma/pcmu/speex).
	 * 
	 * @param video_opt:
	 * if 0: 不推送视频
	 * if 1: 推送编码前视频(NV12/I420/RGBA8888等格式)
	 * if 2: 推送编码后视频(AVC/HEVC)
	 * if 3: 层叠加模式
	 *
	 * <pre>This function must be called firstly.</pre>
	 *
	 * @return the handle of publisher instance
	 */
    public native long SmartPublisherOpen(Object ctx, int audio_opt, int video_opt,  int width, int height);
 
    
     /**
	 * 设置流类型
	 * @param type: 0:表示 live 流, 1:表示 on-demand 流, SDK默认为0(live流)
	 * 注意: 流类型设置当前仅对GB28181媒体流有效
	 * @return {0} if successful
	 */
    public native int SetStreamType(long handle, int type);
 
 
    /**
	 * 投递视频 on demand包, 当前只用于GB28181推送, 注意ByteBuffer对象必须是DirectBuffer
	 *
	 * @param codec_id: 编码id, 当前支持H264和H265, 1:H264, 2:H265
	 *
	 * @param packet: 视频数据, 包格式请参考H264/H265 Annex B Byte stream format, 例如:
	 *                0x00000001 nal_unit 0x00000001 ...
	 *                H264 IDR: 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....
	 *                H265 IDR: 0x00000001 vps 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....
	 *
	 * @param offset: 偏移量
	 * @param size: packet size
	 * @param pts_us: 时间戳, 单位微秒
	 * @param is_pts_discontinuity: 是否时间戳间断，0:未间断，1:间断
	 * @param is_key: 是否是关键帧, 0:非关键帧, 1:关键帧
	 * @param codec_specific_data: 可选参数，可传null, 对于H264关键帧包, 如果packet不含sps和pps, 可传0x00000001 sps 0x00000001 pps
	 *                    ,对于H265关键帧包, 如果packet不含vps,sps和pps, 可传0x00000001 vps 0x00000001 sps 0x00000001 pps
	 * @param codec_specific_data_size: codec_specific_data size
	 * @param width: 图像宽, 可传0
	 * @param height: 图像高, 可传0
	 *
	 * @return {0} if successful
	 */
	public native int PostVideoOnDemandPacketByteBuffer(long handle, int codec_id,
														ByteBuffer packet, int offset, int size, long pts_us, int is_pts_discontinuity, int is_key,
														byte[] codec_specific_data, int codec_specific_data_size,
									  					int width, int height);
 
	
     /**
	 * 投递音频on demand包, 当前只用于GB28181推送, 注意ByteBuffer对象必须是DirectBuffer
	 *
	 * @param codec_id: 编码id, 当前支持PCMA和AAC, 65536:PCMA, 65538:AAC
	 * @param packet: 音频数据
	 * @param offset：packet偏移量
	 * @param size: packet size
	 * @param pts_us: 时间戳, 单位微秒
	 * @param is_pts_discontinuity: 是否时间戳间断，0:未间断，1:间断
	 * @param codec_specific_data: 如果是AAC的话，需要传 Audio Specific Configuration
	 * @param codec_specific_data_size: codec_specific_data size
	 * @param sample_rate: 采样率
	 * @param channels: 通道数
	 *
	 * @return {0} if successful
	 */
	public native int PostAudioOnDemandPacketByteBuffer(long handle, int codec_id,
														ByteBuffer packet, int offset, int size, long pts_us, int is_pts_discontinuity,
														byte[] codec_specific_data, int codec_specific_data_size,
														int sample_rate, int channels);
																											
	/**
	 * on demand source完成seek后, 请调用
	 * @return {0} if successful
	 */
	public native int OnSeekProcessed(long handle);
 
	/**
	 * 启动 GB28181 媒体流
	 *
	 * @return {0} if successful
	 */
	public native int StartGB28181MediaStream(long handle);
 
 
    /**
	 * 停止 GB28181 媒体流
	 *
	 * @return {0} if successful
	 */
	public native int StopGB28181MediaStream(long handle);
 
    
	/**
     * 关闭推送实例，结束时必须调用close接口释放资源
	 *
	 * @return {0} if successful
	 */
    public native int SmartPublisherClose(long handle);
 
}
 
 
/**
* Listener部分实现代码
*/
 
public class PlaybackListenerImpl implements com.gb.ntsignalling.GBSIPAgentPlaybackListener {
 
    /*
     *收到s=Playback的文件下载Invite
     */
    @Override
    public void ntsOnInvitePlayback(long id, String deviceId, SessionDescription sdp) {
        if (!post_task(new PlaybackListenerImpl.OnInviteTask(this.context_, this.is_exit_, this.senders_map_, deviceId, sdp, id))) {
            Log.e(TAG, "ntsOnInvitePlayback post_task failed, " + RecordSender.make_print_tuple(id, deviceId, sdp.getTime().getStartTime(),  sdp.getTime().getStopTime()));
 
            // 这里不发488, 等待事务超时也可以的
            GBSIPAgent agent = this.context_.get_agent();
            if (agent != null)
                agent.respondPlaybackInvite(488, id, deviceId);
        }
    }
 
    /*
     *发送Playback invite response 异常
     */
    @Override
    public void ntsOnPlaybackInviteResponseException(long id, String deviceId, int statusCode, String errorInfo) {
        Log.i(TAG, "ntsOnPlaybackInviteResponseException, status_code:" + statusCode + ", "
                + RecordSender.make_print_tuple(id, deviceId) + ",  error_info:" + errorInfo);
 
        RecordSender sender = senders_map_.remove(id);
        if (null == sender)
            return;
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * 收到CANCEL Playback INVITE请求
     */
    @Override
    public void ntsOnCancelPlayback(long id, String deviceId) {
        Log.i(TAG, "ntsOnCancelPlayback, " + RecordSender.make_print_tuple(id, deviceId));
 
        RecordSender sender = senders_map_.remove(id);
        if (null == sender)
            return;
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * 收到Ack
     */
    @Override
    public void ntsOnAckPlayback(long id, String deviceId) {
        Log.i(TAG, "ntsOnAckPlayback, "+ RecordSender.make_print_tuple(id, deviceId));
 
        RecordSender sender = senders_map_.get(id);
        if (null == sender) {
            Log.e(TAG, "ntsOnAckPlayback get sender is null, " + RecordSender.make_print_tuple(id, deviceId));
 
            GBSIPAgent agent = this.context_.get_agent();
            if (agent != null)
                agent.terminatePlayback(id, deviceId, false);
 
            return;
        }
 
        PlaybackListenerImpl.StartTask task = new PlaybackListenerImpl.StartTask(sender, this.senders_map_);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * 收到Bye
     */
    @Override
    public void ntsOnByePlayback(long id, String deviceId) {
        Log.i(TAG, "ntsOnByePlayback, "+ RecordSender.make_print_tuple(id, deviceId));
 
        RecordSender sender = this.senders_map_.remove(id);
        if (null == sender)
            return;
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * 播放命令
     */
    @Override
    public void ntsOnPlaybackMANSRTSPPlayCommand(long id, String deviceId) {
        RecordSender sender = this.senders_map_.get(id);
        if (null == sender) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPPlayCommand can not get sender " + RecordSender.make_print_tuple(id, deviceId));
            return;
        }
 
        sender.post_play_command();
 
        Log.i(TAG, "ntsOnPlaybackMANSRTSPPlayCommand " + RecordSender.make_print_tuple(id, deviceId));
    }
 
    /*
     * 暂停命令
     */
    @Override
    public void ntsOnPlaybackMANSRTSPPauseCommand(long id, String deviceId) {
        RecordSender sender = this.senders_map_.get(id);
        if (null == sender) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPPauseCommand can not get sender " + RecordSender.make_print_tuple(id, deviceId));
            return;
        }
 
        sender.post_pause_command();
 
        Log.i(TAG, "ntsOnPlaybackMANSRTSPPauseCommand " + RecordSender.make_print_tuple(id, deviceId));
    }
 
    /*
     * 快进/慢进命令
     */
    @Override
    public void ntsOnPlaybackMANSRTSPScaleCommand(long id, String deviceId, double scale) {
        if (scale < 0.01) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPScaleCommand invalid scale:" + scale  + " " + RecordSender.make_print_tuple(id, deviceId));
            return;
        }
 
        RecordSender sender = this.senders_map_.get(id);
        if (null == sender) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPScaleCommand can not get sender, scale:" + scale  + " " + RecordSender.make_print_tuple(id, deviceId));
            return;
        }
 
        sender.post_scale_command(scale);
 
        Log.i(TAG, "ntsOnPlaybackMANSRTSPScaleCommand, scale:" + scale + " " + RecordSender.make_print_tuple(id, deviceId));
    }
 
    /*
     * 随机拖动命令
     */
    @Override
    public void ntsOnPlaybackMANSRTSPSeekCommand(long id, String device_id, double position_sec) {
        if (position_sec < 0.0) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPSeekCommand invalid seek pos:" + position_sec  + ", " + RecordSender.make_print_tuple(id, device_id));
            return;
        }
 
        RecordSender sender = this.senders_map_.get(id);
        if (null == sender) {
            Log.e(TAG, "ntsOnPlaybackMANSRTSPSeekCommand can not get sender " + RecordSender.make_print_tuple(id, device_id));
            return;
        }
 
        long offset_ms = sender.get_file_start_time_offset_ms();
        position_sec += (offset_ms/1000.0);
 
        sender.post_seek_command(position_sec);
 
        Log.i(TAG, "ntsOnPlaybackMANSRTSPSeekCommand seek pos:" + RecordSender.out_point_3(position_sec) + "s, " + RecordSender.make_print_tuple(id, device_id));
    }
 
    /*
     * 停止命令
     */
    @Override
    public void ntsOnPlaybackMANSRTSPTeardownCommand(long id, String device_id) {
        CallTerminatePlaybackTask call_terminate_task =  new CallTerminatePlaybackTask(this.context_, id, device_id, true);
        post_task(call_terminate_task);
 
        RecordSender sender = this.senders_map_.remove(id);
        if (null == sender) {
            Log.w(TAG, "ntsOnPlaybackMANSRTSPTeardownCommand can not remove sender " + RecordSender.make_print_tuple(id, device_id));
            return;
        }
 
        Log.i(TAG, "ntsOnPlaybackMANSRTSPTeardownCommand " + RecordSender.make_print_tuple(id, device_id));
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * 不是在收到BYE Message情况下， 终止Playback
     */
    @Override
    public void ntsOnTerminatePlayback(long id, String deviceId) {
        Log.i(TAG, "ntsOnTerminatePlayback, "+ RecordSender.make_print_tuple(id, deviceId));
 
        RecordSender sender = this.senders_map_.remove(id);
        if (null == sender)
            return;
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
 
    /*
     * Playback会话对应的对话终止, 一般不会触发这个回调,目前只有在响应了200K, 但在64*T1时间后还没收到ACK,才可能会出发
    收到这个, 请做相关清理处理
    */
    @Override
    public void ntsOnPlaybackDialogTerminated(long id, String deviceId) {
        Log.i(TAG, "ntsOnPlaybackDialogTerminated, "+ RecordSender.make_print_tuple(id, deviceId));
 
        RecordSender sender = this.senders_map_.remove(id);
        if (null == sender)
            return;
 
        PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
        if (!post_task(task))
            task.run();
    }
}

支持云台控制和预置位查询

typescript 复制代码 
@Override
public void ntsOnDevicePresetQueryCommand(String fromUserName, String fromUserNameAtDomain, String sn, String deviceId) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "DaniuSDK ntsOnDevicePresetQueryCommand from_user_name:" + from_user_name_ + ", sn:" + sn_ + ", device_id:" + device_id_);

      List<com.gb.ntsignalling.PresetItem> preset_list = new LinkedList<>();


      preset_list.add(new com.gb.ntsignalling.PresetItem("1", "Android PreSet1"));
      preset_list.add(new com.gb.ntsignalling.PresetItem("2", "Android PreSet2"));

      if (gb28181_agent_ != null )
        gb28181_agent_.respondDevicePresetQueryCommand(this.from_user_name_, this.from_user_name_at_domain_, this.sn_, this.device_id_, preset_list);
    }

    private String from_user_name_;
    private String from_user_name_at_domain_;
    private String sn_;
    private String device_id_;

    public Runnable set(String from_user_name, String from_user_name_at_domain,String sn, String device_id) {
      this.from_user_name_ = from_user_name;
      this.from_user_name_at_domain_ = from_user_name_at_domain;
      this.sn_ = sn;
      this.device_id_ = device_id;
      return this;
    }

  }.set(fromUserName, fromUserNameAtDomain, sn, deviceId),0);
}

@Override
public void ntsOnDeviceControlPTZCmd(String deviceId, String typeValue) {
  handler_.postDelayed(new Runnable() {
    @Override
    public void run() {
      Log.i(TAG, "DaniuSDK ntsOnDeviceControlPTZCmd device_id:" + device_id_ + " PTZType:" + ptz_type_);

      if (null == ptz_type_)
        return;

      ptz_type_ = ptz_type_.trim();
      if (ptz_type_.length() != 16)
        return;

      int instruction = hexStringToInt(ptz_type_.substring(6, 8));
      int combination_code2 = hexStringToInt(ptz_type_.substring(12, 14));

      //Android平台GB28181设备接入端，针对性的解析处理即可，这里不再赘述

      private String device_id_;
      private String ptz_type_;

      public Runnable set(String device_id, String ptz_type) {
        this.device_id_ = device_id;
        this.ptz_type_ = ptz_type;
        return this;
      }

    }.set(deviceId, typeValue),0);
  }

[实时水印]支持动态文字水印、png水印

ini 复制代码 
watermarkSelctor = (Spinner) findViewById(R.id.watermarkSelctor);

final String[] watermarks = new String[]{"图片水印", "全部水印", "文字水印", "不加水印"};

ArrayAdapter<String> adapterWatermark = new ArrayAdapter<String>(this,
                                                                 android.R.layout.simple_spinner_item, watermarks);

adapterWatermark.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);

watermarkSelctor.setAdapter(adapterWatermark);

watermarkSelctor.setSelection(3,true);
watemarkType = 3;   //默认不加水印

watermarkSelctor.setOnItemSelectedListener(new AdapterView.OnItemSelectedListener() {

  @Override
  public void onItemSelected(AdapterView<?> parent, View view,
                             int position, long id) {

    watemarkType = position;

    Log.i(TAG, "[水印类型]Currently choosing: " + watermarks[position] + ", watemarkType: " + watemarkType);

    if (layer_post_thread_ != null) {
      layer_post_thread_.enableText(isHasTextWatermark());
      layer_post_thread_.enablePicture(isHasPictureWatermark());
    }
  }

  @Override
  public void onNothingSelected(AdapterView<?> parent) {

  }
});

[实时快照]支持实时快照

typescript 复制代码 
class ButtonCaptureImageListener implements View.OnClickListener {
  @SuppressLint("SimpleDateFormat")
  public void onClick(View v) {
    if(isPushingRtmp || isRecording || isRTSPPublisherRunning || isGB28181StreamRunning)
    {
      String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
      String imageFileName = "dn_" + timeStamp;    //创建以时间命名的文件名称

      String imagePath = imageSavePath + "/" + imageFileName + ".png";

      Log.i(TAG, "imagePath:" + imagePath);

      libPublisher.SmartPublisherSaveCurImage(publisherHandle, imagePath);
    }
    else
    {
      Log.e(TAG, "快照失败，请确保在推送、录像或内置RTSP服务发布状态..");
    }
  }
}

[降噪]支持环境音、手机干扰等引起的噪音降噪处理、自动增益、VAD检测

ini 复制代码 
boolean is_noise_suppression = true;
libPublisher.SmartPublisherSetNoiseSuppression(publisherHandle, is_noise_suppression ? 1 : 0);

boolean is_agc = false;
libPublisher.SmartPublisherSetAGC(publisherHandle, is_agc ? 1 : 0);

int echo_cancel_delay = 0;
libPublisher.SmartPublisherSetEchoCancellation(publisherHandle, 1, echo_cancel_delay);

外部编码前后视频数据对接

  • 编码前数据（目前支持的有YV12/NV21/NV12/I420/RGB24/RGBA32/RGB565等数据类型），其中，Android平台前后摄像头数据，或者屏幕数据，或者Unity拿到的数据，均属编码前数据；
  • 编码后数据（如无人机等264/HEVC数据，或者本地解析的MP4音视频数据）；

[扩展录像功能]支持和录像SDK组合使用，录像相关功能

ini 复制代码 
class ButtonStartRecorderListener implements View.OnClickListener {
  public void onClick(View v) {
    if (isRecording) {
      stopRecorder();

      if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {
        ConfigControlEnable(true);
      }

      btnStartRecorder.setText("实时录像");

      btnPauseRecorder.setText("暂停录像");
      btnPauseRecorder.setEnabled(false);
      isPauseRecording = true;

      return;
    }

    Log.i(TAG, "onClick start recorder..");

    if (libPublisher == null)
      return;

    if (!isPushingRtmp && !isRTSPPublisherRunning&& !isGB28181StreamRunning) {
      InitAndSetConfig();
    }

    ConfigRecorderParam();

    int startRet = libPublisher.SmartPublisherStartRecorder(publisherHandle);
    if (startRet != 0) {
      if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {
        if (publisherHandle != 0) {
          long handle = publisherHandle;
          publisherHandle = 0;
          libPublisher.SmartPublisherClose(handle);
        }
      }

      Log.e(TAG, "Failed to start recorder.");
      return;
    }

    if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {
      CheckInitAudioRecorder();
      ConfigControlEnable(false);
    }

    startLayerPostThread();

    btnStartRecorder.setText("停止录像");
    isRecording = true;

    btnPauseRecorder.setEnabled(true);
    isPauseRecording = true;
  }
}

class ButtonPauseRecorderListener implements View.OnClickListener {
  public void onClick(View v) {
    if (isRecording) {

      if(isPauseRecording)
      {
        int ret = libPublisher.SmartPublisherPauseRecorder(publisherHandle, 1);

        if (ret == 0)
        {
          isPauseRecording = false;
          btnPauseRecorder.setText("恢复录像");
        }
        else if(ret == 3)
        {
          Log.e(TAG, "Pause recorder failed, please re-try again..");
        }
        else
        {
          Log.e(TAG, "Pause recorder failed..");
        }
      }
      else
      {
        int ret = libPublisher.SmartPublisherPauseRecorder(publisherHandle, 0);

        if (ret == 0)
        {
          isPauseRecording = true;
          btnPauseRecorder.setText("暂停录像");
        }
        else if(ret == 3)
        {
          Log.e(TAG, "Resume recorder failed, please re-try again..");
        }
        else
        {
          Log.e(TAG, "Resume recorder failed..");
        }
      }
    }
  }
}

总结

Android平台GB28181设备接入侧模块，如果需要做的更好，上述提到的技术层面的问题解决了还不够，还需要针对各类国标平台适配对接，只有这样，才能更好的为执法记录仪、智能安全帽、智能监控、智慧零售、智慧教育、远程办公、明厨亮灶、智慧交通、智慧工地、雪亮工程、平安乡村、生产运输、车载终端等场景服务。

​

相关推荐
关键帧Keyframe2 天前
音视频面试题集锦第 7 期
音视频开发·视频编码·客户端
关键帧Keyframe2 天前
音视频面试题集锦第 8 期
ios·音视频开发·客户端
蚝油菜花7 天前
MimicTalk：字节跳动和浙江大学联合推出 15 分钟生成 3D 说话人脸视频的生成模型
人工智能·开源·音视频开发
音视频牛哥9 天前
Android平台RTSP|RTMP播放器高效率如何回调YUV或RGB数据？
音视频开发·视频编码·直播
<Sunny>11 天前
MPP音视频总结
音视频开发·1024程序员节·海思mpp
GoFly开发者11 天前
GoFly快速开发框架已集成了RTSP流媒体服务器(直播、录播)代码插件-开发者集成流媒体服务器更加方便
go·音视频开发
aqi0013 天前
FFmpeg开发笔记（六十）使用国产的ijkplayer播放器观看网络视频
android·ffmpeg·音视频·直播·流媒体
linkedbyte13 天前
常用录屏软件功能比较
音视频·直播·录屏
aqi0016 天前
FFmpeg开发笔记（五十九）Linux编译ijkplayer的Android平台so库
android·ffmpeg·音视频·直播·流媒体
热门推荐
01【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总02(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明03组基轨迹建模 GBTM的介绍与实现（Stata 或 R）04【AIGC】重塑未来的科技巨轮05全面解析：构建基于深度学习的安全帽检测系统（UI界面+YOLO代码+数据集）06【经验分享】Ubuntu22.04安装微信(linux官方版)07基于YOLOv10深度学习的CT扫描图像肾结石智能检测系统【python源码+Pyqt5界面+数据集+训练代码】深度学习实战、目标检测08Ubuntu 20.04使用Livox mid 360 测试 FAST_LIO09RAG 实践- Ollama+RagFlow 部署本地知识库10【TC3xx芯片】TC3xx芯片电源管理系统PMS详解