iOS开发-WebRTC本地直播高分辨率不显示画面问题

iOS开发-WebRTC本地直播高分辨率不显示画面问题

在之前使用WebRTC结合ossrs进行推流时候,ossrs的播放端无法看到高分辨率画面问题。根据这个问题,找到了解决方案。

一、WebRTC是什么

WebRTC是什么呢?

WebRTC (Web Real-Time Communications) 是一项实时通讯技术,它允许网络应用或者站点,在不借助中间媒介的情况下,建立浏览器之间点对点(Peer-to-Peer)的连接,实现视频流和(或)音频流或者其他任意数据的传输。

二、ossrs是什么?

ossrs是什么呢?

SRS(Simple Realtime Server)是一个简单高效的实时视频服务器,支持RTMP、WebRTC、HLS、HTTP-FLV、SRT等多种实时流媒体协议。

官网地址:https://ossrs.net/lts/zh-cn/

这里暂时不写iOS Google WebRTC与ossrs实现RTC直播了。暂时值记录一下本地WebRTC直播高分辨率不显示画面问题。

三、高分辨率不显示画面问题解决方案

本地WebRTC直播高分辨率不显示画面问题,这个问题和SDP中的profile-level-id有关。

profile-level-id正好是SPS中的第二至四个字节的base16编码。这三个字节的具体含义是

sps[1] AVCProfileIndication

sps[2] profile_compatibility

sps[3] AVCLevlIndication

http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels

实际设置时,就是level值乘以10,例如level 1.0,设置值就是0x0A。level 3.0,设置值就是0x1E。比较例外的是level 1b,设置值是0x09

图片来源网络(抱歉忘记地址了,如果引用了您的博客图片,请留言)

第三个代表level,比如1f值为31,从图中可以看出3.1,分辨率在720480 720576 1280*720

如果需要高分辨率 19201080 25601920 3840*2160 ,需要设置的level 5.1 16进制值为33,这里设置的值是42e033。

下面是一个SDP示例

"code": 0,

"server": "vid-415v5lz",

"sdp": "v=0\r\no=SRS/4.0.268(Leo) 94003279212192 2 IN IP4 0.0.0.0\r\ns=SRSPublishSession\r\nt=0 0\r\na=ice-lite\r\na=group:BUNDLE 0 1\r\na=msid-semantic: WMS live/livestream\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:q01184s8\r\na=ice-pwd:o25158210twbb093o342910094v0wo5k\r\na=fingerprint:sha-256 6A:66:81:7C:68:91:79:18:05:2C:EE:5F:BF:1B:4B:F4:78:C4:01:06:CC:CC:9E:F0:32:5B:72:21:4A:C2:A1:AA\r\na=setup:passive\r\na=mid:0\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:111 opus/48000/2\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=candidate:0 1 udp 2130706431 10.0.80.128 8000 typ host generation 0\r\na=candidate:1 1 udp 2130706431 112.124.157.141 8000 typ host generation 0\r\nm=video 9 UDP/TLS/RTP/SAVPF 96 127\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:q01184s8\r\na=ice-pwd:o25158210twbb093o342910094v0wo5k\r\na=fingerprint:sha-256 6A:66:81:7C:68:91:79:18:05:2C:EE:5F:BF:1B:4B:F4:78:C4:01:06:CC:CC:9E:F0:32:5B:72:21:4A:C2:A1:AA\r\na=setup:passive\r\na=mid:1\r\na=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\\r\\na=recvonly\\r\\na=rtcp-mux\\r\\na=rtcp-rsize\\r\\na=rtpmap:96 H264/90000\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:127 red/90000\r\na=candidate:0 1 udp 2130706431 10.0.80.128 8000 typ host generation 0\r\na=candidate:1 1 udp 2130706431 112.124.157.141 8000 typ host generation 0\r\n",

"sessionid": "q01184s8:oPvh"

}

可以看到这里的profile-level-id=42e01f

如果在WebRTC中将本地的SDP的profile-level-id替换成42e033

objectivec 复制代码
[weakSelf.webRTCClient offer:^(RTCSessionDescription *sdp) {
        DebugLog(@"changeSDP2Server offer sdp:%@", sdp);
        NSString *offerSDPString = [SDWebRTCSDPUtil setMediaBitrate:sdp.sdp media:@"video" bitrate:(6*1024*1024)];
        DebugLog(@"changeSDP2Server offerSDPString:%@", offerSDPString);
        [weakSelf changeSDP2Server:offerSDPString];
    }];

将调用rtc/v1/publish/接口获得的remoteSDPString中的profile-level-id更改42e033之后再调用

objectivec 复制代码
- (void)setRemoteDescription:(RTC_OBJC_TYPE(RTCSessionDescription) *)sdp
           completionHandler:(nullable void (^)(NSError *_Nullable error))completionHandler;

经过测试,切换高分辨率画面可以正常显示。

objectivec 复制代码
NSString *resultRemoteSDPString = [SDWebRTCSDPUtil setMediaBitrate:remoteSDPString media:@"video" bitrate:(6*1024*1024)];
               DebugLog(@"changeSDP2Server resultRemoteSDPString:%@", resultRemoteSDPString);

               RTCSessionDescription *remoteSDP = [[RTCSessionDescription alloc] initWithType:RTCSdpTypeAnswer sdp:resultRemoteSDPString];
               [weakSelf.webRTCClient setRemoteSdp:remoteSDP completion:^(NSError * error) {
                   DebugLog(@"changeSDP2Server setRemoteDescription error:%@", error);
               }];

四、通过RTCVideoEncoderFactory解决高分辨率不显示画面问题解决方案

通过将SDP中的profile-level-id进行更改并不是我的最终解决方法,我最后使用的是通过指定RTCVideoEncoder的RTCVideoCodecInfo中constrainedHighParams的profile-level-id值为42e033

设置codecs的constrainedHighInfo和constrainedBaselineInfo元素。

objectivec 复制代码
NSDictionary<NSString *, NSString *> *constrainedHighParams = @{
    @"profile-level-id" : kLevelHighConstrainedHigh,
    @"level-asymmetry-allowed" : @"1",
    @"packetization-mode" : @"1",
  };
  RTCVideoCodecInfo *constrainedHighInfo =
      [[RTCVideoCodecInfo alloc] initWithName:kRTCVideoCodecH264Name parameters:constrainedHighParams];
  [codecs addObject:constrainedHighInfo];

  NSDictionary<NSString *, NSString *> *constrainedBaselineParams = @{
    @"profile-level-id" : kLevelHighConstrainedBaseline,
    @"level-asymmetry-allowed" : @"1",
    @"packetization-mode" : @"1",
  };
  RTCVideoCodecInfo *constrainedBaselineInfo =
      [[RTCVideoCodecInfo alloc] initWithName:kRTCVideoCodecH264Name parameters:constrainedBaselineParams];
  [codecs addObject:constrainedBaselineInfo];

完整代码如下

SDRTCVideoEncoderFactory.h

objectivec 复制代码
#import <Foundation/Foundation.h>
#import <WebRTC/WebRTC.h>

/**
 + (NSArray<RTCVideoCodecInfo *> *)supportedCodecs {
   NSDictionary<NSString *, NSString *> *constrainedHighParams = @{
     @"profile-level-id" : kRTCMaxSupportedH264ProfileLevelConstrainedHigh,
     @"level-asymmetry-allowed" : @"1",
     @"packetization-mode" : @"1",
   };
   RTCVideoCodecInfo *constrainedHighInfo =
       [[RTCVideoCodecInfo alloc] initWithName:kRTCVideoCodecH264Name
                                    parameters:constrainedHighParams];
   NSDictionary<NSString *, NSString *> *constrainedBaselineParams = @{
     @"profile-level-id" : kRTCMaxSupportedH264ProfileLevelConstrainedBaseline,
     @"level-asymmetry-allowed" : @"1",
     @"packetization-mode" : @"1",
   };
   RTCVideoCodecInfo *constrainedBaselineInfo =
       [[RTCVideoCodecInfo alloc] initWithName:kRTCVideoCodecH264Name
                                    parameters:constrainedBaselineParams];
   RTCVideoCodecInfo *vp8Info = [[RTCVideoCodecInfo alloc] initWithName:kRTCVideoCodecVp8Name];
 #if defined(RTC_ENABLE_VP9)
   RTCVideoCodecInfo *vp9Info = [[RTCVideoCodecInfo alloc] initWithName:kRTCVideoCodecVp9Name];
 #endif
   return @[
     constrainedHighInfo,
     constrainedBaselineInfo,
     vp8Info,
 #if defined(RTC_ENABLE_VP9)
     vp9Info,
 #endif
   ];
 }
 */

@interface SDRTCVideoEncoderFactory : NSObject<RTCVideoEncoderFactory>

@end

SDRTCVideoEncoderFactory.m

objectivec 复制代码
#import "SDRTCVideoEncoderFactory.h"

static NSString *kLevelHighConstrainedHigh = @"640c33";
static NSString *kLevelHighConstrainedBaseline = @"42e033";

@implementation SDRTCVideoEncoderFactory

- (id<RTCVideoEncoder>)createEncoder:(RTCVideoCodecInfo *)info {
  if ([info.name isEqualToString:kRTCVideoCodecH264Name]) {
    return [[RTCVideoEncoderH264 alloc] initWithCodecInfo:info];
  } else if ([info.name isEqualToString:kRTCVideoCodecVp8Name]) {
    return [RTCVideoEncoderVP8 vp8Encoder];
  } else if ([info.name isEqualToString:kRTCVideoCodecVp9Name]) {
    return [RTCVideoEncoderVP9 vp9Encoder];
  }

  return nil;
}

- (NSArray<RTCVideoCodecInfo *> *)supportedCodecs {
  NSMutableArray<RTCVideoCodecInfo *> *codecs = [NSMutableArray array];

  NSDictionary<NSString *, NSString *> *constrainedHighParams = @{
    @"profile-level-id" : kLevelHighConstrainedHigh,
    @"level-asymmetry-allowed" : @"1",
    @"packetization-mode" : @"1",
  };
  RTCVideoCodecInfo *constrainedHighInfo =
      [[RTCVideoCodecInfo alloc] initWithName:kRTCVideoCodecH264Name parameters:constrainedHighParams];
  [codecs addObject:constrainedHighInfo];

  NSDictionary<NSString *, NSString *> *constrainedBaselineParams = @{
    @"profile-level-id" : kLevelHighConstrainedBaseline,
    @"level-asymmetry-allowed" : @"1",
    @"packetization-mode" : @"1",
  };
  RTCVideoCodecInfo *constrainedBaselineInfo =
      [[RTCVideoCodecInfo alloc] initWithName:kRTCVideoCodecH264Name parameters:constrainedBaselineParams];
  [codecs addObject:constrainedBaselineInfo];

  RTCVideoCodecInfo *vp8Info = [[RTCVideoCodecInfo alloc] initWithName:kRTCVideoCodecVp8Name parameters:nil];
  [codecs addObject:vp8Info];

#if defined(RTC_ENABLE_VP9)
    RTCVideoCodecInfo *vp9Info = [[RTCVideoCodecInfo alloc] initWithName:kRTCVideoCodecVp9Name];
    [codecs addObject:vp9Info];
#endif
  
  return [codecs copy];
}

@end

最后在RTCPeerConnectionFactory初始化的时候做一下设置

objectivec 复制代码
#pragma mark - Lazy
- (RTCPeerConnectionFactory *)factory {
    if (!_factory) {
        RTCInitializeSSL();
        
        SDRTCVideoDecoderFactory *decoderFactory = [[SDRTCVideoDecoderFactory alloc] init];
        SDRTCVideoEncoderFactory *encoderFactory = [[SDRTCVideoEncoderFactory alloc] init];
        
//        for (RTCVideoCodecInfo *codec in encoderFactory.supportedCodecs) {
//            DebugLog(@"RTCVideoCodecInfo codec.parameters:%@", codec.parameters);
//        }
        _factory = [[RTCPeerConnectionFactory alloc] initWithEncoderFactory:encoderFactory decoderFactory:decoderFactory];
    }
    return _factory;
}

经过测试,解决了高分辨率不显示画面的问题

五、小结

iOS开发-WebRTC本地直播高分辨率不显示画面问题,使用WebRTC结合ossrs进行推流时候,ossrs的播放端无法看到高分辨率画面问题。通过设置RTCPeerConnectionFactory的initWithEncoderFactory最后解决了该问题。

https://blog.csdn.net/gloryFlow/article/details/132240952

学习记录,每天不停进步。

相关推荐
Ya-Jun5 小时前
常用第三方库:flutter_boost混合开发
android·flutter·ios
唯独失去了从容8 小时前
WebRTC服务器Coturn服务器中的通信协议
运维·服务器·webrtc
玫瑰花开一片一片18 小时前
Flutter IOS 真机 Widget 错误。Widget 安装后系统中没有
flutter·ios·widget·ios widget
唯独失去了从容20 小时前
WebRTC服务器Coturn服务器部署
webrtc
烎就是我20 小时前
100行代码swift从零实现一个iOS日历
ios·swift
音视频牛哥1 天前
如何实现Android屏幕和音频采集并启动RTSP服务?
音视频开发·视频编码·直播
音视频牛哥1 天前
Linux平台实现低延迟的RTSP、RTMP播放
音视频开发·视频编码·直播
鸿蒙布道师1 天前
鸿蒙NEXT开发通知工具类(ArkTs)
android·ios·华为·harmonyos·arkts·鸿蒙系统·huawei
鸿蒙布道师1 天前
鸿蒙NEXT开发网络相关工具类(ArkTs)
android·ios·华为·harmonyos·arkts·鸿蒙系统·huawei