live555 rtsp服务器实战之doGetNextFrame

注意：该篇文章可能有些绕，最好跟着文章追踪下源码，不了解源码可能就是天书；

概要

live555用于实际项目开发时，createNewStreamSource和doGetNextFrame是必须要实现的两个虚函数，一般会创建两个类来实现这两个函数：假如这两个类为H264LiveVideoServerMediaSubssion和H264FramedLiveSource；H264LiveVideoServerMediaSubssion为实时视频会话类，用于实现createNewStreamSource虚函数；

H264FramedLiveSource为实时视频帧资源类，用户实现doGetNextFrame函数；

那么这两个函数是什么时候被调用以及他们的作用是什么呢？本节将详细介绍；

声明：该文章基于H264视频源为基础分析，其他源类似；

由于这两个类主要是自定义虚函数，所以存在一定的继承关系，首先需要明确这两个类的继承关系(本章只介绍H264FramedLiveSource)：

H264FramedLiveSource:FramedSource:MediaSource:Medium

doGetNextFrame调用流程

doGetNextFrame函数声明于FrameSource类：

cpp 复制代码

virtual void doGetNextFrame() = 0;

该声明为纯虚函数，所以必须有子类对该函数进行实现，H264FramedLiveSource类就是用于实现该函数；doGetNextFrame函数只有一个功能：获取视频帧；

cpp 复制代码

void H264FramedLiveSource::doGetNextFrame()
{
    uint8_t *frame = new uint8_t[1024*1024];
    int len = 0;
    //获取一帧视频帧
    get_frame(&frame, len);
    //将帧长度赋值给父类的成员变量fFrameSize
    fFrameSize = len;
    //将帧数据赋值给父类的成员变量fTo
    memcpy(fTo, frame, len);
    
    delete [] frame;
    // nextTask() = envir().taskScheduler().scheduleDelayedTask(0,(TaskFunc*)FramedSource::afterGetting, this);//表示延迟0秒后再执行 afterGetting 函数
    afterGetting(this);

    return;
}

fFrameSize和fTo的两个父类变量在流传输时会用到；那么问题来了：doGetNextFrame函数在整个流程中在哪里被调用？什么时候调用？

首先doGetNextFrame函数是在PLAY信令交互的时候被调用；调用流程为：

cpp 复制代码

handleCmd_PLAY->startStream->startPlaying(OnDemandServerMediaSubsession)->startPlaying(MediaSink)->continuePlaying(虚函数，子类H264or5VideoRTPSink实现)->continuePlaying(虚函数，子类MultiFramedRTPSink实现)->buildAndSendPacket->packFrame(MultiFramedRTPSink)->getNextFrame->doGetNextFrame

下面详解介绍下这个流程，看过我的另一篇文章："live555 rtsp服务器实战之createNewStreamSource" 的都知道handleRequestBytes函数调用了handleCmd_SETUP；同样handleCmd_PLAY函数也是在这里被调用的；handleRequestBytes函数就是用来处理各种客户端信令交互信息的；handleRequestBytes函数如下：

cpp 复制代码

void RTSPServer::RTSPClientConnection::handleRequestBytes(int newBytesRead)
{
    .
    .
    .
    if (urlIsRTSPS != fOurRTSPServer.fOurConnectionsUseTLS)
      {
#ifdef DEBUG
        fprintf(stderr, "Calling handleCmd_redirect()\n");
#endif
        handleCmd_redirect(urlSuffix);
      }
      else if (strcmp(cmdName, "OPTIONS") == 0)
      {
        // If the "OPTIONS" command included a "Session:" id for a session that doesn't exist,
        // then treat this as an error:
        if (requestIncludedSessionId && clientSession == NULL)
        {
#ifdef DEBUG
          fprintf(stderr, "Calling handleCmd_sessionNotFound() (case 1)\n");
#endif
          handleCmd_sessionNotFound();
        }
        else
        {
          // Normal case:
          handleCmd_OPTIONS();
        }
      }
      else if (urlPreSuffix[0] == '\0' && urlSuffix[0] == '*' && urlSuffix[1] == '\0')
      {
        // The special "*" URL means: an operation on the entire server.  This works only for GET_PARAMETER and SET_PARAMETER:
        if (strcmp(cmdName, "GET_PARAMETER") == 0)
        {
          handleCmd_GET_PARAMETER((char const *)fRequestBuffer);
        }
        else if (strcmp(cmdName, "SET_PARAMETER") == 0)
        {
          handleCmd_SET_PARAMETER((char const *)fRequestBuffer);
        }
        else
        {
          handleCmd_notSupported();
        }
      }
      else if (strcmp(cmdName, "DESCRIBE") == 0)
      {
        handleCmd_DESCRIBE(urlPreSuffix, urlSuffix, (char const *)fRequestBuffer);
      }
      else if (strcmp(cmdName, "SETUP") == 0)
      {
        Boolean areAuthenticated = True;

        if (!requestIncludedSessionId)
        {
          // No session id was present in the request.
          // So create a new "RTSPClientSession" object for this request.

          // But first, make sure that we're authenticated to perform this command:
          char urlTotalSuffix[2 * RTSP_PARAM_STRING_MAX];
          // enough space for urlPreSuffix/urlSuffix'\0'
          urlTotalSuffix[0] = '\0';
          if (urlPreSuffix[0] != '\0')
          {
            strcat(urlTotalSuffix, urlPreSuffix);
            strcat(urlTotalSuffix, "/");
          }
          strcat(urlTotalSuffix, urlSuffix);
          if (authenticationOK("SETUP", urlTotalSuffix, (char const *)fRequestBuffer))
          {
            clientSession = (RTSPServer::RTSPClientSession *)fOurRTSPServer.createNewClientSessionWithId();
          }
          else
          {
            areAuthenticated = False;
          }
        }
        if (clientSession != NULL)
        {
          clientSession->handleCmd_SETUP(this, urlPreSuffix, urlSuffix, (char const *)fRequestBuffer);
          playAfterSetup = clientSession->fStreamAfterSETUP;
        }
        else if (areAuthenticated)
        {
#ifdef DEBUG
          fprintf(stderr, "Calling handleCmd_sessionNotFound() (case 2)\n");
#endif
          handleCmd_sessionNotFound();
        }
      }
      else if (strcmp(cmdName, "TEARDOWN") == 0 || strcmp(cmdName, "PLAY") == 0 || strcmp(cmdName, "PAUSE") == 0 || strcmp(cmdName, "GET_PARAMETER") == 0 || strcmp(cmdName, "SET_PARAMETER") == 0)
      {
        if (clientSession != NULL)
        {
          clientSession->handleCmd_withinSession(this, cmdName, urlPreSuffix, urlSuffix, (char const *)fRequestBuffer);
        }
        else
        {
#ifdef DEBUG
          fprintf(stderr, "Calling handleCmd_sessionNotFound() (case 3)\n");
#endif
          handleCmd_sessionNotFound();
        }
      }
      else if (strcmp(cmdName, "REGISTER") == 0 || strcmp(cmdName, "DEREGISTER") == 0)
      {
        // Because - unlike other commands - an implementation of this command needs
        // the entire URL, we re-parse the command to get it:
        char *url = strDupSize((char *)fRequestBuffer);
        if (sscanf((char *)fRequestBuffer, "%*s %s", url) == 1)
        {
          // Check for special command-specific parameters in a "Transport:" header:
          Boolean reuseConnection, deliverViaTCP;
          char *proxyURLSuffix;
          parseTransportHeaderForREGISTER((const char *)fRequestBuffer, reuseConnection, deliverViaTCP, proxyURLSuffix);

          handleCmd_REGISTER(cmdName, url, urlSuffix, (char const *)fRequestBuffer, reuseConnection, deliverViaTCP, proxyURLSuffix);
          delete[] proxyURLSuffix;
        }
        else
        {
          handleCmd_bad();
        }
        delete[] url;
      }
      else
      {
        // The command is one that we don't handle:
        handleCmd_notSupported();
      }
      .
      .
      .
}

handleCmd_withinSession函数内部就调用了handleCmd_PLAY函数；仅整理流程，调用途中的函数这里不做解释；直接跳到packFrame(MultiFramedRTPSink)函数：

cpp 复制代码

void MultiFramedRTPSink::packFrame()
{
  // Get the next frame.

  // First, skip over the space we'll use for any frame-specific header:
  fCurFrameSpecificHeaderPosition = fOutBuf->curPacketSize();
  fCurFrameSpecificHeaderSize = frameSpecificHeaderSize();
  fOutBuf->skipBytes(fCurFrameSpecificHeaderSize);
  fTotalFrameSpecificHeaderSizes += fCurFrameSpecificHeaderSize;

  // See if we have an overflow frame that was too big for the last pkt
  if (fOutBuf->haveOverflowData())
  {
    // Use this frame before reading a new one from the source
    unsigned frameSize = fOutBuf->overflowDataSize();
    struct timeval presentationTime = fOutBuf->overflowPresentationTime();
    unsigned durationInMicroseconds = fOutBuf->overflowDurationInMicroseconds();
    fOutBuf->useOverflowData();

    afterGettingFrame1(frameSize, 0, presentationTime, durationInMicroseconds);
  }
  else
  {
    // Normal case: we need to read a new frame from the source
    if (fSource == NULL)
      return;

    fSource->getNextFrame(fOutBuf->curPtr(), fOutBuf->totalBytesAvailable(),
                          afterGettingFrame, this, ourHandleClosure, this);
  }
}

这里调用了getNextFrame函数，来看下getNextFrame函数的实现：

cpp 复制代码

void FramedSource::getNextFrame(unsigned char* to, unsigned maxSize,
        afterGettingFunc* afterGettingFunc,
        void* afterGettingClientData,
        onCloseFunc* onCloseFunc,
        void* onCloseClientData) {
  // Make sure we're not already being read:
  if (fIsCurrentlyAwaitingData) {
    envir() << "FramedSource[" << this << "]::getNextFrame(): attempting to read more than once at the same time!\n";
    envir().internalError();
  }

  fTo = to;
  fMaxSize = maxSize;
  fNumTruncatedBytes = 0; // by default; could be changed by doGetNextFrame()
  fDurationInMicroseconds = 0; // by default; could be changed by doGetNextFrame()
  fAfterGettingFunc = afterGettingFunc;
  fAfterGettingClientData = afterGettingClientData;
  fOnCloseFunc = onCloseFunc;
  fOnCloseClientData = onCloseClientData;
  fIsCurrentlyAwaitingData = True;

  doGetNextFrame();
}

发现啦！发现doGetNextFrame函数啦！所以doGetNextFrame函数就是在getNextFrame内被调用的；但是问题来了：搜索可以发现在live555中doGetNextFrame函数很多；怎么就确定这里的doGetNextFrame和我们自定义的doGetNextFrame有关呢？

问的好！那这的关键点就在于fSource->getNextFrame的fSource变量到底是什么类的对象；才能确定doGetNextFrame到底调用的是哪个类的函数；fSource属于MediaSink类的成员变量:

cpp 复制代码

FramedSource* fSource;

但是FrameSource子类很多；所以需要确定fSource到底指向的是哪个子类；

既然fSource是MediaSink类的成员，那么fSource肯定是在MediaSink或其子类中被赋值；因此查找下这些类；首先明确下MediaSink的继承关系,从fSource调用的类开始查找：

cpp 复制代码

//父类
MultiFramedRTPSink:RTPSink:MediaSink:Medium
//子类
H264VideoRTPSink:H264or5VideoRTPSink:VideoRTPSink:MultiFramedRTPSink

因此在这些类中查找即可；根据调用流程可知startPlaying最先赋值，值是startPlaying的第一个参数；

cpp 复制代码

Boolean MediaSink::startPlaying(MediaSource& source,
        afterPlayingFunc* afterFunc,
        void* afterClientData) {
  // Make sure we're not already being played:
  if (fSource != NULL) {
    envir().setResultMsg("This sink is already being played");
    return False;
  }

  // Make sure our source is compatible:
  if (!sourceIsCompatibleWithUs(source)) {
    envir().setResultMsg("MediaSink::startPlaying(): source is not compatible!");
    return False;
  }
  fSource = (FramedSource*)&source;

  fAfterFunc = afterFunc;
  fAfterClientData = afterClientData;
  return continuePlaying();
}

而该函数是在StreamState类中的startPlaying函数中调用；

cpp 复制代码

void StreamState ::startPlaying(Destinations *dests, unsigned clientSessionId,
                                TaskFunc *rtcpRRHandler, void *rtcpRRHandlerClientData,
                                ServerRequestAlternativeByteHandler *serverRequestAlternativeByteHandler,
                                void *serverRequestAlternativeByteHandlerClientData)
{
    .
    .
    .
    if (!fAreCurrentlyPlaying && fMediaSource != NULL)
    {
    if (fRTPSink != NULL)
    {
      fRTPSink->startPlaying(*fMediaSource, afterPlayingStreamState, this);
      fAreCurrentlyPlaying = True;
    }
    else if (fUDPSink != NULL)
    {
      fUDPSink->startPlaying(*fMediaSource, afterPlayingStreamState, this);
      fAreCurrentlyPlaying = True;
    }
  }
}

startPlaying的第一个参数是fMediaSource；而fMediaSource是在StreamState类的构造函数中被赋值的：

cpp 复制代码

StreamState::StreamState(OnDemandServerMediaSubsession &master,
                         Port const &serverRTPPort, Port const &serverRTCPPort,
                         RTPSink *rtpSink, BasicUDPSink *udpSink,
                         unsigned totalBW, FramedSource *mediaSource,
                         Groupsock *rtpGS, Groupsock *rtcpGS)
    : fMaster(master), fAreCurrentlyPlaying(False), fReferenceCount(1),
      fServerRTPPort(serverRTPPort), fServerRTCPPort(serverRTCPPort),
      fRTPSink(rtpSink), fUDPSink(udpSink), fStreamDuration(master.duration()),
      fTotalBW(totalBW), fRTCPInstance(NULL) /* created later */,
      fMediaSource(mediaSource), fStartNPT(0.0), fRTPgs(rtpGS), fRTCPgs(rtcpGS)
{
}

StreamState是在OnDemandServerMediaSubsession类的getStreamParameters函数中被调用：

cpp 复制代码

void OnDemandServerMediaSubsession ::getStreamParameters(unsigned clientSessionId,
                                                         struct sockaddr_storage const &clientAddress,
                                                         Port const &clientRTPPort,
                                                         Port const &clientRTCPPort,
                                                         int tcpSocketNum,
                                                         unsigned char rtpChannelId,
                                                         unsigned char rtcpChannelId,
                                                         TLSState *tlsState,
                                                         struct sockaddr_storage &destinationAddress,
                                                         u_int8_t & /*destinationTTL*/,
                                                         Boolean &isMulticast,
                                                         Port &serverRTPPort,
                                                         Port &serverRTCPPort,
                                                         void *&streamToken)
{
     if (addressIsNull(destinationAddress))
  {
    // normal case - use the client address as the destination address:
    destinationAddress = clientAddress;
  }
  isMulticast = False;

  if (fLastStreamToken != NULL && fReuseFirstSource)
  {
    // Special case: Rather than creating a new 'StreamState',
    // we reuse the one that we've already created:
    serverRTPPort = ((StreamState *)fLastStreamToken)->serverRTPPort();
    serverRTCPPort = ((StreamState *)fLastStreamToken)->serverRTCPPort();
    ++((StreamState *)fLastStreamToken)->referenceCount();
    streamToken = fLastStreamToken;
  }
  else
  {
    // Normal case: Create a new media source:
    unsigned streamBitrate;
    FramedSource *mediaSource = createNewStreamSource(clientSessionId, streamBitrate);
    .
    .
    .
    streamToken = fLastStreamToken = new StreamState(*this, serverRTPPort, serverRTCPPort, rtpSink, udpSink,
                                                     streamBitrate, mediaSource,
                                                     rtpGroupsock, rtcpGroupsock);
   }
}

createNewStreamSource函数加上OnDemandServerMediaSubsession类是不是很熟悉？对的OnDemandServerMediaSubsession就是我们自定义的用于实现createNewStreamSource函数的类H264LiveVideoServerMediaSubssion的父类；因为createNewStreamSource在OnDemandServerMediaSubsession中是纯虚函数，因此该处调用的就是我们自定义的createNewStreamSource函数；

因此在startPlaying函数中将fSource赋值为createNewStreamSource的返回值；我们再看一下createNewStreamSource函数吧：

cpp 复制代码

FramedSource* H264LiveVideoServerMediaSubssion::createNewStreamSource(unsigned clientSessionId, unsigned& estBitrate)
{
    /* Remain to do : assign estBitrate */
    estBitrate = 1000; // kbps, estimate

    //创建视频源
    H264FramedLiveSource* liveSource = H264FramedLiveSource::createNew(envir(), Server_datasize, Server_databuf, Server_dosent);
    if (liveSource == NULL)
    {
        return NULL;
    }

    // Create a framer for the Video Elementary Stream:
    return H264VideoStreamFramer::createNew(envir(), liveSource);
}

则fSource就是H264VideoStreamFramer::createNew(envir(), liveSource); 因此getNextFrame函数中运行的doGetNextFrame就是H264VideoStreamFramer中的doGetNextFrame函数；

等等！发现问题没有：这个并不是我们自定义的H264FramedLiveSource类中获取视频帧的函数doGetNextFrame；这是为什么呢？

别急！返回值中H264VideoStreamFramer::createNew(envir(), liveSource); 将doGetNextFrame的类对象liveSource传递进了H264VideoStreamFramer类中；而liveSource最终赋值给了StreamParser类中成员变量fInputSource和FramedFilter类的成员变量fInputSource；更多细节参考我的另一篇文章；

接着往下说：H264VideoStreamFramer中的doGetNextFrame函数会调用MPEGVideoStreamFramer中的doGetNextFrame函数；在这个函数中有一个ensureValidBytes1第一次调用了自定义的函数doGetNextFrame

cpp 复制代码

void StreamParser::ensureValidBytes1(unsigned numBytesNeeded) {
.
.
.
  fInputSource->getNextFrame(&curBank()[fTotNumValidBytes],
           maxNumBytesToRead,
           afterGettingBytes, this,
           onInputClosure, this);

  throw NO_MORE_BUFFERED_INPUT;
}

这里仅仅是对流的解析；下面才是真正获取视频流：

cpp 复制代码

void H264or5Fragmenter::afterGettingFrame(void* clientData, unsigned frameSize,
            unsigned numTruncatedBytes,
            struct timeval presentationTime,
            unsigned durationInMicroseconds) {
  H264or5Fragmenter* fragmenter = (H264or5Fragmenter*)clientData;
  fragmenter->afterGettingFrame1(frameSize, numTruncatedBytes, presentationTime,
         durationInMicroseconds);
}

void H264or5Fragmenter::afterGettingFrame1(unsigned frameSize,
             unsigned numTruncatedBytes,
             struct timeval presentationTime,
             unsigned durationInMicroseconds) {
  fNumValidDataBytes += frameSize;
  fSaveNumTruncatedBytes = numTruncatedBytes;
  fPresentationTime = presentationTime;
  fDurationInMicroseconds = durationInMicroseconds;

  // Deliver data to the client:
  doGetNextFrame();
}

void H264or5Fragmenter::doGetNextFrame() {
  if (fNumValidDataBytes == 1) {
    // We have no NAL unit data currently in the buffer.  Read a new one:
    fInputSource->getNextFrame(&fInputBuffer[1], fInputBufferSize - 1,
             afterGettingFrame, this,
             FramedSource::handleClosure, this);
  } else {
  ...
  }
}

afterGettingFrame函数才是真正调用doGetNextFrame的函数？那么afterGettingFrame是在哪里被调用的呢？

看一下fSource->getNextFrame函数：

cpp 复制代码

fInputSource->getNextFrame(fOutBuf->curPtr(), fOutBuf->totalBytesAvailable(),
                          afterGettingFrame, this, ourHandleClosure, this);

这里第三个参数就调用了afterGettingFrame函数，再看看getNextFrame函数实现；上面已经写过了为了方便理解这里再写下：

cpp 复制代码

void FramedSource::getNextFrame(unsigned char* to, unsigned maxSize,
        afterGettingFunc* afterGettingFunc,
        void* afterGettingClientData,
        onCloseFunc* onCloseFunc,
        void* onCloseClientData) {
  // Make sure we're not already being read:
  if (fIsCurrentlyAwaitingData) {
    envir() << "FramedSource[" << this << "]::getNextFrame(): attempting to read more than once at the same time!\n";
    envir().internalError();
  }

  fTo = to;
  fMaxSize = maxSize;
  fNumTruncatedBytes = 0; // by default; could be changed by doGetNextFrame()
  fDurationInMicroseconds = 0; // by default; could be changed by doGetNextFrame()
  fAfterGettingFunc = afterGettingFunc;
  fAfterGettingClientData = afterGettingClientData;
  fOnCloseFunc = onCloseFunc;
  fOnCloseClientData = onCloseClientData;
  fIsCurrentlyAwaitingData = True;

  doGetNextFrame();
}

afterGettingFrame被赋值给了fAfterGettingClientData指针；那么fAfterGettingClientData指针什么时候被调用的？

回望doGetNextFrame的实现，每次读完流都会调用:

cpp 复制代码

afterGetting(this);
//函数实现
void FramedSource::afterGetting(FramedSource* source) {
  source->nextTask() = NULL;
  source->fIsCurrentlyAwaitingData = False;
      // indicates that we can be read again
      // Note that this needs to be done here, in case the "fAfterFunc"
      // called below tries to read another frame (which it usually will)

  if (source->fAfterGettingFunc != NULL) {
    (*(source->fAfterGettingFunc))(source->fAfterGettingClientData,
           source->fFrameSize, source->fNumTruncatedBytes,
           source->fPresentationTime,
           source->fDurationInMicroseconds);
  }
}

afterGetting的函数实现中就调用了fAfterGettingFunc指针；这就使doGetNextFrame形成了闭环：

至此函数doGetNextFrame的执行流程已经全部解析完毕，后续还会继续更新关于doGetNextFrame函数获取的帧数据是怎么处理和发送的，H264VideoStreamFramer中的doGetNextFrame函数和MPEGVideoStreamFramer中的doGetNextFrame函数都是什么作用！期待的话关注我，了解最新动态！

该文章持续更新！如果有错误或者模糊的地方欢迎留言探讨！