基于D1开发板和腾讯云nginx服务器构建家庭视频监控方案

腾讯云服务器使用nginx搭建rtmp服务器

什么是nginx？

nginx是一款优秀的反向代理工具，通过nginx可以实现搭建高可用的轻量级web服务器，除此之外，通过Nginx自带的rtmp模块，也可以实现rtmp服务器的搭建。

安装nginx

安装编译 nginx 所需要的库

 sudo apt-get install build-essential libpcre3 libpcre3-dev libssl-dev

下载 nginx-1.21.6.tar.gz

wget http://nginx.org/download/nginx-1.21.6.tar.gz

下载 nginx-rtmp-module

wget https://github.com/arut/nginx-rtmp-module/archive/master.zip

解压nginx文件

tar -zxvf nginx-1.21.6.tar.gz

解压rtmp模块

unzip master.zip

编译

./configure --with-http_ssl_module --add-module=../nginx-rtmp-module-master

安装

make
sudo make install

启动nginx，检测nginx是否能成功运行

sudo /usr/local/nginx/sbin/nginx

配置nginx使用RTMP, /usr/local/nginx/conf/nginx.conf

rtmp {
        server {
                listen 1935;
                chunk_size 4096;
 
                application live {
                        live on;
                        record off;
                        #hls on;
                        #hls_fragment 5s;
                }
        }
}

开放1935端口

• iptables -I INPUT -p tcp --dport 1935 -j ACCEPT
• 在云服务器设置防火墙中打开端口

重启nginx服务器

sudo /usr/local/nginx/sbin/nginx -s stop
sudo /usr/local/nginx/sbin/nginx

确认端口是否开启成功

netstat -npl | grep 1935
netstat -antp | grep 1935

Gstreamer推流

gst-launch-1.0 videotestsrc ! openh264enc ! h264parse ! flvmux ! rtmp2sink location="rtmp://114.132.63.31:1935/live/test"

VLC拉流

rtmp://114.132.63.31:1935/live/test

D1开发板环境搭建

环境准备

• 先准备D1基本环境，确保能正常编译/烧写/运行
• 默认未将gstreamer编译进来
• 通过make menuconfig将gstreamer的相关组件编译进来

奇怪异常临时处理

异常1

/home/fuqiang/workspace/tina-d1-h/out/d1-h-nezha/compile_dir/target/alsa-plugins-1.1.4/ipkg-install/usr/share/alsa/alsa.conf.d/
找不到这个目录的一些文件

解决方案：

cp ./out/d1-h-nezha/compile_dir/target/alsa-plugins-1.1.4/pulse/*  /home/fuqiang/workspace/tina-d1-h/out/d1-h-nezha/compile_dir/target/alsa-plugins-1.1.4/ipkg-install/usr/share/alsa/alsa.conf.d/

在另一个目录中找到，cp过去

异常2

( cd /home/fuqiang/workspace/tina-d1-h/out/d1-h-nezha/compile_dir/target/gst-plugins-bad-1.16.3/ipkg-install; cp -fpR ./usr/lib/libgstwayland-1.0.so.* /home/fuqiang/workspace/tina-d1-h/out/d1-h-nezha/compile_dir/target/gst-plugins-bad-1.16.3/ipkg-sunxi/libgst1wayland/usr/lib/ )

cp: cannot stat './usr/lib/libgstwayland-1.0.so.*': No such file or directory

解决方案：

环境验证

连接wifi

wifi_scan_results_test
wifi_connect_ap_test deecomp deecomp20210701
ping www.baidu.com

FLV

Header

Header 部分记录了FLV的类型、版本、流信息、Header 长度等。一般整个Header占用9个字节，大于9个字节则表示头部信息在这基础之上还存在扩展数据。FLV Header 如下所示：

Body

Body 是由一个个 PreviousTag+Tag 单元组成的。每个Tag下面有一块4个字节的空间，用来记录这个Tag 的长度。PreviousTagSize用于逆向读取处理，表示的是前面的Tag的大小，第一个PreviousTagSize为0，因为他前面没有tag。FLV Body 的信息排布如下：

Tag

每个Tag 也是由两部分组成的：Tag Header 和 Tag Data

Tag Header

Tag Header 存放了当前Tag的类型，数据长度、时间戳、时间戳扩展、StreamsID等信息，然后再接着数据区Tag Data。Tag的排布如下：

Tag Data

Audio Tag Data

Video Tag Data

H.264/AVC：

RTMP

• 1.RTMP(实时消息传输协议)是Adobe 公司开发的一个基于TCP的应用层协议
• 2.RTMP协议中基本的数据单元称为消息（Message）
• 3.当RTMP协议在互联网中传输数据的时候，消息会被拆分成更小的单元，称为消息块（Chunk）
• 基于TCP
• 包括RTMP基本协议及RTMPT/RTMPS/RTMPE等多种变种

librtmp（rtmpdump:RTMPDump (mplayerhq.hu)）

librtmp库实现了rtmp协议的客户端功能，以及少数服务端功能，是学习rtmp的一个良好的工具。

librtmp API的使用流程如下：

• 分配并初始化一个RTMP实例
• 设置RTMP实例的参数
• 调用RTMP_Connect函数连接到服务器
• 调用RTMP_ConnectStream函数创建一个rtmp流
• 使用RTMP_Read或RTMP_Write进行读写，或者对rtmp流进行控制（如暂停、停止等）
• 读写完成后，关闭并释放RTMP

rtmpsink

Gstreamer的rtmpsink是基于librtmp实现的

rtmp2sink（gst-plugins-bad/gst/rtmp2）

fuqiang@ubuntu:~/workspace/gstreamer/subprojects/gst-plugins-bad/gst/rtmp2$ tree
.
├── gstrtmp2.c  //plugin注册
├── gstrtmp2element.c  //element注册
├── gstrtmp2elements.h
├── gstrtmp2locationhandler.c  //对location参数的解析处理
├── gstrtmp2locationhandler.h
├── gstrtmp2sink.c  //sink端实现
├── gstrtmp2sink.h
├── gstrtmp2src.c  //src端实现
├── gstrtmp2src.h
├── meson.build
├── rtmp  //rtmp协议具体实现
│   ├── amf.c
│   ├── amf.h
│   ├── rtmpchunkstream.c
│   ├── rtmpchunkstream.h
│   ├── rtmpclient.c
│   ├── rtmpclient.h
│   ├── rtmpconnection.c
│   ├── rtmpconnection.h
│   ├── rtmphandshake.c
│   ├── rtmphandshake.h
│   ├── rtmpmessage.c
│   ├── rtmpmessage.h
│   ├── rtmputils.c
│   └── rtmputils.h
└── TODO

1 directory, 25 files

rtmp2sink的源码分析：

typedef struct
{
  GstBaseSink parent_instance;

  /* properties */
  GstRtmpLocation location;  //推流地址
  gboolean async_connect;
  guint peak_kbps;
  guint32 chunk_size;
  GstRtmpStopCommands stop_commands;
  GstStructure *stats;

  /* If both self->lock and OBJECT_LOCK are needed,
   * self->lock must be taken first */
  GMutex lock;
  GCond cond;

  gboolean running, flushing;

  GstTask *task;
  GRecMutex task_lock;

  GMainLoop *loop;
  GMainContext *context;

  GCancellable *cancellable;
  GstRtmpConnection *connection;
  guint32 stream_id;

  GPtrArray *headers;
  guint64 last_ts, base_ts;     /* timestamp fixup */
} GstRtmp2Sink;
...
static GstStaticPadTemplate gst_rtmp2_sink_sink_template =  //接收数据格式只能为flv
GST_STATIC_PAD_TEMPLATE ("sink",
    GST_PAD_SINK,
    GST_PAD_ALWAYS,
    GST_STATIC_CAPS ("video/x-flv")
    );
...
static void
gst_rtmp2_sink_init (GstRtmp2Sink * self)  //实例初始化
{
  self->location.flash_ver = g_strdup ("FMLE/3.0 (compatible; FMSc/1.0)");
  self->location.publish = TRUE;
  self->async_connect = TRUE;
  self->chunk_size = GST_RTMP_DEFAULT_CHUNK_SIZE;
  self->stop_commands = GST_RTMP_DEFAULT_STOP_COMMANDS;

  g_mutex_init (&self->lock);
  g_cond_init (&self->cond);

  self->task = gst_task_new (gst_rtmp2_sink_task_func, self, NULL);  //task线程
  g_rec_mutex_init (&self->task_lock);
  gst_task_set_lock (self->task, &self->task_lock);

  self->headers = g_ptr_array_new_with_free_func
      ((GDestroyNotify) gst_mini_object_unref);
}

static gboolean
gst_rtmp2_sink_start (GstBaseSink * sink)
{
  GstRtmp2Sink *self = GST_RTMP2_SINK (sink);
  gboolean async;

  GST_OBJECT_LOCK (self);
  async = self->async_connect;
  GST_OBJECT_UNLOCK (self);

  GST_INFO_OBJECT (self, "Starting (%s)", async ? "async" : "delayed");

  g_clear_object (&self->cancellable);

  self->running = TRUE;
  self->cancellable = g_cancellable_new ();
  self->stream_id = 0;
  self->last_ts = 0;
  self->base_ts = 0;

  if (async) {
    gst_task_start (self->task);
 //task线程开启，这里面主要做connect的工作
  }

  return TRUE;
}


static GstFlowReturn
gst_rtmp2_sink_render(GstBaseSink * sink, GstBuffer * buffer)  //父类gstbasesink的chain调下来，用于传输buffer
{
  GstRtmp2Sink *self = GST_RTMP2_SINK (sink);
  GstBuffer *message;
  GstFlowReturn ret;

  if (G_UNLIKELY (should_drop_header (self, buffer))) {
    GST_DEBUG_OBJECT (self, "Skipping header %" GST_PTR_FORMAT, buffer);
    return GST_FLOW_OK;
  }

  GST_LOG_OBJECT (self, "render %" GST_PTR_FORMAT, buffer);

  if (G_UNLIKELY (!buffer_to_message (self, buffer, &message))) {  //FLV--->RTMP
    GST_ELEMENT_ERROR (self, STREAM, FAILED, ("Failed to convert FLV to RTMP"),
        ("Failed to convert %" GST_PTR_FORMAT, message));
    return GST_FLOW_ERROR;
  }

  if (G_UNLIKELY (!message)) {
    GST_DEBUG_OBJECT (self, "Skipping %" GST_PTR_FORMAT, buffer);
    return GST_FLOW_OK;
  }

  g_mutex_lock (&self->lock);

  if (G_UNLIKELY (is_running (self) && self->cancellable &&
          gst_task_get_state (self->task) != GST_TASK_STARTED)) {
    GST_DEBUG_OBJECT (self, "Starting connect");
    gst_task_start (self->task);
  }

  while (G_UNLIKELY (is_running (self) && !self->connection)) {
    GST_DEBUG_OBJECT (self, "Waiting for connection");
    g_cond_wait (&self->cond, &self->lock);
  }

  while (G_UNLIKELY (is_running (self) && self->connection &&
          gst_rtmp_connection_get_num_queued (self->connection) > 3)) {
    GST_LOG_OBJECT (self, "Waiting for queue");
    g_cond_wait (&self->cond, &self->lock);
  }

  if (G_UNLIKELY (!is_running (self))) {
    gst_buffer_unref (message);
    ret = GST_FLOW_FLUSHING;
  } else if (G_UNLIKELY (!self->connection)) {
    gst_buffer_unref (message);
    /* send_connect_error has sent an ERROR message */
    ret = GST_FLOW_ERROR;
  } else {
    send_streamheader (self);
send_message (self, message);  //传输数据
    ret = GST_FLOW_OK;
  }

  g_mutex_unlock (&self->lock);
  return ret;
}

static void
send_message (GstRtmp2Sink * self, GstBuffer * message)
{
  GstRtmpMeta *meta = gst_buffer_get_rtmp_meta (message);

  g_return_if_fail (meta != NULL);
  g_return_if_fail (self->stream_id != 0);

  meta->mstream = self->stream_id;

  if (gst_rtmp_message_is_metadata (message)) {
    gst_rtmp_connection_set_data_frame (self->connection, message);  //传输数据
  } else {
    gst_rtmp_connection_queue_message (self->connection, message);
  }
}

FLV转为RTMP主要实现如下：

static gboolean
buffer_to_message(GstRtmp2Sink * self, GstBuffer * buffer, GstBuffer ** outbuf)
{
  GstBuffer *message;
  GstRtmpFlvTagHeader header;
  guint64 timestamp;
  guint32 cstream;

  {
    GstMapInfo info;

    if (G_UNLIKELY (!gst_buffer_map (buffer, &info, GST_MAP_READ))) {
      GST_ERROR_OBJECT (self, "map failed: %" GST_PTR_FORMAT, buffer);
      return FALSE;
    }

    /* FIXME: This is ugly and only works behind flvmux.
     *        Implement true RTMP muxing. */

    if (G_UNLIKELY (info.size >= 4 && memcmp (info.data, "FLV", 3) == 0)) {  //确实是否为FLV格式的数据输入
      /* drop the header, we don't need it */
      GST_DEBUG_OBJECT (self, "ignoring FLV header: %" GST_PTR_FORMAT, buffer);
      gst_buffer_unmap (buffer, &info);
      *outbuf = NULL;
      return TRUE;
    }

    if (!gst_rtmp_flv_tag_parse_header (&header, info.data, info.size)) {  //解析tag header
      GST_ERROR_OBJECT (self, "too small for tag header: %" GST_PTR_FORMAT,
          buffer);
      gst_buffer_unmap (buffer, &info);
      return FALSE;
    }

    if (info.size < header.total_size) {
      GST_ERROR_OBJECT (self, "too small for tag body: buffer %" G_GSIZE_FORMAT
          ", tag %" G_GSIZE_FORMAT, info.size, header.total_size);
      gst_buffer_unmap (buffer, &info);
      return FALSE;
    }

    /* flvmux timestamps roll over after about 49 days */  //时间戳调整
    timestamp = header.timestamp;
    if (timestamp + self->base_ts + G_MAXINT32 < self->last_ts) {
      GST_WARNING_OBJECT (self, "Timestamp regression %" G_GUINT64_FORMAT
          " -> %" G_GUINT64_FORMAT "; assuming overflow", self->last_ts,
          timestamp + self->base_ts);
      self->base_ts += G_MAXUINT32;
      self->base_ts += 1;
    } else if (timestamp + self->base_ts > self->last_ts + G_MAXINT32) {
      GST_WARNING_OBJECT (self, "Timestamp jump %" G_GUINT64_FORMAT
          " -> %" G_GUINT64_FORMAT "; assuming underflow", self->last_ts,
          timestamp + self->base_ts);
      if (self->base_ts > 0) {
        self->base_ts -= G_MAXUINT32;
        self->base_ts -= 1;
      } else {
        GST_WARNING_OBJECT (self, "Cannot regress further;"
            " forcing timestamp to zero");
        timestamp = 0;
      }
    }
    timestamp += self->base_ts;
    self->last_ts = timestamp;

    gst_buffer_unmap (buffer, &info);
  }

  switch (header.type) {  //获取stream的类型
    case GST_RTMP_MESSAGE_TYPE_DATA_AMF0:
      cstream = 4;
      break;

    case GST_RTMP_MESSAGE_TYPE_AUDIO:
      cstream = 5;
      break;

    case GST_RTMP_MESSAGE_TYPE_VIDEO:
      cstream = 6;
      break;

    default:
      GST_ERROR_OBJECT (self, "unknown tag type %d", header.type);
      return FALSE;
  }

  /* May not know stream ID yet; set later */
  message = gst_rtmp_message_new (header.type, cstream, 0);  //创建新buffer
  message = gst_buffer_append_region (message, gst_buffer_ref (buffer),
GST_RTMP_FLV_TAG_HEADER_SIZE, header.payload_size);  //根据payload_size的大小来分配内存

  GST_BUFFER_DTS (message) = timestamp * GST_MSECOND;

  *outbuf = message;
  return TRUE;
}

gboolean
gst_rtmp_flv_tag_parse_header(GstRtmpFlvTagHeader * header,  //解析tag header
    const guint8 * data, gsize size)
{
  g_return_val_if_fail (header, FALSE);
  g_return_val_if_fail (data, FALSE);

  /* Parse FLVTAG header as described in
   * video_file_format_spec_v10.pdf page 5 (page 9 of the PDF) */

  if (size < GST_RTMP_FLV_TAG_HEADER_SIZE) {
    return FALSE;
  }

  /* TagType UI8 */
  header->type = GST_READ_UINT8 (data); //1字节

  /* DataSize UI24 */
  header->payload_size = GST_READ_UINT24_BE (data + 1);  //3字节

  /* 4 bytes for the PreviousTagSize UI32 following every tag */
  header->total_size = GST_RTMP_FLV_TAG_HEADER_SIZE + header->payload_size + 4;

  /* Timestamp UI24 + TimestampExtended UI8 */
  header->timestamp = GST_READ_UINT24_BE (data + 4);
header->timestamp |= (guint32) GST_READ_UINT8 (data + 7) << 24;

  /* Skip StreamID UI24. It's "always 0" for FLV files and for aggregated RTMP
* messages we're supposed to use the Stream ID from the AGGREGATE. */

  return TRUE;
}