一套rk3588 rtsp服务器推流的 github 方案及记录 -03(完结)

define_mine2023-12-23 4:47

opencv 解码记录

解码库使用的时候发现瑞芯微以前做过解码库对ffmpeg和gstreamer的支持

然后最近实在不想再调试Rtsp浪费时间了，就从这中间找了一个比较快的方案

ffmpeg 带硬解码库编译

编译流程参考文献

https://blog.csdn.net/T__zxt/article/details/123424359

编译的ffmpeg是这个版本

https://github.com/jjm2473

安装了下面这些东西：

bash 复制代码 
sudo apt-get -y install \
  autoconf \
  automake \
  build-essential \
  cmake \
  git-core \
  libass-dev \
  libfreetype6-dev \
  libsdl2-dev \
  libtool \
  libva-dev \
  libvdpau-dev \
  libvorbis-dev \
  libxcb1-dev \
  libxcb-shm0-dev \
  libxcb-xfixes0-dev \
  pkg-config \
  texinfo \
  wget \
  zlib1g-dev

编译选项参考的是这哥们的语句：

https://www.reddit.com/r/OrangePI/comments/14f725v/comment/jpcezy2/?context=3

下面这句话，哪个不能用删哪个，一定要有的是 --enable-rkmpp

bash 复制代码 
./configure --enable-nonfree --enable-gpl --enable-version3 --enable-libx264 --enable-libdrm --enable-rkmpp --enable-librtmp --enable-shared --enable-static --enable-librga --enable-libx265 --enable-libmp3lame --enable-libpulse --enable-openssl --enable-libopus --enable-libvorbis
# 我最后用的 应该是
 sudo apt-get -y install   autoconf   automake   build-essential   cmake   git-core   libass-dev   libfreetype6-dev   libsdl2-dev   libtool   libva-dev   libvdpau-dev   libvorbis-dev   libxcb1-dev   libxcb-shm0-dev   libxcb-xfixes0-dev   pkg-config   texinfo   wget   zlib1g-dev  
./configure --enable-nonfree --enable-gpl --enable-version3 --enable-libx264 --enable-libdrm --enable-rkmpp  --enable-shared --enable-static --enable-librga --enable-libx265  --enable-libpulse --enable-openssl --enable-libvorbis

使用板上编译的时候记得多等一会，有点慢，不要急

编译出来的结果需要使用下面的语句去看ffmpeg是否正常

bash 复制代码 
ffmpeg -decoders |grep h264

如果编译之后，ffmpeg不支持，

修改

bash 复制代码 
sudo vim /etc/ld.so.conf
shell 复制代码 
include /etc/ld.so.conf.d/*.conf
/usr/local/lib

打开之前ldd看一眼是不是所有的链接库都找到地方了

bash 复制代码 
ffmpeg-rk-enc-5.1$ ldd ffmpeg
        linux-vdso.so.1 (0x0000007fbb328000)
        libavdevice.so.59 => /usr/local/lib/libavdevice.so.59 (0x0000007fbb230000)
        libavfilter.so.8 => /usr/local/lib/libavfilter.so.8 (0x0000007fbae60000)
        libavformat.so.59 => /usr/local/lib/libavformat.so.59 (0x0000007fbabe0000)
        libavcodec.so.59 => /usr/local/lib/libavcodec.so.59 (0x0000007fb9ae0000)
        libpostproc.so.56 => /usr/local/lib/libpostproc.so.56 (0x0000007fb9ac0000)
        libswresample.so.4 => /usr/local/lib/libswresample.so.4 (0x0000007fb9a90000)
        libswscale.so.6 => /usr/local/lib/libswscale.so.6 (0x0000007fb9a10000)
        libavutil.so.57 => /usr/local/lib/libavutil.so.57 (0x0000007fb9830000)
        libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007fb9790000)
        libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007fb95e0000)
        /lib/ld-linux-aarch64.so.1 (0x0000007fbb2ef000)
        libdrm.so.2 => /lib/aarch64-linux-gnu/libdrm.so.2 (0x0000007fb95b0000)
        libxcb.so.1 => /lib/aarch64-linux-gnu/libxcb.so.1 (0x0000007fb9570000)
        libxcb-shm.so.0 => /lib/aarch64-linux-gnu/libxcb-shm.so.0 (0x0000007fb9550000)
        libxcb-shape.so.0 => /lib/aarch64-linux-gnu/libxcb-shape.so.0 (0x0000007fb9530000)
        libxcb-xfixes.so.0 => /lib/aarch64-linux-gnu/libxcb-xfixes.so.0 (0x0000007fb9510000)
        libasound.so.2 => /lib/aarch64-linux-gnu/libasound.so.2 (0x0000007fb93f0000)
        libpulse.so.0 => /lib/aarch64-linux-gnu/libpulse.so.0 (0x0000007fb9390000)
        libSDL2-2.0.so.0 => /lib/aarch64-linux-gnu/libSDL2-2.0.so.0 (0x0000007fb91f0000)
        libsndio.so.7 => /lib/aarch64-linux-gnu/libsndio.so.7 (0x0000007fb91c0000)
        libXv.so.1 => /lib/aarch64-linux-gnu/libXv.so.1 (0x0000007fb91a0000)
        libX11.so.6 => /lib/aarch64-linux-gnu/libX11.so.6 (0x0000007fb9050000)
        libXext.so.6 => /lib/aarch64-linux-gnu/libXext.so.6 (0x0000007fb9020000)
        libavrkmpp.so.1 => /usr/local/lib/libavrkmpp.so.1 (0x0000007fb9000000)
        libva.so.2 => /lib/aarch64-linux-gnu/libva.so.2 (0x0000007fb8fc0000)
        libz.so.1 => /lib/aarch64-linux-gnu/libz.so.1 (0x0000007fb8f90000)
        libssl.so.3 => /lib/aarch64-linux-gnu/libssl.so.3 (0x0000007fb8ee0000)
        libcrypto.so.3 => /lib/aarch64-linux-gnu/libcrypto.so.3 (0x0000007fb8af0000)
        liblzma.so.5 => /lib/aarch64-linux-gnu/liblzma.so.5 (0x0000007fb8ab0000)
        librockchip_mpp.so.1 => /lib/aarch64-linux-gnu/librockchip_mpp.so.1 (0x0000007fb88b0000)
        libvorbis.so.0 => /lib/aarch64-linux-gnu/libvorbis.so.0 (0x0000007fb8870000)
        libvorbisenc.so.2 => /lib/aarch64-linux-gnu/libvorbisenc.so.2 (0x0000007fb87b0000)
        libx264.so.163 => /lib/aarch64-linux-gnu/libx264.so.163 (0x0000007fb8560000)
        libx265.so.199 => /lib/aarch64-linux-gnu/libx265.so.199 (0x0000007fb82a0000)
        librga.so.2 => /lib/aarch64-linux-gnu/librga.so.2 (0x0000007fb8270000)
        libva-drm.so.2 => /lib/aarch64-linux-gnu/libva-drm.so.2 (0x0000007fb8250000)
        libva-x11.so.2 => /lib/aarch64-linux-gnu/libva-x11.so.2 (0x0000007fb8230000)
        libvdpau.so.1 => /lib/aarch64-linux-gnu/libvdpau.so.1 (0x0000007fb8210000)
        libXau.so.6 => /lib/aarch64-linux-gnu/libXau.so.6 (0x0000007fb81f0000)
        libXdmcp.so.6 => /lib/aarch64-linux-gnu/libXdmcp.so.6 (0x0000007fb81d0000)
        libpulsecommon-15.99.so => /usr/lib/aarch64-linux-gnu/pulseaudio/libpulsecommon-15.99.so (0x0000007fb8140000)
        libdbus-1.so.3 => /lib/aarch64-linux-gnu/libdbus-1.so.3 (0x0000007fb80e0000)
        libXcursor.so.1 => /lib/aarch64-linux-gnu/libXcursor.so.1 (0x0000007fb80c0000)
        libXinerama.so.1 => /lib/aarch64-linux-gnu/libXinerama.so.1 (0x0000007fb80a0000)
        libXi.so.6 => /lib/aarch64-linux-gnu/libXi.so.6 (0x0000007fb8070000)
        libXfixes.so.3 => /lib/aarch64-linux-gnu/libXfixes.so.3 (0x0000007fb8050000)
        libXrandr.so.2 => /lib/aarch64-linux-gnu/libXrandr.so.2 (0x0000007fb8030000)
        libXss.so.1 => /lib/aarch64-linux-gnu/libXss.so.1 (0x0000007fb8010000)
        libXxf86vm.so.1 => /lib/aarch64-linux-gnu/libXxf86vm.so.1 (0x0000007fb7ff0000)
        libgbm.so.1 => /lib/aarch64-linux-gnu/libgbm.so.1 (0x0000007fb7fd0000)
        libwayland-egl.so.1 => /lib/aarch64-linux-gnu/libwayland-egl.so.1 (0x0000007fb7fb0000)
        libwayland-client.so.0 => /lib/aarch64-linux-gnu/libwayland-client.so.0 (0x0000007fb7f90000)
        libwayland-cursor.so.0 => /lib/aarch64-linux-gnu/libwayland-cursor.so.0 (0x0000007fb7f70000)
        libxkbcommon.so.0 => /lib/aarch64-linux-gnu/libxkbcommon.so.0 (0x0000007fb7f10000)
        libdecor-0.so.0 => /lib/aarch64-linux-gnu/libdecor-0.so.0 (0x0000007fb7ef0000)
        libbsd.so.0 => /lib/aarch64-linux-gnu/libbsd.so.0 (0x0000007fb7ec0000)
        libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007fb7c90000)
        libogg.so.0 => /lib/aarch64-linux-gnu/libogg.so.0 (0x0000007fb7c70000)
        libnuma.so.1 => /lib/aarch64-linux-gnu/libnuma.so.1 (0x0000007fb7c40000)
        libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007fb7c10000)
        libsndfile.so.1 => /lib/aarch64-linux-gnu/libsndfile.so.1 (0x0000007fb7b80000)
        libX11-xcb.so.1 => /lib/aarch64-linux-gnu/libX11-xcb.so.1 (0x0000007fb7b60000)
        libsystemd.so.0 => /lib/aarch64-linux-gnu/libsystemd.so.0 (0x0000007fb7a80000)
        libasyncns.so.0 => /lib/aarch64-linux-gnu/libasyncns.so.0 (0x0000007fb7a60000)
        libapparmor.so.1 => /lib/aarch64-linux-gnu/libapparmor.so.1 (0x0000007fb7a30000)
        libXrender.so.1 => /lib/aarch64-linux-gnu/libXrender.so.1 (0x0000007fb7a10000)
        libwayland-server.so.0 => /lib/aarch64-linux-gnu/libwayland-server.so.0 (0x0000007fb79e0000)
        libexpat.so.1 => /lib/aarch64-linux-gnu/libexpat.so.1 (0x0000007fb79a0000)
        libffi.so.8 => /lib/aarch64-linux-gnu/libffi.so.8 (0x0000007fb7980000)
        libmd.so.0 => /lib/aarch64-linux-gnu/libmd.so.0 (0x0000007fb7960000)
        libFLAC.so.8 => /lib/aarch64-linux-gnu/libFLAC.so.8 (0x0000007fb7910000)
        libopus.so.0 => /lib/aarch64-linux-gnu/libopus.so.0 (0x0000007fb78a0000)
        libzstd.so.1 => /opt/ros/humble/lib/libzstd.so.1 (0x0000007fb7810000)
        liblz4.so.1 => /lib/aarch64-linux-gnu/liblz4.so.1 (0x0000007fb77e0000)
        libcap.so.2 => /lib/aarch64-linux-gnu/libcap.so.2 (0x0000007fb77c0000)
        libgcrypt.so.20 => /lib/aarch64-linux-gnu/libgcrypt.so.20 (0x0000007fb76d0000)
        libgpg-error.so.0 => /lib/aarch64-linux-gnu/libgpg-error.so.0 (0x0000007fb7690000)

至此，编译好了一个支持硬件解码的ffmpeg，记得make install ，然后去编译opencv,让opencv支持ffmpeg

opencv编译带硬解码ffmpeg

原来的opencv是这样的：

shell 复制代码 
OpenCV FFmpeg version: 
General configuration for OpenCV 4.5.5 =====================================
  Version control:               unknown

  Platform:
    Timestamp:                   2023-09-25T04:22:20Z
    Host:                        Linux 5.10.110-rockchip-rk3588 aarch64
    CMake:                       3.22.1
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/gmake
    Configuration:               Release

  CPU/HW features:
    Baseline:                    NEON FP16

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                /usr/bin/c++  (ver 11.4.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--gc-sections -Wl,--as-needed  
    Linker flags (Debug):        -Wl,--gc-sections -Wl,--as-needed  
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          dl m pthread rt
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python3 stitching ts video videoio
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 java python2
    Applications:                tests perf_tests apps
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:                           QT5
    QT:                          YES (ver 5.15.3 )
      QT OpenGL support:         YES (Qt5::OpenGL 5.15.3)
    GTK+:                        YES (ver 3.24.33)
      GThread :                  YES (ver 2.72.4)
      GtkGlExt:                  NO
    OpenGL support:              YES (/usr/lib/aarch64-linux-gnu/libGL.so /usr/lib/aarch64-linux-gnu/libGLU.so)
    VTK support:                 YES (ver 9.1.0)

  Media I/O: 
    ZLib:                        /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
    WEBP:                        /usr/lib/aarch64-linux-gnu/libwebp.so (ver encoder: 0x020f)
    PNG:                         /lib/libpng.so (ver 1.6.37)
    TIFF:                        /usr/lib/aarch64-linux-gnu/libtiff.so (ver 42 / 4.3.0)
    JPEG 2000:                   OpenJPEG (ver 2.4.0)
    OpenEXR:                     /usr/lib/aarch64-linux-gnu/libImath-2_5.so /usr/lib/aarch64-linux-gnu/libIlmImf-2_5.so /usr/lib/aarch64-linux-gnu/libIex-2_5.so /usr/lib/aarch64-linux-gnu/libHalf-2_5.so /usr/lib/aarch64-linux-gnu/libIlmThread-2_5.so (ver 2_5)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      YES (2.2.6)
    FFMPEG:                      YES
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                NO
    GStreamer:                   YES (1.20.3)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Lapack:                      NO
    Eigen:                       YES (ver 3.4.0)
    Custom HAL:                  YES (carotene (ver 0.0.1))
    Protobuf:                    build (3.19.1)

  OpenCL:                        YES (no extra features)
    Include path:                /home/orangepi/opencv-4.5.5/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.10.12)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython3.10.so (ver 3.10.12)
    numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.21.5)
    install path:                lib/python3.10/dist-packages/cv2/python-3.10

  Python (for build):            /usr/bin/python2.7

  Java:                          
    ant:                         NO
    JNI:                         /usr/lib/jvm/default-java/include /usr/lib/jvm/default-java/include/linux /usr/lib/jvm/default-java/include
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /usr/local

opencv 4.5.5来讲，这个ffmpeg太新了，编译不过。只能以毒攻毒，去下了opencv4.8.0

回头把这个打包上传到csdn资源，防止它小更新不能用

4.8.0是编译通过的

一样也是板端编译，cmake-gui配置,然后去build文件夹下面 make make install

参考文献：

https://blog.csdn.net/Flag_ing/article/details/109508374

注意 ，如果使用qt，一定要把qt勾选，否则在qt里运行会报错

还有opencl他也不是自动勾选的

其他的应该随意

然后去创建工程使用、测试opencv是不是能正常用

测试代码

c++ 复制代码 
// RTSP Server

#include "xop/RtspServer.h"
#include "net/Timer.h"
#include <thread>
#include <memory>
#include <iostream>
#include <string>


#include <opencv2/opencv.hpp>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>

// 编码库 
#include <encoder.h>
// 解码库
#include <decoder.h>
// 海康拉流库
#include "HCNetSDK.h"
#include <chrono>
#include "yolo.h"
// 查看264 二进制 文件
#include <sstream>
#include <fstream>


using namespace std;
using namespace cv;
// 定义一个缓冲池结构体
struct Buffer {
    Mat buf1;
    Mat buf2;
};

// 定义互斥锁
std::mutex mymutex;
Buffer matBuf;
VideoCapture videocapture;
int captureOpen = 0;
/**
 * @brief 读图
 * 
 * @return Mat 
 */
Mat captureThread()
{
	string rtspIN =   "rtsp://admin:s745112822@192.168.2.110:554/h264/ch0/main/av_stream";
	if (captureOpen == 0 )
	{	

		videocapture.open(rtspIN,CAP_FFMPEG);
		captureOpen = 1;
		//打开图像
		
		// capture.open(url);
	}
	while (1)
	{
		//加锁
		mymutex.lock();
		// buf2->buf1

		if(matBuf.buf1.empty())
			videocapture.read(matBuf.buf1); 
		else if(matBuf.buf2.empty())
			videocapture.read(matBuf.buf2);
		else{
			matBuf.buf1 = matBuf.buf2;
			videocapture.read(matBuf.buf2);
		}
		// 没有读到数据 网络问题 重启rtsp
		if(matBuf.buf1.empty()){
			videocapture.release();
			videocapture.open(rtspIN,CAP_FFMPEG);
		}
		//读图成功
		// imgready = true;
 
		//解锁
		mymutex.unlock();
 
		xop::Timer::Sleep(10);
	}
}

void writeCharPointerToFile(const char* data, std::size_t size, const std::string& filename) {
    // 打开文件
    std::ofstream file(filename, ios::binary);

    // 检查文件是否成功打开
    if (!file.is_open()) {
        std::cerr << "无法打开文件: " << filename << std::endl;
        return;
    }

    // 写入数据
    file.write(data, size);

    // 关闭文件
    file.close();
}

bool IsKeyFrame(const char* data, uint32_t size)
{
	if (size > 4) {
		//0x67:sps ,0x65:IDR, 0x6: SEI
		if (data[4] == 0x67 || data[4] == 0x65 || 
			data[4] == 0x6 || data[4] == 0x27) {
			return true;
		}
	}

	return false;
}
// encoder e ;
	
void SendFrameThread(xop::RtspServer* rtsp_server, xop::MediaSessionId session_id, int& clients);

int main(int argc, char **argv)
{	
	// 读图线程 
	thread t1(captureThread);
	int clients = 1;
	
	std::string ip = "0.0.0.0";
	std::string rtsp_url = "rtsp://127.0.0.1:5514/live";
	// e.init();
	std::shared_ptr<xop::EventLoop> event_loop(new xop::EventLoop());  
	std::shared_ptr<xop::RtspServer> server = xop::RtspServer::Create(event_loop.get());
	if (!server->Start(ip, 5514)) {
		return -1;
	}
	
#ifdef AUTH_CONFIG
	server->SetAuthConfig("-_-", "admin", "12345");
#endif

	 
	xop::MediaSession *session = xop::MediaSession::CreateNew("live"); // url: rtsp://ip/live
	session->AddSource(xop::channel_0, xop::H264Source::CreateNew()); 
	session->AddSource(xop::channel_1, xop::AACSource::CreateNew(44100,2));
	// session->startMulticast(); /* 开启组播(ip,端口随机生成), 默认使用 RTP_OVER_UDP, RTP_OVER_RTSP */

	session->AddNotifyConnectedCallback([] (xop::MediaSessionId sessionId, std::string peer_ip, uint16_t peer_port){
		printf("RTSP client connect, ip=%s, port=%hu \n", peer_ip.c_str(), peer_port);
	});
   
	session->AddNotifyDisconnectedCallback([](xop::MediaSessionId sessionId, std::string peer_ip, uint16_t peer_port) {
		printf("RTSP client disconnect, ip=%s, port=%hu \n", peer_ip.c_str(), peer_port);
	});

	std::cout << "URL: " << rtsp_url << std::endl;
        
	xop::MediaSessionId session_id = server->AddSession(session); 
	//server->removeMeidaSession(session_id); /* 取消会话, 接口线程安全 */
         
	std::thread thread(SendFrameThread, server.get(), session_id, std::ref(clients));
	thread.detach();

	while(1) {
		xop::Timer::Sleep(100);
	}

	getchar();
	return 0;
}

void SendFrameThread(xop::RtspServer* rtsp_server, xop::MediaSessionId session_id, int& clients)
{       
	encoder e;
	// decoder d;
    // cv::VideoCapture *videocapture = new cv::VideoCapture(0);
	// std::string rtspIN = "rtsp://admin:s745112822@192.168.2.110:554/h264/ch1/sub/av_stream";
	
	
    // yolo处理
	// char* modelname = "/home/orangepi/code/realsense/cap2/yolov5s-640-640.rknn";
	// yolo yolo5;
	// yolo5.load_model(modelname);
  //   yolo5.model_init();
//  decoder相关 结果回传
	// d.init();
	std::vector<cv::Mat> resultArray;
	int lenofmat = 0;
//  encoder相关 内存拷贝
	int size = 0;
	char* buffer ;
	cv::Mat colorBar;
	int i = 0;
	while(1)
	{
		
		if(clients > 0) /* 会话有客户端在线, 发送音视频数据 */
		{
			{     
				
				xop::AVFrame videoFrame = {0};
				// opencv 拿图 后续需要优化
				
				auto start_time = std::chrono::high_resolution_clock::now();			
				// videocapture->read(colorBar); // 拿图 
				while(matBuf.buf1.empty());
				mymutex.lock();
				colorBar = matBuf.buf1;
				matBuf.buf1 = matBuf.buf2;
				matBuf.buf2.release();
				mymutex.unlock();
				auto end_time1 = std::chrono::high_resolution_clock::now();
				// d.poststream("264stream pointer",lenofmat,resultArray);
				// printf("width is %d, height is %d",colorBar.rows,colorBar.cols);
				// yolo 处理

				// yolo5.load_Mat(colorBar);
				// yolo5.run();
				// colorBar = yolo5.getPicResult();
				auto end_time2 = std::chrono::high_resolution_clock::now();
				if(0==i){
					// 第一帧有sps信息 给他两帧拼一起
					char *buffer1;
					int size1;
					e.init(buffer1,size1);
					videoFrame.size = size1;
					e.postAframe(colorBar,buffer,size);
					videoFrame.size += size;					
					videoFrame.buffer.reset(new uint8_t[videoFrame.size]);
					memcpy(videoFrame.buffer.get(), buffer1, size1);
					memcpy(videoFrame.buffer.get()+size1, buffer, size);
					i++;
				}		
				else{
					e.postAframe(colorBar,buffer,size);
					videoFrame.size = size;  // 视频帧大小 
					videoFrame.buffer.reset(new uint8_t[videoFrame.size]);
					memcpy(videoFrame.buffer.get(), buffer, videoFrame.size);
				}

				auto end_time3 = std::chrono::high_resolution_clock::now();

				auto duration = std::chrono::duration_cast<std::chrono::milliseconds>(end_time1 - start_time);
				cout<<"===============================A==frame================================================="<<endl;
				cout << "decode mat took: " << duration.count() <<"ms\r\n";
				duration = std::chrono::duration_cast<std::chrono::milliseconds>(end_time2 - end_time1);
				cout << "rknn took: " << duration.count() <<"ms\r\n";
				duration = std::chrono::duration_cast<std::chrono::milliseconds>(end_time3 - end_time2);
				cout << "encode took: " << duration.count() <<"ms\r\n";
				videoFrame.type = IsKeyFrame(buffer, size) ? xop::VIDEO_FRAME_I : xop::VIDEO_FRAME_P;
				// videoFrame.type = 0; // 建议确定帧类型。I帧(xop::VIDEO_FRAME_I) P帧(xop::VIDEO_FRAME_P)

				videoFrame.timestamp = xop::H264Source::GetTimestamp(); // 时间戳, 建议使用编码器提供的时间戳
							
				// writeCharPointerToFile((char *)videoFrame.buffer.get(), videoFrame.size,  "filename.txt");		
				rtsp_server->PushFrame(session_id, xop::channel_0, videoFrame); //送到服务器进行转发, 接口线程安全
				
				/*
				//获取一帧 H264, 打包
				xop::AVFrame videoFrame = {0};
				videoFrame.type = 0; // 建议确定帧类型。I帧(xop::VIDEO_FRAME_I) P帧(xop::VIDEO_FRAME_P)
				videoFrame.size = video frame size;  // 视频帧大小 
				videoFrame.timestamp = xop::H264Source::GetTimestamp(); // 时间戳, 建议使用编码器提供的时间戳
				videoFrame.buffer.reset(new uint8_t[videoFrame.size]);                    
				memcpy(videoFrame.buffer.get(), video frame data, videoFrame.size);					
                   
				rtsp_server->PushFrame(session_id, xop::channel_0, videoFrame); //送到服务器进行转发, 接口线程安全
				*/
			}
                    
			{				
				/*
				//获取一帧 AAC, 打包
				xop::AVFrame audioFrame = {0};
				audioFrame.type = xop::AUDIO_FRAME;
				audioFrame.size = audio frame size;  /* 音频帧大小 
				audioFrame.timestamp = xop::AACSource::GetTimestamp(44100); // 时间戳
				audioFrame.buffer.reset(new uint8_t[audioFrame.size]);                    
				memcpy(audioFrame.buffer.get(), audio frame data, audioFrame.size);

				rtsp_server->PushFrame(session_id, xop::channel_1, audioFrame); // 送到服务器进行转发, 接口线程安全
				*/
			}		
		}
		// xop::Timer::Sleep(20);  /* 实际使用需要根据帧率计算延时! */
	}
	videocapture.release();
	e.deinit((MPP_RET)0);
	// d.deinit();
	// yolo5.deinit();

}

至此 一帧图片怎么都能推出去了

相关推荐
吃个糖糖1 分钟前
35 Opencv 亚像素角点检测
人工智能·opencv·计算机视觉
良许Linux4 分钟前
0.96寸OLED显示屏详解
linux·服务器·后端·互联网
蜜獾云15 分钟前
docker 安装雷池WAF防火墙 守护Web服务器
linux·运维·服务器·网络·网络安全·docker·容器
小屁不止是运维16 分钟前
麒麟操作系统服务架构保姆级教程（五）NGINX中间件详解
linux·运维·服务器·nginx·中间件·架构
Hacker_Oldv21 分钟前
WPS 认证机制
运维·服务器·wps
bitcsljl29 分钟前
Linux 命令行快捷键
linux·运维·服务器
ac.char32 分钟前
在 Ubuntu 下使用 Tauri 打包 EXE 应用
linux·运维·ubuntu
IT古董1 小时前
【漫话机器学习系列】017.大O算法（Big-O Notation）
人工智能·机器学习
凯哥是个大帅比1 小时前
人工智能ACA（五）--深度学习基础
人工智能·深度学习
Youkiup1 小时前
【linux 常用命令】
linux·运维·服务器
热门推荐
01〔AI 绘画〕Stable Diffusion 之 解决绘制多人或面部很小的人物时面部崩坏问题 篇02ARM学习（31）编译器对overlay方式的支持03玄机平台应急响应—webshell查杀04DisplayPort AltMode On Type-c协议_Ver2.0(学习笔记)05校验 GPT-4 真实性的三个经典问题：快速区分 GPT-3.5 与 GPT-4，并提供免费测试网站06Docker 夺命连环 15 问076个主流的工业3D管道设计软件08基于大语言模型（LLM）的合成数据生成、策展和评估的综述09(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明10红米手机使用google play