Linux 36.3 + JetPack v6.0@jetson-inference之视频操作

Linux 36.3 + JetPack v6.0@jetson-inference之视频操作

  • [1. 源由](#1. 源由)
  • [2. 输入输出源](#2. 输入输出源)
    • [2.1 输入](#2.1 输入)
    • [2.2 输出](#2.2 输出)
  • [3. 示例](#3. 示例)
    • [3.1 MIPI CSI 摄像头](#3.1 MIPI CSI 摄像头)
    • [3.2 V4L2 摄像头](#3.2 V4L2 摄像头)
    • [3.3 WebRTC](#3.3 WebRTC)
    • [3.4 RTP](#3.4 RTP)
    • [3.5 RTSP](#3.5 RTSP)
    • [3.6 Video 文件](#3.6 Video 文件)
    • [3.7 Image 文件](#3.7 Image 文件)
  • [4. 代码分析](#4. 代码分析)
    • [4.1 Python](#4.1 Python)
    • [4.2 C++](#4.2 C++)
  • [5. 参考资料](#5. 参考资料)

1. 源由

对jetson-inference示例代码完成编译之后,首先确保视频、图像方面的操作没有问题。

因为是所有示例基本上都是针对视频或者图像进行深度机器学习操作的。

2. 输入输出源

2.1 输入

Input Protocol Resource URI Notes
MIPI CSI camera csi:// csi://0 CSI camera 0 (substitute other camera numbers for 0)
V4L2 camera v4l2:// v4l2:///dev/video0 V4L2 device 0 (substitute other camera numbers for 0)
WebRTC stream webrtc:// webrtc://@:8554/my_input From browser webcam to localhost, port 8554 (requires HTTPS/SSL)
RTP stream rtp:// rtp://@:1234 localhost, port 1234 (requires additional configuration)
RTSP stream rtsp:// rtsp://:1234 Replace with remote host's IP or hostname
Video file file:// file://my_video.mp4 Supports loading MP4, MKV, AVI, FLV (see codecs below)
Image file file:// file://my_image.jpg Supports loading JPG, PNG, TGA, BMP, GIF, etc.
Image sequence file:// file://my_directory/ Searches for images in alphanumeric order
    input                resource URI of the input stream, for example:
                             * /dev/video0               (V4L2 camera #0)
                             * csi://0                   (MIPI CSI camera #0)
                             * rtp://@:1234              (RTP stream)
                             * rtsp://user:pass@ip:1234  (RTSP stream)
                             * webrtc://@:1234/my_stream (WebRTC stream)
                             * file://my_image.jpg       (image file)
                             * file://my_video.mp4       (video file)
                             * file://my_directory/      (directory of images)
  --input-width=WIDTH    explicitly request a width of the stream (optional)
  --input-height=HEIGHT  explicitly request a height of the stream (optional)
  --input-rate=RATE      explicitly request a framerate of the stream (optional)
  --input-save=FILE      path to video file for saving the input stream to disk
  --input-codec=CODEC    RTP requires the codec to be set, one of these:
                             * h264, h265
                             * vp8, vp9
                             * mpeg2, mpeg4
                             * mjpeg
  --input-decoder=TYPE   the decoder engine to use, one of these:
                             * cpu
                             * omx  (aarch64/JetPack4 only)
                             * v4l2 (aarch64/JetPack5 only)
  --input-flip=FLIP      flip method to apply to input:
                             * none (default)
                             * counterclockwise
                             * rotate-180
                             * clockwise
                             * horizontal
                             * vertical
                             * upper-right-diagonal
                             * upper-left-diagonal
  --input-loop=LOOP      for file-based inputs, the number of loops to run:
                             * -1 = loop forever
                             *  0 = don't loop (default)
                             * >0 = set number of loops

2.2 输出

Output Protocol Resource URI Notes
WebRTC stream webrtc:// webrtc://@:8554/my_output Send to browser, port 8554, stream name "my_output"
RTP stream rtp:// rtp://:1234 Replace with remote host's IP or hostname
RTSP stream rtsp:// rtsp://@:1234/my_output Reachable at rtsp://:1234/my_output
Video file file:// file://my_video.mp4 Supports saving MP4, MKV, AVI, FLV (see codecs below)
Image file file:// file://my_image.jpg Supports saving JPG, PNG, TGA, BMP
Image sequence file:// file://image_%i.jpg %i is replaced by the image number in the sequence
OpenGL window display:// display://0 Creates GUI window on screen 0
    output               resource URI of the output stream, for example:
                             * file://my_image.jpg       (image file)
                             * file://my_video.mp4       (video file)
                             * file://my_directory/      (directory of images)
                             * rtp://<remote-ip>:1234    (RTP stream)
                             * rtsp://@:8554/my_stream   (RTSP stream)
                             * webrtc://@:1234/my_stream (WebRTC stream)
                             * display://0               (OpenGL window)
  --output-codec=CODEC   desired codec for compressed output streams:
                            * h264 (default), h265
                            * vp8, vp9
                            * mpeg2, mpeg4
                            * mjpeg
  --output-encoder=TYPE  the encoder engine to use, one of these:
                            * cpu
                            * omx  (aarch64/JetPack4 only)
                            * v4l2 (aarch64/JetPack5 only)
  --output-save=FILE     path to a video file for saving the compressed stream
                         to disk, in addition to the primary output above
  --bitrate=BITRATE      desired target VBR bitrate for compressed streams,
                         in bits per second. The default is 4000000 (4 Mbps)
  --headless             don't create a default OpenGL GUI window

3. 示例

3.1 MIPI CSI 摄像头

bash 复制代码
$ video-viewer csi://0                        # MIPI CSI camera 0 (substitue other camera numbers)
$ video-viewer csi://0 output.mp4             # save output stream to MP4 file (H.264 by default)
$ video-viewer csi://0 rtp://<remote-ip>:1234 # broadcast output stream over RTP to <remote-ip>

3.2 V4L2 摄像头

bash 复制代码
$ video-viewer v4l2:///dev/video0                 # /dev/video0 can be replaced with /dev/video1, ect.
$ video-viewer /dev/video0                        # dropping the v4l2:// protocol prefix is fine
$ video-viewer /dev/video0 output.mp4             # save output stream to MP4 file (H.264 by default)
$ video-viewer /dev/video0 rtp://<remote-ip>:1234 # broadcast output stream over RTP to <remote-ip>

3.3 WebRTC

bash 复制代码
$ video-viewer /dev/video0 webrtc://@:8554/my_output               # send V4L2 webcam to browser
$ video-viewer webrtc://@:8554/my_input output.mp4                 # receive browser webcam (requires HTTPS/SSL) and save to MP4
$ video-viewer webrtc://@:8554/my_input webrtc://@:8554/my_output  # receieve + send (full-duplex loopback)

3.4 RTP

bash 复制代码
$ video-viewer --input-codec=h264 rtp://@:1234         # recieve on localhost port 1234
$ video-viewer --input-codec=h264 rtp://224.0.0.0:1234 # subscribe to multicast group
$ video-viewer --bitrate=1000000 csi://0 rtp://<remote-ip>:1234         # transmit camera over RTP, encoded as H.264 @ 1Mbps 
$ video-viewer --output-codec=h265 my_video.mp4 rtp://<remote-ip>:1234  # transmit a video file over RTP, encoded as H.265

3.5 RTSP

bash 复制代码
$ video-viewer rtsp://<remote-ip>:1234 my_video.mp4      # subscribe to RTSP feed from <remote-ip>, port 1234 (and save it to file)
$ video-viewer rtsp://username:password@<remote-ip>:1234 # with authentication (replace username/password with credentials)
$ video-viewer /dev/video0 rtsp://@:1234/my_output                 # stream a V4L2 camera out over RTSP 
$ video-viewer rtsp://<remote-ip>:1234/input rtsp://@:1234/output  # subscribe to an RTSP feed, and relay it (loopback)

3.6 Video 文件

bash 复制代码
# playback
$ video-viewer my_video.mp4                              # display the video file
$ video-viewer my_video.mp4 rtp://<remote-ip>:1234       # transmit the video over RTP

# recording
$ video-viewer csi://0 my_video.mp4                      # record CSI camera to video file
$ video-viewer /dev/video0 my_video.mp4                  # record V4L2 camera to video file

3.7 Image 文件

bash 复制代码
$ video-viewer input.jpg output.jpg	# load/save an image
$ video-viewer input_dir/ output_dir/   # load all images from input_dir and save them to output_dir
$ video-viewer "*.jpg" output_%i.jpg    # load all jpg images and save them to output_0.jpg, output_1.jpg, ect

4. 代码分析

4.1 Python

代码:video-viewer.py

python 复制代码
Root
├── Imports
│   ├── import sys
│   ├── import argparse
│   └── from jetson_utils import videoSource, videoOutput, Log
├── Command-line Argument Parsing
│   ├── Create Argument Parser
│   │   ├── Description: "View various types of video streams"
│   │   ├── Formatter Class: argparse.RawTextHelpFormatter
│   │   └── Epilog: videoSource.Usage() + videoOutput.Usage() + Log.Usage()
│   ├── Add Arguments
│   │   ├── input (URI of the input stream)
│   │   └── output (URI of the output stream, optional)
│   └── Parse Arguments
│       ├── args = parser.parse_known_args()[0]
│       └── Handle Parsing Exceptions
│           ├── Print Help
│           └── Exit
├── Create Video Sources & Outputs
│   ├── input = videoSource(args.input, argv=sys.argv)
│   └── output = videoOutput(args.output, argv=sys.argv)
├── Capture Frames Loop
    ├── Initialize Frame Counter
    │   └── numFrames = 0
    ├── while True Loop
        ├── Capture Image
        │   └── img = input.Capture()
        ├── Handle Timeout
        │   └── if img is None: continue
        ├── Log Verbose Information
        │   └── if numFrames % 25 == 0 or numFrames < 15:
        │       └── Log.Verbose(...)
        ├── Increment Frame Counter
        │   └── numFrames += 1
        ├── Render Image
        │   └── output.Render(img)
        ├── Update Title Bar
        │   └── output.SetStatus(...)
        └── Exit on EOS
            ├── if not input.IsStreaming() or not output.IsStreaming():
            └── break

4.2 C++

代码:video-viewer.cpp

cpp 复制代码
#include statements
├── "videoSource.h"
├── "videoOutput.h"
├── "logging.h"
├── "commandLine.h"
└── <signal.h>

Global variables
└── bool signal_recieved = false;

Function definitions
├── void sig_handler(int signo)
│   └── if (signo == SIGINT)
│       ├── LogInfo("received SIGINT\n");
│       └── signal_recieved = true;
└── int usage()
    ├── printf("usage: video-viewer [--help] input_URI [output_URI]\n\n");
    ├── printf("View/output a video or image stream.\n");
    ├── printf("See below for additional arguments that may not be shown above.\n\n");
    ├── printf("positional arguments:\n");
    ├── printf("    input_URI       resource URI of input stream  (see videoSource below)\n");
    ├── printf("    output_URI      resource URI of output stream (see videoOutput below)\n\n");
    ├── printf("%s", videoSource::Usage());
    ├── printf("%s", videoOutput::Usage());
    └── printf("%s", Log::Usage());

main function
├── Parse command line
│   ├── commandLine cmdLine(argc, argv);
│   └── if (cmdLine.GetFlag("help"))
│       └── return usage();
├── Attach signal handler
│   └── if (signal(SIGINT, sig_handler) == SIG_ERR)
│       └── LogError("can't catch SIGINT\n");
├── Create input video stream
│   ├── videoSource* input = videoSource::Create(cmdLine, ARG_POSITION(0));
│   └── if (!input)
│       ├── LogError("video-viewer:  failed to create input stream\n");
│       └── return 0;
├── Create output video stream
│   ├── videoOutput* output = videoOutput::Create(cmdLine, ARG_POSITION(1));
│   └── if (!output)
│       ├── LogError("video-viewer:  failed to create output stream\n");
│       └── return 0;
├── Capture/display loop
│   ├── uint32_t numFrames = 0;
│   └── while (!signal_recieved)
│       ├── uchar3* image = NULL;
│       ├── int status = 0;
│       ├── if (!input->Capture(&image, &status))
│       │   └── if (status == videoSource::TIMEOUT)
│       │       └── continue;
│       │   └── break; // EOS
│       ├── if (numFrames % 25 == 0 || numFrames < 15)
│       │   └── LogVerbose("video-viewer:  captured %u frames (%ux%u)\n", numFrames, input->GetWidth(), input->GetHeight());
│       ├── numFrames++;
│       └── if (output != NULL)
│           ├── output->Render(image, input->GetWidth(), input->GetHeight());
│           ├── char str[256];
│           ├── sprintf(str, "Video Viewer (%ux%u) | %.1f FPS", input->GetWidth(), input->GetHeight(), output->GetFrameRate());
│           ├── output->SetStatus(str);
│           └── if (!output->IsStreaming())
│               └── break;
├── Destroy resources
│   ├── printf("video-viewer:  shutting down...\n");
│   ├── SAFE_DELETE(input);
│   └── SAFE_DELETE(output);
└── printf("video-viewer:  shutdown complete\n");

5. 参考资料

【1】jetson-inference - Camera Streaming and Multimedia

相关推荐
vip45127 分钟前
Linux 经典面试八股文
linux
大霞上仙29 分钟前
Ubuntu系统电脑没有WiFi适配器
linux·运维·电脑
Karoku0661 小时前
【企业级分布式系统】Zabbix监控系统与部署安装
运维·服务器·数据库·redis·mysql·zabbix
为什么这亚子1 小时前
九、Go语言快速入门之map
运维·开发语言·后端·算法·云原生·golang·云计算
布值倒区什么name1 小时前
bug日常记录responded with a status of 413 (Request Entity Too Large)
运维·服务器·bug
孤客网络科技工作室2 小时前
VMware 虚拟机使用教程及 Kali Linux 安装指南
linux·虚拟机·kali linux
。puppy2 小时前
HCIP--3实验- 链路聚合,VLAN间通讯,Super VLAN,MSTP,VRRPip配置,OSPF(静态路由,环回,缺省,空接口),NAT
运维·服务器
颇有几分姿色2 小时前
深入理解 Linux 内存管理:free 命令详解
linux·运维·服务器
光芒再现dev3 小时前
已解决,部署GPTSoVITS报错‘AsyncRequest‘ object has no attribute ‘_json_response_data‘
运维·python·gpt·语言模型·自然语言处理
AndyFrank3 小时前
mac crontab 不能使用问题简记
linux·运维·macos