Linux 36.3 + JetPack v6.0@jetson-inference之视频操作
- [1. 源由](#1. 源由)
- [2. 输入输出源](#2. 输入输出源)
-
- [2.1 输入](#2.1 输入)
- [2.2 输出](#2.2 输出)
- [3. 示例](#3. 示例)
-
- [3.1 MIPI CSI 摄像头](#3.1 MIPI CSI 摄像头)
- [3.2 V4L2 摄像头](#3.2 V4L2 摄像头)
- [3.3 WebRTC](#3.3 WebRTC)
- [3.4 RTP](#3.4 RTP)
- [3.5 RTSP](#3.5 RTSP)
- [3.6 Video 文件](#3.6 Video 文件)
- [3.7 Image 文件](#3.7 Image 文件)
- [4. 代码分析](#4. 代码分析)
-
- [4.1 Python](#4.1 Python)
- [4.2 C++](#4.2 C++)
- [5. 参考资料](#5. 参考资料)
1. 源由
对jetson-inference示例代码完成编译之后，首先确保视频、图像方面的操作没有问题。
因为是所有示例基本上都是针对视频或者图像进行深度机器学习操作的。
2. 输入输出源
2.1 输入
| Input | Protocol | Resource URI | Notes |
|---|---|---|---|
| MIPI CSI camera | csi:// | csi://0 | CSI camera 0 (substitute other camera numbers for 0) |
| V4L2 camera | v4l2:// | v4l2:///dev/video0 | V4L2 device 0 (substitute other camera numbers for 0) |
| WebRTC stream | webrtc:// | webrtc://@:8554/my_input | From browser webcam to localhost, port 8554 (requires HTTPS/SSL) |
| RTP stream | rtp:// | rtp://@:1234 | localhost, port 1234 (requires additional configuration) |
| RTSP stream | rtsp:// | rtsp://:1234 | Replace with remote host's IP or hostname |
| Video file | file:// | file://my_video.mp4 | Supports loading MP4, MKV, AVI, FLV (see codecs below) |
| Image file | file:// | file://my_image.jpg | Supports loading JPG, PNG, TGA, BMP, GIF, etc. |
| Image sequence | file:// | file://my_directory/ | Searches for images in alphanumeric order |
input resource URI of the input stream, for example:
* /dev/video0 (V4L2 camera #0)
* csi://0 (MIPI CSI camera #0)
* rtp://@:1234 (RTP stream)
* rtsp://user:pass@ip:1234 (RTSP stream)
* webrtc://@:1234/my_stream (WebRTC stream)
* file://my_image.jpg (image file)
* file://my_video.mp4 (video file)
* file://my_directory/ (directory of images)
--input-width=WIDTH explicitly request a width of the stream (optional)
--input-height=HEIGHT explicitly request a height of the stream (optional)
--input-rate=RATE explicitly request a framerate of the stream (optional)
--input-save=FILE path to video file for saving the input stream to disk
--input-codec=CODEC RTP requires the codec to be set, one of these:
* h264, h265
* vp8, vp9
* mpeg2, mpeg4
* mjpeg
--input-decoder=TYPE the decoder engine to use, one of these:
* cpu
* omx (aarch64/JetPack4 only)
* v4l2 (aarch64/JetPack5 only)
--input-flip=FLIP flip method to apply to input:
* none (default)
* counterclockwise
* rotate-180
* clockwise
* horizontal
* vertical
* upper-right-diagonal
* upper-left-diagonal
--input-loop=LOOP for file-based inputs, the number of loops to run:
* -1 = loop forever
* 0 = don't loop (default)
* >0 = set number of loops
2.2 输出
| Output | Protocol | Resource URI | Notes |
|---|---|---|---|
| WebRTC stream | webrtc:// | webrtc://@:8554/my_output | Send to browser, port 8554, stream name "my_output" |
| RTP stream | rtp:// | rtp://:1234 | Replace with remote host's IP or hostname |
| RTSP stream | rtsp:// | rtsp://@:1234/my_output | Reachable at rtsp://:1234/my_output |
| Video file | file:// | file://my_video.mp4 | Supports saving MP4, MKV, AVI, FLV (see codecs below) |
| Image file | file:// | file://my_image.jpg | Supports saving JPG, PNG, TGA, BMP |
| Image sequence | file:// | file://image_%i.jpg | %i is replaced by the image number in the sequence |
| OpenGL window | display:// | display://0 | Creates GUI window on screen 0 |
output resource URI of the output stream, for example:
* file://my_image.jpg (image file)
* file://my_video.mp4 (video file)
* file://my_directory/ (directory of images)
* rtp://<remote-ip>:1234 (RTP stream)
* rtsp://@:8554/my_stream (RTSP stream)
* webrtc://@:1234/my_stream (WebRTC stream)
* display://0 (OpenGL window)
--output-codec=CODEC desired codec for compressed output streams:
* h264 (default), h265
* vp8, vp9
* mpeg2, mpeg4
* mjpeg
--output-encoder=TYPE the encoder engine to use, one of these:
* cpu
* omx (aarch64/JetPack4 only)
* v4l2 (aarch64/JetPack5 only)
--output-save=FILE path to a video file for saving the compressed stream
to disk, in addition to the primary output above
--bitrate=BITRATE desired target VBR bitrate for compressed streams,
in bits per second. The default is 4000000 (4 Mbps)
--headless don't create a default OpenGL GUI window
3. 示例
3.1 MIPI CSI 摄像头
bash
$ video-viewer csi://0 # MIPI CSI camera 0 (substitue other camera numbers)
$ video-viewer csi://0 output.mp4 # save output stream to MP4 file (H.264 by default)
$ video-viewer csi://0 rtp://<remote-ip>:1234 # broadcast output stream over RTP to <remote-ip>3.2 V4L2 摄像头
bash
$ video-viewer v4l2:///dev/video0 # /dev/video0 can be replaced with /dev/video1, ect.
$ video-viewer /dev/video0 # dropping the v4l2:// protocol prefix is fine
$ video-viewer /dev/video0 output.mp4 # save output stream to MP4 file (H.264 by default)
$ video-viewer /dev/video0 rtp://<remote-ip>:1234 # broadcast output stream over RTP to <remote-ip>3.3 WebRTC
bash
$ video-viewer /dev/video0 webrtc://@:8554/my_output # send V4L2 webcam to browser
$ video-viewer webrtc://@:8554/my_input output.mp4 # receive browser webcam (requires HTTPS/SSL) and save to MP4
$ video-viewer webrtc://@:8554/my_input webrtc://@:8554/my_output # receieve + send (full-duplex loopback)3.4 RTP
bash
$ video-viewer --input-codec=h264 rtp://@:1234 # recieve on localhost port 1234
$ video-viewer --input-codec=h264 rtp://224.0.0.0:1234 # subscribe to multicast group
$ video-viewer --bitrate=1000000 csi://0 rtp://<remote-ip>:1234 # transmit camera over RTP, encoded as H.264 @ 1Mbps
$ video-viewer --output-codec=h265 my_video.mp4 rtp://<remote-ip>:1234 # transmit a video file over RTP, encoded as H.2653.5 RTSP
bash
$ video-viewer rtsp://<remote-ip>:1234 my_video.mp4 # subscribe to RTSP feed from <remote-ip>, port 1234 (and save it to file)
$ video-viewer rtsp://username:password@<remote-ip>:1234 # with authentication (replace username/password with credentials)
$ video-viewer /dev/video0 rtsp://@:1234/my_output # stream a V4L2 camera out over RTSP
$ video-viewer rtsp://<remote-ip>:1234/input rtsp://@:1234/output # subscribe to an RTSP feed, and relay it (loopback)3.6 Video 文件
bash
# playback
$ video-viewer my_video.mp4 # display the video file
$ video-viewer my_video.mp4 rtp://<remote-ip>:1234 # transmit the video over RTP
# recording
$ video-viewer csi://0 my_video.mp4 # record CSI camera to video file
$ video-viewer /dev/video0 my_video.mp4 # record V4L2 camera to video file3.7 Image 文件
bash
$ video-viewer input.jpg output.jpg # load/save an image
$ video-viewer input_dir/ output_dir/ # load all images from input_dir and save them to output_dir
$ video-viewer "*.jpg" output_%i.jpg # load all jpg images and save them to output_0.jpg, output_1.jpg, ect4. 代码分析
4.1 Python
python
Root
├── Imports
│ ├── import sys
│ ├── import argparse
│ └── from jetson_utils import videoSource, videoOutput, Log
├── Command-line Argument Parsing
│ ├── Create Argument Parser
│ │ ├── Description: "View various types of video streams"
│ │ ├── Formatter Class: argparse.RawTextHelpFormatter
│ │ └── Epilog: videoSource.Usage() + videoOutput.Usage() + Log.Usage()
│ ├── Add Arguments
│ │ ├── input (URI of the input stream)
│ │ └── output (URI of the output stream, optional)
│ └── Parse Arguments
│ ├── args = parser.parse_known_args()[0]
│ └── Handle Parsing Exceptions
│ ├── Print Help
│ └── Exit
├── Create Video Sources & Outputs
│ ├── input = videoSource(args.input, argv=sys.argv)
│ └── output = videoOutput(args.output, argv=sys.argv)
├── Capture Frames Loop
├── Initialize Frame Counter
│ └── numFrames = 0
├── while True Loop
├── Capture Image
│ └── img = input.Capture()
├── Handle Timeout
│ └── if img is None: continue
├── Log Verbose Information
│ └── if numFrames % 25 == 0 or numFrames < 15:
│ └── Log.Verbose(...)
├── Increment Frame Counter
│ └── numFrames += 1
├── Render Image
│ └── output.Render(img)
├── Update Title Bar
│ └── output.SetStatus(...)
└── Exit on EOS
├── if not input.IsStreaming() or not output.IsStreaming():
└── break4.2 C++
cpp
#include statements
├── "videoSource.h"
├── "videoOutput.h"
├── "logging.h"
├── "commandLine.h"
└── <signal.h>
Global variables
└── bool signal_recieved = false;
Function definitions
├── void sig_handler(int signo)
│ └── if (signo == SIGINT)
│ ├── LogInfo("received SIGINT\n");
│ └── signal_recieved = true;
└── int usage()
├── printf("usage: video-viewer [--help] input_URI [output_URI]\n\n");
├── printf("View/output a video or image stream.\n");
├── printf("See below for additional arguments that may not be shown above.\n\n");
├── printf("positional arguments:\n");
├── printf(" input_URI resource URI of input stream (see videoSource below)\n");
├── printf(" output_URI resource URI of output stream (see videoOutput below)\n\n");
├── printf("%s", videoSource::Usage());
├── printf("%s", videoOutput::Usage());
└── printf("%s", Log::Usage());
main function
├── Parse command line
│ ├── commandLine cmdLine(argc, argv);
│ └── if (cmdLine.GetFlag("help"))
│ └── return usage();
├── Attach signal handler
│ └── if (signal(SIGINT, sig_handler) == SIG_ERR)
│ └── LogError("can't catch SIGINT\n");
├── Create input video stream
│ ├── videoSource* input = videoSource::Create(cmdLine, ARG_POSITION(0));
│ └── if (!input)
│ ├── LogError("video-viewer: failed to create input stream\n");
│ └── return 0;
├── Create output video stream
│ ├── videoOutput* output = videoOutput::Create(cmdLine, ARG_POSITION(1));
│ └── if (!output)
│ ├── LogError("video-viewer: failed to create output stream\n");
│ └── return 0;
├── Capture/display loop
│ ├── uint32_t numFrames = 0;
│ └── while (!signal_recieved)
│ ├── uchar3* image = NULL;
│ ├── int status = 0;
│ ├── if (!input->Capture(&image, &status))
│ │ └── if (status == videoSource::TIMEOUT)
│ │ └── continue;
│ │ └── break; // EOS
│ ├── if (numFrames % 25 == 0 || numFrames < 15)
│ │ └── LogVerbose("video-viewer: captured %u frames (%ux%u)\n", numFrames, input->GetWidth(), input->GetHeight());
│ ├── numFrames++;
│ └── if (output != NULL)
│ ├── output->Render(image, input->GetWidth(), input->GetHeight());
│ ├── char str[256];
│ ├── sprintf(str, "Video Viewer (%ux%u) | %.1f FPS", input->GetWidth(), input->GetHeight(), output->GetFrameRate());
│ ├── output->SetStatus(str);
│ └── if (!output->IsStreaming())
│ └── break;
├── Destroy resources
│ ├── printf("video-viewer: shutting down...\n");
│ ├── SAFE_DELETE(input);
│ └── SAFE_DELETE(output);
└── printf("video-viewer: shutdown complete\n");