ESXI下有一个Linux虚拟机，其中通过Docker跑了个Frigate作为家庭NVR，同时用于人脸识别给中控屏做人来亮屏。由于我的宿主机的核心是3代i5的3317u，比较羸弱，所以Frigate中ffmpeg软解视频比较吃力，CPU使用率就飙上去了，所以考虑将GPU核显利用起来。本文测试使用的码流为TP摄像头创建的640*480的H.264
将GPU直通给Linux
首先设置GPU直通并重启ESXI，并给Linux虚拟机添加GPU为PCI设备
接着进入ESXI的ssh，让ESXI启动时不去获取显卡控制权
esxcli system settings kernel set -s vga -v FALSE
重启ESXI，此时即使机器插着显示器，ESXI启动也不输出画面了
接着配置Linux虚拟机：
禁用虚拟显卡：将svga.present由TRUE改为FALSE
让GPU驱动不知道在虚拟机中运行：添加hypervisor.cpuid.v0为FALSE
重启虚拟机，svga画面输出已经消失了。hypervisor.cpuid.v0一定要配置，否则Linux虚拟机无法完成启动。
启动后，通过ssh连接虚拟机，验证直通效果：sudo lshw -C display或者lspci -vnn | grep VGA -A 12
yaml
$ sudo lshw -C display
*-display
description: VGA compatible controller
product: 3rd Gen Core processor Graphics Controller
vendor: Intel Corporation
physical id: 0
bus info: pci@0000:1b:00.0
version: 09
width: 64 bits
clock: 33MHz
capabilities: msi pm vga_controller bus_master cap_list
configuration: driver=i915 latency=64
resources: irq:66 memory:fc800000-fcbfffff memory:d0000000-dfffffff ioport:2000(size=64)
yaml
$ lspci -vnn | grep VGA -A 12
1b:00.0 VGA compatible controller [0300]: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:0166] (rev 09) (prog-if 00 [VGA controller])
DeviceName: pciPassthru0
Subsystem: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:2010]
Physical Slot: 256
Flags: bus master, fast devsel, latency 64, IRQ 66
Memory at fc800000 (64-bit, non-prefetchable) [size=4M]
Memory at d0000000 (64-bit, prefetchable) [size=256M]
I/O ports at 2000 [size=64]
Capabilities: <access denied>
Kernel driver in use: i915
Kernel modules: i915可见GPU直通成功了。
配置Frigate
docs.frigate.video/configurati...
docs.frigate.video/configurati...
主要就是需要在docker参数中加上--device /dev/dri/renderD128和--privileged，然后在Frigate配置文件中加上hwaccel_args: preset-vaapi
进入Frigate查看效果：
GPU硬件加速成功，但是intel_gpu_top执行失败，这个原因未知，网上很多人有类似的问题。
Frigate log：
yaml
2024-03-12 15:21:21.941076931 [2024-03-12 15:21:21] frigate.util.services ERROR : Unable to poll intel GPU stats: Failed to detect engines! (No such file or directory)
2024-03-12 15:21:21.941085403 (Kernel 4.16 or newer is required for i915 PMU support.)
2024-03-12 15:21:21.941089240 timeout: the monitored command dumped core直接运行intel_gpu_top：
Failed to initialize PMU! (Permission denied)
将Detectors交给GPU
需要使用openvino。openvino也支持CPU，效率比默认的CPU Detector高，但是如果要使用CPU，只支持6代以上的CPU，GPU的话没有要求。
docs.frigate.video/configurati...
修改Frigate配置文件：
yaml
detectors:
ov:
type: openvino
device: AUTO
model:
path: /openvino-model/ssdlite_mobilenet_v2.xml
model:
width: 300
height: 300
input_tensor: nhwc
input_pixel_format: bgr
labelmap_path: /openvino-model/coco_91cl_bkgr.txt启动成功：
yaml
2024-03-12 15:28:42.137142043 [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO : Model Input Shape: [1,300,300,3]
2024-03-12 15:28:42.137157934 [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO : Model Output-0 Shape: [1,1,100,7]
2024-03-12 15:28:42.137176631 [2024-03-12 15:28:42] frigate.detectors.plugins.openvino INFO : Model has 1 Output Tensors使用CPU推理：
可见延迟有287ms。CPU占用非常高，FPS才1.4
使用GPU推理：
可见延迟下降到80ms，CPU占用降低了很多，FPS达到了4
后续
试着将上述openvino的device从AUTO改为GPU，发现竟然启动失败了：
arduino
2024-03-12 15:50:55.499156863 Process detector:ov:
2024-03-12 15:50:55.502277448 Traceback (most recent call last):
2024-03-12 15:50:55.502450200 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-03-12 15:50:55.502457812 self.run()
2024-03-12 15:50:55.502467808 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-03-12 15:50:55.502478795 self._target(*self._args, **self._kwargs)
2024-03-12 15:50:55.502486642 File "/opt/frigate/frigate/object_detection.py", line 102, in run_detector
2024-03-12 15:50:55.502521519 object_detector = LocalObjectDetector(detector_config=detector_config)
2024-03-12 15:50:55.502526047 File "/opt/frigate/frigate/object_detection.py", line 53, in __init__
2024-03-12 15:50:55.502530992 self.detect_api = create_detector(detector_config)
2024-03-12 15:50:55.502534675 File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
2024-03-12 15:50:55.502540247 return api(detector_config)
2024-03-12 15:50:55.502545383 File "/opt/frigate/frigate/detectors/plugins/openvino.py", line 32, in __init__
2024-03-12 15:50:55.502585844 self.interpreter = self.ov_core.compile_model(
2024-03-12 15:50:55.502634947 File "/usr/local/lib/python3.9/dist-packages/openvino/runtime/ie_api.py", line 399, in compile_model
2024-03-12 15:50:55.502646924 super().compile_model(model, device_name, {} if config is None else config),
2024-03-12 15:50:55.502661726 RuntimeError: Failed to create plugin /usr/local/lib/python3.9/dist-packages/openvino/libs/libopenvino_intel_gpu_plugin.so for device GPU
2024-03-12 15:50:55.502667511 Please, check your environment
2024-03-12 15:50:55.502671411 Check 'error_code == 0' failed at src/plugins/intel_gpu/src/runtime/ocl/ocl_device_detector.cpp:194:
2024-03-12 15:50:55.502677424 [GPU] No supported OCL devices found or unexpected error happened during devices query.
2024-03-12 15:50:55.502682322 [GPU] Please check OpenVINO documentation for GPU drivers setup guide.
2024-03-12 15:50:55.502686670 [GPU] clGetPlatformIDs error code: -1001所以可能上面openvino一直使用的是CPU？可是官网写着openvino要使用CPU的话，需要6代以上的CPU。。