基于ONNXRuntime C#实现的高性能YOLO推理框架

YoloSharpOnnx是一个高性能、内存复用、跨平台的 C# YOLO 推理库，基于 OpenCV 和 ONNX 运行时实现。

背景

刚开始做工业检测项目时，在网上找了一些现成的yOLO推理库，发现都不怎么好用，性能也一般，并且部署不是很方便，只支持少数几种硬件部署，另外项目需要使用批量检测的功能，网上的一些YOLO推理框架都没有批量推理的功能，于是干脆就自己动手封装一个YOLO推理框架，方便修改优化，可以随时根据需求调整功能或添加功能。

完成基本的目标检测功能后，后面一个月时间，陆陆续续把YOLO所有的功能都完善了，就把代码放到GitHub上面开源了

功能特色

• 支持所有YOLO任务：目标检测、图片分类、实例分割、姿势估计、OBB
• 支持多种部署方案：CPU, CUDA / TensorRT, OpenVINO, CoreML, DirectML
• 可以批量检测图片，性能大幅优于单线程串行执行
• 图像处理使用OpenCVSharp
• 推理引擎：ONNX Runtime 是一个跨平台的机器学习模型加速器

示例Demo

|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Object Detection | Image Classification |
| | |

|---------------------------------------------------------------------------------------------------------|
| |
| |
[Instance Segmentation]

|------------------------------------------------------------------------------------------------------|
| |
| |
[Pose Estimation]

|-----------------------------------------------------------------------------------------------------|
| |
| |
[OBB Detection]

使用示例

1 导出模型为onnx格式

from ultralytics import YOLO

# Load a model
model = YOLO('path/to/best.pt')

# Export the model to ONNX format
model.export(format='onnx')

2 YoloSharpOnnx初始化

安装Nuget 包YoloSharpOnnx, OnnxRuntime, OpenCvSharp4.runtime

CPU推理

dotnet add package YoloSharpOnnx
dotnet add package OpenCvSharp4.runtime.win
dotnet add package Microsoft.ML.OnnxRuntime

using YoloSharp yolo = new YoloSharp(new ExecutionProviderCPU("yolo11n.onnx"));

CoreML推理

dotnet add package YoloSharpOnnx
dotnet add package OpenCvSharp4.runtime.osx.10.15-x64
dotnet add package Microsoft.ML.OnnxRuntime

using YoloSharp yolo = new YoloSharp(new ExecutionProviderCoreML("yolo11n.onnx"));

CUDA/TensorRT推理

dotnet add package YoloSharpOnnx
dotnet add package OpenCvSharp4.runtime.win
dotnet add package Microsoft.ML.OnnxRuntime.Gpu.Windows

using YoloSharp yolo = new YoloSharp(new ExecutionProviderCUDA("yolo11n.onnx",0));
using YoloSharp yolo = new YoloSharp(new ExecutionProviderTensorRT("yolo11n.onnx",0));

DirectML推理

dotnet add package YoloSharpOnnx
dotnet add package OpenCvSharp4.runtime.win
dotnet add package Microsoft.ML.OnnxRuntime.DirectML

using YoloSharp yolo = new YoloSharp(new ExecutionProviderDirectML("yolo11n.onnx",0));

OpenVINO Inference

dotnet add package YoloSharpOnnx
dotnet add package OpenCvSharp4.runtime.win
dotnet add package Intel.ML.OnnxRuntime.OpenVino

using YoloSharp yolo = new YoloSharp(new ExecutionProviderOpenVINO("yolo11n.onnx", IntelDeviceType.NPU));

基本的API，加载模型并进行预测

using Mat image = Cv2.ImRead("bus.jpg");
using YoloSharp yolo = new YoloSharp(new ExecutionProviderCPU("yolo11n.onnx"));

List<DetectionResult> res = yolo.RunDetect(image);

yolo.DrawDetections(image,res);
Cv2.ImWrite("bus_res.jpg", image);

string printString = res.Summary();
Console.WriteLine(printString);

性能测试API

using Mat image = Cv2.ImRead("bus.jpg");
     
using YoloSharp yolo = new YoloSharp(new ExecutionProviderDirectML("yolo11n.onnx",1));
var res = yolo.RunDetectWithTime(item.FullName);

Console.WriteLine($"{res.ToString()}, {res.SpeedResult.ToString()}");

配置参数

using Mat image = Cv2.ImRead("bus.jpg");
using YoloSharp yolo = new YoloSharp(new ExecutionProviderCPU("yolo11n.onnx"));
yolo.YoloConfiguration.IoU = 0.4f;
yolo.YoloConfiguration.Confidence = 0.3f;
yolo.YoloConfiguration.ResizeAlgorithm = InterpolationFlags.Linear;
yolo.YoloConfiguration.ImageExtsBatch = [".jpg", ".png"];
var res = yolo.RunDetect(image);

批量处理API

private static void TestBatchInfer()
{
    string modelPath = @"D:\code\model\best.onnx";
    string dir = @"D:\code\model\TestImages"
    DirectoryInfo directory = new DirectoryInfo(dir);
    var files = directory.GetFiles()
    System.Diagnostics.Stopwatch _stopwatch = new System.Diagnostics.Stopwatch();
    _stopwatch.Start();
    int num=files.Length;
    using (YoloSharp yolo = new YoloSharp(new ExecutionProviderDirectML(modelPath, 0)))
    {
        var list = yolo.RunBatchDetect(dir,new ProcessCallback(), ReceiveProcess, 30)
    }
    _stopwatch.Stop()
    Console.WriteLine($"detect {num} images, time:{_stopwatch.Elapsed}");

}
private static void ReceiveProcess(DetectionBatchResult e)
{
    string res = e.Results.Summary();
}
internal class ProcessCallback : IBatchProcessCallback
{
   
    public void ReceiveProcessResult(DetectionBatchResult e)
    {
       
        string res = e.Results.Summary();
      
    }
}

批量处理Foreach API

private static async Task TestBatchForeachInfer()
{
    var files = Directory.GetFiles(dir);
    System.Diagnostics.Stopwatch _stopwatch = new System.Diagnostics.Stopwatch();
    _stopwatch.Start();
    int num = files.Length;
    using (YoloSharp yolo = new YoloSharp(new ExecutionProviderDirectML(modelPath, _deviceId)))
    {
        yolo.YoloConfiguration.BatchPoolSize = 30;
        await foreach (var item in yolo.BatchDetectForeachAsync(files.ToList()))
        {
            Console.WriteLine($"{item.ImagePath} {item.Results.Summary()}");
        }
    }
    _stopwatch.Stop();
    Console.WriteLine($"detect {num} images, time:{_stopwatch.Elapsed}");
}

性能测试

Yolo C# inference library	Version	Image Processing library	Image Resize Algorithm	Sequence inference	Batch inference
YoloSharp	6.1.0	SixLabors.ImageSharp 3.1.12	Triangle(Bilinear)	support	not support
YoloDotNet	4.2.0	SkiaSharp 3.119.1	Linear(Bilinear)	support	support
YoloSharpOnnx	1.3.3	OpenCvSharp4 4.13.0.20260318	Linear(Bilinear)	support	support

测试工具

YoloOnnxWinform

测试电脑

Hardware	Summary
Windows	Windows 10 OS Version 19045.6466
CPU	AMD Ryzen 7 5800X 8-Core Processor 3.8GHz
RAM	DDR4 3200 MHz 32GB
GPU	AMD Radeom RX6800 16GB
Storage	SSD 2TB

测试数据

Images: 300 images (image size: 2480x3494)

Yolo Model: Yolo11n.onnx InputShape float32[1,3,1280,1280]

Inference Provider: DirectML Inference Microsoft.ML.OnnxRuntime.DirectML 1.24.3

测试结果

Yolo C# inference library	Version	Image Processing library	Image Resize Algorithm	Sequence inference (Time/Memory)	Batch inference (Time/Memory)
YoloSharp	6.1.0	SixLabors.ImageSharp 3.1.12	Triangle(Bilinear)	18.707s, 1374M	-
YoloDotNet	4.2.0	SkiaSharp 3.119.1	Linear(Bilinear)	17.665s, 169M	10.587s, 639M
YoloSharpOnnx	1.3.3	OpenCvSharp4 4.13.0.20260318	Linear(Bilinear)	13.693s, 169M	2.980s, 601M

后续需要完善的功能

RT-DETR、YoloE、Yolo-World、SAM等检测模型的API开发