目录
[为什么选择 FP16](#为什么选择 FP16)
说明
在 OCR 项目落地中,很多人都会遇到同一个问题:模型效果不错,但推理速度、部署便利性、GPU 兼容性、内存稳定性,很难同时兼顾。
最近我们基于 PP-OCR 系列模型,完成了一套新的 TensorRT 推理版本:
lw.TensorRT.PPOCRSharp
它采用 C++ 实现底层高性能推理 DLL,C# 负责界面调用和业务集成,适合 Windows 环境下对 OCR 速度有要求的场景。
效果

NVIDIA GeForce RTX4060 Laptop GPU (8 GB)
CUDA 12.4
TensorRT-10.16.1.11
项目特点
- TensorRT GPU 推理加速
底层使用 NVIDIA TensorRT 直接加载转换后的 .engine 文件进行推理,相比通用推理框架,在 NVIDIA GPU 上更容易获得更低延迟。
- C++ DLL 封装,C# 简单调用
核心推理逻辑封装在 C++ DLL 中,C# 端通过 P/Invoke 调用,既保证性能,也方便 WinForms、WPF、业务系统集成。
- 支持 PP-OCR 多版本模型
当前测试程序中已加入多组模型选择:
-
PP-OCRv5 mobile
-
PP-OCRv5 server
-
PP-OCRv6 tiny
-
PP-OCRv6 small
-
PP-OCRv6 medium
可以根据速度、精度、显存占用灵活选择。
- 支持 det / rec / cls 三阶段 OCR
完整支持文本检测、文本识别、方向分类模型调用。实际使用时也可以关闭 cls,进一步提升速度。
- 自动检测 TensorRT engine 文件
程序会直接加载已经转换好的 .engine 文件。
如果缺少 engine,C# 程序会自动生成 trtexec_convert_commands.txt 说明文档,提示用户如何使用本机 trtexec.exe 转换 ONNX 模型。
因为不同电脑 TensorRT 安装路径不同,文档中使用 <TRTEXEC> 占位,用户只需要替换成本机路径即可。
- GPU ID 可配置
界面保留 use_gpu 和 gpu_id 设置,适合多显卡环境。底层 C++ 已经处理 CUDA device 绑定,避免多线程推理时跑错 GPU。
- 多 predictor 并发识别
识别模型支持多个 predictor 并行处理,适合一张图中存在多行文本的场景。实际部署时可以根据显卡性能调整 rec_predictor_num,并不是越大越好,建议从 1、2、4 逐步测试。
为什么选择 FP16
TensorRT 支持 FP16 和 INT8,但 OCR 模型尤其是 rec 识别模型,对量化非常敏感。
在实际测试中,未经充分校准的 INT8 可能出现识别为空、准确率明显下降等问题。因此当前版本选择 FP16 作为默认和推荐方案:
-
速度提升明显
-
精度更稳定
-
转换流程简单
-
更适合通用部署
适用场景
这套方案适合:
-
Windows 桌面 OCR 软件
-
工业视觉字符识别
-
票据、表单、截图 OCR
-
批量图片文字识别
-
C# 项目需要调用高性能 OCR DLL
-
NVIDIA GPU 环境下追求更低延迟的 OCR 推理
测试体验
在 TensorRT FP16 engine 准备好之后,程序启动后即可初始化模型并进行识别。
典型流程:
-
选择 OCR 模型
-
设置 GPU ID、batch、predictor 数量
-
点击初始化
-
选择图片
-
点击识别
程序会输出 det、rec、整体 OCR 耗时,方便对不同模型、不同参数进行性能对比。
部署建议
为了获得更稳定的速度,建议:
-
使用 NVIDIA 独立显卡
-
设置 NVIDIA 电源管理模式为"最高性能优先"
-
Windows 电源模式设置为高性能
-
初始化后先预热几次,再统计速度
-
不建议跨机器复用 TensorRT engine
-
不同显卡、驱动、CUDA、TensorRT 版本建议重新转换 engine
时间不稳定设置
1、查询GPU最高频率

2、锁定GPU频率

3、低延时模式设置为超高性

4、电源管理模式设置为最高性能优先

C#调用源码
using Newtonsoft.Json;
using OpenCvSharp;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Drawing;
using System.IO;
using System.Runtime.InteropServices;
using System.Text;
using System.Windows.Forms;
namespace OCRTest
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
const string DllName = "lw.TensorRT.PPOCRSharp.dll";
//初始化
[DllImport(DllName, EntryPoint = "init", CallingConvention = CallingConvention.Cdecl)]
public extern static int init(ref IntPtr engine
, [MarshalAs(UnmanagedType.I1)] bool use_gpu
, int gpu_id
, string det_model_dir
, int limit_side_len
, double det_db_thresh
, double det_db_box_thresh
, double det_db_unclip_ratio
, [MarshalAs(UnmanagedType.I1)] bool use_dilation
, [MarshalAs(UnmanagedType.I1)] bool cls
, [MarshalAs(UnmanagedType.I1)] bool use_angle_cls
, string cls_model_dir
, double cls_thresh
, double cls_batch_num
, string rec_model_dir
, string rec_char_dict_path
, int rec_batch_num
, int rec_img_h
, int rec_img_w
, int predictor_num
, StringBuilder msg);
//识别
[DllImport(DllName, EntryPoint = "ocr", CallingConvention = CallingConvention.Cdecl)]
public extern static int ocr(IntPtr engine, IntPtr image, StringBuilder msg, out IntPtr ocr_result, out int ocr_result_len);
//识别,按图像内存传入,避免托管层依赖 C++ Mat ABI
[DllImport(DllName, EntryPoint = "ocr2", CallingConvention = CallingConvention.Cdecl)]
public extern static int ocr2(IntPtr engine, int rows, int cols, int channels, IntPtr data, StringBuilder msg, out IntPtr ocr_result, out int ocr_result_len);
//释放
[DllImport(DllName, EntryPoint = "destroy", CallingConvention = CallingConvention.Cdecl)]
public extern static int destroy(IntPtr engine, StringBuilder msg);
static IntPtr OCREngine;
private Bitmap bmp;
private String imgPath = null;
private List<OCRResult> ltOCRResult;
private string fileFilter = "*.*|*.bmp;*.jpg;*.jpeg;*.tiff;*.tif;*.png";
private StringBuilder OCRResultInfo = new StringBuilder();
private StringBuilder OCRResultAllInfo = new StringBuilder();
Pen pen = new Pen(Brushes.Red, 2f);
/// <summary>
/// 选择图片
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void button1_Click(object sender, EventArgs e)
{
OpenFileDialog ofd = new OpenFileDialog();
ofd.Filter = fileFilter;
if (ofd.ShowDialog() == DialogResult.OK)
{
imgPath = ofd.FileName;
bmp?.Dispose();
bmp = new Bitmap(imgPath);
pictureBox1.Image = bmp;
richTextBox1.Clear();
button2_Click(null, null);
}
}
/// <summary>
/// 识别
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void button2_Click(object sender, EventArgs e)
{
if (OCREngine == IntPtr.Zero)
{
MessageBox.Show("请先初始化!!!");
return;
}
if (imgPath == null)
{
MessageBox.Show("请先选择图片!!!");
return;
}
button1.Enabled = false;
button2.Enabled = false;
richTextBox1.Clear();
OCRResultInfo.Clear();
OCRResultAllInfo.Clear();
StringBuilder msgTemp = new StringBuilder(128);
Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
IntPtr strPtr = IntPtr.Zero;
int ocr_result_len = 0;
string ocr_result = string.Empty;
int res = -1;
try
{
using (Mat img = Cv2.ImRead(imgPath, ImreadModes.Color))
{
res = ocr2(OCREngine, img.Rows, img.Cols, img.Channels(), img.Data, msgTemp, out strPtr, out ocr_result_len);
}
if (strPtr != IntPtr.Zero && ocr_result_len > 0)
{
byte[] buffer = new byte[ocr_result_len];
Marshal.Copy(strPtr, buffer, 0, ocr_result_len);
ocr_result = Encoding.UTF8.GetString(buffer);
}
stopwatch.Stop();
double totalTime = stopwatch.Elapsed.TotalMilliseconds;
OCRResultAllInfo.AppendLine($"耗时: {totalTime:F2}ms");
OCRResultAllInfo.AppendLine("---------------------------");
OCRResultInfo.AppendLine($"耗时: {totalTime:F2}ms");
OCRResultInfo.AppendLine("---------------------------");
if (res == 0)
{
ltOCRResult = Newtonsoft.Json.JsonConvert.DeserializeObject<List<OCRResult>>(ocr_result);
OCRResultAllInfo.Append(JsonConvert.SerializeObject(ltOCRResult, Newtonsoft.Json.Formatting.Indented));
using (Graphics graphics = Graphics.FromImage(bmp))
{
foreach (OCRResult item in ltOCRResult)
{
OCRResultInfo.AppendLine(item.text);
System.Drawing.Point[] pt = new System.Drawing.Point[] {
new System.Drawing.Point(item.x1, item.y1)
, new System.Drawing.Point(item.x2, item.y2)
, new System.Drawing.Point(item.x3, item.y3)
, new System.Drawing.Point(item.x4, item.y4)
};
graphics.DrawPolygon(pen, pt);
}
}
if (checkBox1.Checked)
{
richTextBox1.Text = OCRResultAllInfo.ToString();
}
else
{
richTextBox1.Text = OCRResultInfo.ToString();
}
pictureBox1.Image = null;
pictureBox1.Image = bmp;
}
else
{
MessageBox.Show("识别失败," + msgTemp.ToString());
}
}
finally
{
stopwatch.Stop();
if (strPtr != IntPtr.Zero)
{
Marshal.FreeCoTaskMem(strPtr);
}
button1.Enabled = true;
button2.Enabled = true;
}
}
/// <summary>
/// 初始化
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void Form1_Load(object sender, EventArgs e)
{
rdov6tiny.Checked = true;
chkcls.Checked = false;
chkuse_gpu.Checked = true;
txtgpu_id.Text = "0";
LoadDefaultImage();
}
private void checkBox1_CheckedChanged(object sender, EventArgs e)
{
richTextBox1.Clear();
if (checkBox1.Checked)
{
richTextBox1.Text = OCRResultAllInfo.ToString();
}
else
{
richTextBox1.Text = OCRResultInfo.ToString();
}
}
private void chkcls_CheckedChanged(object sender, EventArgs e)
{
if (OCREngine == IntPtr.Zero)
{
return;
}
richTextBox1.Text = "";
Console.WriteLine("cls设置已改变,正在重新初始化模型...");
UnloadModel();
LoadModel();
}
private void radioButton1_CheckedChanged(object sender, EventArgs e)
{
RadioButton rb = sender as RadioButton;
}
private void Form1_FormClosing(object sender, FormClosingEventArgs e)
{
UnloadModel();
}
private void btnDestroy_Click(object sender, EventArgs e)
{
UnloadModel();
}
void UnloadModel()
{
if (OCREngine != IntPtr.Zero)
{
StringBuilder msgTemp = new StringBuilder(128);
destroy(OCREngine, msgTemp);
Console.WriteLine("释放成功:" + msgTemp.ToString());
OCREngine = IntPtr.Zero;
}
}
private void btnInit_Click(object sender, EventArgs e)
{
if (OCREngine != IntPtr.Zero)
{
StringBuilder msgTemp = new StringBuilder(128);
destroy(OCREngine, msgTemp);
Console.WriteLine("释放成功:" + msgTemp.ToString());
OCREngine = IntPtr.Zero;
LoadModel();
}
else
{
LoadModel();
}
}
void LoadModel()
{
StringBuilder msgTemp = new StringBuilder(128);
bool use_gpu = true;
int gpu_id = 0;
string det_model_dir = "";
int limit_side_len = 960;
double det_db_thresh = 0.3;
double det_db_box_thresh = 0.6;
double det_db_unclip_ratio = 1.2;
bool use_dilation = false;
bool cls = true;
bool use_angle_cls = true;
string cls_model_dir = "inference_trt/PP-OCRv5_mobile_cls_onnx.engine";
double cls_thresh = 0.9;
int cls_batch_num = 1;
string rec_model_dir = "";
string rec_char_dict_path = "inference/ppocrv5_dict.txt";
int rec_batch_num = 8;
int rec_img_h = 48;
int rec_img_w = 320;
int predictor_num = 4;
//赋值
if (chkuse_gpu.Checked == true)
{
use_gpu = true;
}
else
{
use_gpu = false;
}
gpu_id = Convert.ToInt32(txtgpu_id.Text.ToString());
det_db_thresh = Convert.ToDouble(txtdet_db_thresh.Text.ToString());
det_db_box_thresh = Convert.ToDouble(txtdet_db_box_thresh.Text.ToString());
det_db_unclip_ratio = Convert.ToDouble(txtdet_db_unclip_ratio.Text.ToString());
if (chkcls.Checked == true)
{
cls = true;
}
else
{
cls = false;
}
cls_batch_num = Convert.ToInt32(txtcls_batch_num.Text.ToString());
rec_batch_num = Convert.ToInt32(txtrec_batch_num.Text.ToString());
predictor_num = Convert.ToInt32(txtpredictor_num.Text.ToString());
if (rdomobile.Checked)
{
det_model_dir = BuildEnginePath("PP-OCRv5_mobile_det_onnx");
rec_model_dir = BuildEnginePath("PP-OCRv5_mobile_rec_onnx");
cls_model_dir = BuildEnginePath("PP-OCRv5_mobile_cls_onnx");
}
elseif (rdoserver.Checked)
{
det_model_dir = BuildEnginePath("PP-OCRv5_server_det_infer");
rec_model_dir = BuildEnginePath("PP-OCRv5_server_rec_infer");
cls_model_dir = BuildEnginePath("PP-OCRv5_server_cls_infer");
}
elseif (rdov6small.Checked)
{
det_model_dir = BuildEnginePath("PP-OCRv6_small_det");
rec_model_dir = BuildEnginePath("PP-OCRv6_small_rec");
rec_char_dict_path = "inference/PP-OCRv6_small_rec_dict.txt";
}
elseif (rdov6medium.Checked)
{
det_model_dir = BuildEnginePath("PP-OCRv6_medium_det");
rec_model_dir = BuildEnginePath("PP-OCRv6_medium_rec");
rec_char_dict_path = "inference/PP-OCRv6_medium_rec_dict.txt";
}
else
{
det_model_dir = BuildEnginePath("PP-OCRv6_tiny_det");
rec_model_dir = BuildEnginePath("PP-OCRv6_tiny_rec");
rec_char_dict_path = "inference/PP-OCRv6_tiny_rec_dict.txt";
}
if (!EnsureTensorRtEngines(det_model_dir, rec_model_dir, cls ? cls_model_dir : null, rec_batch_num))
{
return;
}
Console.WriteLine("TensorRT precision: FP16");
Console.WriteLine("det model: " + det_model_dir);
if (cls && !File.Exists(Path.Combine(Application.StartupPath, cls_model_dir.Replace('/', Path.DirectorySeparatorChar))))
{
Console.WriteLine("cls model missing, disabled: " + cls_model_dir);
cls = false;
use_angle_cls = false;
}
Console.WriteLine("cls model: " + (cls ? cls_model_dir : "disabled"));
Console.WriteLine("rec model: " + rec_model_dir);
Console.WriteLine("rec dict : " + rec_char_dict_path);
int res = init(ref OCREngine
, use_gpu
, gpu_id
, det_model_dir
, limit_side_len
, det_db_thresh
, det_db_box_thresh
, det_db_unclip_ratio
, use_dilation
, cls
, use_angle_cls
, cls_model_dir
, cls_thresh
, cls_batch_num
, rec_model_dir
, rec_char_dict_path
, rec_batch_num
, rec_img_h
, rec_img_w
, predictor_num
, msgTemp);
if (res == 0)
{
Console.WriteLine("模型加载成功!");
}
else
{
string msg = msgTemp.ToString();
Console.WriteLine("模型加载失败," + msg);
}
}
private bool EnsureTensorRtEngines(string detEngine, string recEngine, string clsEngine, int recBatchNum)
{
List<string> missing = new List<string>();
foreach (string engine in new[] { detEngine, recEngine, clsEngine })
{
if (string.IsNullOrWhiteSpace(engine)) continue;
string fullPath = Path.Combine(Application.StartupPath, engine.Replace('/', Path.DirectorySeparatorChar));
if (!File.Exists(fullPath)) missing.Add(engine);
}
if (missing.Count == 0)
{
returntrue;
}
StringBuilder sb = new StringBuilder();
sb.AppendLine("TensorRT FP16 engine 转换说明");
sb.AppendLine("=======================================");
sb.AppendLine();
sb.AppendLine("当前缺少以下 engine 文件,请先使用 trtexec.exe 转换。");
sb.AppendLine("不同电脑的 trtexec.exe 路径可能不同,请把下面命令中的 <TRTEXEC> 替换为本机 trtexec.exe 的完整路径,或把 TensorRT bin 目录加入 PATH 后替换为 trtexec.exe。");
sb.AppendLine();
sb.AppendLine("常见位置示例:");
sb.AppendLine(@"C:\TensorRT-xx\bin\trtexec.exe");
sb.AppendLine(@"项目目录\TensorRT-10.16.1.11.Windows.amd64.cuda-12.9\TensorRT-10.16.1.11\bin\trtexec.exe");
sb.AppendLine();
sb.AppendLine("建议在 exe 所在目录执行以下命令:");
sb.AppendLine(Application.StartupPath);
sb.AppendLine();
foreach (string engine in missing)
{
string onnx = EngineToOnnxPath(engine);
string outDir = Path.GetDirectoryName(Path.Combine(Application.StartupPath, engine.Replace('/', Path.DirectorySeparatorChar)));
Directory.CreateDirectory(outDir);
string shape = BuildTrtShapeArgs(engine, recBatchNum);
sb.AppendLine("缺少:" + engine);
sb.AppendLine("对应 ONNX:" + onnx);
sb.AppendLine("转换命令:");
sb.AppendLine($"\"<TRTEXEC>\" --onnx=\"{onnx.Replace('/', Path.DirectorySeparatorChar)}\" --saveEngine=\"{engine.Replace('/', Path.DirectorySeparatorChar)}\" --fp16 {shape}");
sb.AppendLine();
}
sb.AppendLine("转换完成后重新点击初始化。");
sb.AppendLine("注意:engine 和显卡型号、驱动、CUDA、TensorRT 版本绑定,不建议跨机器复用。");
string commandFile = Path.Combine(Application.StartupPath, "trtexec_convert_commands.txt");
try
{
File.WriteAllText(commandFile, sb.ToString(), Encoding.UTF8);
sb.AppendLine();
sb.AppendLine("转换命令已写入:");
sb.AppendLine(commandFile);
}
catch (Exception ex)
{
sb.AppendLine();
sb.AppendLine("转换命令写入 txt 失败:");
sb.AppendLine(ex.Message);
}
MessageBox.Show(sb.ToString(), "缺少 TensorRT engine", MessageBoxButtons.OK, MessageBoxIcon.Information);
Console.WriteLine(sb.ToString());
returnfalse;
}
private string BuildEnginePath(string modelName)
{
return"inference_trt/" + modelName + "_fp16.engine";
}
private string EngineToOnnxPath(string engine)
{
string file = Path.GetFileNameWithoutExtension(engine);
if (file.EndsWith("_fp16", StringComparison.OrdinalIgnoreCase))
{
file = file.Substring(0, file.Length - 5);
}
return"inference/" + file + ".onnx";
}
private string BuildTrtShapeArgs(string engine, int recBatchNum)
{
string name = Path.GetFileNameWithoutExtension(engine).ToLowerInvariant();
if (name.Contains("rec"))
{
int optBatch = Math.Max(1, recBatchNum);
int maxBatch = Math.Max(optBatch, 16);
return $"--minShapes=x:1x3x48x32 --optShapes=x:{optBatch}x3x48x320 --maxShapes=x:{maxBatch}x3x48x1280";
}
if (name.Contains("cls"))
{
return"--minShapes=x:1x3x80x160 --optShapes=x:8x3x80x160 --maxShapes=x:16x3x80x160";
}
return"--minShapes=x:1x3x32x32 --optShapes=x:1x3x960x960 --maxShapes=x:1x3x960x960";
}
private void LoadDefaultImage()
{
string defaultImagePath = Path.Combine(Application.StartupPath, "3.jpg");
if (!File.Exists(defaultImagePath))
{
defaultImagePath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "3.jpg");
}
if (!File.Exists(defaultImagePath))
{
return;
}
imgPath = defaultImagePath;
bmp?.Dispose();
bmp = new Bitmap(imgPath);
pictureBox1.Image = bmp;
}
}
}
下载
通过网盘分享的文件:lw.TensorRT.PPOCRSharp_Test 链接: https://pan.baidu.com/s/1bG5n0ZgJ7mW2fQ4dn9sjaQ 提取码: cmvf