[深度学习]--分类问题的排查错误的流程

原因复现:

原生的.pt 好使, 转化后的 CoreML不好使, 分类有问题。

yolov8 格式的支持情况

复制代码
                   Format     Argument           Suffix    CPU    GPU
0                 PyTorch            -              .pt   True   True
1             TorchScript  torchscript     .torchscript   True   True
2                    ONNX         onnx            .onnx   True   True
3                OpenVINO     openvino  _openvino_model   True  False
4                TensorRT       engine          .engine  False   True
5                  CoreML       coreml       .mlpackage   True  False
6   TensorFlow SavedModel  saved_model     _saved_model   True   True
7     TensorFlow GraphDef           pb              .pb   True   True
8         TensorFlow Lite       tflite          .tflite   True  False
9     TensorFlow Edge TPU      edgetpu  _edgetpu.tflite   True  False
10          TensorFlow.js         tfjs       _web_model   True  False
11           PaddlePaddle       paddle    _paddle_model   True   True
12                   NCNN         ncnn      _ncnn_model   True   True

这里可以看到CoreML 只支持cpu, 尼玛tflite也是只支持cpu的

python 复制代码
def test_coreml():
    detect_weight = '/home/justin/Desktop/code/python_project/Jersey-Number/runs/detect/train64/weights/best.pt'
    model_detect = YOLO(detect_weight)
    results = model_detect(source="/home/justin/Desktop/code/python_project/Jersey-Number/zr_yz.MP4",stream=True,classes=[3])

    class_weight = '/home/justin/Desktop/code/python_project/Jersey-Number/runs/classify/train7/weights/best.mlpackage'
    class_weight = '/home/justin/Desktop/code/python_project/Jersey-Number/runs/classify/train7/weights/best.mlpackage'
    model_class = YOLO(class_weight)
    # 要使用的字体
    fontFace = cv2.FONT_HERSHEY_SIMPLEX
    fontScale = 3
    thickness = 1
    img_count = 0

    for result in results:
        img_count+=1
        if img_count == 6:
            a = 1
        boxes = result.boxes  # Boxes object for bounding box outputs
        for box in boxes:
            cls = box.cls.item()
            conf = box.conf.item()
            if conf > 0.5:
                x1,y1,x2,y2 = box.xyxy.tolist()[0]
                x1,y1,x2,y2 = int(x1),int(y1),int(x2),int(y2)
                orig_img = result.orig_img
                # H,W = orig_img.orig_shape
                cv2.imwrite("/home/justin/Desktop/code/python_project/Jersey-Number/runs/imgs"+"{:06d}-raw.jpg".format(img_count),orig_img)
                cropped_image = orig_img[y1:y2,x1:x2]
                # res_number_class = model_class(cropped_image,save_txt=True,save=True)
                res_number_class = model_class(cropped_image, device = "cpu")
                cv2.rectangle(orig_img, (int(x1), int(y1)), (int(x2), int(y2)), (255, 0, 0), 2) 
                for r in res_number_class:
                    if hasattr(r,"probs"):
                        if r.probs.top1conf.item() > 0.2:
                            class_name = r.names[r.probs.top1]
                            (width, height), bottom = cv2.getTextSize(class_name, fontFace, fontScale=fontScale, thickness=thickness)
                            cv2.putText(orig_img, class_name+" conf:"+str(r.probs.top1conf.item()), (x1 - width, y1-height), fontFace, fontScale, color=(0, 0, 255), thickness=thickness,
                                            lineType=cv2.LINE_AA)
                cv2.imwrite("/home/justin/Desktop/code/python_project/Jersey-Number/runs/imgs"+"{:06d}.jpg".format(img_count),orig_img)

报错的这句话值得看一眼:

sklearn不支持,tensorflow和torch没测试过,可能会有问题。 先跑跑再说吧

复制代码
Loading /home/justin/Desktop/code/python_project/Jersey-Number/runs/classify/train7/weights/best.mlpackage for CoreML inference...
scikit-learn version 1.4.2 is not supported. Minimum required version: 0.17. Maximum required version: 1.1.2. Disabling scikit-learn conversion API.
TensorFlow version 2.13.1 has not been tested with coremltools. You may run into unexpected errors. TensorFlow 2.12.0 is the most recent version that has been tested.
Torch version 2.3.0+cu121 has not been tested with coremltools. You may run into unexpected errors. Torch 2.1.0 is the most recent version that has been tested.

所以还要降级,真是麻烦,tensorflow是因为要转android侧的模型。

这里要给个参数,来指定cpu复现

res_number_class = model_class(cropped_image, device = "cpu")

这意思是不能用pytorch 跑了吗? @todo, 然后用pytorch 2.0的环境试一下看看怎么样?@todo,

核心诉求是要把coreml的模型加载起来,看看是不是存在一样的错误

复制代码
Exception has occurred: Exception
Model prediction is only supported on macOS version 10.13 or later.
  File "/home/justin/Desktop/code/python_project/Jersey-Number/zr_yz.py", line 76, in test_coreml
    res_number_class = model_class(cropped_image, device = "cpu")
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/justin/Desktop/code/python_project/Jersey-Number/zr_yz.py", line 88, in <module>
    test_coreml()
Exception: Model prediction is only supported on macOS version 10.13 or later.
bash 复制代码
detect 参数
detect_conf = 0.5172230005264282
切割位置: x1,y1,x2,y2
1. 原始位置:[1648.0953369140625, 882.2176513671875, 1682.9732666015625, 980.842041015625]
2.强制转成int 为后面切出这个区域做准备(1648, 882, 1682, 980)

分类输出结果:

top1:64

top1conf:tensor(0.9994, device='cuda:0')

top5:[64, 53, 10, 0, 20]

top5conf:tensor([9.9943e-01, 4.8942e-04, 1.9284e-05, 1.8095e-05, 8.8464e-06], device='cuda:0')

垃圾

shit CoreML模型只能在mac 上跑, 而且只能用CoreMl 跑么??? @todo???

确实只能在mac上跑

ref:

coreml的文档:

https://developer.apple.com/documentation/coreml

coremltool的文档:

https://apple.github.io/coremltools/docs-guides/

一段python代码:

python 复制代码
import coremltools as ct
import PIL
import torch
import numpy as np

def get_x1y1x2y2(coordinate,img):
    width,height = img.size()
    center_x = int(coordinate[0] * width)
    center_y = int(coordinate[1] * height)
    img_w = int(coordinate[2]*width)
    img_h = int(coordinate[3]*height)
    return center_x, center_y, img_w, img_h

def ml_test_detect():
    mlmodel = ct.models.MLModel('/Users/smkj/Desktop/Code/justin/head_person_hoop_number_v8n.mlpackage')
    print(mlmodel)
    img = PIL.Image.open("/Users/smkj/Desktop/Code/justin/imgs000006-raw.jpg").resize((640,384))
    res = mlmodel.predict({"image":img})
    confidence_max_list = list(np.array(res['confidence']).argmax(axis=1))
    # array([0.86775684, 0.8630705 , 0.01861118, 0.09405255], dtype=float32)
    for row_index, class_id in enumerate(confidence_max_list):
        if class_id == 3:
            coordinate = res['coordinates'][row_index]
            x1,y1,x2,y2 = 555 - 12 / 2, 333  - 36 / 2, 555 + 12/2, 333 + 36/2
            im=img.crop((x1, y1, x2, y2))
            im.save("bbb.jpg")
    print(res)
# print(img)
def ml_test_classify():
    img = PIL.Image.open("/Users/smkj/Desktop/Code/justin/bbb.jpg").resize((64,64))

    mlmodel = ct.models.MLModel('/Users/smkj/Desktop/Code/justin/classification.mlpackage')
    res = mlmodel.predict({"image":img})
    max_key = max(res['classLabel_probs'], key=res['classLabel_probs'].get)
    print("class_name:",max_key, "confidence:",res['classLabel_probs'].get(max_key))
    a = 1
ml_test_classify()

在mac上安装opencv实在是太费劲了,各位自求多福吧!

用这个可以替代opencv: pip install pillow


置信度也是99.99

coreml不爽的点是必须要固定尺寸??? @todo 也许是我用惯了动态尺寸的原因。 anyway,今天调试了一天,在两个电脑上装了环境,算是搞定了。!!!

相关推荐
贾全11 分钟前
第十章:HIL-SERL 真实机器人训练实战
人工智能·深度学习·算法·机器学习·机器人
我是小哪吒2.037 分钟前
书籍推荐-《对抗机器学习:攻击面、防御机制与人工智能中的学习理论》
人工智能·深度学习·学习·机器学习·ai·语言模型·大模型
慕婉030741 分钟前
深度学习前置知识全面解析:从机器学习到深度学习的进阶之路
人工智能·深度学习·机器学习
埃菲尔铁塔_CV算法3 小时前
基于 TOF 图像高频信息恢复 RGB 图像的原理、应用与实现
人工智能·深度学习·数码相机·算法·目标检测·计算机视觉
中杯可乐多加冰5 小时前
【AI落地应用实战】AIGC赋能职场PPT汇报:从效率工具到辅助优化
人工智能·深度学习·神经网络·aigc·powerpoint·ai赋能
烟锁池塘柳05 小时前
【大模型】解码策略:Greedy Search、Beam Search、Top-k/Top-p、Temperature Sampling等
人工智能·深度学习·机器学习
zzc9216 小时前
时频图数据集更正程序,去除坐标轴白边及调整对应的标签值
人工智能·深度学习·数据集·标签·时频图·更正·白边
Blossom.1187 小时前
机器学习在智能供应链中的应用:需求预测与物流优化
人工智能·深度学习·神经网络·机器学习·计算机视觉·机器人·语音识别
Gyoku Mint7 小时前
深度学习×第4卷:Pytorch实战——她第一次用张量去拟合你的轨迹
人工智能·pytorch·python·深度学习·神经网络·算法·聚类
m0_751336399 小时前
突破性进展:超短等离子体脉冲实现单电子量子干涉,为飞行量子比特奠定基础
人工智能·深度学习·量子计算·材料科学·光子器件·光子学·无线电电子