Candle - HuggingFace Rust AI 框架 - 小记

伊织code2024-03-24 13:27

文章目录

    • [关于 Candle](#关于 Candle)
      • 结构
      • 安装
        • [1、With Cuda support :](#1、With Cuda support :)
        • [2、Without Cuda support :](#2、Without Cuda support :)
        • [3、With mkl support](#3、With mkl support)
    • [二、基本使用 Hello world!](#二、基本使用 Hello world!)
      • [1、处理 MNIST 数据集](#1、处理 MNIST 数据集)
      • [2、使用一个 `Linear` 层](#2、使用一个 Linear 层)
      • [3、使用 `candle_nn`](#3、使用 candle_nn)
    • [三、Pytorch cheatsheet](#三、Pytorch cheatsheet)

关于 Candle

Candle is a minimalist ML framework for Rust with a focus on performance (including GPU support) and ease of use.

你可以尝试在线 demos: whisper,LLaMA2, T5, yolo, Segment Anything.

相关文章、教程

结构

Candle 结构包括：

  • Candle-core：核心操作、设备和 Tensor 结构定义。
  • Candle-nn：构建真实模型的工具。
  • Candle-examples：在实际设置中使用库的示例。
  • Candle-kernels：CUDA 自定义内核；
  • Candle-datasets：数据集和数据加载器。
  • Candle-Transformers：与 Transformers 相关的实用程序。
  • Candle-flash-attn：Flash attention v2 层。
  • candle-onnx: ONNX 模型评估。

安装

https://huggingface.github.io/candle/guide/installation.html

安装 Rust: https://blog.csdn.net/lovechris00/article/details/124808034

1、With Cuda support :

1.1 首先，确保 Cuda 被正确安装了

  • nvcc --version 应该打印有关Cuda编译器驱动程序的信息。
  • nvidia-smi --query-gpu=compute_cap --format=csv 应该打印您的GPU计算能力，例如：
bash 复制代码 
compute_cap
8.9

您还可以使用 CUDA_COMPUTE_CAP=<compute cap> 环境变量为特定的计算编译 Cuda内核。

如果以上任何命令出错，请确保更新您的Cuda版本。

1.2 创建一个新的app，添加 candle-core 来增加 Cuda 支持。

从创建一个新的 cargo 开始 :

bash 复制代码 
cargo new myapp
cd myapp

Make sure to add the candle-core crate with the cuda feature:

确保添加具有cuda功能的 candle-core 被创建：

shell 复制代码 
cargo add --git https://github.com/huggingface/candle.git candle-core --features "cuda"

运行 cargo build 来保证所有被正确编译

shell 复制代码 
cargo build
2、Without Cuda support :

创建一个新的 app，并添加 candle-core 如下：

shell 复制代码 
cargo new myapp
cd myapp
cargo add --git https://github.com/huggingface/candle.git candle-core

最后，运行 cargo build 来保证所有被正确编译

shell 复制代码 
cargo build
3、With mkl support

You can also see the mkl feature which could be interesting to get faster inference on CPU. Using mkl

二、基本使用 Hello world!

转载自：Hello world!

1、处理 MNIST 数据集

We will now create the hello world of the ML world, building a model capable of solving MNIST dataset.

Open src/main.rs and fill in this content:

rust 复制代码 
use candle_core::{Device, Result, Tensor};

struct Model {
    first: Tensor,
    second: Tensor,
}

impl Model {
    fn forward(&self, image: &Tensor) -> Result<Tensor> {
        let x = image.matmul(&self.first)?;
        let x = x.relu()?;
        x.matmul(&self.second)
    }
}

fn main() -> Result<()> {
    // Use Device::new_cuda(0)?; to use the GPU.
    let device = Device::Cpu;

    let first = Tensor::randn(0f32, 1.0, (784, 100), &device)?;
    let second = Tensor::randn(0f32, 1.0, (100, 10), &device)?;
    let model = Model { first, second };

    let dummy_image = Tensor::randn(0f32, 1.0, (1, 784), &device)?;

    let digit = model.forward(&dummy_image)?;
    println!("Digit {digit:?} digit");
    Ok(())
}

Everything should now run with:

bash 复制代码 
cargo run --release

2、使用一个 Linear 层

Now that we have this, we might want to complexify things a bit, for instance by adding bias and creating the classical Linear layer. We can do as such

rust 复制代码 
struct Linear{
    weight: Tensor,
    bias: Tensor,
}
impl Linear{
    fn forward(&self, x: &Tensor) -> Result<Tensor> {
        let x = x.matmul(&self.weight)?;
        x.broadcast_add(&self.bias)
    }
}

struct Model {
    first: Linear,
    second: Linear,
}

impl Model {
    fn forward(&self, image: &Tensor) -> Result<Tensor> {
        let x = self.first.forward(image)?;
        let x = x.relu()?;
        self.second.forward(&x)
    }
}

This will change the model running code into a new function

rust 复制代码 
fn main() -> Result<()> {
    // Use Device::new_cuda(0)?; to use the GPU.
    // Use Device::Cpu; to use the CPU.
    let device = Device::cuda_if_available(0)?;

    // Creating a dummy model
    let weight = Tensor::randn(0f32, 1.0, (784, 100), &device)?;
    let bias = Tensor::randn(0f32, 1.0, (100, ), &device)?;
    let first = Linear{weight, bias};
    let weight = Tensor::randn(0f32, 1.0, (100, 10), &device)?;
    let bias = Tensor::randn(0f32, 1.0, (10, ), &device)?;
    let second = Linear{weight, bias};
    let model = Model { first, second };

    let dummy_image = Tensor::randn(0f32, 1.0, (1, 784), &device)?;

    // Inference on the model
    let digit = model.forward(&dummy_image)?;
    println!("Digit {digit:?} digit");
    Ok(())
}

Now it works, it is a great way to create your own layers. But most of the classical layers are already implemented in candle-nn.

3、使用 candle_nn

For instance Linear is already there. This Linear is coded with PyTorch layout in mind, to reuse better existing models out there, so it uses the transpose of the weights and not the weights directly.

So instead we can simplify our example:

bash 复制代码 
cargo add --git https://github.com/huggingface/candle.git candle-nn

And rewrite our examples using it

rust 复制代码 
use candle_core::{Device, Result, Tensor};
use candle_nn::{Linear, Module};

struct Model {
    first: Linear,
    second: Linear,
}

impl Model {
    fn forward(&self, image: &Tensor) -> Result<Tensor> {
        let x = self.first.forward(image)?;
        let x = x.relu()?;
        self.second.forward(&x)
    }
}

fn main() -> Result<()> {
    // Use Device::new_cuda(0)?; to use the GPU.
    let device = Device::Cpu;

    // This has changed (784, 100) -> (100, 784) !
    let weight = Tensor::randn(0f32, 1.0, (100, 784), &device)?;
    let bias = Tensor::randn(0f32, 1.0, (100, ), &device)?;
    let first = Linear::new(weight, Some(bias));
    let weight = Tensor::randn(0f32, 1.0, (10, 100), &device)?;
    let bias = Tensor::randn(0f32, 1.0, (10, ), &device)?;
    let second = Linear::new(weight, Some(bias));
    let model = Model { first, second };

    let dummy_image = Tensor::randn(0f32, 1.0, (1, 784), &device)?;

    let digit = model.forward(&dummy_image)?;
    println!("Digit {digit:?} digit");
    Ok(())
}

Feel free to modify this example to use Conv2d to create a classical convnet instead.

Now that we have the running dummy code we can get to more advanced topics:

三、Pytorch cheatsheet

https://huggingface.github.io/candle/guide/cheatsheet.html#pytorch-cheatsheet

Using PyTorch Using Candle
Creation torch.Tensor([[1, 2], [3, 4]]) Tensor::new(&[[1f32, 2.], [3., 4.]], &Device::Cpu)?
Creation torch.zeros((2, 2)) Tensor::zeros((2, 2), DType::F32, &Device::Cpu)?
Indexing tensor[:, :4] tensor.i((.., ..4))?
Operations tensor.view((2, 2)) tensor.reshape((2, 2))?
Operations a.matmul(b) a.matmul(&b)?
Arithmetic a + b &a + &b
Device tensor.to(device="cuda") tensor.to_device(&Device::new_cuda(0)?)?
Dtype tensor.to(dtype=torch.float16) tensor.to_dtype(&DType::F16)?
Saving torch.save({"A": A}, "model.bin") candle::safetensors::save(&HashMap::from([("A", A)]), "model.safetensors")?
Loading weights = torch.load("model.bin") candle::safetensors::load("model.safetensors", &device)

伊织 2024-03-23

相关推荐
羞儿40 分钟前
【读点论文】基于二维伽马函数的光照不均匀图像自适应校正算法
人工智能·算法·计算机视觉
Enaium1 小时前
Rust入门实战 编写Minecraft启动器#1启动方法
java·后端·rust
算法金「全网同名」1 小时前
算法金 | 时间序列预测真的需要深度学习模型吗？是的，我需要。不，你不需要？
深度学习·机器学习·数据分析
SEU-WYL1 小时前
基于深度学习的文本框检测
人工智能·深度学习·dnn
B站计算机毕业设计超人1 小时前
计算机毕业设计Python深度学习美食推荐系统 美食可视化 美食数据分析大屏 美食爬虫 美团爬虫 机器学习 大数据毕业设计 Django Vue.js
大数据·python·深度学习·机器学习·数据分析·课程设计·推荐算法
电商运营花2 小时前
告别盲目跟风！1688竞品数据分析实战指南（图文解析）
大数据·人工智能·经验分享·笔记·数据挖掘·数据分析
Rjdeng2 小时前
【AI大模型】驱动的未来：穿戴设备如何革新血液、皮肤检测与营养健康管理
人工智能·ai·穿戴设备·血液分析·营养健康
baivfhpwxf20232 小时前
select_shape 借助形状特征选择区域
人工智能·笔记
夕小瑶2 小时前
守口如瓶，OpenAI刻意隐瞒的黑客攻击事件时隔一年被证实
人工智能·深度学习·机器学习·自然语言处理
啊取名真困难2 小时前
AI艺术创作机器人：探索科技与艺术的交汇点
人工智能·科技·机器人
热门推荐
01Coze扣子平台完整体验和实践（附国内和国际版对比）02yolov8实战第五天——yolov8+ffmpg实时视频流检测并进行实时推流——（推流，保姆教学）03组基轨迹建模 GBTM的介绍与实现（Stata 或 R）042024年江苏省研究生数学建模科研创新实践大赛C题气象数据高精度融合技术研究论文和代码分析05【经验分享】Ubuntu22.04安装微信(linux官方版)06【立创EDA-PCB设计基础】6.布线铺铜实战及细节详解07通达信神奇九转指标原理及选股公式，无未来函数，数字不消失082024亚太赛（中文赛）数学建模竞赛选题建议+初步分析09浏览器插件——ModHeader 的分享和学习102024 年第十四届 APMCM 亚太地区大学生数学建模竞赛B题超详细解题思路+数据预处理问题一代码分享