【AIGC】HF-Mirror 使用说明(MacOS 版)

今天这篇文章主要是记录 HF-Mirror 的使用过程,官网上提供了 Linux 和 Windows 的使用方式。本人作为 MacOS 用户也将自己的使用办法记录一下,希望能够帮助到其他人。

HF-Mirror 是什么?

HF-Mirror 是大神 padeoe 开源的 huggingface 镜像站,网站属于公益性质,它帮助国内 AI 开发者能够快速、稳定地下载 huggingface 上模型和数据集。网站地址:hf-mirror.com/

怎么使用呢?

在下载模型前,需要对系统终端进行一些配置。

安装依赖

bash 复制代码
(base) yuanzhenhui@MacBook-Pro python % pip install -U huggingface_hub
Requirement already satisfied: huggingface_hub in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (0.15.1)
Collecting huggingface_hub
  Obtaining dependency information for huggingface_hub from https://files.pythonhosted.org/packages/3d/c8/c3342c97848896df5d78d18abd94c558e457a4f02feec99a79989d8c30e0/huggingface_hub-0.21.2-py3-none-any.whl.metadata
  Downloading huggingface_hub-0.21.2-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: filelock in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (3.9.0)
Collecting fsspec>=2023.5.0 (from huggingface_hub)
  Obtaining dependency information for fsspec>=2023.5.0 from https://files.pythonhosted.org/packages/ad/30/2281c062222dc39328843bd1ddd30ff3005ef8e30b2fd09c4d2792766061/fsspec-2024.2.0-py3-none-any.whl.metadata
  Downloading fsspec-2024.2.0-py3-none-any.whl.metadata (6.8 kB)
Requirement already satisfied: requests in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (2.31.0)
Requirement already satisfied: tqdm>=4.42.1 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (4.65.0)
Requirement already satisfied: pyyaml>=5.1 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (6.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (4.7.1)
Requirement already satisfied: packaging>=20.9 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (23.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (2023.11.17)
Downloading huggingface_hub-0.21.2-py3-none-any.whl (346 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 346.2/346.2 kB 644.8 kB/s eta 0:00:00
Downloading fsspec-2024.2.0-py3-none-any.whl (170 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 170.9/170.9 kB 2.7 MB/s eta 0:00:00
Installing collected packages: fsspec, huggingface_hub
  Attempting uninstall: fsspec
    Found existing installation: fsspec 2023.4.0
    Uninstalling fsspec-2023.4.0:
      Successfully uninstalled fsspec-2023.4.0
  Attempting uninstall: huggingface_hub
    Found existing installation: huggingface-hub 0.15.1
    Uninstalling huggingface-hub-0.15.1:
      Successfully uninstalled huggingface-hub-0.15.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
s3fs 2023.4.0 requires fsspec==2023.4.0, but you have fsspec 2024.2.0 which is incompatible.
Successfully installed fsspec-2024.2.0 huggingface_hub-0.21.2

设置环境变量

bash 复制代码
(base) yuanzhenhui@MacBook-Pro python % cd
(base) yuanzhenhui@MacBook-Pro python % vim .bash_profile

# 在 bash_profile 文件中随便找一行将下面的命令记录一下

export HF_ENDPOINT=https://hf-mirror.com

如果你止步于此,那么在下载时就会跳转到如下页面: 点击"前往官方入口"按钮会跳转到 Huggingface 官网,在这里你需要生成 access token (按照提示跟着走就可以了...),如下图:

使用 token 下载

拷贝后就可以通过以下命令进行下载,以 google/gemma-7b 为例:

bash 复制代码
(base) yuanzhenhui@MacBook-Pro python % huggingface-cli download \
--token hf_xxxxxxxxxxxxxxxxxxx \
--resume-download google/gemma-7b \
--local-dir gemma

至于下载后怎么用就留待后面的文章进行描述吧。

相关推荐
深度学习实战训练营27 分钟前
基于CNN-RNN的影像报告生成
人工智能·深度学习
昨日之日20062 小时前
Moonshine - 新型开源ASR(语音识别)模型,体积小,速度快,比OpenAI Whisper快五倍 本地一键整合包下载
人工智能·whisper·语音识别
浮生如梦_2 小时前
Halcon基于laws纹理特征的SVM分类
图像处理·人工智能·算法·支持向量机·计算机视觉·分类·视觉检测
深度学习lover2 小时前
<项目代码>YOLOv8 苹果腐烂识别<目标检测>
人工智能·python·yolo·目标检测·计算机视觉·苹果腐烂识别
热爱跑步的恒川3 小时前
【论文复现】基于图卷积网络的轻量化推荐模型
网络·人工智能·开源·aigc·ai编程
阡之尘埃5 小时前
Python数据分析案例61——信贷风控评分卡模型(A卡)(scorecardpy 全面解析)
人工智能·python·机器学习·数据分析·智能风控·信贷风控
孙同学要努力7 小时前
全连接神经网络案例——手写数字识别
人工智能·深度学习·神经网络
Eric.Lee20217 小时前
yolo v5 开源项目
人工智能·yolo·目标检测·计算机视觉
其实吧38 小时前
基于Matlab的图像融合研究设计
人工智能·计算机视觉·matlab
丕羽9 小时前
【Pytorch】基本语法
人工智能·pytorch·python