【AIGC】HF-Mirror 使用说明(MacOS 版)

今天这篇文章主要是记录 HF-Mirror 的使用过程,官网上提供了 Linux 和 Windows 的使用方式。本人作为 MacOS 用户也将自己的使用办法记录一下,希望能够帮助到其他人。

HF-Mirror 是什么?

HF-Mirror 是大神 padeoe 开源的 huggingface 镜像站,网站属于公益性质,它帮助国内 AI 开发者能够快速、稳定地下载 huggingface 上模型和数据集。网站地址:hf-mirror.com/

怎么使用呢?

在下载模型前,需要对系统终端进行一些配置。

安装依赖

bash 复制代码
(base) yuanzhenhui@MacBook-Pro python % pip install -U huggingface_hub
Requirement already satisfied: huggingface_hub in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (0.15.1)
Collecting huggingface_hub
  Obtaining dependency information for huggingface_hub from https://files.pythonhosted.org/packages/3d/c8/c3342c97848896df5d78d18abd94c558e457a4f02feec99a79989d8c30e0/huggingface_hub-0.21.2-py3-none-any.whl.metadata
  Downloading huggingface_hub-0.21.2-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: filelock in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (3.9.0)
Collecting fsspec>=2023.5.0 (from huggingface_hub)
  Obtaining dependency information for fsspec>=2023.5.0 from https://files.pythonhosted.org/packages/ad/30/2281c062222dc39328843bd1ddd30ff3005ef8e30b2fd09c4d2792766061/fsspec-2024.2.0-py3-none-any.whl.metadata
  Downloading fsspec-2024.2.0-py3-none-any.whl.metadata (6.8 kB)
Requirement already satisfied: requests in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (2.31.0)
Requirement already satisfied: tqdm>=4.42.1 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (4.65.0)
Requirement already satisfied: pyyaml>=5.1 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (6.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (4.7.1)
Requirement already satisfied: packaging>=20.9 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (23.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (2023.11.17)
Downloading huggingface_hub-0.21.2-py3-none-any.whl (346 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 346.2/346.2 kB 644.8 kB/s eta 0:00:00
Downloading fsspec-2024.2.0-py3-none-any.whl (170 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 170.9/170.9 kB 2.7 MB/s eta 0:00:00
Installing collected packages: fsspec, huggingface_hub
  Attempting uninstall: fsspec
    Found existing installation: fsspec 2023.4.0
    Uninstalling fsspec-2023.4.0:
      Successfully uninstalled fsspec-2023.4.0
  Attempting uninstall: huggingface_hub
    Found existing installation: huggingface-hub 0.15.1
    Uninstalling huggingface-hub-0.15.1:
      Successfully uninstalled huggingface-hub-0.15.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
s3fs 2023.4.0 requires fsspec==2023.4.0, but you have fsspec 2024.2.0 which is incompatible.
Successfully installed fsspec-2024.2.0 huggingface_hub-0.21.2

设置环境变量

bash 复制代码
(base) yuanzhenhui@MacBook-Pro python % cd
(base) yuanzhenhui@MacBook-Pro python % vim .bash_profile

# 在 bash_profile 文件中随便找一行将下面的命令记录一下

export HF_ENDPOINT=https://hf-mirror.com

如果你止步于此,那么在下载时就会跳转到如下页面: 点击"前往官方入口"按钮会跳转到 Huggingface 官网,在这里你需要生成 access token (按照提示跟着走就可以了...),如下图:

使用 token 下载

拷贝后就可以通过以下命令进行下载,以 google/gemma-7b 为例:

bash 复制代码
(base) yuanzhenhui@MacBook-Pro python % huggingface-cli download \
--token hf_xxxxxxxxxxxxxxxxxxx \
--resume-download google/gemma-7b \
--local-dir gemma

至于下载后怎么用就留待后面的文章进行描述吧。

相关推荐
徐小夕@趣谈前端9 分钟前
拒绝重复造轮子?我们偏偏花365天,用Vue3写了款AI协同的Word编辑器
人工智能·编辑器·word
阿里云大数据AI技术9 分钟前
全模态、多引擎、一体化,阿里云DLF3.0构建Data+AI驱动的智能湖仓平台
人工智能·阿里云·云计算
陈天伟教授10 分钟前
人工智能应用- 语言理解:05.大语言模型
人工智能·语言模型·自然语言处理
池央11 分钟前
CANN GE 深度解析:图编译器的核心优化策略、执行流调度与模型下沉技术原理
人工智能·ci/cd·自动化
七月稻草人14 分钟前
CANN ops-nn:AIGC底层神经网络算力的核心优化引擎
人工智能·神经网络·aigc·cann
种时光的人14 分钟前
CANN仓库核心解读:ops-nn打造AIGC模型的神经网络算子核心支撑
人工智能·神经网络·aigc
晚霞的不甘16 分钟前
守护智能边界:CANN 的 AI 安全机制深度解析
人工智能·安全·语言模型·自然语言处理·前端框架
谢璞18 分钟前
中国AI最疯狂的一周:50亿金元肉搏,争夺未来的突围之战
人工智能
池央18 分钟前
CANN 算子生态的深度演进:稀疏计算支持与 PyPTO 范式的抽象层级
运维·人工智能·信号处理
方见华Richard19 分钟前
世毫九实验室(Shardy Lab)研究成果清单(2025版)
人工智能·经验分享·交互·原型模式·空间计算