【AIGC】HF-Mirror 使用说明(MacOS 版)

今天这篇文章主要是记录 HF-Mirror 的使用过程,官网上提供了 Linux 和 Windows 的使用方式。本人作为 MacOS 用户也将自己的使用办法记录一下,希望能够帮助到其他人。

HF-Mirror 是什么?

HF-Mirror 是大神 padeoe 开源的 huggingface 镜像站,网站属于公益性质,它帮助国内 AI 开发者能够快速、稳定地下载 huggingface 上模型和数据集。网站地址:hf-mirror.com/

怎么使用呢?

在下载模型前,需要对系统终端进行一些配置。

安装依赖

bash 复制代码
(base) yuanzhenhui@MacBook-Pro python % pip install -U huggingface_hub
Requirement already satisfied: huggingface_hub in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (0.15.1)
Collecting huggingface_hub
  Obtaining dependency information for huggingface_hub from https://files.pythonhosted.org/packages/3d/c8/c3342c97848896df5d78d18abd94c558e457a4f02feec99a79989d8c30e0/huggingface_hub-0.21.2-py3-none-any.whl.metadata
  Downloading huggingface_hub-0.21.2-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: filelock in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (3.9.0)
Collecting fsspec>=2023.5.0 (from huggingface_hub)
  Obtaining dependency information for fsspec>=2023.5.0 from https://files.pythonhosted.org/packages/ad/30/2281c062222dc39328843bd1ddd30ff3005ef8e30b2fd09c4d2792766061/fsspec-2024.2.0-py3-none-any.whl.metadata
  Downloading fsspec-2024.2.0-py3-none-any.whl.metadata (6.8 kB)
Requirement already satisfied: requests in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (2.31.0)
Requirement already satisfied: tqdm>=4.42.1 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (4.65.0)
Requirement already satisfied: pyyaml>=5.1 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (6.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (4.7.1)
Requirement already satisfied: packaging>=20.9 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from huggingface_hub) (23.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /Users/yuanzhenhui/anaconda3/lib/python3.11/site-packages (from requests->huggingface_hub) (2023.11.17)
Downloading huggingface_hub-0.21.2-py3-none-any.whl (346 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 346.2/346.2 kB 644.8 kB/s eta 0:00:00
Downloading fsspec-2024.2.0-py3-none-any.whl (170 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 170.9/170.9 kB 2.7 MB/s eta 0:00:00
Installing collected packages: fsspec, huggingface_hub
  Attempting uninstall: fsspec
    Found existing installation: fsspec 2023.4.0
    Uninstalling fsspec-2023.4.0:
      Successfully uninstalled fsspec-2023.4.0
  Attempting uninstall: huggingface_hub
    Found existing installation: huggingface-hub 0.15.1
    Uninstalling huggingface-hub-0.15.1:
      Successfully uninstalled huggingface-hub-0.15.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
s3fs 2023.4.0 requires fsspec==2023.4.0, but you have fsspec 2024.2.0 which is incompatible.
Successfully installed fsspec-2024.2.0 huggingface_hub-0.21.2

设置环境变量

bash 复制代码
(base) yuanzhenhui@MacBook-Pro python % cd
(base) yuanzhenhui@MacBook-Pro python % vim .bash_profile

# 在 bash_profile 文件中随便找一行将下面的命令记录一下

export HF_ENDPOINT=https://hf-mirror.com

如果你止步于此,那么在下载时就会跳转到如下页面: 点击"前往官方入口"按钮会跳转到 Huggingface 官网,在这里你需要生成 access token (按照提示跟着走就可以了...),如下图:

使用 token 下载

拷贝后就可以通过以下命令进行下载,以 google/gemma-7b 为例:

bash 复制代码
(base) yuanzhenhui@MacBook-Pro python % huggingface-cli download \
--token hf_xxxxxxxxxxxxxxxxxxx \
--resume-download google/gemma-7b \
--local-dir gemma

至于下载后怎么用就留待后面的文章进行描述吧。

相关推荐
星期天要睡觉23 分钟前
计算机视觉(opencv)实战十八——图像透视转换
人工智能·opencv·计算机视觉
Morning的呀1 小时前
Class48 GRU
人工智能·深度学习·gru
拾零吖3 小时前
李宏毅 Deep Learning
人工智能·深度学习·机器学习
华芯邦3 小时前
广东充电芯片助力新能源汽车车载系统升级
人工智能·科技·车载系统·汽车·制造
时空无限4 小时前
说说transformer 中的掩码矩阵以及为什么能掩盖住词语
人工智能·矩阵·transformer
查里王4 小时前
AI 3D 生成工具知识库:当前产品格局与测评总结
人工智能·3d
武子康5 小时前
AI-调查研究-76-具身智能 当机器人走进生活:具身智能对就业与社会结构的深远影响
人工智能·程序人生·ai·职场和发展·机器人·生活·具身智能
小鹿清扫日记5 小时前
从蛮力清扫到 “会看路”:室外清洁机器人的文明进阶
人工智能·ai·机器人·扫地机器人·具身智能·连合直租·有鹿巡扫机器人
fanstuck5 小时前
Prompt提示工程上手指南(六):AI避免“幻觉”(Hallucination)策略下的Prompt
人工智能·语言模型·自然语言处理·nlp·prompt
zhangfeng11336 小时前
win7 R 4.4.0和RStudio1.25的版本兼容性以及系统区域设置有关 导致Plots绘图面板被禁用,但是单独页面显示
开发语言·人工智能·r语言·生物信息