【NLP】32. Transformers (HuggingFace Pipelines 实战)

🤖 Transformers (HuggingFace Pipelines 实战)

本教程基于 Hugging Face 的 transformers 库,展示如何使用预训练模型完成以下任务:

  • 情感分析(Sentiment Analysis)
  • 文本生成(Text Generation)
  • 翻译(Translation)
  • 掩码填空(Masked Language Modeling)
  • 零样本分类(Zero-shot Classification)
  • 特定任务模型推理(Text2Text Generation, e.g., T5)
  • 文本摘要(Summarization)
  • 自定义分类模型载入与推理(如 BERT)

📦 安装依赖

python 复制代码
!pip install datasets evaluate transformers sentencepiece -q

✅ 情感分析

python 复制代码
from transformers import pipeline

classifier = pipeline("sentiment-analysis")

# 单个句子
classifier("I really enjoy learning new things every day.")

# 多个句子
classifier([
    "I really enjoy learning new things every day.",
    "I dislike rainy weather on weekends."
])

✍️ 文本生成(默认模型)

python 复制代码
from transformers import pipeline

generator = pipeline("text-generation")
generator("Today we are going to explore something exciting")

🎯 文本生成(指定参数)

python 复制代码
# 两句话,每句最多15词
generator("Today we are going to explore something exciting", num_return_sequences=2, max_length=15)

⚙️ 使用 distilgpt2 模型生成文本

python 复制代码
from transformers import pipeline

generator = pipeline("text-generation", model="distilgpt2")
generator(
    "Today we are going to explore something exciting",
    max_length=30,
    num_return_sequences=2,
)

🌍 翻译(英语→德语)

python 复制代码
from transformers import pipeline

translator = pipeline("translation_en_to_de")
translator("This is a great day to learn something new!")

🧩 掩码填空任务(填词)

python 复制代码
from transformers import pipeline

unmasker = pipeline("fill-mask")
unmasker("Life is like a <mask> of chocolates.")

🧠 零样本分类(Zero-shot Classification)

python 复制代码
from transformers import pipeline

classifier = pipeline("zero-shot-classification")
classifier(
    "This tutorial is about machine learning and natural language processing.",
    candidate_labels=["education", "sports", "politics"]
)

🔁 T5 模型(Text2Text)

python 复制代码
from transformers import pipeline

text2text = pipeline("text2text-generation")
text2text("Translate English to French: How are you today?")

✂️ 文本摘要(Summarization)

python 复制代码
from transformers import pipeline

summarizer = pipeline("summarization")
summarizer(
    "Machine learning is a field of artificial intelligence that focuses on enabling machines to learn from data..."
)

🧪 使用自己的模型(以 BERT 为例)

python 复制代码
from transformers import BertTokenizer, BertForSequenceClassification
from transformers import pipeline

model_name = "nlptown/bert-base-multilingual-uncased-sentiment"
tokenizer = BertTokenizer.from_pretrained(model_name)
model = BertForSequenceClassification.from_pretrained(model_name)

pipe = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)
pipe("I had an amazing experience with this product!")
相关推荐
AIbase202412 分钟前
GEO排名优化:AI如何重新定义搜索引擎的可见性规则?
人工智能·搜索引擎
hanniuniu1313 分钟前
F5发布后量子API安全解决方案,以AI驱动全面防护应对量子计算威胁
人工智能·安全·量子计算
fsnine15 分钟前
机器学习回顾——决策树详解
人工智能·决策树·机器学习
新智元33 分钟前
刚刚,OpenAI 任命新印裔 CTO!11 亿美金连公司打包收购,接管 ChatGPT 产品核心
人工智能·openai
益莱储中国41 分钟前
前沿科技竞速:脑机接口、AI芯片与半导体生态上的新突破
人工智能·科技
知孤云出岫42 分钟前
IFA 2025 展望:人工智能无处不在,计算、游戏与家庭科技的未来
人工智能·科技·游戏
聚客AI1 小时前
95%企业AI失败?揭秘LangGraph+OceanBase融合数据层如何破局!
大数据·图像处理·人工智能·机器学习·语言模型·自然语言处理·数据分析
Christo31 小时前
TSMC-1987《Convergence Theory for Fuzzy c-Means: Counterexamples and Repairs》
人工智能·算法·机器学习·kmeans
LeonDL1681 小时前
【3D通用视觉框架】基于Qt5开发的3D视觉框架软件,纯底层,全套源码,开箱即用
人工智能·通用3d视觉框架软件·机器视觉软件框架·3d视觉测量与检测软件系统·qt/c++·3d测高、平面度等测量工具·测高、断差、共面度、轮廓等功能
聚客AI1 小时前
✅5大实战技巧:优化RAG召回质量,避免“召回不足”灾难!
人工智能·llm·增强现实