1:爬虫爬网页内容 + AI自动总结
场景:爬文章/新闻/文案,AI一键精简、提炼观点、写摘要
python
import requests
from bs4 import BeautifulSoup
from openai import OpenAI
1. 简单爬虫获取网页正文
url = "https://www.example.com/article.html" # 替换目标网址
headers = {"User-Agent": "Mozilla/5.0"}
res = requests.get(url, headers=headers)
soup = BeautifulSoup(res.text, "html.parser")
提取正文(根据网页改标签)
content = soup.get_text(strip=True)[:2000] # 限制长度防超限
2. 调用AI总结(可用GPT/豆包/通义千问任意大模型)
client = OpenAI(
api_key="你的API密钥",
base_url="https://api.doubao.com/v1" # 豆包/中转接口,免费好用
)
prompt = f"""
请精简总结下面这段内容,通俗易懂:
{content}
"""
response = client.chat.completions.create(
model="doubao-pro",
messages=[{"role":"user", "content":prompt}]
)
输出结果
print("AI总结:")
print(response.choices[0].message.content)
用法2:爬虫采集榜单/文案 + AI改写伪原创
适合情感博主、自媒体:爬热门文案,AI改写洗稿、换风格、治愈/清醒文案二创
python
import requests
from bs4 import BeautifulSoup
from openai import OpenAI
爬取热门短句/情感文案
url = "文案网站地址"
html = requests.get(url).text
soup = BeautifulSoup(html, "html.parser")
text_list = [p.get_text() for p in soup.find_all("p")]
AI批量改写
client = OpenAI(api_key="你的密钥", base_url="对应接口")
for text in text_list[:5]:
if len(text) > 5:
res = client.chat.completions.create(
model="doubao-pro",
messages=[{"role":"user", "content":f"改成高级治愈风情感文案:{text}"}]
)
print("原文案:", text)
print("AI改写:", res.choices[0].message.content)
print("-"*50)
用法3:爬虫抓数据 + AI分类/情绪分析
结合你做情感博主刚需:
爬评论、私信、网友情绪文案,AI自动判断:焦虑/委屈/恋爱脑/回避型情绪
python
from openai import OpenAI
假设爬虫爬到的用户留言
comment_list = [
"他总是忽冷忽热,我每天内耗睡不着",
"我习惯性讨好别人,害怕被抛弃",
"不想谈恋爱,只想一个人独处"
]
client = OpenAI(api_key="你的密钥", base_url="接口地址")
for msg in comment_list:
prompt = f"判断这段文字的情绪和依恋类型:{msg}"
ans = client.chat.completions.create(
model="doubao-pro",
messages=[{"role":"user", "content":prompt}]
)
print(msg, "→", ans.choices[0].message.content)