03、爬取资料---但是失败,仅作为记录

1、找网址

进入直播间,里面的用户被设置不对外查看。

如图,找url

2、伪装

user-agent 用户代理

cookie 用户登录后保留的信息

登录信息:找cookie

浏览器信息:找user-agent

user-agent 用户代理

cookie 用户登录后保留的信息

代码:

java 复制代码
# 导入请求模块
import re
import requests
# 用户列表的链接
url = ''
# 伪装程序,相当于有一个假身份证用于访问  浏览器的信息:user-agent   登录信息:cookie
headers = {'user-agent': '', 'cookie': ''}
# 请求访问网站,得到响应
res = requests.get(url, headers=headers)
# 转化为json数据
js = res.json()
print(js)
# 在 js 的 data 里面的ranks里面
userList = js['data']['ranks']
print(userList)
# 遍历功能
for user in userList:
    # 用户的 user 里面的 pay_grade 里面的 level
    userPay = user['user']['pay_grade']['level']
    userHomePage = '' + user['user']['sec_uid']
    res = requests.get(userHomePage, headers=headers)
    # 文本数据
    text = res.text
    nickName = re.findall('(.*?)的主页', text)
    douyinNum = re.findall('抖音号是(.*?)', text)
    print(nickName, douyinNum, userPay, userHomePage)

错误原因:

显示访问太多次了,这样容易被封ip,就没尝试了。

java 复制代码
urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:1131)

The above exception was the direct cause of the following exception:

urllib3.exceptions.ProxyError: ('Unable to connect to proxy', SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)')))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "E:\install\python3.8\install\lib\site-packages\requests\adapters.py", line 486, in send
    resp = conn.urlopen(
  File "E:\install\python3.8\install\lib\site-packages\urllib3\connectionpool.py", line 845, in urlopen
    retries = retries.increment(
  File "E:\install\python3.8\install\lib\site-packages\urllib3\util\retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='live.douyin.com', port=443): Max retries exceeded with url: /webcast/ranklist/audience/?aid=6383&app_name=douyin_web&live_id=1&device_platform=web&language=zh-CN&enter_from=web_search&cookie_enabled=true&screen_width=1313&screen_height=821&browser_language=zh-CN&browser_platform=Win32&browser_name=Chrome&browser_version=117.0.0.0&webcast_sdk_version=2450&room_id=7291676063342643994&anchor_id=2581300722537251&sec_anchor_id=MS4wLjABAAAAkY_WtKOYqH-5zSWzQSFe9tXTCirrA8sLJBKNrspoORbkdrMBXqAnyjV2f75mX4lk&rank_type=30&msToken=MeH2AD_j6RbHpbqPWUyyoVMzuw63sALL5xx4Y13yp6nnq8D6sknHWmfpY9j_YujnW01p_EmbfcfuNIZw-Py6bwV8Oz1j3LXX1dn8WOsZ1EEwaQVPFop9rYhPenbTrOCy&X-Bogus=DFSzswVENasANeogtYIDvKXAIQ-X&_signature=_02B4Z6wo000012ZO15QAAIDCBUQX.a0biutmTtMAALyult0a6-ftHDP.4JgbIybvVd-fj4v.dHUHYgVoxsNh8DR7dFJG7wgVBodOHrVp-kXOMztMjLuxQ1QdaFLM5hvBFWOWIIff9gOunxqM66 (Caused by ProxyError('Unable to connect to proxy', SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)'))))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\install\python3.8\install\lib\site-packages\requests\api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "E:\install\python3.8\install\lib\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "E:\install\python3.8\install\lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "E:\install\python3.8\install\lib\site-packages\requests\adapters.py", line 513, in send
    raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='live.douyin.com', port=443): Max retries exceeded with url: /webcast/ranklist/audience/?aid=6383&app_name=douyin_web&live_id=1&device_platform=web&language=zh-CN&enter_from=web_search&cookie_enabled=true&screen_width=1313&screen_height=821&browser_language=zh-CN&browser_platform=Win32&browser_name=Chrome&browser_version=117.0.0.0&webcast_sdk_version=2450&room_id=7291676063342643994&anchor_id=2581300722537251&sec_anchor_id=MS4wLjABAAAAkY_WtKOYqH-5zSWzQSFe9tXTCirrA8sLJBKNrspoORbkdrMBXqAnyjV2f75mX4lk&rank_type=30&msToken=MeH2AD_j6RbHpbqPWUyyoVMzuw63sALL5xx4Y13yp6nnq8D6sknHWmfpY9j_YujnW01p_EmbfcfuNIZw-Py6bwV8Oz1j3LXX1dn8WOsZ1EEwaQVPFop9rYhPenbTrOCy&X-Bogus=DFSzswVENasANeogtYIDvKXAIQ-X&_signature=_02B4Z6wo000012ZO15QAAIDCBUQX.a0biutmTtMAALyult0a6-ftHDP.4JgbIybvVd-fj4v.dHUHYgVoxsNh8DR7dFJG7wgVBodOHrVp-kXOMztMjLuxQ1QdaFLM5hvBFWOWIIff9gOunxqM66 (Caused by ProxyError('Unable to connect to proxy', SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)'))))

Process finished with exit code 1
相关推荐
The Future is mine25 分钟前
Python计算经纬度两点之间距离
开发语言·python
Enti7c25 分钟前
HTML5和CSS3的一些特性
开发语言·css3
九月镇灵将27 分钟前
GitPython库快速应用入门
git·python·gitpython
爱吃巧克力的程序媛33 分钟前
在 Qt 创建项目时,Qt Quick Application (Compat) 和 Qt Quick Application
开发语言·qt
兔子的洋葱圈1 小时前
【django】1-2 django项目的请求处理流程(详细)
后端·python·django
独好紫罗兰1 小时前
洛谷题单3-P5719 【深基4.例3】分类平均-python-流程图重构
开发语言·python·算法
27669582921 小时前
美团民宿 mtgsig 小程序 mtgsig1.2 分析
java·python·小程序·美团·mtgsig·mtgsig1.2·美团民宿
橘子在努力1 小时前
【橘子大模型】关于PromptTemplate
python·ai·llama
篝火悟者1 小时前
自学-C语言-基础-数组、函数、指针、结构体和共同体、文件
c语言·开发语言
SheepMeMe2 小时前
蓝桥杯2024省赛PythonB组——日期问题
python·算法·蓝桥杯