爬取去哪网旅游攻略信息

代码展现:

python 复制代码
import requests
import parsel
import csv
import time
f = open('旅游去哪攻略.csv',mode='a',encoding='utf-8',newline='')
csv_writer = csv.writer(f)
csv_writer.writerow(['标题','浏览量','日期','天数','人物','人均价格','玩法'])
for page in range(1,5):
    url = f'https://travel.qunar.com/travelbook/list.htm?page={page}&order=hot_heat'
    headers = {
            'Cookie': 'QN1=0000f180306c5a8fd1604d35; QN300=s%3Dbing; QN99=4383; qunar-assist{%22version%22:%2220211215173359.925%22%2C%22show%22:false%2C%22audio%22:false%2C%22speed%22:%22middle%22%2C%22zomm%22:1%2C%22cursor%22:false%2C%22pointer%22:false%2C%22bigtext%22:false%2C%22overead%22:false%2C%22readscreen%22:false%2C%22theme%22:%22default%22}; QunarGlobal=10.68.76.77_2234d452_18ce6a55a1c_3f52|1704686636774; QN205=s%3Dbing; QN277=s%3Dbing; csrfToken=BlDdq9XhjNUjJAscPT4v8cy32cW9i8oB; QN601=8db3942458d11f928c6b2dd8c1aa2279; QN163=0; _i=ueHd8ZkXXXVXomXy-xZtrutbuUoX; _vi=-FhfAqdNLwBmA7eEf04ekxQMaabajPL5jFd9ieQfDCRjLTXWWK7LdR_IvNWGcF29uIil1Zdss74CLcjh9nkEXxxColSCvCaRdcM203OwfiovKYZg9z51kh2199uQrg1Tzx1FNh2Gufhwxf-x7L65h_yAbVYUi9bptgoqjAor959u; QN269=F267AAA1ADDA11EE8FF326DBBC301320; QN48=3919f823-181d-40ef-9953-1539bcb491a3; fid=d2b33715-a247-4127-91d0-9b501fbc4863; Hm_lvt_c56a2b5278263aa647778d304009eafc=1704686638; viewbook=7824809|7826165|7826165|7825949|7825196; JSESSIONID=A4FAADF972AB80EA863A36B7828EE793; ariaDefaultTheme=undefined; Hm_lpvt_c56a2b5278263aa647778d304009eafc=1704686730; QN271=34057a3a-f730-4fdc-92f1-a9672688d3f6; QN267=0896993248d1fd56b2',
            'Referer': 'https://travel.qunar.com/?from=header',
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
        }
    response = requests.get(url=url,headers=headers)
    response.encoding = response.apparent_encoding
    html_data = response.text
    selector_data = parsel.Selector(html_data)
    link_list = selector_data.css('.b_strategy_list .list_item .tit a::attr(href)').getall()
    origin_title = selector_data.css('.b_strategy_list .list_item .tit a::text').getall()
    time.sleep(5)
    for link in link_list:
        href = 'https://travel.qunar.com/travelbook/note'+link.replace('/youji','')
        new_html_data = requests.get(url=href,headers=headers)
        new_html_data.encoding = new_html_data.apparent_encoding
        data = parsel.Selector(new_html_data.text)
        title = data.css('.user_info #booktitle::text').get()
        view_count = data.css('.e_line2 .clrfix .date .view_count::text').get()
        date = data.css('#js_mainleft > div.b_foreword > ul > li.f_item.when > p > span.data::text').get()
        days = data.css('#js_mainleft > div.b_foreword > ul > li.f_item.howlong > p > span.data::text').get()
        character = data.css('#js_mainleft > div.b_foreword > ul > li.f_item.who > p > span.data::text').get()
        price = data.css('#js_mainleft > div.b_foreword > ul > li.f_item.howmuch > p > span.data::text').get()
        play_list = data.css('#js_mainleft > div.b_foreword > ul > li.f_item.how > p > span.data >span::text').getall()
        play = ' '.join(play_list)
        print(title, view_count, date, days, character, price, play)
        csv_writer.writerow([title,view_count,date,days,character,price,play])

结果展现:

总结:

1.进一步熟悉了用css选择器去解析代码,这是两个静态网页信息提取

2.学到了如何快速提取列表中的全部元素,当作一个字符串的方法

c=['自驾游', '旅行团',' 暴走']

a=' '.join(c)

print(a)

结果为

自驾游 旅行团 暴走

相关推荐
我的ID配享太庙呀2 天前
基于 CentOS 7 的 LVS+DR+Web+NFS 旅游攻略分享平台部署
centos·旅游·lvs
说私域2 天前
基于定制开发开源AI智能名片与S2B2C商城小程序的旅游日志创新应用研究
人工智能·小程序·旅游
xinxunkandian3 天前
需求升级,创新破局!苏州金龙赋能旅游客运新生态
旅游
kjyxs4 天前
Tourism Management and Technology Economy,旅游管理与技术经济知网期刊
旅游
Deng9452013144 天前
基于Python的旅游数据可视化应用
python·numpy·pandas·旅游·数据可视化技术
java1234_小锋4 天前
基于Python的旅游推荐协同过滤算法系统(去哪儿网数据分析及可视化(Django+echarts))
python·数据分析·旅游
来自宇宙的曹先生5 天前
使用策略模式 + 自动注册机制来构建旅游点评系统的搜索模块
旅游·策略模式
paopaokaka_luck6 天前
基于Spring Boot+Vue的巴彦淖尔旅游网站(AI问答、腾讯地图API、WebSocket及时通讯、支付宝沙盒支付)
数据库·vue.js·spring boot·websocket·mysql·毕业设计·旅游
飞翔的佩奇6 天前
Java项目:基于SSM框架实现的旅游协会管理系统【ssm+B/S架构+源码+数据库+毕业论文】
java·数据库·mysql·毕业设计·ssm·旅游·jsp
qq_12498707531 个月前
基于Spring Boot+vue框架的武隆旅游网站设计与实现(源码+论文+调试+安装+售后)
vue.js·spring boot·毕业设计·旅游