python爬虫教程：selenium常用API用法和浏览器控制

Cachel wood2023-11-22 22:00

文章目录

selenium api

selenium新版本(4.8.2)很多函数，包括元素定位、很多API方法均发生变化，本文记录以selenium4.8.2为准。

webdriver常用API
方法 描述
get(String url) 访问目标url地址，打开网页
current_url 获取当前页面url地址
title 获取页面标题
page_source 获取页面源代码
close() 关闭浏览器当前打开的窗口
quit() 关闭浏览器所有的窗口
find_element(by) 查找单个元素
find_elements(by) 查找元素列表，返回一个集合
window_handles 获取当前窗口句柄
python 复制代码 
from selenium import webdriver

browser = webdriver.Chrome()
browser.get('https://www.baidu.com') 

# 网页标题
print(browser.title)
# 当前网址
print(browser.current_url)
# 浏览器名称
print(browser.name)
# cookie
print(browser.get_cookie)
# 当前窗口句柄
print(browser.window_handles)
# 网页源码
print(browser.page_source)
python 复制代码 
百度一下，你就知道
https://www.baidu.com/
chrome
<bound method WebDriver.get_cookie of <selenium.webdriver.chrome.webdriver.WebDriver (session="381859b7996708e36891008a494c290e")>>
['01CD0AD3FA1C37889F3BE65C61160946']
<html style="--bubble-width: 140px; --bubble-padding-left: 164px;"><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"><meta content="always" name="referrer"><meta name="theme-color" content="#ffffff"><meta name="description" content="全球领先的中文搜索引擎、致力于让网民更便捷地获取信息，找到所求。百度超过千亿的中文网页数据库，可以瞬间找到相关的搜索结果。"><link rel="shortcut icon" href="https://www.baidu.com/favicon.ico" type="image/x-icon"><link rel="search" type="application/opensearchdescription+xml" href="/content-search.xml" title="百度搜索"><link rel="icon" sizes="any" mask="" href="https://www.baidu.com/favicon.ico"><link rel="dns-prefetch" href="//dss0.bdstatic.com"><link rel="dns-prefetch" href="//dss1.bdstatic.com"><link rel="dns-prefetch" href="//ss1.bdstatic.com"><link rel="dns-prefetch" href="//sp0.baidu.com"><link rel="dns-prefetch" href="//sp1.baidu.com"><link rel="dns-prefetch" href="//sp2.baidu.com"><link rel="dns-prefetch" href="//pss.bdstatic.com"><link rel="apple-touch-icon-precomposed" href="https://psstatic.cdn.bcebos.com/video/wiseindex/aa6eef91f8b5b1a33b454c401_1660835115000.png"><title>百度一下，你就知道</title><style index="newi" type="text/css">#form .bdsug{top:39px}.bdsug{display:none;position:absolute;width:535px;background:#fff;border:1px solid #ccc!important;_overflow:hidden;box-shadow:1px 1px 3px #ededed;-webkit-box-shadow:1px 1px 3px #ededed;-moz-box-shadow:1px 1px 3px #ededed;-o-box-shadow:1px 1px 3px #ededed}.bdsug li{width:519px;color:#000;font:14px arial;line-height:25px;padding:0 8px;position:relative;cursor:default}.bdsug li.bdsug-s{background:#f0f0f0}.bdsug-store span,.bdsug-store b{color:#7A77C8}.bdsug-store-del{font-size:12px;color:#666;text-decoration:underline;position:absolute;right:8px;top:0;cursor:pointer;display:none}.bdsug-s .bdsug-store-del{display:inline-block}.bdsug-ala{display:inline-block;border-bottom:1px solid #e6e6e6}.bdsug-ala h3{line-height:14px;background:url(//www.baidu.com/img/sug_bd.png?v=09816787.png) no-repeat left center;margin:6px 0 4px;font-size:12px;font-weight:400;color:#7B7B7B;padding-left:20px}.bdsug-ala p{font-size:14px;font-weight:700;padding-left:20px}#m .bdsug .bdsug-direct p{color:#00c;font-weight:700;line-height:34px;padding:0 8px;margin-top:0;cursor:pointer;white-space:nowrap;overflow:hidden}#m .bdsug .bdsug-direct p img{width:16px;height:16px;margin:7px 6px 9px 0;vertical-align:middle}#m .bdsug .bdsug-direct p span{margin-left:8px}#form .bdsug .bdsug-direct{width:auto;padding:0;border-bottom:1px solid #f1f1f1}#form .bdsug .bdsug-direct p i{font-size:12px;line-height:100%;font-style:normal;font-weight:400;color:#fff;background-color:#2b99ff;display:inline;text-align:center;padding:1px 5px;*padding:2px 5px 0;margin-left:8px;overflow:hidden}.bdsug .bdsug-pcDirect{color:#000;font-size:14px;line-height:30px;height:30px;background-color:#f8f8f8}.bdsug .bdsug-pc-direct-tip{position:absolute;right:15px;top:8px;width:55px;height:15px;display:block;background:url(https://pss.bdstatic.com/r/www/cache/static/protocol/https/global/img/pc_direct_42d6311.png) no-repeat 0 0}.bdsug li.bdsug-pcDirect-s{background-color:#f0f0f0}.bdsug .bdsug-pcDirect-is{color:#000;font-size:14px;line-height:22px;background-color:#f5f5f5}.bdsug .bdsug-pc-direct-tip-is{position:absolute;right:15px;top:3px;width:55px;height:15px;display:block;background:url(https://pss.bdstatic.com/r/www/cache/static/protocol/https/global/img/pc_direct_42d6311.png) no-repeat 0 0}.bdsug li.bdsug-pcDirect-is-s{background-color:#f0f0f0}.bdsug .bdsug-pcDirect-s .bdsug-pc-direct-tip,.bdsug .bdsug-pcDirect-is-s .bdsug-pc-direct-tip-is{background-position:0 -15px}.bdsug .bdsug-newicon{color:#929292;opacity:.7;font-size:12px;display:inline-block;line-height:22px;letter-spacing:2px}.bdsug .bdsug-s .bdsug-newicon{opacity:1}.bdsug .bdsug-newicon i{letter-spacing:0;font-style:normal}.bdsug .bdsug-feedback-wrap{display:none}.toggle-underline{text-decoration:none}.toggle-underline:hover{text-decoration:underline}.bdpfmenu,.usermenu{border:1px solid #d1d1d1;position:absolute;width:105px;top:36px;z-index:302;box-shadow:1px 1px 5px #d1d1d1;-webkit-box-shadow:1px 1px 5px #d1d1d1;-moz-box-shadow:1px 1px 5px #d1d1d1;-o-box-shadow:1px 1px 5px #d1d1d1}.bdpfmenu{font-size:12px;background-color:#fff}.bdpfmenu a,.usermenu a{display:block;text-align:left;margin:0!important;padding:0 9px;line-height:26px;text-decoration:none}.briiconsbg{background-repeat:no-repeat;background-size:300px 18px;background-image:url(https://pss.bdstatic.com/r/www/cache/static/protocol/https/home/img/icons_0c37e9b.png);background-image:url(https://pss.bdstatic.com/r/www/cache/static/protocol/https/home/img/icons_809ae65.gif)\9}.bdpfmenu a:link,.bdpfmenu a:visited,#u .usermenu a:link,#u .usermenu a:visited{background:#fff;color:#333}.bdpfmenu a:hover,.bdpfmenu a:active,#u .usermenu a:hover,#u .usermenu a:active{background:#38f;text-decoration:none;color:#fff}.bdpfmenu{width:70px}#wrapper .bdnuarrow{width:0;height:0;font-size:0;line-height:0;display:block;position:absolute;top:-10px;left:50%;margin-left:-5px}#wrapper .bdnuarrow em,#wrapper .bdnuarrow i{width:0;height:0;font-size:0;line-height:0;display:block;position:absolute;border:5px solid transparent;border-style:dashed dashed solid}#wrapper .bdnuarrow em{border-bottom-color:#d8d8d8;top:-1px}#wrapper .bdnuarrow i{border-bottom-color:#fff;top:0}#gxszHead .prefpanelclose{cursor:pointer;width:16px;height:16px;float:right;margin-top:7px;background-position:-248px 0}#gxszHead .prefpanelclose:hover{background-position:-264px 0}.s_ipt::-webkit-input-placeholder{padding-left:3px;color:#aaa;font-size:13px}.s_ipt::-moz-placeholder{padding-left:3px;color:#aaa;font-size:13px}.s_ipt:-ms-input-placeholder{padding-left:3px;color:#aaa;font-size:13px}.s_ipt::placeholder{padding-left:3px;color:#aaa;font-size:13px}.kw-placeholder{position:absolute;top:0;left:0;color:#aaa;font-size:13px;height:40px;line-height:40px;padding-left:10px;max-width:360px;z-index:99;pointer-events:none}.kw-placeholder.kw-placehlder-high{height:40px;line-height:40px}.kw-placeholder.placeholders-hidden{visibility:hidden}#head_wrapper #form .bdsug-new{width:544px;top:35px;border-radius:0 0 10px 10px;border:2px solid #4E6EF2!important;border-top:0!important;box-shadow:none;font-family:Arial,sans-serif;z-index:1}#head_wrapper.sam_head_wrapper2 #form .bdsug-new{width:545px;z-index:1;border:1px solid #4E6EF2!important;border-top:0!important}#head_wrapper #form .bdsug-new ul{margin:7px 14px 0;padding:8px 0 7px;background:0 0;border-top:2px solid #f5f5f6}#head_wrapper #form .bdsug-new ul li{width:auto;padding-left:14px;margin-left:-14px;margin-right:-14px;color:#626675;line-height:28px;background:0 0;font-family:Arial,sans-serif}#head_wrapper #form .bdsug-new ul li .sug-search-icon,#head_wrapper #form .bdsug-new ul li .sug-history-icon{margin-right:4px;color:#222}#head_wrapper #form .bdsug-new ul li span{color:#626675}#head_wrapper #form .bdsug-new ul li b{font-weight:400;color:#222}#head_wrapper #form .bdsug-new .bdsug-store-del{font-size:13px;text-decoration:none;color:#9195A3;right:16px}#head_wrapper #form .bdsug-new .bdsug-store-del:hover{color:#315EFB;cursor:pointer}#head_wrapper #form .bdsug-new ul li:hover,#head_wrapper #form .bdsug-new ul li:hover span,#head_wrapper #form .bdsug-new ul li:hover b{cursor:pointer}.wrapper_new #form .bdsug-new .bdsug-s{background-color:#F5F5F6!important}.wrapper_new #form .sam_search .bdsug-new .bdsug-s{background-color:#F1F3FD!important}#head_wrapper #form .sam_search .bdsug-new .bdsug-s{background-color:#F1F3FD!important}#head .s-down #form .bdsug-new{top:32px}.s-skin-hasbg #head_wrapper #form .bdsug-new{border-color:#4569ff!important;border-top:0!important}.s-skin-hasbg #head_wrapper.s-down #form .bdsug-new{border-color:#4e6ef2!important;border-top:0!important}.s-skin-hasbg #head_wrapper.s-down #form.sam_search .bdsug-new{border-color:rgba(0,0,0,.05)!important;border-top:1px solid rgba(0,0,0,.05)!important;top:54px!important}#head_wrapper #form .bdsug-new .bdsug-s,#head_wrapper #form .bdsug-new .bdsug-s span,#head_wrapper #form .bdsug-new .bdsug-s b{color:#315EFB}#head_wrapper #form .bdsug-new>div span:hover,#head_wrapper #form .bdsug-new>div a:hover{color:#315EFB!important}#head_wrapper #form #kw.new-ipt-focus{border-color:#4e6ef2}#head_wrapper #form .bdsug-new ul li{}#head_wrapper #form .bdsug-new ul li .sug-hot-orange,#head_wrapper #form .bdsug-new ul li .sug-hot-grey,#head_wrapper #form .bdsug-new ul li .sug-hot-blue{display:inline-block;width:12px;height:12px;font-size:12px;line-height:12px;padding:2px;text-align:center;font-weight:500;margin-left:6px;vertical-align:text-bottom}#head_wrapper #form .bdsug-new ul li .sug-hot-orange{display:inline-block;color:#fff;background:#F60;border-radius:4px}#head_wrapper #form .bdsug-new ul li .sug-new-tag{text-align:center;margin-left:6px;box-sizing:border-box;font-size:12px;line-height:14px;padding:1px 4px;font-weight:500}#head_wrapper #form .bdsug-new ul li .sug-new-tag-grey{color:#858585;border:1px solid rgba(133,133,133,.5);border-radius:4px}#head_wrapper #form .bdsug-new ul li .sug-new-tag-blue{color:#36F;border:1px solid rgba(51,102,255,.4);border-radius:4px}#head_wrapper #form .bdsug-new ul li .sug-new-tag-orange{color:#F33;border:1px solid rgba(255,51,51,.4);border-radius:4px}#head_wrapper #form .bdsug-new ul li .sug-tag-text{display:inline-block;box-sizing:border-box;margin-left:6px;height:18px;padding:2px 4px;font-size:12px;line-height:12px;font-weight:500;text-align:center;border-radius:4px;vertical-align:middle;border-width:1px;border-style:solid}#head_wrapper #form .bdsug-new ul li .sug-tag-img{height:18px;max-width:80px;vertical-align:middle;margin-left:6px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap{display:block;height:40px;padding:7px 0;color:#222}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap:visited{color:#222}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .left-img-wrap{position:relative}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .left-img-wrap::before{content:"";width:38px;height:38px;position:absolute;top:0;right:0;left:0;bottom:0;border:1px solid rgba(0,0,0,.06);border-radius:9px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .left-img{box-sizing:border-box;float:left;width:40px;height:40px;border:1px solid rgba(0,0,0,.06);border-radius:9px;margin-right:6px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .content{float:left}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .header{margin-bottom:6px;line-height:18px;height:18px;vertical-align:middle}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .name{float:left;font-size:18px;color:#222;font-weight:500}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .tag-img{width:18px;margin-left:4px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .more{line-height:14px;width:240px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .brief,#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .info{font-size:14px;color:#222;font-weight:400}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .brief{margin-right:6px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .right-btn{float:right;margin-top:5px;margin-right:6px;width:60px;height:24px;text-align:center;font-size:14px;color:#36F;line-height:24px;background-image:linear-gradient(111deg,#e8f7ff 0,#edf0ff 100%);border-radius:12px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .right-btn:hover{background:#315efb;color:#fff}#head_wrapper #form .bdsug-new ul li:hover .direct-sug-wrap .name{color:#315EFB}#head_wrapper #form .bdsug-new ul li:hover .direct-sug-wrap .brief{color:#222}#head_wrapper.s-down #form .sam-bdsug.bdsug-new{top:52px}#head_wrapper #form .sam-bdsug.bdsug-new{width:100%;box-shadow:0 4px 4px 0 rgba(0,0,0,.1);border:1px solid rgba(0,0,0,.05)!important;border-radius:12px;top:56px}#head_wrapper #form .sam-bdsug.bdsug-new ul{border:0;padding:0 0 7px}#head_wrapper #form .sam-bdsug.bdsug-new ul li{line-height:32px}#head_wrapper #form .sam-bdsug.bdsug-new ul .bdsug-s{background-color:#F1F3FD!important}#head_wrapper #form .sam-bdsug.bdsug-new .bdsug-store-del{right:15px}.sam_search .sam_search_rec,.sam_search .sam_search_soutu{z-index:1;display:none;position:absolute;top:50%;margin-top:-12px;font-size:24px;color:#4E6EF2;height:24px;line-height:24px;width:24px;cursor:pointer;-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0);transition:transform .3s ease}.sam_search .sam_search_rec{right:54px}.sam_search .sam_search_soutu{right:14px}.sam_search .sam_search_rec:hover,.sam_search .sam_search_soutu:hover{color:#1D4FFF!important;transform:scale(1.08,1.08)}.sam_search .sam_search_rec_hover,.sam_search .sam_search_soutu_hover{background:#626675;border-radius:8px;height:32px;width:76px;text-align:center;line-height:32px;font-size:13px;color:#FFF;position:absolute;z-index:2;top:50px}.sam_search .sam_search_rec_hover:before,.sam_search .sam_search_soutu_hover:before{content:'';border:4px solid transparent;border-bottom:4px solid #626675;position:absolute;left:50%;top:-8px;margin-left:-4px}.sam_search .sam_search_rec_hover{right:29px}.sam_search .sam_search_soutu_hover{display:none;right:-12px}</style><style type="text/css" index="superbase">blockquote,body,button,dd,dl,dt,fieldset,form,h1,h2,h3,h4,h5,h6,hr,input,legend,li,ol,p,pre,td,textarea,th,ul{margin:0;padding:0}
html{color:#000;overflow-y:scroll;overflow:-moz-scrollbars}
body,button,input,select,textarea{font-size:12px;font-family:Arial,sans-serif}
h1,h2,h3,h4,h5,h6{font-size:100%}
em{font-style:normal}
small{font-size:12px}
ol,ul{list-style:none}
a{text-decoration:none}
a:hover{text-decoration:underline}
legend{color:#000}
fieldset,img{border:0}
button,input,select,textarea{font-size:100%}
table{border-collapse:collapse;border-spacing:0}
img{-ms-interpolation-mode:bicubic}
textarea{resize:vertical}
.left{float:left}
.right{float:right}
.overflow{overflow:hidden}
.hide{display:none}
.block{display:block}

与之前的函数名称相比，函数前面的get都被去掉，然后函数的命名方法也从驼峰命名法 变为python常用的下划线命名法。

webelement常用API

通过 WebElement 实现与网站页面上元素的交互，这些元素包含文本框、文本域、按钮、单选框、div等。

方法 描述
click() 对元素进行点击
clear() 清空内容（如文本框内容）
send_keys() 写入内容与模拟按键操作
is_displayed() 元素是否可见（true:可见）
is_enabled 元素是否启用
is_selected() 元素是否已选择
tag_name 获取元素标签名
get_attritube() 获取元素对应的属性值
text 获取元素文本值（可见状态下才能获取到）
submit() 表单提交
  • 代码示例
python 复制代码 
from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()

url = 'https://www.51bmj.cn/policyNotice/%E5%B9%BF%E4%B8%9C%E7%9C%81/%E5%B9%BF%E5%B7%9E%E5%B8%82/1/%E5%88%B6%E9%80%A0'

driver.get(url)

xpath = '//*[@id="__layout"]/div/div/div[2]/div[3]/div[2]/div[1]/div[1]/a'
print(xpath)

element = driver.find_element(by = By.XPATH, value=xpath)


# is_displayed 是否可见
print(element.is_displayed())
# is_enabled 是否启用
print(element.is_enabled())
# is_selected 是否选择
print(element.is_selected())
# name 元素标签名
print(element.tag_name)
# get_attritube 获取元素属性名
print(element.get_attribute('href'))
# text 获取元素文本值
print(element.text)
python 复制代码 
//*[@id="__layout"]/div/div/div[2]/div[3]/div[2]/div[1]/div[1]/a
False
True
False
a
https://51bmj.cn/inform/3247500
控制浏览器
  • 代码示例
python 复制代码 
from selenium import webdriver
import time  

browser = webdriver.Chrome()

# 设置浏览器全屏
browser.maximize_window()   
browser.get('https://www.baidu.com')  
time.sleep(2)

# 打开淘宝页面
browser.get('https://www.bilibili.com/')  
time.sleep(2)

# 后退到百度页面
browser.back()  
time.sleep(2)

# 前进的淘宝页面
browser.forward() 
time.sleep(2)

# 关闭浏览器
browser.close()
python 复制代码 
from selenium import webdriver
import time  

driver = webdriver.Chrome()

url = 'https://bootapi.51bmj.cn/bmj-api/api/es/Policy/queryPolicyList.json?domain=undefined'

# 设置浏览器大小：全屏
driver.maximize_window()   
driver.get('https://www.51bmj.cn/policyNotice/%E5%B9%BF%E4%B8%9C%E7%9C%81/%E5%B9%BF%E5%B7%9E%E5%B8%82/1/%E5%88%B6%E9%80%A0')  
#time.sleep(2)

# 设置分辨率 500*500
driver.set_window_size(500,500)  

# 关闭浏览器
#driver.close()
相关推荐
龙哥说跨境4 分钟前
如何利用指纹浏览器爬虫绕过Cloudflare的防护？
服务器·网络·python·网络爬虫
科技探秘人4 分钟前
Chrome与火狐哪个浏览器的隐私追踪功能更好
前端·chrome
科技探秘人5 分钟前
Chrome与傲游浏览器性能与功能的深度对比
前端·chrome
JerryXZR10 分钟前
前端开发中ES6的技术细节二
前端·javascript·es6
七星静香12 分钟前
laravel chunkById 分块查询 使用时的问题
java·前端·laravel
q24985969315 分钟前
前端预览word、excel、ppt
前端·word·excel
小白学大数据20 分钟前
正则表达式在Kotlin中的应用：提取图片链接
开发语言·python·selenium·正则表达式·kotlin
小华同学ai20 分钟前
wflow-web：开源啦 ，高仿钉钉、飞书、企业微信的审批流程设计器，轻松打造属于你的工作流设计器
前端·钉钉·飞书
flashman91121 分钟前
python在word中插入图片
python·microsoft·自动化·word
菜鸟的人工智能之路24 分钟前
桑基图在医学数据分析中的更复杂应用示例
python·数据分析·健康医疗
热门推荐
01【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总02(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明03组基轨迹建模 GBTM的介绍与实现（Stata 或 R）04【AIGC】重塑未来的科技巨轮05全面解析：构建基于深度学习的安全帽检测系统（UI界面+YOLO代码+数据集）06【经验分享】Ubuntu22.04安装微信(linux官方版)07基于YOLOv10深度学习的CT扫描图像肾结石智能检测系统【python源码+Pyqt5界面+数据集+训练代码】深度学习实战、目标检测08Ubuntu 20.04使用Livox mid 360 测试 FAST_LIO09RAG 实践- Ollama+RagFlow 部署本地知识库10红米手机使用google play