运维开发宝典035-Shell脚本自动化运维编程Shell运维脚本实战1

云计算磊哥@2026-06-19 8:22

大家好，我是云计算磊哥，从业20年的IT老鸟。运维培训15年，总结了一套从入门到精通的全运维开发宝典手册。准备用300天时间写一套博文，手把手从安装软件讲起，从linux系统管理，shell脚本编程，mysql运维架构备份核心技术，Apache/nginx/tomcatWEB服务器管理，ansible自动化运维，redis集群哨兵，LVM/HAproxy/keepalived集群架构，rabbitMQ消息队列，docker&K8S集群资源管理，K8S自愈，K8S自动扩容，PYTHON编程，PYTHON自动化运维，从行业到产品，从过去到未来，从理论到操作，从视频到文档工具，100+篇系列文章一站式发布。从零基础入门到30k运维开发工程师岗位诸多就业问题。多方位全方面的给你讲清楚云计算这个行业该如何做。关注我。后续AI大模型开发课程更精彩。

实战项目

1.sed实现网络配置
shell 复制代码 
[root@xulei ~]# sed -i 's/\(IP=\)\S\S*/\1102.117.2.23/' filename
2.sed实现sshd配置
shell 复制代码 
[root@xulei ~]# cat testfile 
abc
dec
AllowUsers
[root@xulei ~]# sed '/^AllowUsers/s/$/,what you want add/' testfile 
AllowUsers,what you want add
china
unix
3.sed实现nginx配置
shell 复制代码 
#!/bin/bash
#function name
function_name=$1
#pool name
pool_name=$2
#pool corresponding ip list
pool_ip_lists=$3
#pool corresponding tomcat port
pool_tomcat_port=$4
#upstream file location
ngx_upstream_file=$5


#检测pool在nginx upstream配置文件中是否存在
function check_pool_in_ngx_upstream() {
    grep -E "${pool_name}[^-]" ${ngx_upstream_file} >> /dev/null
    if [ $? -eq 0 ];then
        echo -e "\033[36m the ${pool_name} in ${ngx_upstream_file}. \033[0m"
    else
        echo -e "\033[31m the ${pool_name} not in ${ngx_upstream_file}. \033[0m"
        exit 1
    fi
}

#显示pool在nginx upstream配置文件中对应内容
function show_pool_in_ngx_upstream() {
    pool_name_first_line=`egrep -n "${pool_name}[^-]" ${ngx_upstream_file} | cut -d ":" -f1`
    line_list=`grep -n "^}" ${ngx_upstream_file} | cut -d ":" -f1`
    pool_name_end_line=${pool_name_first_line}
    for line in ${line_list[*]};do
        if [ $line -gt ${pool_name_first_line} ];then
            pool_name_end_line=${line}
            break;
        fi
    done
    sed -n "${pool_name_first_line},${pool_name_end_line}p" ${ngx_upstream_file}
}

#增加pool进nginx upstream配置文件
function add_pool_to_upstream() {
    #pool对应ip地址列表,多个ip以逗号改开
    pool_ip=`awk 'BEGIN{list="'${pool_ip_lists}'";split(list,ip_list,",");for(ip in ip_list){print ip_list[ip];}}'`
    for ip in ${pool_ip[*]};do
        echo "add ${pool_name} ${ip} in ${ngx_upstream_file}"
        sed -i '/upstream '${pool_name}'[^-]*{/a\\tserver '${ip}':'${pool_tomcat_port}';' ${ngx_upstream_file}
    done
    echo -e "\033[31m ====添加完成如下:==== \033[0m"
}

#在nginx upstream配置文件删除pool对应的ip地址
function delete_ip_from_upstream() {
    pool_name_first_line=`egrep -n "${pool_name}[^-]" ${ngx_upstream_file} | cut -d ":" -f1`
    line_list=`grep -n "^}" ${ngx_upstream_file} | cut -d ":" -f1`
    pool_name_end_line=${pool_name_first_line}
    for line in ${line_list[*]};do
        if [ $line -gt ${pool_name_first_line} ];then
            pool_name_end_line=${line}
            break;
        fi
    done
    #获取pool对应配置行数
    line_count=`sed -n "${pool_name_first_line},${pool_name_end_line}p" ${ngx_upstream_file} | wc -l`
    #如果某个pool的配置行数等于3,则不能进行删除操作
    if [ ${line_count} -eq 3 ];then
        echo -e "\033[31m this is lowest configure. \033[0m"
    fi
    #删除pool_ip_lists中包含的ip地址
    for ((i=${pool_name_first_line};i<=${pool_name_end_line};i++));do
        pool_ip=`awk 'BEGIN{list="'${pool_ip_lists}'";split(list,ip_list,",");for(ip in ip_list){print ip_list[ip];}}'`
        line_context=`sed -n ''${i}'p' ${ngx_upstream_file}`
        for ip in ${pool_ip[*]};do
            echo "this line ${line_context} has ${ip}" | egrep "${ip}:${pool_tomcat_port}"
            if [ $? -eq 0 ];then
                #将包含删除ip的行,替换为空行
                sed -i ''${i}'s/.*'${ip}':'${pool_tomcat_port}'.*//ig' ${ngx_upstream_file}
                #sed -i ''${i}'d' ${ngx_upstream_file}
                echo -e "\033[36m delete ${pool_name} from ${ngx_upstream_file} where ip = ${ip}. \033[0m"
            fi
        done
    done
    #删除文件中的空行
    sed -i '/^$/d' ${ngx_upstream_file}
    echo -e "\033[31m ====删除完成如下:==== \033[0m"
}


#调用方法
if [ $# -eq 5 ];then
    case $1 in
        add)
            check_pool_in_ngx_upstream;
            show_pool_in_ngx_upstream;
            add_pool_to_upstream;
            show_pool_in_ngx_upstream;
            ;;
        delete)
            check_pool_in_ngx_upstream;
            show_pool_in_ngx_upstream;
            delete_ip_from_upstream;
            show_pool_in_ngx_upstream;
            ;;
        *)
            $"Usage: {sh change_nginx_upstream_conf.sh add chat-frontier-web 10.10.13.194 8080 /etc/nginx/conf.d/upstream.conf|sh change_nginx_upstream_conf.sh add chat-frontier-web 10.10.13.194 8080 /etc/nginx/conf.d/upstream.conf}"
            exit 3
    esac
else
    echo "variables count not eq 5.please check the usage."
fi
4.关闭本机selinux功能
shell 复制代码 
[root@xulei ~]# sed -i 's/XELINUX=/cXELINUX=disabled/' /etc/selinux/config
6.将固定文件的内容添加到nginx配置文件中
shell 复制代码 
[root@xulei ~]#sed -i 's/location/www.xulei.com/'

####7.zabbix_agentd.conf配置文件修改

shell 复制代码 
[root@xulei ~]# sed -i 's/Hostname=$clientname/Hostname=new/g' /etc/zabbix/zabbix.conf
8.awk 统计/etc/password 各种 shell 数量
shell 复制代码 
[root@xulei ~]# awk -F: '{shells[$NF]++} END{ for(i in shells){print i,shells[i]} }' /etc/passwd
9.awk 统计网站访问各种状态数量
shell 复制代码 
[root@xulei ~]# netstat -ant |grep :80 |awk '{access_stat[$NF]++} END{for(i in access_stat ){print i,access_stat[i]}}'
TIME_WAIT 1064
ESTABLISHED 1
LISTEN 1
10.awk 统计当前访问的每个 IP 的数量
shell 复制代码 
[root@xulei ~]# netstat -ant |grep :80 |awk -F: '{ip_count[$8]++} END{for(i in ip_count){print i,ip_count[i]} }' |sort
172.16.130.16 289
172.16.130.33 254
172.16.130.44 158
172.16.130.99 4
11.统计 Nginx 日志中某一天的 PV 量
shell 复制代码 
[root@xulei log]# grep '22/Mar/2026' cd.mobiletrain.org.log |wc -l
1646
12.获取获得内存使用情况
shell 复制代码 
[root@xulei ~]# free -m | awk '{print $2]'
13: 基于时间戳的备份程序课后作业
14: 批量主机软件部署程序课后作业
15. Web 日志访问量分析程序
15.1统计2026年9月5日 每个IP访问状态码数量（$status）
shell 复制代码 
[root@xulei log]#grep '05/Sep/2026' sz.mobiletrain.org.log | awk '{ip_code[$1 " "$9]++} END {for (i in ip_code){print i,ip_code[1]}}' | sort -k1 -rn | head -n10
222.112.25.173 404 1
222.112.25.173 304 11
184.123.12.231 304 4
123.12.123.122 200 2
123.15.23.111 304 10
16.编写系统初始化脚本
shell 复制代码 
#!/bin/bash
ip_master="10.18.44.126"

netcard=`ls /etc/sysconfig/network-scripts/ifcfg* | awk -F - 'NR==1{print $3}'`

ip_local=`ip a l dev $netcard | awk -F ' +|/' 'NR==3{print $3}'`
public_key='sh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCARFyaCnqLKkFKtbMMnynnLi1cIk+M1txLZDE/+YeEmvvYDrDzI32t/Icbw/q9CbxAr9k4hQq3aMTu7X5wK0wBwLS0VAxETy9iDeWIsMr6eYWMelpzqMibh8K96SbW9xia0/fW2Dsc+oaFrvOFFxAhoDmL/G2gtvtD1LbuX0Q32dswtFS0hSvhjKlEjsJvSyEH07FyHLCBUP8AK4qlfjOHJKJu7J4XzSRvRplpPhTOOfqzv6ob1LtPmCLihgjFl0bQdKyOapd+q3buflt9y4PfFWm4/lG/4FUfQHnrnV/uEG7NL/LeO4XX7FkCACQbjUJB3kGGw+PGRhZN7wf43jh admin@example.com'
#测试网络
network_test(){
	ping -c 1 www.baidu.com & > /dev/null
	[ $? -eq 0 ] && echo 网络测试成功 || exit 2
}
#配置centos yum
config_centos_yum(){
	sed -i 's/^mirror/#mirror/;s/^#baseurl/baseurl/' /etc/yum.repos.d/CentOS-Base.repo
	}
#配置epel
config_epel_yum(){
	yum install epel-release -y
	sed -i 's/^metalink/#metalink/;s/^#baseurl/baseurl/' /etc/yum.repos.d/epel.repo
	}
#安装vim lftp openssh-clients elinks git
soft_init(){
	yum install expect vim lftp openssh-clients elinks git -y
	}
#记录现有所有固定IP并给新机器配置固定ip地址
ip_add(){
	mv /etc/sysconfig/network-scripts/ifcfg-$netcard /etc/sysconfig/network-scripts/ifcfg-${netcard}.bak
	cat >> /etc/sysconfig/network-scripts/ifcfg-$netcard <<-eof
	TYPE="Ethernet"
	BOOTPROTO="static"
	NAME=$netcard
	DEVICE=$netcard
	ONBOOT="yes"
	IPADDR=$ip_local
	NETMASK=255.255.255.0
	DNS1=10.18.40.100
	GATEWAY=10.0.0.2
	eof
	systemctl stop network;systemctl start network
	}
#设置主机名称
hostname_set(){
	read -p "请设置这台机器的主机名称:" host_name
	hostnamectl set-hostname $host_name
		}
#给管理机器添加新机器的域名解析
hostname_add_to_master(){
	echo $ip_local $host_name >> hosts
	\cp hosts /etc/hosts

	/usr/bin/expect <<-eof
	set timeout 10
	spawn scp hosts 10.18.44.126:/etc/hosts
	expect {
	"yes/no" { send "yes\n";exp_continue }
	"password" { send "1\n" }
	}
	expect eof
	eof
	}
#关闭防火墙和selinux#开机关闭防火墙和selinux
se_firewall_close(){
	sed -i 's/^SELINUX=enforcing/SELINUX=disabled/' /etc/sysconfig/selinux
	systemctl disable firewalld
	setenforce 0
	systemctl stop firewalld
	}

#添加管理机器到新机器的公钥
public_key_add(){
	if [ -d /root/.ssh ];then
		:
	else
		mkdir /root/.ssh
	fi

	/usr/bin/expect <<-eof
	set timeout 10
	spawn ssh 10.0.0.189
	expect {
		"yes/no" { send "yes\n";exp_continue }
		"password" { send "1\n" }
		}
	expect "root@wing"
	send "/var/ftp/gitlocal/sysinit/public_key.sh\n"

	expect eof
	eof
	}

network_test
#config_centos_yum
#config_epel_yum
#soft_init
hostname_set
hostname_add_to_master
se_firewall_close
public_key_add
ip_add
17.LAMP 终级部署
shell 复制代码 
#!/bin/bash
#install edusoho
Mysql_Pass=123

#update
#yum update

#epel
yum -y install epel-release

#env mini vi
yum -y install bash-completion vim wget

systemctl restart firewalld
systemctl enable firewalld

firewall-cmd --permanent --add-service=http  #80
firewall-cmd --permanent --add-service=https #443
firewall-cmd --permanent --add-port=8080/tcp #8080
firewall-cmd --reload

#LAMP linux(centos7.4)+apache+mysql(mariadb)+php
yum -y install httpd
yum -y install php php-cli php-curl \
php-fpm \
php-intl \
php-mcrypt \
php-mysql \
php-gd \
php-mbstring \
php-xml \
php-dom

yum -y install mariadb-server mariadb
yum -y install http://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/m/mod_xsendfile-0.12-10.el7.x86_64.rpm
#wget http://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/m/mod_xsendfile-0.12-10.el7.x86_64.rpm
#rpm -ivh mod_xsendfile-0.12-10.el7.x86_64.rpm
#apache
rm -rf /etc/httpd/conf.d/welcome.conf
#sed -ri 's/Listen 80/Listen 8080/' /etc/httpd/conf/httpd.conf
systemctl enable httpd

#mysql
systemctl start mariadb
systemctl enable mariadb

mysqladmin -uroot password "$Mysql_Pass"
mysql -uroot -p"$Mysql_Pass" -e "create database edusoho"

#php sed awk 编辑文本 分析日志 SPLUNK ELK 替换/// ###
sed -ri 's/post_max_size = 8M/post_max_size = 1024M/' /etc/php.ini
sed -ri 's/memory_limit = 128M/memory_limit = 1024M/' /etc/php.ini
sed -ri 's/upload_max_filesize = 2M/upload_max_filesize = 1024M/' /etc/php.ini
sed -ri 's#;date.timezone =#date.timezone = Asia/ShangHai#' /etc/php.ini

#sed -ri  's/post_max_size = 8M/post_max_size = 1024M/;s/memory_limit = 128M/memory_limit = 1024M/;s/upload_max_filesize = 2M/upload_max_filesize = 1024M/;s#;date.timezone =#date.timezone = Asia/ShangHai#' /etc/php.ini

#9000
systemctl start php-fpm
systemctl enable php-fpm

#edusoho
wget http://download.edusoho.com/edusoho-7.5.12.tar.gz
tar xvzf edusoho-7.5.12.tar.gz
cp -rf edusoho /var/www/
chown -R apache.apache /var/www/edusoho/

rm -rf /var/www/html/index.html
sed -ri 's#DocumentRoot "/var/www/html"#DocumentRoot "/var/www/edusoho/web"#' /etc/httpd/conf/httpd.conf

cat >>/etc/httpd/conf/httpd.conf <<EOF
<Directory "/var/www/edusoho/web">
AllowOverride All
Require all granted
</Directory>
EOF

systemctl restart httpd
systemctl restart php-fpm
18.Linux 系统状态收集及分析
shell 复制代码 
#!/bin/sh
#requires the following
# free, hostname, grep, cut, awk, uname, sar, ps, netstat
HOSTNAME=`hostname -s`
#memory
MEMORY=`free | grep Mem | awk '{print $2}'`
 
#cpu info
CPUS=`cat /proc/cpuinfo | grep processor | wc -l | awk '{print $1}'`
CPU_MHZ=`cat /proc/cpuinfo | grep MHz | tail -n1 | awk '{print $4}'`
CPU_TYPE=`cat /proc/cpuinfo | grep vendor_id | tail -n 1 | awk '{print $3}'`
CPU_TYPE2=`uname -m`
 
OS_NAME=`uname -s`
OS_KERNEL=`uname -r`
UPTIME=`uptime`
PROC_COUNT=`ps -ef | wc -l`
 
body() {
    IFS= read -r header
    printf '%s\n' "$header"
    "$@"
}
 
#print it out
echo "概要信息" `date +'%Y-%m-%d %H:%S'`
echo "----------------------------------"
echo "主机名            : $HOSTNAME"
echo "内存大小          : $MEMORY"
echo "CPU核数           : $CPUS"
echo "CPU类型           : $CPU_TYPE $CPU_TYPE2 $CPU_MHZ MHz"
echo "操作系统          : $OS_NAME"
echo "内核版本          : $OS_KERNEL"
echo "进程总数          : $PROC_COUNT"
echo "启动时间及负载    : $UPTIME"
echo
echo "内存使用情况"
echo "----------------------------------"
free -m
echo 
echo "磁盘使用情况"
echo "----------------------------------"
df -h
echo 
echo "网络连接情况"
echo "----------------------------------"
#过滤了127.0.0.1
netstat -n |grep -v '127.0.0.1'| awk '/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}'
echo 
echo "网络监听情况"
echo "----------------------------------"
netstat -tnpl | awk 'NR>2 {printf "%-20s %-15s \n",$4,$7}'
echo 
echo "内存占用Top 10"
echo "----------------------------------"
ps -eo rss,pmem,pcpu,vsize,args |body sort -k 1 -r -n | head -n 10
echo 
echo "CPU占用Top 10"
echo "----------------------------------"
ps -eo rss,pmem,pcpu,vsize,args |body sort -k 3 -r -n | head -n 10
echo 
echo "最近1小时网络流量统计"
echo "----------------------------------"
sar -n DEV -s `date -d "1 hour ago" +%H:%M:%S`
echo 
echo "最近1小时cpu使用统计"
echo "----------------------------------"
sar -u -s `date -d "1 hour ago" +%H:%M:%S`
echo 
echo "最近1小时磁盘IO统计"
echo "----------------------------------"
sar -b -s `date -d "1 hour ago" +%H:%M:%S`
echo 
echo "最近1小时进程队列和平均负载统计"
echo "----------------------------------"
sar -q -s `date -d "1 hour ago" +%H:%M:%S`
echo 
echo "最近1小时内存和交换空间的统计统计"
echo "----------------------------------"
sar -r -s `date -d "1 hour ago" +%H:%M:%S`
echo
19.Web 访问日志全文分析
19.1统计2026年9月5日 PV量
shell 复制代码 
[root@xulei log]#grep '05/Sep/2026' cd.mobiletrain.org.log | wc -l
1260

[root@xulei log]#awk '$4>="[05/Sep/2026:08:00:00" && $4<="[05/Sep/2026:09:00:00" {print $0}' sz.mobiletrain.org.log | wc -l
53
19.2统计2026年9月5日 一天内访问最多的10个IP（ip top10）
shell 复制代码 
[root@xulei log]#grep '05/Sep/2026' cd.mobiletrain.org.log | awk '{ ips[$1]++ } END {for (i in ips){print i,ips[i]}}' | sort -k2 -rn | head -n10 
182.140.217.111 138
121.12.22.33 100
10.19.3.2 90
23.29.112.23 80
121.31.30.189 45
187.23.43.123 40
19.3统计2026年9月5日 访问最多的10个页面（$request top 10）
shell 复制代码 
[root@xulei log]#grep '05/Sep/2026' sz.mobiletrain.org.log | awk '{ urls[$7]++} END{for (i in urls){print urls[i],i}}' | sort -k1 -rn | head -n10

44 /
34 /e/admin/DoTimeRepage.php
25 /skin/sz/js/minkh.php
25 /js/jquery-1.11.3.min.js
22 /skin/sz/js/page/jquery-1.4.2.min.js
21 /skin/sz/js/page/ready.js
19 /skin/sz/js/table.js
18 /skin/sz/js/page/core.js
16 /img/sz_home/sz_js.png
16 /img/kbxx_bg.png
热门推荐
012026年6月AI大模型全景报告：GPT-5.6、Claude Opus 4.8、Gemini 3.5，中美AI三足鼎立谁主沉浮？022026年6月AI行业全景：从百模大战到Agent元年，这30天发生了什么？03【AI】2026 年具身智能模型和世界模型总结042026 年 AI 编程工具终极横评：Cursor vs Claude Code vs Copilot vs Windsurf05Claude Code、Codex、Cursor三分天下：2026年AI编程Agent生态全景剖析06GitHub 镜像站点07【AI总结】2026年6月 主流国内外大模型总结08AI科技热点日报 | 2026年6月1日09AI科技热点日报 | 2026年6月22日10AI一周事件 · 2026-06-03 至 2026-06-09