背景
接wiki 【服务器断电后,重启服务器,gitlab无法访问】https://blog.csdn.net/nobigdeal00/article/details/144280761
最近把不小心把服务器重启,每次重启后,都会出现gitlab无法访问
分析
查看系统正在运行的任务
bash
adminpc@adminpc:~/go_git/scripts$ systemctl list-jobs
JOB UNIT TYPE STATE
177 system-getty.slice start waiting
19 setvtrgb.service start waiting
2 multi-user.target start waiting
161 gitlab-runsvdir.service start waiting
202 systemd-update-utmp-runlevel.service start waiting
1 graphical.target start waiting
200 plymouth-quit-wait.service start running
7 jobs listed.
发现gitlab-runsvdir.service状态处于waiting,而前面有个任务plymouth-quit-wait.service一直处于running,阻塞了其他任务的执行
- plymouth-quit-wait.service
plymouth-quit-wait.service 是 systemd 的一个服务单元,与 Plymouth 引导屏幕管理器有关。而Plymouth 是 Linux 系统中用于显示图形化启动和关闭画面的工具,由于服务器没接入显示器,这里可能对Plymouth的正常启动产生了影响,因为 Plymouth 通常依赖于图形输出设备来确认启动画面的显示状态,这里它可能在一直等待图形模式初始化完成
手动将该任务停掉
bash
adminpc@adminpc:~/go_git/scripts$ sudo systemctl stop plymouth-quit-wait.service
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ====
Authentication is required to stop 'plymouth-quit-wait.service'.
==== AUTHENTICATION COMPLETE ====
再次查看任务列表,剩余任务已经执行完成
bash
adminpc@adminpc:~/go_git/scripts$ sudo systemctl list-jobs
No jobs running.
查看gitlab状态,gitlab-runsvdir可正常运行
bash
adminpc@adminpc:~$ sudo gitlab-ctl status
run: alertmanager: (pid 4627) 6s; run: log: (pid 4601) 6s
run: gitaly: (pid 4620) 6s; run: log: (pid 4597) 6s
run: gitlab-exporter: (pid 4617) 6s; run: log: (pid 4602) 6s
run: gitlab-kas: (pid 4621) 6s; run: log: (pid 4605) 6s
run: gitlab-workhorse: (pid 4624) 6s; run: log: (pid 4608) 6s
run: logrotate: (pid 4628) 6s; run: log: (pid 4609) 6s
run: nginx: (pid 4612) 6s; run: log: (pid 4599) 6s
run: node-exporter: (pid 4619) 6s; run: log: (pid 4611) 6s
run: postgres-exporter: (pid 4623) 6s; run: log: (pid 4613) 6s
run: postgresql: (pid 4610) 6s; run: log: (pid 4600) 6s
run: prometheus: (pid 4618) 6s; run: log: (pid 4604) 6s
run: puma: (pid 4615) 6s; run: log: (pid 4603) 6s
run: redis: (pid 4614) 6s; run: log: (pid 4607) 6s
run: redis-exporter: (pid 4616) 6s; run: log: (pid 4598) 6s
run: sidekiq: (pid 4622) 6s; run: log: (pid 4606) 6s
查看runsv状态,与gitlab相关的服务均已监控起来
bash
adminpc@adminpc:~/go_git/scripts$ ps aux | grep runsv
root 431032 0.0 0.0 2716 1408 ? Ss 20:12 0:00 runsvdir -P /opt/gitlab/service log: ...........................................................................................................................................................................................................................................................................................................................................................................................................
root 431033 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv gitlab-kas
root 431034 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv postgresql
root 431035 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv redis
root 431036 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv gitlab-exporter
root 431037 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv redis-exporter
root 431038 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv puma
root 431040 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv logrotate
root 431041 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv prometheus
root 431042 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv alertmanager
root 431043 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv nginx
root 431044 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv node-exporter
root 431045 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv sidekiq
root 431046 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv gitaly
root 431047 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv postgres-exporter
root 431048 0.0 0.0 2564 1408 ? Ss 20:12 0:00 runsv gitlab-workhorse
adminpc 432194 0.0 0.0 9640 2432 pts/0 S+ 20:14 0:00 grep --color=auto runsv
解决方案
修改 GRUB 启动参数,绕过 Plymouth 的图形启动画面,直接文本启动
编辑 /etc/default/grub 文件
bash
sudo vim/etc/default/grub
修改 GRUB_CMDLINE_LINUX_DEFAULT,移除 splash 参数(splash参数会启动图形化的启动画面,即开机动画,对学习开发来说,开机动画其实没那么重要)并添加 text
bash
...
# GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX_DEFAULT="quiet text"
...
更新 GRUB 配置并重启系统
bash
sudo update-grub
sudo reboot
重启系统后,查看gitlab状态,均正常
bash
adminpc@adminpc-M600:~$ sudo gitlab-ctl status
run: alertmanager: (pid 2785) 332s; run: log: (pid 2776) 332s
run: gitaly: (pid 2800) 332s; run: log: (pid 2782) 332s
run: gitlab-exporter: (pid 2797) 332s; run: log: (pid 2784) 332s
run: gitlab-kas: (pid 2796) 332s; run: log: (pid 2778) 332s
run: gitlab-workhorse: (pid 2807) 332s; run: log: (pid 2791) 332s
run: logrotate: (pid 2795) 332s; run: log: (pid 2779) 332s
run: nginx: (pid 2794) 332s; run: log: (pid 2787) 332s
run: node-exporter: (pid 2806) 332s; run: log: (pid 2804) 332s
run: postgres-exporter: (pid 2802) 332s; run: log: (pid 2798) 332s
run: postgresql: (pid 2790) 332s; run: log: (pid 2781) 332s
run: prometheus: (pid 2786) 332s; run: log: (pid 2775) 332s
run: puma: (pid 2792) 332s; run: log: (pid 2780) 332s
run: redis: (pid 2789) 332s; run: log: (pid 2777) 332s
run: redis-exporter: (pid 2799) 332s; run: log: (pid 2788) 332s
run: sidekiq: (pid 2793) 332s; run: log: (pid 2783) 332s
给服务器接上显示器,正常启动也ok,只是没了开机画面,取而代之的是启动脚本执行状态