task oracle_110944_k:110944 blocked for more than 120 seconds

Jul 14 09:21:38 db systemd-logind: Failed to abandon session scope: Connection timed out

Jul 14 09:26:38 db systemd-logind: Failed to start session scope session-2283.scope: Connection timed out

Jul 14 09:28:45 db systemd: Failed to start Session 2283 of user root.

Jul 14 09:29:52 db systemd-logind: Failed to start session scope session-2285.scope: Connection timed out

Jul 14 09:31:08 db systemd: Failed to start Session 2285 of user root.

Jul 14 09:32:04 db systemd-logind: Failed to start session scope session-2286.scope: Connection timed out

Jul 14 09:32:46 db kernel: INFO: task oracle_110944_k:110944 blocked for more than 120 seconds.

Jul 14 09:33:38 db kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

Jul 14 09:33:38 db kernel: oracle_110944_k D ffff94e117266eb0 0 110944 1 0x00000080

Jul 14 09:33:58 db kernel: Call Trace:

Jul 14 09:34:29 db kernel: \ schedule_preempt_disabled+0x29/0x70

Jul 14 09:35:44 db kernel: \ __mutex_lock_slowpath+0xc7/0x1d0

Jul 14 09:35:44 db kernel: \ mutex_lock+0x1f/0x2f

Jul 14 09:35:44 db kernel: \ do_last+0x28f/0x12c0

Jul 14 09:35:44 db kernel: \ path_openat+0xd7/0x640

Jul 14 09:35:44 db kernel: \ ? __do_fault.isra.58+0x8a/0x100

Jul 14 09:35:44 db kernel: \ do_filp_open+0x4d/0xb0

Jul 14 09:48:14 db systemd: Started Session 2299 of user root.

Jul 14 09:50:32 db systemd-logind: Failed to start session scope session-2300.scope: Connection timed out

Jul 14 09:52:34 db systemd-logind: Failed to start session scope session-2301.scope: Connection timed out

Jul 14 09:52:48 db su: (to oracle) root on tty1

Jul 14 09:53:12 db systemd-logind: Failed to start session scope session-2302.scope: Connection timed out

Jul 14 09:53:36 db kernel: drm:vmw_kms_update_layout_ioctl \[vmwgfx] *ERROR* Topology is beyond supported limits.

Jul 14 17:54:46 db kernel: Initializing cgroup subsys cpuset

Jul 14 17:54:46 db kernel: Initializing cgroup subsys cpu

Jul 14 17:54:46 db kernel: Initializing cgroup subsys cpuacct

Jul 14 17:54:46 db kernel: Linux version 3.10.0-862.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) ) #1 SMP Fri Apr 20 16:44:24 UTC 2018

Jul 14 17:54:46 db kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-862.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8

Jul 14 09:20:07 db systemd: Started Session 2280 of user root.

Jul 14 09:20:07 db systemd: Started Session 2281 of user root.

Jul 14 09:21:38 db systemd-logind: Failed to abandon session scope: Connection timed out

Jul 14 09:26:38 db systemd-logind: Failed to start session scope session-2283.scope: Connection timed out

Jul 14 09:28:45 db systemd: Failed to start Session 2283 of user root.

Jul 14 09:29:52 db systemd-logind: Failed to start session scope session-2285.scope: Connection timed out

Jul 14 09:31:08 db systemd: Failed to start Session 2285 of user root.

Jul 14 09:32:04 db systemd-logind: Failed to start session scope session-2286.scope: Connection timed out

Jul 14 09:32:46 db kernel: INFO: task oracle_110944_k:110944 blocked for more than 120 seconds.

Jul 14 09:33:38 db kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

Jul 14 09:33:38 db kernel: oracle_110944_k D ffff94e117266eb0 0 110944 1 0x00000080

Jul 14 09:33:58 db kernel: Call Trace:

Jul 14 09:34:29 db kernel: \ schedule_preempt_disabled+0x29/0x70

Jul 14 09:35:44 db kernel: \ __mutex_lock_slowpath+0xc7/0x1d0

Jul 14 09:35:44 db kernel: \ mutex_lock+0x1f/0x2f

Jul 14 09:35:44 db kernel: \ do_last+0x28f/0x12c0

Jul 14 09:35:44 db kernel: \ path_openat+0xd7/0x640

Jul 14 09:35:44 db kernel: \ ? __do_fault.isra.58+0x8a/0x100

处理办法

1、首先操作 # echo 0 > /proc/sys/kernel/hung_task_timeout_secs

2、根据mos:Linux: Node Reboots After "kernel: INFO: task ocssd.bin:<pid> blocked for more than 120 seconds." in /var/log/messages (Doc ID 1423693.1)

做如下操作:

echo 'vm.min_free_kbytes = 262144' >> /etc/sysctl.conf

echo 'vm.swappiness=100' >> /etc/sysctl.conf

sysctl -p

3、配置hugepage并禁用Auto Memory Management (AMM)

相关推荐
AI多Agent协作实战派2 分钟前
AI多Agent协作系统实战(二十八):顶栏统一战争——从1个页面异常到31个页面全量对齐
数据库·人工智能·uni-app
吴声子夜歌3 分钟前
MongoDB 8.0——可视化管理工具
数据库·mongodb
zdl68614 分钟前
EasyMarkets:“芯片波动考验科技估值”
数据库·人工智能·科技
翼龙云_cloud38 分钟前
阿里云国际站代理商:ECS弹性伸缩 自动应对流量高峰
运维·网络·数据库·阿里云·架构
吴声子夜歌39 分钟前
MongoDB 4.2——安装 MongoDB
数据库·mongodb
吴声子夜歌42 分钟前
MongoDB 8.0——MongoDB的安装
数据库·mongodb
韭菜炒鸡肝天1 小时前
Django模型迁移指南:从命令用法到最佳实践
数据库·django·sqlite
鹿鸣天涯1 小时前
应急响应:MySQL日志分析
mysql·安全·adb·oracle
阿维的博客日记2 小时前
Java事务传播级别REQUIRES_NEW,REQUIRED_NEW区别
java·数据库·事务
toooooop813 小时前
如何用 ss + ps 精准定位本机 Redis 的“隐形”消费者?
linux·数据库·redis·缓存