task oracle_110944_k:110944 blocked for more than 120 seconds

Jul 14 09:21:38 db systemd-logind: Failed to abandon session scope: Connection timed out

Jul 14 09:26:38 db systemd-logind: Failed to start session scope session-2283.scope: Connection timed out

Jul 14 09:28:45 db systemd: Failed to start Session 2283 of user root.

Jul 14 09:29:52 db systemd-logind: Failed to start session scope session-2285.scope: Connection timed out

Jul 14 09:31:08 db systemd: Failed to start Session 2285 of user root.

Jul 14 09:32:04 db systemd-logind: Failed to start session scope session-2286.scope: Connection timed out

Jul 14 09:32:46 db kernel: INFO: task oracle_110944_k:110944 blocked for more than 120 seconds.

Jul 14 09:33:38 db kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

Jul 14 09:33:38 db kernel: oracle_110944_k D ffff94e117266eb0 0 110944 1 0x00000080

Jul 14 09:33:58 db kernel: Call Trace:

Jul 14 09:34:29 db kernel: [<ffffffffaa513e69>] schedule_preempt_disabled+0x29/0x70

Jul 14 09:35:44 db kernel: [<ffffffffaa511c27>] __mutex_lock_slowpath+0xc7/0x1d0

Jul 14 09:35:44 db kernel: [<ffffffffaa51100f>] mutex_lock+0x1f/0x2f

Jul 14 09:35:44 db kernel: [<ffffffffaa02abbf>] do_last+0x28f/0x12c0

Jul 14 09:35:44 db kernel: [<ffffffffaa02bcc7>] path_openat+0xd7/0x640

Jul 14 09:35:44 db kernel: [<ffffffffa9fc05fa>] ? __do_fault.isra.58+0x8a/0x100

Jul 14 09:35:44 db kernel: [<ffffffffaa02d85d>] do_filp_open+0x4d/0xb0

Jul 14 09:48:14 db systemd: Started Session 2299 of user root.

Jul 14 09:50:32 db systemd-logind: Failed to start session scope session-2300.scope: Connection timed out

Jul 14 09:52:34 db systemd-logind: Failed to start session scope session-2301.scope: Connection timed out

Jul 14 09:52:48 db su: (to oracle) root on tty1

Jul 14 09:53:12 db systemd-logind: Failed to start session scope session-2302.scope: Connection timed out

Jul 14 09:53:36 db kernel: [drm:vmw_kms_update_layout_ioctl [vmwgfx]] *ERROR* Topology is beyond supported limits.

Jul 14 17:54:46 db kernel: Initializing cgroup subsys cpuset

Jul 14 17:54:46 db kernel: Initializing cgroup subsys cpu

Jul 14 17:54:46 db kernel: Initializing cgroup subsys cpuacct

Jul 14 17:54:46 db kernel: Linux version 3.10.0-862.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) ) #1 SMP Fri Apr 20 16:44:24 UTC 2018

Jul 14 17:54:46 db kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-862.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8

Jul 14 09:20:07 db systemd: Started Session 2280 of user root.

Jul 14 09:20:07 db systemd: Started Session 2281 of user root.

Jul 14 09:21:38 db systemd-logind: Failed to abandon session scope: Connection timed out

Jul 14 09:26:38 db systemd-logind: Failed to start session scope session-2283.scope: Connection timed out

Jul 14 09:28:45 db systemd: Failed to start Session 2283 of user root.

Jul 14 09:29:52 db systemd-logind: Failed to start session scope session-2285.scope: Connection timed out

Jul 14 09:31:08 db systemd: Failed to start Session 2285 of user root.

Jul 14 09:32:04 db systemd-logind: Failed to start session scope session-2286.scope: Connection timed out

Jul 14 09:32:46 db kernel: INFO: task oracle_110944_k:110944 blocked for more than 120 seconds.

Jul 14 09:33:38 db kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

Jul 14 09:33:38 db kernel: oracle_110944_k D ffff94e117266eb0 0 110944 1 0x00000080

Jul 14 09:33:58 db kernel: Call Trace:

Jul 14 09:34:29 db kernel: [<ffffffffaa513e69>] schedule_preempt_disabled+0x29/0x70

Jul 14 09:35:44 db kernel: [<ffffffffaa511c27>] __mutex_lock_slowpath+0xc7/0x1d0

Jul 14 09:35:44 db kernel: [<ffffffffaa51100f>] mutex_lock+0x1f/0x2f

Jul 14 09:35:44 db kernel: [<ffffffffaa02abbf>] do_last+0x28f/0x12c0

Jul 14 09:35:44 db kernel: [<ffffffffaa02bcc7>] path_openat+0xd7/0x640

Jul 14 09:35:44 db kernel: [<ffffffffa9fc05fa>] ? __do_fault.isra.58+0x8a/0x100

处理办法

1、首先操作 # echo 0 > /proc/sys/kernel/hung_task_timeout_secs

2、根据mos:Linux: Node Reboots After "kernel: INFO: task ocssd.bin:<pid> blocked for more than 120 seconds." in /var/log/messages (Doc ID 1423693.1)

做如下操作:

echo 'vm.min_free_kbytes = 262144' >> /etc/sysctl.conf

echo 'vm.swappiness=100' >> /etc/sysctl.conf

sysctl -p

3、配置hugepage并禁用Auto Memory Management (AMM)

相关推荐
广州腾科助你拿下华为认证19 分钟前
PostgreSQL认证_PGCM考试难度有多大?
数据库·postgresql
代码的余温21 分钟前
Oracle RAC认证矩阵:规避风险的关键指南
数据库·oracle·矩阵
白鲸开源22 分钟前
一行代码引发 12G 内存 5 分钟爆仓!SeaTunnel Kafka 连接器"内存溢出"元凶抓到了
数据库·kafka·开源
cellurw1 小时前
Day39 SQLite数据库操作与HTML核心API及页面构建
数据库·sqlite·html
小蒜学长1 小时前
旅行社旅游管理系统的设计与实现(代码+数据库+LW)
java·数据库·spring boot·后端·旅游
Mr_hwt_1231 小时前
基于mybatis-plus动态数据源实现mysql集群读写分离和从库负载均衡教程(详细案例)
数据库·spring boot·mysql·mybatis·mysql集群
万邦科技Lafite1 小时前
实战演练:通过API获取商品详情并展示
大数据·数据库·python·开放api接口
黄焖鸡能干四碗2 小时前
智慧教育,智慧校园,智慧安防学校建设解决方案(PPT+WORD)
java·大数据·开发语言·数据库·人工智能
敲上瘾2 小时前
Docker 存储卷(Volume)核心概念、类型与操作指南
linux·服务器·数据库·docker·容器·架构
DemonAvenger2 小时前
MySQL内存优化:缓冲池与查询缓存调优实战指南
数据库·mysql·性能优化