记录一次死锁检测

  1. 现象是这样的,服务有时启动正常,有时启动失败,最后排查发现是mq消费者造成的。 启动dubbo过程中,mq消费者正常消费,调用某个dubbo服务。此时dubbo application正处于starting状态,造成了死锁。

2.模拟死锁过程,获取到的线程信息,已省略无关的信息。

java 复制代码
Found one Java-level deadlock:
=============================
"Thread-2":
  waiting to lock monitor 0x00000136bfd3bb88 (object 0x00000005c4c594f0, a org.apache.dubbo.config.deploy.DefaultModuleDeployer),
  which is held by "main"
"main":
  waiting to lock monitor 0x00000136ba6465d8 (object 0x00000005c46287e0, a java.util.concurrent.ConcurrentHashMap),
  which is held by "Thread-2"

Java stack information for the threads listed above:
===================================================
"Thread-2":
        at org.apache.dubbo.config.deploy.DefaultModuleDeployer.startSync(DefaultModuleDeployer.java:143)
        - waiting to lock <0x00000005c4c594f0> (a org.apache.dubbo.config.deploy.DefaultModuleDeployer)
        at org.apache.dubbo.config.deploy.DefaultModuleDeployer.start(DefaultModuleDeployer.java:139)
        at org.apache.dubbo.config.ReferenceConfig.get(ReferenceConfig.java:228)
        at org.apache.dubbo.config.spring.ReferenceBean.getCallProxy(ReferenceBean.java:351)
        - locked <0x00000005c46287e0> (a java.util.concurrent.ConcurrentHashMap)
        at org.apache.dubbo.config.spring.ReferenceBean.access$100(ReferenceBean.java:100)
        at org.apache.dubbo.config.spring.ReferenceBean$DubboReferenceLazyInitTargetSource.createObject(ReferenceBean.java:359)
        at org.springframework.aop.target.AbstractLazyCreationTargetSource.getTarget(AbstractLazyCreationTargetSource.java:86)
        - locked <0x00000005c98d68d0> (a org.apache.dubbo.config.spring.ReferenceBean$DubboReferenceLazyInitTargetSource)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:192)
        ....
"main":
        at org.springframework.context.event.AbstractApplicationEventMulticaster.getApplicationListeners(AbstractApplicationEventMulticaster.java:187)
        - waiting to lock <0x00000005c46287e0> (a java.util.concurrent.ConcurrentHashMap)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:128)
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:393)
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:347)
        at org.apache.dubbo.config.spring.ServiceBean.publishExportEvent(ServiceBean.java:133)
        at org.apache.dubbo.config.spring.ServiceBean.exported(ServiceBean.java:125)
        at org.apache.dubbo.config.ServiceConfig.doExport(ServiceConfig.java:392)
        - locked <0x00000005c9398570> (a org.apache.dubbo.config.spring.ServiceBean)
        at org.apache.dubbo.config.ServiceConfig.export(ServiceConfig.java:243)
        - locked <0x00000005c9398570> (a org.apache.dubbo.config.spring.ServiceBean)
        at org.apache.dubbo.config.deploy.DefaultModuleDeployer.exportServiceInternal(DefaultModuleDeployer.java:350)
        at org.apache.dubbo.config.deploy.DefaultModuleDeployer.exportServices(DefaultModuleDeployer.java:322)
        at org.apache.dubbo.config.deploy.DefaultModuleDeployer.startSync(DefaultModuleDeployer.java:158)
        - locked <0x00000005c4c594f0> (a org.apache.dubbo.config.deploy.DefaultModuleDeployer)
        at org.apache.dubbo.config.deploy.DefaultModuleDeployer.start(DefaultModuleDeployer.java:139)
        at org.apache.dubbo.config.spring.context.DubboDeployApplicationListener.onContextRefreshedEvent(DubboDeployApplicationListener.java:113)
        at org.apache.dubbo.config.spring.context.DubboDeployApplicationListener.onApplicationEvent(DubboDeployApplicationListener.java:102)
        at org.apache.dubbo.config.spring.context.DubboDeployApplicationListener.onApplicationEvent(DubboDeployApplicationListener.java:47)
		...

Found 1 deadlock.
  1. 解决方案
java 复制代码
    @EventListener
    public void onDubboStarted(DubboApplicationStateEvent event) {
        if (event.getState() == DeployState.STARTED) {
            doSomething();
        }
    }
相关推荐
AI人工智能+电脑小能手1 小时前
【大白话说Java面试题 第192题】【08_Kafka篇】第8题:死信队列是什么?延时队列是什么?
java·kafka·消息队列·死信队列·延时队列
meng半颗糖1 小时前
3.Java流程控制语句
java·开发语言·intellij-idea
大模型码小白1 小时前
企业级检索增强后端集成:Java 服务如何管理知识库版本
java·服务器·开发语言·人工智能·python·microsoft
jvmind_dev2 小时前
1c1g 容器莫名 OOM Killed?排查 Metaspace 持续增长与脚本引擎的坑
java·后端
AI多Agent协作实战派2 小时前
AI多Agent协作系统实战(二十二):从6列到12列——任务监控报告的进化之路
java·人工智能·uni-app·bug
用户3126874877202 小时前
@Transactional 注了等于没用?Spring 事务失效的 7 种场景,你踩过几个
java
千桐科技2 小时前
DataX 执行引擎正式加入:qData 开源版 v1.6.0 新增 Quartz + DataX 轻量运行模式!
java·大数据·开源·数据治理·数据中台·qdata
Irene19912 小时前
Oracle PL/SQL Developer 版本差异(11g、14)实战总结
java·开发语言
AI人工智能+电脑小能手2 小时前
【大白话说Java面试题 第191题】【08_Kafka篇】第7题:消息队列的优缺点
java·消息队列·系统设计·分布式架构·技术选型