[FIXED] Deadlock while creating bean in DefaultSingletonBeanRegistry and CachingConnectionFactory

Issue

My application finds deadlock while bean initialization. Two beans are waiting and holding resources required by one another in different threads. During the bean initialization "BeanA" class has taken lock of "connectionMonitor" object by calling method [CachingConnectionFactory >> createConnection()] and waiting for lock of ConcurrentHashMap "singletonObjects" in DefaultSingletonBeanRegistry. Whereas "BeanB" has taken lock of ConcurrentHashMap "singletonObjects" by calling method [DefaultSingletonBeanRegistry >> getSingleton()] and is waiting for lock of "connectionMonitor" object in CachingConnectionFactory.

Thread dump :-

Thread 1:-

   java.lang.Thread.State: BLOCKED (on object monitor)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:187)
- waiting to lock <0x000008001202c048> (a java.util.concurrent.ConcurrentHashMap)
at org.springframework.beans.factory.support.AbstractBeanFactory.isSingleton(AbstractBeanFactory.java:399)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doGetBeanNamesForType(DefaultListableBeanFactory.java:431)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:395)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeansOfType(DefaultListableBeanFactory.java:515)
at org.springframework.context.support.AbstractApplicationContext.getBeansOfType(AbstractApplicationContext.java:1197)
at org.springframework.amqp.rabbit.core.RabbitAdmin.initialize(RabbitAdmin.java:457)
at org.springframework.amqp.rabbit.core.RabbitAdmin$11.onCreate(RabbitAdmin.java:419)
at org.springframework.amqp.rabbit.connection.CompositeConnectionListener.onCreate(CompositeConnectionListener.java:33)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:553)
- locked <0x000009fff2a3a620> (a java.lang.Object)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils$1.createConnection(ConnectionFactoryUtils.java:90)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.doGetTransactionalResourceHolder(ConnectionFactoryUtils.java:140)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.getTransactionalResourceHolder(ConnectionFactoryUtils.java:76)
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:496)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1331)
at java.lang.Thread.run(Thread.java:748)

Thread 2:-

   java.lang.Thread.State: BLOCKED (on object monitor)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:544)
- waiting to lock <0x000009fff2a3a620> (a java.lang.Object)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1431)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1412)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1388)
at org.springframework.amqp.rabbit.core.RabbitAdmin.declareQueue(RabbitAdmin.java:207)
<some call stack lines removed from here for privacy>
- locked <0x000008001202c048> (a java.util.concurrent.ConcurrentHashMap)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:372)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1173)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1067)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)

I am trying to establish connection to rabbit in the @PostConstruct method something like below:-

BeanA {
    
    @PostConstruct
    public void init() {
      <call to CachingConnectionFactory.createConnection is made using some code)>
    }
}

BeanB {
    
    @PostConstruct
    public void init() {
      <call to CachingConnectionFactory.createConnection is made using some code)>
    }
}

Please suggest what needs to be done. Thanks

Solution

It is too early to get access to low-level resources like connection from the initialization phase.

Consider to implement a SmartLifecycle and call that createConnection() from the start() implementation instead of that @PostConstruct.

Answered By - Artem Bilan
Answer Checked By - Mary Flores (JavaFixing Volunteer)