[FIXED] TestContainers SQL too many connections exception

Issue

I am using Spring Boot with 3 SQL datasources defined. I am using the SpringBoot properties to make the 3 TestContainers for the Datasources.

When I run my tests, the containers are getting slower and eventually I've finally gotten a Caused by: java.sql.SQLNonTransientConnectionException: Too many connections exception.

An example test is:

    @Test
    @DisplayName("Require Client Access.")
    @Transactional(transactionManager = "thirdTransactionManager")
    @Sql(
        scripts = "/db/sql/some_test_data_basic.sql",
        config = @SqlConfig(dataSource = "thirdDatasource", transactionManager = "thirdTransactionManager"))
    void requireCustomerAccess() throws CompClientAccessException {

         //Assertions
     }

I am using FlyWay to migrate schema so I am unsure if between each test class either Hikari or Flyway is not closing its Connection Pool connections after each method class leading to the too many connections?

I have 3 TestContainers started via Spring's props like below. I can see them in docker.

datasource.jdbc.url=jdbc:tc:mysql:8.0.22:///databasename?TC_TEMPFS=/testtempfs:rw
datasource.username=dev
datasource.password=password

A snippet of the DatasourceConf (one of the three):

   @Qualifier("thirdDatasource")
    @Bean(name = "thirdDatasource")
    public HikariDataSource thirdDatasource() {
        HikariDataSource hikariDataSource = new HikariDataSource();
        hikariDataSource.setMaximumPoolSize(3);
        hikariDataSource.setJdbcUrl(thirdUrl);
        hikariDataSource.setUsername(thirdUsername);
        hikariDataSource.setPassword(thirdUsername);
        hikariDataSource.setDriverClassName(driverClassName);
        return hikariDataSource;
    }

    @Bean(name="thirdTransactionManager")
    @Qualifier("thirdTransactionManager")
    public PlatformTransactionManager thirdTransactionManager(

        final @Qualifier("thirdEntityManagerFactory") LocalContainerEntityManagerFactoryBean thirdEntityManagerFactory) {
        return new JpaTransactionManager(thirdEntityManagerFactory.getObject());
    }

I am guessing the HikariPools are being remade for each Test class and not shutting down?

 o.h.e.j.s.SqlExceptionHelper   : HikariPool-45 - Connection is not available, request timed out after 30000ms.

Thank you for any help.

Solution

Thanks to the help from M. Deinum:

Application contexts are cached between tests to help speed up tests: https://docs.spring.io/spring-framework/docs/current/reference/html/testing.html#testcontext-ctx-management-caching

A new context is created say for example a different set of @MockBeans are used, different Profiles are used.

Due to this, there were more than 16 Hikari datasources cached in my test environment which all at least used 3-10 connections. This was more than the default for the TestContainer so by the time a certain number of tests with differing contexts had been reached/cached, the connection pools (as a sum of ) would not be able to create new connections to the datasource(s) giving the SQL connection errors.

Takeaway:

Think about test overlap more. I.e. does it need to be a full @SpringBootTest if some logic has already been done or will a MocMvc or DataJPATest suffice.

The number of cached contexts is 32 by default, but that can be set via a JVM system arg (see docs).

Answered By - Jcov
Answer Checked By - Katrina (JavaFixing Volunteer)