[FIXED] How to Test a Spring batch Job

Issue

I'm trying to test a job but I can't get to the bottom of it, basically my step consists of a single reader and a composite Processor and a Composite writer, all this because I have to read from a single .CSV and write on MongoDB and Mysql, here is my Job Configuration class:

@Configuration
@EnableBatchProcessing
public class BatchConfiguration {

    @Autowired
    public JobBuilderFactory jobBuilderFactory;
    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Autowired
    public  MarketDataMySqlRepository marketDataMySqlRepository;


    @Value("${file.input}")
    private String fileInput;




    @Bean
    public TaskExecutor taskExecutor(){
        SimpleAsyncTaskExecutor taskExecutor=new SimpleAsyncTaskExecutor("customerInfoThreads-");
        return taskExecutor;
    }


    @Bean
    public FlatFileItemReader<MarketDataDto> reader() {
        return new FlatFileItemReaderBuilder<MarketDataDto>().name("ItemReader")
                .resource(new PathResource(fileInput)).delimited()
                .names("time","id", "price", "quantity", "isbuyermaker")
                .fieldSetMapper(new BeanWrapperFieldSetMapper<MarketDataDto>() {
                    {
                        setTargetType(MarketDataDto.class);
                    }
                }).build();
    }

    @Bean
    public MongoItemWriter<MarketDataMongo> writer(MongoTemplate mongoTemplate) {

        return new MongoItemWriterBuilder<MarketDataMongo>().template(mongoTemplate).collection("marketdata")
                .build();

    }
    @Bean
    public RepositoryItemWriter<MarketDataMySql> writer1() {
        RepositoryItemWriter<MarketDataMySql> writer = new RepositoryItemWriter<>();
        writer.setRepository(marketDataMySqlRepository);
        writer.setMethodName("save");
        return writer;
    }

    @Bean
    public CompositeItemWriter CompositeItemWriter(MongoTemplate mongoTemplate){
        CompositeItemWriter writer = new CompositeItemWriter();
        writer.setDelegates(Arrays.asList(writer(mongoTemplate),writer1()));
        return writer;
    }


    @Bean
    public UserItemProcessorMongo MongoProcessor() {
        return new UserItemProcessorMongo();
    }
    @Bean
    public UserItemProcessorMySql MySqlProcessor() {return new UserItemProcessorMySql();}


    @Bean
    public CompositeItemProcessor compositeProcessor() throws Exception {
        List<ItemProcessor<MarketDataDto,MarketDataMongo>> itemProcessors = new ArrayList<>(1);
        List<ItemProcessor<MarketDataDto,MarketDataMySql>> itemProcessors1 = new ArrayList<>(1);
        itemProcessors1.add(MySqlProcessor());
        itemProcessors.add(MongoProcessor());
        CompositeItemProcessor processor = new CompositeItemProcessor();
        processor.setDelegates(itemProcessors);
        processor.setDelegates(itemProcessors1);
        //processor.afterPropertiesSet();
        return processor;

    }

    @Bean
    public Step step1(FlatFileItemReader<MarketDataDto> itemReader, MongoItemWriter<MarketDataMongo> itemWriter, MongoTemplate mongoTemplate)
            throws Exception {
        return this.stepBuilderFactory.get("step1").chunk(200000).reader(itemReader)
                .processor(compositeProcessor()).writer(CompositeItemWriter(mongoTemplate)).build();

    }

    @Bean
    public Job updateUserJob(JobCompletionNotificationListener listener, Step step1)
            throws Exception {

        return this.jobBuilderFactory.get("updateMarketJob").incrementer(new RunIdIncrementer())
                .listener(listener).start(step1).build();
    }

    }

And this is my TestClass :

@RunWith(SpringRunner.class)
@SpringBatchTest
@ContextConfiguration(classes= {BatchConfiguration.class, BatchAutoConfiguration.class})
public class BatchConfigurationTest {


    @Autowired
    private JobLauncherTestUtils jobLauncherTestUtils;

    @Autowired
    private JobRepositoryTestUtils jobRepositoryTestUtils;

    @Autowired
    private MarketDataMySqlRepository marketDataMySqlRepository;




    @Test
    public void readerTest() throws Exception {
        JobParameters jobParameters = this.jobLauncherTestUtils.getUniqueJobParameters();

      
        JobExecution jobExecution = this.jobLauncherTestUtils.launchJob(jobParameters);
        Assert.assertEquals(ExitStatus.COMPLETED, jobExecution.getExitStatus());

    }
}

When i try to launch my test i riceive this Exception :

java.lang.IllegalStateException: Failed to load ApplicationContext

    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:98)
    at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:124)
    at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:118)
    at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:83)
    at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:248)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:227)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:289)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:291)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:246)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97)
    at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
    at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
    at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
    at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
    at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
    at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'batchConfiguration': Unsatisfied dependency expressed through field 'marketDataMySqlRepository'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'com.aitho.springbatch.repository.MarketDataMySqlRepository' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:659)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:639)
    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
    at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:127)
    at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:60)
    at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.delegateLoading(AbstractDelegatingSmartContextLoader.java:276)
    at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.loadContext(AbstractDelegatingSmartContextLoader.java:244)
    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:141)
    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:90)
    ... 27 more
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'com.aitho.springbatch.repository.MarketDataMySqlRepository' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoMatchingBeanFound(DefaultListableBeanFactory.java:1801)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1357)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:656)
    ... 46 more

Solution

According to the error you shared, you should define a bean of type com.aitho.springbatch.repository.MarketDataMySqlRepository in your test context.

There is an autowired field of that type in BatchConfigurationTest but no bean of such type is defined in your context.

Answered By - Mahmoud Ben Hassine
Answer Checked By - David Goodson (JavaFixing Volunteer)