[FIXED] Make Spring 4 Batch service repeatable on Scheduled execution

Issue

I've been trying to solve this issue for few days and I couldn't make it. I'm quite new with Spring 4, so maybe someone with experience can tell me how to do it.

I was trying to learn how to code a Spring Batch with the latest version of the framework. I followed the Getting Started Tutorial: https://spring.io/guides/gs/batch-processing/

Once the job was executing, I wanted to make it execute periodically. Therefore, I went for a scheduled task: https://spring.io/guides/gs/scheduling-tasks/

It looked easy to put both ideas together. In practise, it was not possible. I read a little bit and I added some code to make the jobExecution unique every time. Still, the coded step, once finished, never executes again.

I investigated a little bit and I've read about RepeatTemplate, but I don't see it clear how to make it fit with my code. Here you have the 3 relevant methods for the execution:

@Scheduled(cron="0 0/2 * * * ?")
public void run() throws Exception {
    System.out.println("Job Started at :" + new Date());

    JobParameters param = new JobParametersBuilder().addString("newsSyncJob",
            String.valueOf(System.currentTimeMillis())).toJobParameters();

    NewsJobCompletionListener listener = new NewsJobCompletionListener();
    JobExecution execution = jobLauncher.run(newsSyncJob(listener), param);

    System.out.println("Job finished with status :" + execution.getStatus());
}


/**
 * Execution of the job
 * @param listener
 * @return
 */
@Bean
public Job newsSyncJob(NewsJobCompletionListener listener) {
    log.debug("newsSyncJob execution started");
    this.init();
    return jobBuilderFactory.get("newsSyncJob")
            .incrementer(new RunIdIncrementer())
            .listener(listener)
            .flow(step1())
            .end()
            .build();
}

@Bean
protected Step step1() {
    return stepBuilderFactory.get("step1")
            .<NewsSync, NewsSync> chunk(4)
            .reader(filesReader()).
            processor(newsProcessor()).
            writer(stateWriter()).
            build();
}

Any ideas how to make the step reexecute successfully? Thank you in advance!

Solution

After M.Deinum's comment I made a little more investigation and I could solve it myself. Here is the explanation step by step:

The configuration was wrong because newsSyncJob() should be only executing once. It builds the Job and the bean keeps created as singleton. A singleton that can be executed as many times as you want. Same happens with the step bean. step1() is only called once to create the singleton bean. As part of the Job, it may execute when the Job is executed... as far as it is configured properly. For now, the two methods look like this:

@Bean
public Job newsSyncJob() {
    log.debug("Building batch newsSyncJob...");
    this.init();
    return jobBuilderFactory.get("newsSyncJob")
            .incrementer(new RunIdIncrementer())
            .start(step1())
            .build();
}

@Bean
protected Step step1() {
    return stepBuilderFactory.get("step1")
            .<NewsSync, NewsSync> chunk(4)
            .reader(filesReader()).
            processor(newsProcessor()).
            writer(stateWriter()).
            build();
}

Now, to make the step execute every time we run the job, its scope needs to be changed. The step is composed by an itemReader + itemProcessor + itemWriter. By default, their scope is Prototype. This means that it keeps the state of the previous execution and it won't run again. To make it run again, the scope needs to be changed to Step scope. The code will look like this:

@Bean
@StepScope
ItemReader<NewsSync> filesReader() {
    return new NewsSyncReader(srcPath);
}

@Bean
@StepScope
ItemProcessor<NewsSync, NewsSync> newsProcessor() {
    return new NewsSyncProcessor(jdbcTemplate, newsRepository);
}

@Bean
@StepScope
ItemWriter<NewsSync> stateWriter() {
    return new NewsSyncWriter(jdbcTemplate);
}

Now to finish, the scheduled call will be configured in a different class using the dependency injection of the Beans created in the previous class. Something like this:

@EnableScheduling
@Service
public class BatchScheduled {

    private static final Logger log = LoggerFactory.getLogger(BatchScheduled.class);

    @Autowired
    private JobLauncher jobLauncher;

    @Autowired
    @Qualifier("newsSyncJob")
    private Job newsJobSync;

    /**
     * Scheduled run of the batch
     * @throws Exception
     */
    @Scheduled(cron="0 0/2 * * * ?")
    public void run() throws Exception {
        log.info("newsJobSync started");
        System.out.println("Job Started at :" + new Date());

        JobParameters param = new JobParametersBuilder().addString("newsSyncJob",
                String.valueOf(System.currentTimeMillis())).toJobParameters();

        JobExecution execution = jobLauncher.run(newsJobSync, param);

        log.info("newsJobSync finished with status " + execution.getExitStatus());
    }
}

We finally have a Spring Batch executing with a scheduled expression.

Answered By - Marc Loan
Answer Checked By - David Goodson (JavaFixing Volunteer)