Issue
I'm reading a csv a file using a multiResourceItemReader and I've kept the skip limit to be 10. When the limit exceeds I want to catch the SkipLimitExceedException
and throw my own customized exception with a message like "Invalid csv" ,where or how do i catch it?
try {
log.info("Running job to insert batch fcm: {} into database.", id);
jobLauncher
.run(importJob, new JobParametersBuilder()
.addString("fullPathFileName", TMP_DIR)
.addString("batch_fcm_id", String.valueOf(id))
.addLong("time",System.currentTimeMillis())
.toJobParameters());
}
catch(...){...}
I cannot catch it here, is it because I'm using MultiResourceItemReader
and the asynchronous process doesn't allow me to catch it here?
my job is as follows
@Bean(name = "fcmJob")
Job importJob(@Qualifier(MR_ITEM_READER) Reader reader,
@Qualifier(JDBC_WRITER) JdbcBatchItemWriter jdbcBatchItemWriter,
@Qualifier("fcmTaskExecutor") TaskExecutor taskExecutor) {
Step writeToDatabase = stepBuilderFactory.get("file-database")//name of step
.<FcmIdResource, FcmIdResource>chunk(csvChunkSize) // <input as, output as>
.reader(reader)
.faultTolerant()
.skipLimit(10)
.skip(UncategorizedSQLException.class)
.noSkip(FileNotFoundException.class)
.writer(jdbcBatchItemWriter)
.taskExecutor(taskExecutor)
.throttleLimit(20)
.build();
return jobBuilderFactory.get("jobBuilderFactory") //Name of job builder factory
.incrementer(new RunIdIncrementer())
.start(writeToDatabase)
.on("*")
.to(deleteTemporaryFiles())
.end()
.build();
}
I have tried using ItemReaderListener, SkipPolicy, SkipListener, but they cannot throw an exception, is there any other way?
Solution
The exception you are looking for is not thrown by the job, you can get it from the job execution using JobExecution#getAllFailureExceptions
.
So in your example, instead of doing:
try {
jobLauncher.run(job, new JobParameters());
} catch (Exception e) {
//...
}
You should do:
JobExecution jobExecution = jobLauncher.run(job, new JobParameters());
List<Throwable> allFailureExceptions = jobExecution.getFailureExceptions();
In your case, SkipLimitExceedException
will be one of allFailureExceptions
.
EDIT: Adding an example showing that SkipLimitExceedException
is part of allFailureExceptions
:
import java.util.Arrays;
import java.util.List;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.support.ListItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
@EnableBatchProcessing
public class MyJob {
@Autowired
private JobBuilderFactory jobs;
@Autowired
private StepBuilderFactory steps;
@Bean
public ItemReader<Integer> itemReader() {
return new ListItemReader<>(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10));
}
@Bean
public ItemProcessor<Integer, Integer> itemProcessor() {
return item -> {
if (item % 3 == 0) {
throw new IllegalArgumentException("no multiples of three here! " + item);
}
return item;
};
}
@Bean
public ItemWriter<Integer> itemWriter() {
return items -> {
for (Integer item : items) {
System.out.println("item = " + item);
}
};
}
@Bean
public Step step() {
return steps.get("step")
.<Integer, Integer>chunk(2)
.reader(itemReader())
.processor(itemProcessor())
.writer(itemWriter())
.faultTolerant()
.skip(IllegalArgumentException.class)
.skipLimit(2)
.build();
}
@Bean
public Job job() {
return jobs.get("job")
.start(step())
.build();
}
public static void main(String[] args) throws Exception {
ApplicationContext context = new AnnotationConfigApplicationContext(MyJob.class);
JobLauncher jobLauncher = context.getBean(JobLauncher.class);
Job job = context.getBean(Job.class);
JobExecution jobExecution = jobLauncher.run(job, new JobParameters());
List<Throwable> allFailureExceptions = jobExecution.getAllFailureExceptions();
for (Throwable failureException : allFailureExceptions) {
System.out.println("failureException = " + failureException);
}
}
}
This sample prints:
item = 1
item = 2
item = 4
item = 5
item = 7
item = 8
failureException = org.springframework.batch.core.step.skip.SkipLimitExceededException: Skip limit of '2' exceeded
Answered By - Mahmoud Ben Hassine
Answer Checked By - Pedro (JavaFixing Volunteer)