[FIXED] How to resolve ClassCastException in MultiResourceItemReader Spring Batch

Issue

I'm reading multiple files from the S3 bucket using MultiResourceItemReader, I'm getting ClassCastException before executing the myReader() method, Something wrong with MultiResourceItemReader not sure what's going wrong here.

Please find my code below:

                @Bean
                public MultiResourceItemReader<String> multiResourceReader()
                {
                    String bucket = "mybucket;
                    String key = "/myfiles";
                
                    List<InputStream> resourceList = s3Client.getFiles(bucket, key);
                    List<InputStreamResource> inputStreamResourceList = new ArrayList<>();
                    for (InputStream s: resourceList) {
                        inputStreamResourceList.add(new InputStreamResource(s));
                    }
            
            Resource[] resources = inputStreamResourceList.toArray(new InputStreamResource[inputStreamResourceList.size()]);
        //InputStreamResource[] resources = inputStreamResourceList.toArray(new InputStreamResource[inputStreamResourceList.size()]);
            
            // I'm getting all the stream content - I verified my stream is not null
                    for (int i = 0; i < resources.length; i++) {
                        try {
                            InputStream s  = resources[i].getInputStream();
                            String result = IOUtils.toString(s, StandardCharsets.UTF_8);
                            System.out.println(result);
                        } catch (IOException e) {
                            e.printStackTrace();
                        }
                    }
            
                    MultiResourceItemReader<String> resourceItemReader = new MultiResourceItemReader<>();
                    resourceItemReader.setResources(resources);
                    resourceItemReader.setDelegate(myReader());
                    
    resourceItemReader.setDelegate((ResourceAwareItemReaderItemStream<? extends String>) new CustomComparator()); 
                    return resourceItemReader;
                }
        
        
          

Exception:

Caused by: java.lang.ClassCastException: class CustomComparator cannot be cast to class org.springframework.batch.item.file.ResourceAwareItemReaderItemStream (CustomComparator and org.springframework.batch.item.file.ResourceAwareItemReaderItemStream are in unnamed module of loader org.springframework.boot.loader.LaunchedURLClassLoader @cc285f4)
        at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
        at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
        ... 65 common frames omitted

Can someone please help me to resolve this issue. Appreciated your help in advance. Thanks.

Solution

The reason you see the NullPointerException is due to the default comparator used by the MultiResourceItemReader to sort the resources after loading them.

The default compare behavior calls the getFilename() method of the InputStreamResource.

Refer - https://github.com/spring-projects/spring-batch/blob/115c3022147692155d45e23cdd5cef84895bf9f5/spring-batch-infrastructure/src/main/java/org/springframework/batch/item/file/MultiResourceItemReader.java#L82

But the InputStreamResource just inherits the getFileName() method from its parent AbstractResource, which just returns null. https://github.com/spring-projects/spring-framework/blob/316e84f04f3dbec3ea5ab8563cc920fb21f49749/spring-core/src/main/java/org/springframework/core/io/AbstractResource.java#L220

The solution is to provide a custom comparator for the MultiResourceItemReader. Here is a simple example, assuming you do not want to sort the resources in a specific way before processing:

public class CustomComparator implements Comparator<InputStream>{

        @Override
        public int compare(InputStream is1, InputStream is2) {
       //comparing based on last modified time
            return Long.compare(is1.hashCode(),is2.hashCode());
   }
}

MultiResourceItemReader<String> resourceItemReader = new MultiResourceItemReader<>();
resourceItemReader.setResources(resources);
resourceItemReader.setDelegate(myReader());
//UPDATED with correction - set custom Comparator
resourceItemReader.setComparator(new CustomComparator());

Refer this answer for how a Comparator is used by Spring Batch MultiResourceItemReader.

File processing order with Spring Batch

Answered By - Shankar P S
Answer Checked By - Gilberto Lyons (JavaFixing Admin)