[FIXED] How do you create multiple Spring @RestControllers each with their own thread pool?

Issue

I have a Spring RESTful service using a Tomcat web servlet that processes 2 different types of data and therefore has 2 rest controllers for each type of data. Controller #1 has the potential to perform an intensive task using lots of memory so I would like to allow up to, for instance, 10 connections on this controller. But if all 10 connections are processing on controller #1 I would also like controller #2 to have its own thread pool so it can continue processing while controller #1 is full.

The proper way to configure Tomcat is set its properties in the application.yml as described here in the spring docs. To set the total number of max connection one would use:

server.tomcat.max-connections: 10
server.tomcat.max-threads: 10

But, this configures the maximum number of connections/threads for the entire application, both controllers combined. I would need each controller to have its own thread pool and its own number of maximum connections. Is this possible?

Solution

You can't *. Spring Boot sets up an embedded Tomcat servlet container and registers a DispatcherServlet. The entire Tomcat pool of threads is used to handle all requests going through the DispatcherServlet (or any other servlets/filters registered).

* You should create a ThreadPoolTaskExecutor or ExecutorService bean for each type of data, then inject them into your @Controller beans appropriately and dispatch all the work to them.

@Controlller
class FirstController {
    private final ExecutorService threadPool;

    public FirstController(@Qualifier("first-type-data") ExecutorService threadPool) {
        this.threadPool = threadPool;
    }

    @RequestMapping("/endpoint1")
    public CompletableFuture<Foo> handleEndpoint() {
        CompletableFuture<Foo> foo = new CompletableFuture<>();
        threadPool.submit(() -> {
            // handle all your business logic
            foo.complete(...);
        });
        return foo;
    }
}

The Spring MVC "user space stack" doesn't really know about connections. You can pass around the HttpServletRequest and maintain your own count. Once you hit a threshold, you could send back an appropriate response directly without starting any of the business logic.

Answered By - Savior
Answer Checked By - Marilyn (JavaFixing Volunteer)