Issue
My understanding was that spring-kafka was created to interact with Kafka Client APIs, and later on, spring-cloud-stream project was created for "building highly scalable event-driven microservices connected with shared messaging systems", and this project includes a couple of binders, one of them is a binder that allows the interaction with Kafka Stream API:
spring-cloud-stream-binder-kafka-streams
So it was clear to me that if I want to interact with Kafka Stream API, I will use the spring-cloud-stream approach with the appropriate binder.
But, I found out that you can interact with Kafka Stream API also with the spring-kafka approach.
Need the below two dependencies. One example is here.
'org.springframework.kafka:spring-kafka'
'org.apache.kafka:kafka-streams'
So my question is - if both the approaches allow interaction with Kafka Stream API, what are the differences between the approaches?
Solution
As Gary pointed out in the comments above, spring-kafka is the lower level library that provides the building blocks for Spring Cloud Stream Kafka Streams binder (spring-cloud-stream-binder-kafka-streams). The binder provides a programming model using which you can write your Kafka Streams processor as a java.util.funciton.Funciton or java.util.function.Consumer. You can have multiple such functions and each of them will build its own Kafka Streams topologies. Behind the scenes, the binder uses Spring-Kafka to build the Kafka Streams StreamsBuilder objet using the StreamsBuilderFactoryBean. Binder also allows you to compose various functions. The functional model comes largely from Spring Cloud Function, but it is adapted for the Kafka Streams in the binder implementation. The short answer is that both spring-Kafka and the Spring Cloud Stream Kafka Streams binder will work, but the binder gives a programming model and extra features consistent with Spring Cloud Stream whereas spring-kafka gives various low-level building blocks.
Answered By - sobychacko
