[FIXED] How to create a persistent connection to Kafka through Spring Boot web application

Issue

Please excuse the ambiguity as I'm trying to be as clear as possible with the little knowledge that I have.

We have a Kafka cluster and the idea is to have a Spring Boot application to serve as the bridge between the Producers, Consumers and Kafka.

I've worked with simple REST, CRUD applications, in which I basically interact with the server on a request basis.

---

Question:

How do I go about creating a persistent connection between Spring and Kafka to send and receive messages?

Does the protocol matter?

Solution

Kafka uses its own binary protocol over TCP for publishing messages to topics for polling messages with subscriptions. For being able to connect to a Kafka cluster from a Spring Boot project, you may want to use spring-kafka which provides an implementation for this protocol:

<dependency>
  <groupId>org.springframework.kafka</groupId>
  <artifactId>spring-kafka</artifactId>
  <version>2.8.0</version>
</dependency>

A minimalist set-up for a publisher and a consumer can be found in the official spring-kafka docs.

Answered By - Ervin Szilagyi
Answer Checked By - David Goodson (JavaFixing Volunteer)