[FIXED] Can't connect to my kafka running on docker from spring boot application running via intellij

Issue

I have a docker-compose file with Kafka, zookeeper, and spring boot application. while I run the entire file everything works fine. when I run it without my spring boot application in order to debug it via intellij It cannot connect to Kafka and doesn't work properly.

my docker-compose file:

 version: "3.5" services:   # Install Zookeeper.   zookeeper:
    container_name: zookeeper
    image: debezium/zookeeper:1.2
    networks:
      - mynetwork
    ports:
      - 2181:2181
      - 2888:2888
      - 3888:3888   # Install Kafka.   kafka:
    container_name: kafka
    image: debezium/kafka:1.2
    depends_on:
      - zookeeper
    ports:
      - 9092:9092
      - 29092:29092
    networks:
      - mynetwork
    extra_hosts:
      - "host.docker.internal:host-gateway"
    environment:
      - ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP= INTERNAL:PLAINTEXT,PLAINTEXT:PLAINTEXT,EXTERNAL_SAME_HOST:PLAINTEXT
      - KAFKA_ADVERTISED_LISTENERS= INTERNAL://kafka:9092,EXTERNAL_SAME_HOST://localhost:29092
      - KAFKA_LISTENERS= EXTERNAL_SAME_HOST://:29092,INTERNAL://:9092
      - KAFKA_INTER_BROKER_LISTENER_NAME= PLAINTEXT   # Install Postgres.   postgres:
    container_name: postgres
    image: debezium/postgres:12
    volumes:
      - ./sql/init.sql:/docker-entrypoint-initdb.d/init.sql
    ports:
      - 5432:5432
    networks:
      - mynetwork
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=postgres   kafka-ui:
    container_name: kafka-ui
    image: provectuslabs/kafka-ui:0.2.1
    ports:
      - 8080:8080
    networks:
      - mynetwork
    environment:
      - KAFKA_CLUSTERS_0_NAME=local
      - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:9092   #Deploy a Consumer.   consumer:
    build:
      context: .
    container_name: pledge-consumer
    environment:
      - SPRING_DATASOURCE_URL=jdbc:postgresql://postgres:5432/postgres
    ports:
      - 8101:8080
    networks:
      - mynetwork
    image: isber/ssm-pledgeservice:v1
    depends_on:
      - zookeeper
      - kafka
      - postgres

networks:   mynetwork:
    external: true

In the application I tried:

spring.kafka.bootstrap-servers=kafka:9092

which works when I run it via docker but not from intellij

I also tried when running with intellij:

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.bootstrap-servers=localhost:29092

Solution

I found the problem, the image I used:

image: debezium/kafka:1.2

had a problem and it didn't read any of the parameters of the environment I added. I upgraded to:

image: debezium/kafka:1.4

everything works.

Answered By - israel berko
Answer Checked By - Mildred Charles (JavaFixing Admin)