An example developed using Spring Boot 3. This example explained about spring boot 3 with kafka with multi project setup. Also will split such as: common, producer and consumer.
Eang Sopheaktra
April 06 2024 02:05 pm
Message queues play a crucial role in modern distributed systems. Let’s explore their benefits:
Better Performance:
Increased Reliability:
Granular Scalability:
Simplified Decoupling:
Apache Kafka is a powerful distributed streaming platform that has gained popularity for various use cases. Let’s explore why you might consider using Kafka:
Real-Time Data Streams:
Durability and Fault Tolerance:
Scalability and High Throughput:
Guaranteed Order and At-Least-Once Delivery:
Complex Event Processing (CEP):
Internet of Things (IoT) and Automation:
Scalability:
Durability:
High Performance:
Real-Time Data Streaming:
Complexity:
Operational Overhead:
Message Ordering:
Message Size Limitations:
Dependency on Zookeeper:
The fully fledged server uses the following:
There are a number of third-party dependencies used in the project. Browse the Maven pom.xml file for details of libraries and versions used.
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
On this step I will not introduce more specific on how to create multiple project while it depends on your IDE or sometimes manaul by hard copy and patse lol.
package com.tra22.kafka.dto;
import lombok.*;
import java.io.Serializable;
@Getter
@Setter
@Builder
@ToString
@NoArgsConstructor
@AllArgsConstructor
public class OrderDto implements Serializable {
private int orderId;
private double price;
}
package com.tra22.kafka.producer;
import com.tra22.kafka.dto.OrderDto;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;
@Service
@Slf4j
@RequiredArgsConstructor
public class KafkaProducerService {
private final KafkaTemplate<String, OrderDto> kafkaTemplate;
public void send(String topicName, String key, OrderDto value) {
var future = kafkaTemplate.send(topicName, key, value);
future.whenComplete((sendResult, exception) -> {
if (exception != null) {
future.completeExceptionally(exception);
log.error("error: ", exception);
} else {
future.complete(sendResult);
log.info("result: {}", sendResult);
}
log.info("order producer : {} ", value);
});
}
}
server:
port: 8089
spring:
kafka:
bootstrap-servers: localhost:9092
# producer config
producer:
key-serializer: org.apache.kafka.common.serialization.StringSerializer
value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
package com.tra22.kafka;
import com.tra22.kafka.dto.OrderDto;
import com.tra22.kafka.producer.KafkaProducerService;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
@SpringBootApplication
//@ComponentScan(basePackages = {"com.tra22.common"})
public class Main {
public static void main(String[] args) {
SpringApplication.run(Main.class, args);
}
@Bean
public CommandLineRunner commandLineRunner(
KafkaProducerService kafkaTemplate
) {
return args -> {
kafkaTemplate.send("order-topic", "orderId", OrderDto.builder().orderId(1).price(100.23).build());
};
}
}
server:
port: 8082
spring:
kafka:
bootstrap-servers: localhost:9092
# consumer config
consumer:
key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
value-deserializer: org.springframework.kafka.support.serializer.JsonDeserializer
properties:
spring.json.trusted.packages: "*"
MessagingMessageListenerAdapter
configured with various features, such as converters to convert the data, if necessary, to match the method parameters.package com.tra22.kafka.event;
import com.tra22.kafka.dto.OrderDto;
import lombok.extern.slf4j.Slf4j;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;
@Component
@Slf4j
public class KafKaTopicListeners {
@KafkaListener(topics = {"order-topic"}, groupId = "order-group")
public void consume(OrderDto orderDto) {
log.info("order consumer : {} ", orderDto);
}
}
You will need:
Clone the project and use Maven to build the server
$ mvn clean install
Download the source code for the sample application implementing an Producer and Consumer. Also you will learn with multi module project on spring boot 3 (Maven Project) below: