Blog Archives

01: 15+ Apache Kafka must-know basics interview Q&As – Part 1

Apache Kafka is used in Micro Services Architecture (i.e. MSA) to Big Data & Low Latency application architectures.

Q01. What is Apache Kafka?
A01. Apache Kafka is a message broker like ActiveMQ, IBM WebSphere MQ, RabitMQ, etc. Apache Kafka is distributed messaging broker with the purpose of being a super fast data high way for moving data from source systems to target systems.… Read more ...



01: 15+ Apache Kafka must-know basics interview Q&As – Part 2

This extends 8 Apache Kafka must-know basics interview Q&As – Part 1. Q07. What do you understand by the term “data is presented to Kafka as stream”? A07. This means either the Data is acquired from source systems in real time or as a scheduled extract process, the data is…

Read more ...


01: 15+ Apache Kafka must-know basics interview Q&As – Part 3

This extends Apache Kafka must-know basics interview Q&As – Part 2. Q15. What do you understand by the terms Kafka Consumer Groups & group.id? A15. Consumers read from any single partition, allowing you to scale throughput of message consumption as depicted below. Consumers can also be organised into consumer groups…

Read more ...


01: Apache Kafka example with Java – getting started tutorial

Apache Kafka with Java getting started tutorial demonstrates how quickly you can get started with Kafka using Docker.

Step 1: Make sure Docker engine is installed on your computer. For example on a Mac OS $ brew cask install docker or on Windows.

Step 2: Start the Docker engine on your operating system.

Kafka services on Docker

Step 3: Create the below docker-compose.ymlRead more ...



02: Apache Kafka example with Java Producer & Consumer Tutorial

Apache Kafka with Java getting started tutorial demonstrates how quickly you can get started with Kafka using Docker. This extends Apache Kafka example with Java – getting started tutorial – Part 1.

Step 1: As discussed in part-1, stand-up the Kafka, Zookeeper & Apache Kafka Cluster Visualization (AKHQ) on a Docker container.

Java Producer class ProducerApp.java

Read more ...


03: Apache Kafka example with toxiproxy & Producer timeout handling

This extends Apache Kafka example with Java Producer & Consumer Tutorial – Part 2. This covers Apache Kafka example with toxiproxy & producer timeout. The producer code needs to be robust to handle any network issues like timeouts & latency issues. In this tutorial a tool named toxiproxy is used to simulate network issues.

Kafka server on Docker

Step 1: As discussed in part-1, stand-up the Kafka, Zookeeper & Apache Kafka Cluster Visualization (AKHQ) on a Docker container.… Read more ...



05: Apache Kafka Streaming with JSON & Java Tutorial

This extends Apache Kafka JSON example with Java Producer & Consumer Tutorial. In this tutorial, let’s look at KafkaStreams, which enables you to consume from Kafka topics, analyse, transform or aggregate data, and potentially, send it to another Kafka topic.

This kafka tutorial is stateless, which means there is no interaction between individual messages. In the ongoing tutorials we will look at stateful processing like count, aggregate, windowing, etc.… Read more ...



06: Apache Kafka Streaming with JSON & Java Tutorial – stateful operations groupByKey, windowing & aggregate

This extends Apache Kafka Streaming with JSON & Java Tutorial and you can either use Kafka on Docker as per Apache Kafka example with Java – getting started tutorial or install Kafka on Mac (i.e Getting started with Apache Kafka on Mac Tutorial) or Windows. This kafka tutorial is stateful…

Read more ...


07: Apache Kafka Streaming with stateless operations

This extends 06: Apache Kafka Streaming with JSON & Java Tutorial – stateful operations groupByKey, windowing & aggregate and Getting started with Apache Kafka on Mac Tutorial.

Kafka Streams DSL (i.e. Domain Specific Language) is built on top of the Streams Processor API. It is recommended to use DSL as most data processing operations can be expressed in just a few lines of DSL code.… Read more ...



Java Developer & Architect Q&As

Big Data Engineer & Architect Q&As

16+ Key Areas & 13+ Techs to fast-track