Apache Kafka Review
Does real-time streaming and persistence into distributed nodes. It provides a mechanism to create, publish, and subscribe.

Valuable Features:

Real-time streaming and persistence into distributed nodes. It provides a simple mechanism to create, publish, and subscribe.

Improvements to My Organization:

We are using Kafka as part of our product. It is one of the messaging layers used to interact between various layers of software modules. This provides a clear separation of modules and leverages it for development and testing of different modules.

Room for Improvement:

The management tools are getting mature. When we have thousands of topics, it is hard to visualize.

Use of Solution:

I’ve been using Kafka for two years.

Stability Issues:

We have not encountered any stability issues.

Scalability Issues:

We have to balance the nodes when topics partition across cluster nodes. As it assumes they are of equal sizes, sometimes some nodes may not be allocated similar resources. Reassignment moves all the partitions of specified topics which may be an issue when not planned for.

Technical Support:

We have the source code to make changes if necessary.

Previous Solutions:

Kafka rendered itself suitable for our product offering. It supports all the necessary requirements for a real-time pipeline.

Initial Setup:

Setting up was easy with ZooKeeper.

Cost and Licensing Advice:

With paid support from Confluent, you get the additional benefit of Kafka Connect.

Other Solutions Considered:

We used Akka Streams for faster communication, but it would require additional configuration and setup for persistence. Kafka provides those by default.

Other Advice:

Kafka provides distributed persistence and streaming layers. The user has flexibility in managing as a consumer on how to consume messages if they have to handle resilience in their code. It requires ZooKeeper.

Disclosure: I am a real user, and this review is based on my own experience and opinions.


Anonymous avatar x30

Have A Question About Apache Kafka?

Our experts can help. 228,931 professionals have used our research on 5,837 solutions.
Why do you like it?

Sign Up with Email