Apache Kafka Review

Does real-time streaming and persistence into distributed nodes. It provides a mechanism to create, publish, and subscribe.


What is most valuable?

Real-time streaming and persistence into distributed nodes. It provides a simple mechanism to create, publish, and subscribe.

How has it helped my organization?

We are using Kafka as part of our product. It is one of the messaging layers used to interact between various layers of software modules. This provides a clear separation of modules and leverages it for development and testing of different modules.

What needs improvement?

The management tools are getting mature. When we have thousands of topics, it is hard to visualize.

For how long have I used the solution?

I’ve been using Kafka for two years.

What do I think about the stability of the solution?

We have not encountered any stability issues.

What do I think about the scalability of the solution?

We have to balance the nodes when topics partition across cluster nodes. As it assumes they are of equal sizes, sometimes some nodes may not be allocated similar resources. Reassignment moves all the partitions of specified topics which may be an issue when not planned for.

How is customer service and technical support?

We have the source code to make changes if necessary.

Which solutions did we use previously?

Kafka rendered itself suitable for our product offering. It supports all the necessary requirements for a real-time pipeline.

How was the initial setup?

Setting up was easy with ZooKeeper.

What's my experience with pricing, setup cost, and licensing?

With paid support from Confluent, you get the additional benefit of Kafka Connect.

Which other solutions did I evaluate?

We used Akka Streams for faster communication, but it would require additional configuration and setup for persistence. Kafka provides those by default.

What other advice do I have?

Kafka provides distributed persistence and streaming layers. The user has flexibility in managing as a consumer on how to consume messages if they have to handle resilience in their code. It requires ZooKeeper.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Add a Comment
Guest
Sign Up with Email