What is most valuable?
- Message Retention: Unlike regular message queues, messages stay in Kafka after clients consume them. A message can be consumed over and over again by the same or a different client until topic retention (by max data size or oldest message timestamp) kicks in and the oldest messages get deleted. This can be very handy in many scenarios: handling bugs in software, testing code, simple distribution of message processing, and routing messages to many different consumers simultaneously.
- Horizontal Scalability: To add more capacity, both in terms of storage and performance to a Kafka cluster, you just need to add more servers. Regular message queues usually work in a master-slave configuration and do not scale very well horizontally.
- Simplicity in operations.
How has it helped my organization?
It has become dead simple to connect different application and services, saving a lot of development hours.
What needs improvement?
The standard Kafka Java library, which is shipped with the product, is too complex for inexperienced users. At my company, engineering teams ended up writing wrapper libraries to solve complex issues. Kafka client libraries in general are complex, regardless of language. This is the price Kafka users have to pay for having simple, yet robust, server-side code.
What could be improved is the hard dependency on ZooKeeper. The work in this direction has already been started, though. Overall, the project is moving forward at a very good pace
For how long have I used the solution?
I have used Kafka for three years.
What do I think about the stability of the solution?
Sometimes we have stability issues, but not often.
What do I think about the scalability of the solution?
We have not had any scalability issues.
How is customer service and technical support?
There is no official technical support as the product is 100% open source.
Which solutions did we use previously?
We used RabbitMQ before. It does not scale well.
How was the initial setup?
The setup was pretty straightforward.
What's my experience with pricing, setup cost, and licensing?
There is no pricing and licensing.
Which other solutions did I evaluate?
We didn't evaluate any other options.
What other advice do I have?
Go ahead. It's a great product.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
May 24 2017