What is most valuable?
The most valuable features to me are replication, partitioning and easy integration with Apache Spark, which we use quite a bit for distributed processing.
Replication is good for high availability. It provides additional safety for data in case of node failure or data center outage. Partitioning is a really useful feature for parallelizing processing. We use Apache Spark to process data from a Kafka queue, and Spark is able to assign one executor to each Kafka partition. The more partitions we have, the more threads we can use to process data in parallel. This helps us achieve really good throughput.
How has it helped my organization?
It will help us build a scalable platform. This will allow the company to provide better customer service.
What needs improvement?
It’s pretty easy to use for now. I haven’t had any difficulty or problems that I can complain about. Maybe they can add a UI to the configure queues and to display statistics about data stores.
For how long have I used the solution?
I have used Kafka for about a year.
What do I think about the stability of the solution?
So far, we have not encountered any stability issues.
What do I think about the scalability of the solution?
We have not had any scalability issues. The product is horizontally scalable, so adding extra hardware is all that is needed.
How are customer service and technical support?
We haven’t needed technical support with the product yet.
Which solution did I use previously and why did I switch?
I think performance-wise, the product is very good and fits in our use case. We used other distributed message queues, but all products have their own use case
How was the initial setup?
Initial setup wasn’t really complex. We use Kafka through Hortonworks Suite, which comes with many other big data tools. Ambari makes it easy to setup
What's my experience with pricing, setup cost, and licensing?
Licensing and pricing was handled by my management, so I don’t have much knowledge there.
What other advice do I have?
Give it a try. It’s a valuable, high-performance, distributed processing tool.