What is our primary use case?
Our company provides services and we use Apache Kafka as part of the solution that we provide to clients.
One of the use cases is to collect all of the data from multiple endpoints and provide it to the users. Our application integrates with Kafka as a consumer using the API, and then sends information to the users who connect.
What is most valuable?
The most valuable feature is that it can handle high volume.
Apache Kafa is open-source and some of our clients are interested in becoming more involved in that.
What needs improvement?
Kafka does not provide control over the message queue, so we do not know whether we are experiencing lost or duplicate messages. Better control over the message queue would be an improvement. Solutions such as ActiveMQ do afford better control. Because of this, there is sometimes a gap in the results where we have either lost messages, or there are duplicates.
We have had problems when there was an imbalance because all of the messages were being sent back.
For how long have I used the solution?
I'm a beginner with Apache Kafka.
What do I think about the stability of the solution?
I cannot judge stability without having better control over the message queue, although I feel that it is not 100% stable.
How are customer service and technical support?
We have not been in contact with technical support. For our first implementation with it, Kafka was already set up and running. When we did our PoC, I was not part of the team who was facing issues and it was they who were in contact with support.
Which solution did I use previously and why did I switch?
I also have experience with IBM MQ.
How was the initial setup?
We had problems when we were setting up Kafka ourselves to conduct our PoC internally. Kafka would not start and it was related to parameters or property settings in Java. We were able to work around it, but we had problems like adding certificates.
What about the implementation team?
In one case, we were using Kafka after it had already been set up, externally. It worked fine and we just had to configure some of the connectors that we wanted to try out.
What's my experience with pricing, setup cost, and licensing?
Apache Kafka is open-source and can be used free of charge.
What other advice do I have?
In this type of solution, you need to be able to accept a high volume of messages, but not lose any, and not have any duplicates. Because we are unable to control the queue in Kafka, I cannot say that this works 100%.
The suitability of this solution depends on the use cases. There are two or three things that we are worried about, and we will be very careful in choosing solutions. In cases where the messages are well organized, or there is no worry that there will be duplicate or dropped messages, then I recommend using Kafka. Also, I recommend this solution for those looking to get involved with open-source applications.
Other than the problems with having no control over the queue, Apache Kafka is wonderful.
I would rate this solution an eight out of ten.
Which deployment model are you using for this solution?