Apache Kafka Review

Open source, granular message retention options, and good third party support

What is our primary use case?

I am a user, as well as an integrator for our clients. This is one of the products that we implement for others.

What is most valuable?

The most valuable features of this solution is the architectural style of messaging or event streaming. First is important to understand that anything can be represented as an event=message e.g new order, status change, confirmation, information from IoT or from monitoring system. Each message can be transport very quickly from the source (producer) into consumer(s). Messages can be changed in the fly - streaming messaging.

Messages can be process exactly at once or at least once. Ordering of messages depends on just configuration setup. 

When comparing it with other messaging and integration platforms, this is one of the best rated. Message store is out of the box functionality. Messages are automatically stored based on parameter setup of retention policy. Messages can remain for longer, which can be configured from a few milliseconds up to years. Scalability and availability of messages can be changed with zero maintenance window.

You don't need extra clusters to achieve high availability for the messaging system like Veritas, PowerHA or other.

One platform for classical messaging, real time messaging, ETL, message streaming. 

ETL can be realised with connectors into external systems which also could run in more instances. Exists a lot of ready to use connectors and new ones can be developed.

What needs improvement?

The model where you create the integration or the integration scenario needs improvement. It contains fewer developer words or maintaining words where someone prepared the topics, the connectors, or the streaming platforms. You would first need to have a control center from a third party for managing. 

If you would like to prepare something that is a more sophisticated integration scenario, where you use one microservice to provide the event or a second to several that consumed these microservices, then this needs to be modeled elsewhere. 

Also, when comparing to the traditional ESD for data mixing, you can create a scenario that could be deployed with inputs and some outputs.

Most business like the topics, but for me, I think that it is a problem that messaging platforms have, there is no design tool with IDE for creating.

It would be helpful to create a more complex solution for several types of styles, and not just for one provider or for one customer. That would be easier, but if you have more than one consumer then it could be a more complex scenario. It would be like events that go to several microservers to create orders, validate orders, and creating words. This would be helpful.

In the next release, adding some IDE or developing tools, for creating better integration scenarios, even though it already a developer-oriented solution, would be helpful. It would also be helpful for the auto-deployment.

Having a governance style would also be helpful to understand. 

It would be beneficial to have a repository of all of the topics, data types that exist, or data structures.

For how long have I used the solution?

I have been working with this solution for one year.

What do I think about the stability of the solution?

It's a stable solution.

What do I think about the scalability of the solution?

Kafka is very scalable, which is an important feature of it.

Our clients have approximately ten applications in their companies that communicate with Kafka.

How are customer service and technical support?

I have not contacted technical support through Kafka, I communicate directly with Confluence. Confluence is the company that developed the open-source platform and they provide support.

The communication is very good and they are very capable of assisting you with all technical inquiries.

There are direct contacts that make it easy.

Which solution did I use previously and why did I switch?

Previously, I worked with IBM MQ, a different type of messaging platform.

How was the initial setup?

For the most part, the initial setup is easy, but if you need a more sophisticated infrastructure or if you have to set up the topics, then you have to be careful and you have to be more knowledgable in Kafka. You will have to know the parameters for the rotations, the size of the message, and the timeouts, as an example.

For a developer it is easy, but for an administrator and production, it requires more.

What's my experience with pricing, setup cost, and licensing?

Apache Kafka is an open-source solution and there are no fees, but there are fees associated with confluence, which are based on subscription.

What other advice do I have?

I would recommend trying this solution. 

Take the time to understand it because it is a different style when it comes to working with data.

I would rate this solution a nine out of ten.

Which deployment model are you using for this solution?

**Disclosure: My company has a business relationship with this vendor other than being a customer: partner
More Apache Kafka reviews from users
...who work at a Financial Services Firm
...who compared it with IBM MQ
Add a Comment