Apache Kafka Reviews

We use Kafka for event monitoring.

Everything in Kafka is amazing.

The most valuable feature for us is the event monitoring.

It is easy to configure.

This solution could be made easier to manage.

Compatibility with other solutions and integration with other tools can be improved.

We cannot apply all of our security requirements because it is hard to upload them.

We have not experienced any bugs or glitches.

It is easy to scale. This is a new project so we only have about five users right now.

I have not been in contact with technical support.

I have also used IBM MQ and Kafka is much easier to use. However, IBM MQ is better for large deployments.

The initial setup was straightforward.

We deployed this solution ourselves.

Apache Kafka is a good solution with many good features but for large deployments, I would choose IBM MQ over Kafka.

I would rate this solution a seven out of ten.

We get amazing throughput. We don't get any delay. 

The manageability should be improved. There are lots of things we need to manage and it should have a function that enables us to manage them all cohesively. There should be a default property. It's really hard to manage all these things.

I have been using Apache Kafka for around six months. 

It is really stable. The stability is really good. I just installed it on my local machine. 

It is scalable. 

I use it 24/7. We have to change both packets to Kafka and from Kafka it receives data. We retain the packet for only six seconds.

I haven't needed to contact their customer support. It's quite simple. I didn't need to contact them to manage my application. 

It's wasn't so easy for me to deploy. 

My advice would be to go through the documents and understand the topics. Learn what its effects are and take care of partitioning. 

Based on my experience, I would rate it an eight out of ten. It's quite complicated and the configuration requires a lot of effort. As a developer it is quite hard to go into all these things.

The most valuable features are topic-based eventing, scalability, and retention periods.

My organization is transforming by using the new SOA/eventing-based architecture. The application depends on the employees’ information events. Kafka is very helpful in implementing this. It increases the performance and gives the details to multiple external/internal teams using Kafka topics in an asynchronous manner.

For example, if someone is moving from one office to another one, we have to update the software. While updating it, the system puts that event in a topic so that all other consumers can update that person’s new location. This can include the payroll team, the insurance team, and the hospital network.

The retention period helps us retain the data in the topic for the configured number of days. In this example, if any of the consumers fail to consume the message from the topic, then that message will be there until the retention period ends.

I would like to see a more user-friendly GUI.

We have used this solution since December, 2015.

If you are using the same group ID for multiple topics, it may shut down the application. We have faced this issue before.

We have not had any scalability issues.

I would give technical support a rating of 6 out of 10.

We were using ActiveMQ, which is just a messaging system. We are changing because of Kafka’s added value of scalability, retention, and high payload support.

The installation was somewhat straightforward.

The solution is worth the money.

This is the best tool I have ever used for asynchronous, event-based solutions.

• Supports more than 10,000 events/second.
• Scalability
• Replication

It is a good product for event-driven architecture.

We use Kafka for reactive architecture, track and trace, mail and parcel.

A good free monitor tool would be great for Apache Kafka (from Apache foundation).

We used Kafka 0.8 for 2 years and Kafka 0.10 for 3 months.

We have not encountered any stability issues.

We have not encountered any scalability issues.

We haven’t used technical support.

Apache MQ is different. It is a message bus (log rotate) than can manage more than 10,000 events/sec.

The basic configuration is quite good. We have built a Hadoop cluster and the Kafka service was included.

We use a community version.

Kafka processes asynchronous exchanges, so there are no transactional interactions.

I am a user, as well as an integrator for our clients. This is one of the products that we implement for others.

The most valuable features of this solution is the architectural style of messaging or event streaming. First is important to understand that anything can be represented as an event=message e.g new order, status change, confirmation, information from IoT or from monitoring system. Each message can be transport very quickly from the source (producer) into consumer(s). Messages can be changed in the fly - streaming messaging.

Messages can be process exactly at once or at least once. Ordering of messages depends on just configuration setup. 

When comparing it with other messaging and integration platforms, this is one of the best rated. Message store is out of the box functionality. Messages are automatically stored based on parameter setup of retention policy. Messages can remain for longer, which can be configured from a few milliseconds up to years. Scalability and availability of messages can be changed with zero maintenance window.

You don't need extra clusters to achieve high availability for the messaging system like Veritas, PowerHA or other.

One platform for classical messaging, real time messaging, ETL, message streaming. 

ETL can be realised with connectors into external systems which also could run in more instances. Exists a lot of ready to use connectors and new ones can be developed.

The model where you create the integration or the integration scenario needs improvement. It contains fewer developer words or maintaining words where someone prepared the topics, the connectors, or the streaming platforms. You would first need to have a control center from a third party for managing. 

If you would like to prepare something that is a more sophisticated integration scenario, where you use one microservice to provide the event or a second to several that consumed these microservices, then this needs to be modeled elsewhere. 

Also, when comparing to the traditional ESD for data mixing, you can create a scenario that could be deployed with inputs and some outputs.

Most business like the topics, but for me, I think that it is a problem that messaging platforms have, there is no design tool with IDE for creating.

It would be helpful to create a more complex solution for several types of styles, and not just for one provider or for one customer. That would be easier, but if you have more than one consumer then it could be a more complex scenario. It would be like events that go to several microservers to create orders, validate orders, and creating words. This would be helpful.

In the next release, adding some IDE or developing tools, for creating better integration scenarios, even though it already a developer-oriented solution, would be helpful. It would also be helpful for the auto-deployment.

Having a governance style would also be helpful to understand. 

It would be beneficial to have a repository of all of the topics, data types that exist, or data structures.

I have been working with this solution for one year.

It's a stable solution.

Kafka is very scalable, which is an important feature of it.

Our clients have approximately ten applications in their companies that communicate with Kafka.

I have not contacted technical support through Kafka, I communicate directly with Confluence. Confluence is the company that developed the open-source platform and they provide support.

The communication is very good and they are very capable of assisting you with all technical inquiries.

There are direct contacts that make it easy.

Previously, I worked with IBM MQ, a different type of messaging platform.

For the most part, the initial setup is easy, but if you need a more sophisticated infrastructure or if you have to set up the topics, then you have to be careful and you have to be more knowledgable in Kafka. You will have to know the parameters for the rotations, the size of the message, and the timeouts, as an example.

For a developer it is easy, but for an administrator and production, it requires more.

Apache Kafka is an open-source solution and there are no fees, but there are fees associated with confluence, which are based on subscription.

I would recommend trying this solution. 

Take the time to understand it because it is a different style when it comes to working with data.

I would rate this solution a nine out of ten.

We are currently using this solution on our cloud-based clusters.

We use Kafka as part of our services. Our product (cloud clusters) has many components and Kafka is one of them.

For example, we use Kafka as a data integration tool. If you take Oracle GoldenGate as a typical use case, what happens is GoldenGate collects the data for the replication and sends this data to the Kafka servers. We collect the data on the Kafka servers, and we create some transformations, some operations, from that data. We then copy the data to the HTTP or hub site.

Previously, when I worked at Nokia, we were collected data using Kafka and then we stored the data on the Kafka servers. We did all transformations through Kafka streaming. Later, Kafka moved data over to the HP site. 

Kafka has a good storage layer on its side. I can store this data if it's streaming, and, if we do encounter any error, for example, on the network or server, we can later use the data to do some analytics on it using the Kafka server.

Kafka provides us with a way to store the data used for analytics. That's the big selling point. There's very good log management. 

Kafka provides many APIs that can be flexible and can be placed or expanded using the development life cycle. For example, using Java, I can customize the API according to our customers' demands. I can expand the functionality according to our customer demands as well. It's also possible to create some models. It allows for more flexibility than much of the competition.

If the graphical user interface was easier for the Kafka administration it would be much better. Right now, you need to use the program with a command-line interface. If the graphical user interface was easier, it could be a better product.

I've been using the solution for more than three years.

The solution can be quite stable. We haven't encountered any issues on the Kafka side. However, Creating custom stabilizations would be good for dealing with stabilizing issues.

The scalability of the solution is very good. You can analyze system events horizontally and the cluster can be brought over to the cloud side with the Kafka user's server.

We use the solution for both small and medium-sized organizations, but also larger enterprises. Some of our clients are in the banking and financial sector.

Officially, I did not create any Kafka support tasks on the configuration support that is offered. I have created some questions on the stack overflow, however. Technical support is very good and I've found their response is very quick, giving you an answer within a day.

We didn't previously use a different solution. We did some applications with Java for the consumer content but not the application function within that. We did objects instead.

The initial setup isn't too complex. I know Kafka very well and don't find it to be overly difficult. There's also very good documentation which users can take advantage of.

Deployment, including security integration, only took about one day.

Two people handled the deployment. One person created the authentification group and after creating groups and users, another handled topic authentification and user definition for the customer.

I handled the implementation for cloud-based clusters. I defined the broker nodes and other nodes for Kafka. We are a cloud integrator, so we handled it ourselves.

I'm unaware of the costs surrounding licensing and setup.

We're using the 2.1.30 version of the solution for our cloud-based clusters. We use the on-premises deployment model. Most customers use the on-premise solution for cloud-based clusters.

Kafka is a very good solution for log management. If you need anything done related to log management, Kafka can do it. Kafka can also store the data in the brokers. This prevents data loss as well as the duplication of data. It's quite comprehensive.

I'd rate the solution seven out of ten. If the solution could provide a user interface I'd rate it higher. This is important for managing Kafka's clusters on the administration side. It would also be helpful if two to three files could be minimized to one configuration file.

I like Kafka's flexibility, stability, reliability, and robustness.

Kafka has a lot of monitors, but sometimes it's most important to just have a simple monitor. Improvements to Kafka's management would be nice, but it's not so necessary for me. There are a lot of consoles that offer a better view than Kafka. Some are free, and some are paid, but I'm thinking about streaming. For example, if you connect more streams to a component in the same queue, how will it integrate to recognize the flow and the message?

I've been using Kafka for more than two years.

Kafka is stable. Defining our user base is hard because Kafka influences the whole company, so you could say around 100 users. Kafka is a core system, so it affects all users we choose to link to the primary key.

I previously used IBM MQ. 

The first time we tried to deploy Kafka, it seemed a little complicated, but the second try went better. Sometimes it isn't easy to set up the necessary communication or estimate how many partitions we need. Some applications have a vast amount of data, so we have to consider how to improve the performance and not increase the transfer times.

I rate Apache Kafka nine out of 10. I think it's one of the best tools on the internet.

• Distributed
• Persistence
• Offset management by consumer

This is the base streaming component of our IoT platform.

In case of disaster recovery, we mirror the data in the cluster by maintaining the offsets and store the data within Hadoop 2.8 HDFS.

• It needs a separate cluster and a separate administrator to manage the Kafka cluster, adding an extra cost.
• It is challenging when data is moved to a mirror cluster, in the case of disaster recovery. It doesn't keep the offset.

I have used this solution for one year.

The open source community is very strong. Also, distributors like Cloudera and Hortonworks provide paid support.

For big data, we did not have a previous solution. I have used Microsoft MQ for building traditional systems.

The setup was straightforward.

This is open source with the cost of a cluster administrator.

We did not look at anything else. At that time, this was already accepted by the industry for streaming data processing.

If the Hadoop distribution is MapR, then consider MapR Streaming. MapR Streaming has overcome these fundamental issues. It stores data within the MapR-FS itself. So there is extra overhead, but with a licensing cost.