Apache Kafka Overview

Apache Kafka is the #2 ranked solution in our list of top Message Queue Software. It is most often compared to IBM MQ: Apache Kafka vs IBM MQ

What is Apache Kafka?

Apache Kafka is a distributed streaming platform, with the following capabilities:

  • It lets you publish and subscribe to streams of records. In this respect it is similar to a message queue or enterprise messaging system.
  • It lets you store streams of records in a fault-tolerant way.
  • It lets you process streams of records as they occur.

Apache Kafka gets used for two broad classes of application:

  • Building real-time streaming data pipelines that reliably get data between systems or applications.
  • Building real-time streaming applications that transform or react to the streams of data.
Apache Kafka Buyer's Guide

Download the Apache Kafka Buyer's Guide including reviews and more. Updated: June 2021

Apache Kafka Customers
Uber, Netflix, Activision, Spotify, Slack, Pinterest
Apache Kafka Video

Pricing Advice

What users are saying about Apache Kafka pricing:
  • "Apache Kafka is an open-source solution and there are no fees, but there are fees associated with confluence, which are based on subscription."
  • "The solution is open source; it's free to use."
  • "Apache Kafka is open-source and can be used free of charge."
  • "Kafka is open-source and it is cheaper than any other product."

Filter Reviews

Filter by:
Filter Reviews
Industry
Loading...
Filter Unavailable
Company Size
Loading...
Filter Unavailable
Job Level
Loading...
Filter Unavailable
Rating
Loading...
Filter Unavailable
Considered
Loading...
Filter Unavailable
Order by:
Loading...
  • Date
  • Highest Rating
  • Lowest Rating
  • Review Length
Search:
Showingreviews based on the current filters. Reset all filters
RH
Enterprise Architect at a tech services company with 11-50 employees
Real User
Top 5Leaderboard
Open source, granular message retention options, and good third party support

What is our primary use case?

I am a user, as well as an integrator for our clients. This is one of the products that we implement for others.

Pros and Cons

  • "When comparing it with other messaging and integration platforms, this is one of the best rated."
  • "The model where you create the integration or the integration scenario needs improvement."

What other advice do I have?

I would recommend trying this solution. Take the time to understand it because it is a different style when it comes to working with data. I would rate this solution a nine out of ten.
OnurTokat
Senior Big Data Developer | Cloudera at Dilisim
Real User
Good scalability and excellent for storing data used for analytics but lacks a user interface

What is our primary use case?

We are currently using this solution on our cloud-based clusters.

Pros and Cons

  • "Kafka provides us with a way to store the data used for analytics. That's the big selling point. There's very good log management."
  • "If the graphical user interface was easier for the Kafka administration it would be much better. Right now, you need to use the program with the command-line interface. If the graphical user interface was easier, it could be a better product."

What other advice do I have?

We're using the 2.1.30 version of the solution for our cloud-based clusters. We use the on-premises deployment model. Most customers use the on-premise solution for cloud-based clusters. Kafka is a very good solution for log management. If you need anything done related to log management, Kafka can do it. Kafka can also store the data in the brokers. This prevents data loss as well as the duplication of data. It's quite comprehensive. I'd rate the solution seven out of ten. If the solution could provide a user interface I'd rate it higher. This is important for managing Kafka's clusters on the…
Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: June 2021.
512,221 professionals have used our research since 2012.
KS
Solution Architect at a manufacturing company with 10,001+ employees
Real User
Top 20
Good performance when a high throughput is required, but they need to implement a portal

What is our primary use case?

I am a solution architect and I used Apache Kafka in this role.

Pros and Cons

  • "The processing power of Apache Kafka is good when you have requirements for high throughput and a large number of consumers."
  • "They need to have a proper portal to do everything because, at this moment, Kafka is lagging in this regard."

What other advice do I have?

I think that many people are using Apache Kafka just as a publishing and subscription model, but I feel that Kafka is better than that. Furthermore, Confluent Kafka is even more than that. Confluent Kafka is offering features that are equal to those of a data lake. You can do lots with data, and huge data can be persisted. However, many people are not using that feature. Rather than make use of persistence logic, they are pushing the messages and consuming them. Maybe if people were using it for persistence, they would see the impact or real power of Kafka. I would rate this solution a seven…
MS
Senior Consultant at a tech services company with 51-200 employees
Consultant
Top 10
Stable, free to use, and offers good stream processing

What is our primary use case?

Apache Kafka is used for stream processing, metric and log aggregations, and as a message queue for connecting different microservices.

Pros and Cons

  • "The stream processing is a very valuable aspect of the solution for us."
  • "The solution could always add a few more features to enhance its usage."

What other advice do I have?

What happens in our company is a little different. We basically provide services to other companies through Kafka, like our management services. It doesn't necessarily mean we're using the solution ourselves, however, we will be going and deploying Kafka for companies, like a systems integrator. The version of the solution is normally 2.4, however, it depends on the requirements. Our cloud providers are always different due to the fact that the countries that we work with are all different. For example, in the US it could Amazon, Azure, or Google. It varies. I'd advise other organizations…
JJ
Technology Lead at a computer software company with 10,001+ employees
Vendor
A cost-effective solution for high volume, multi-source data collection

What is our primary use case?

Our company provides services and we use Apache Kafka as part of the solution that we provide to clients. One of the use cases is to collect all of the data from multiple endpoints and provide it to the users. Our application integrates with Kafka as a consumer using the API, and then sends information to the users who connect.

Pros and Cons

  • "The most valuable feature is that it can handle high volume."
  • "Kafka does not provide control over the message queue, so we do not know whether we are experiencing lost or duplicate messages."

What other advice do I have?

In this type of solution, you need to be able to accept a high volume of messages, but not lose any, and not have any duplicates. Because we are unable to control the queue in Kafka, I cannot say that this works 100%. The suitability of this solution depends on the use cases. There are two or three things that we are worried about, and we will be very careful in choosing solutions. In cases where the messages are well organized, or there is no worry that there will be duplicate or dropped messages, then I recommend using Kafka. Also, I recommend this solution for those looking to get involved…
MoulaliNaguri
Project Engineer at Wipro Technologies
Real User
Free to use, mature, and offers good scalability

What is our primary use case?

We primarily use the solution for big data. We often get a million messages per second, and with such a high output we use Kafka to help us handle it.

Pros and Cons

  • "It's an open-source product, which means it doesn't cost us anything to use it."
  • "The UI is based on command line. It would be helpful if they could come up with a simpler user interface."

What other advice do I have?

I'd rate the solution eight out of ten. It's good at scaling, and, performance-wise, it's excellent. If they could add upon the UI and allow for easier configuration, I'd rate them higher.
LP
Owner at Binarylogicworks.com.au
Real User
Good performance and resilience, but it is complex and has a learning curve

What is our primary use case?

I am a solution architect and this is one of the products that I implement for my customers. Kafka works well when subscribes want to stream data for specific topics.

Pros and Cons

  • "The most valuable feature is the performance."
  • "Kafka is complex and there is a little bit of a learning curve."

What other advice do I have?

This is currently the product that I am recommending to customers. Some customers want an open-source solution. There are some newer products that are coming on to the market that are even faster than Kafka but this solution is very resilient. In the long run, I think that open-source will dominate the pace. I would rate this solution a seven out of ten.
SergeyGoncharov
Developer Infrastructure at Outbrain
Real User
Very easy to install, stable, and has good scaling options

Pros and Cons

  • "It's very easy to keep to install and it's pretty stable."
  • "The third party is not very stable and sometimes you have problems with this component. There are some developments in newer versions and we're about to try them out, but I'm not sure if it closes the gap."

What other advice do I have?

I would definitely recommend Kafka. In our current position, we use it to move a lot of data and I think it's definitely working well. I would definitely recommend it. I would rate it an eight out of ten.
See 8 more Apache Kafka Reviews