We just raised a $30M Series A: Read our story
2018-09-11T08:28:00Z

What needs improvement with Apache Kafka?

7

Please share with the community what you think needs improvement with Apache Kafka.

What are its weaknesses? What would you like to see changed in a future version?

ITCS user
Guest
1818 Answers

author avatar
Top 10Real User

Kafka can allow for duplicates, which isn't as helpful in some of our scenarios. They need to work on their duplicate management capabilities but for now developers should ensure idempotent operations for such scenarios. While the solution scales well and easily, you need to understand your future needs and prep for the peaks.

2021-08-07T05:24:00Z
author avatar
Top 5LeaderboardReal User

I would like to see real-time event-based consumption of messages rather than the traditional way through a loop. The traditional messaging system works by listing and looping with a small wait to check to see what the messages are. A push system is where you have something that is ready to receive a message and when the message comes in and hits the partition, it goes straight to the consumer versus the consumer having to pull. I believe this consumer approach is something they are working on and may come in an upcoming release. However, that is message consumption versus message listening. Confluent created the KSQL language, but they gave it to the open-source community. I would like to see KSQL be able to be used on raw data versus structured and semi-structured data.

2021-06-26T01:12:49Z
author avatar
Top 20Real User

The graphical user environment is currently lacking in Apache. It's not available within the solution and needs to be built from scratch. Some of the open source products of this solution have limitations.

2021-05-12T12:29:06Z
author avatar
Top 5LeaderboardReal User

The initial setup and deployment could be less complex. Integration is one of the main concerns that we have.

2021-02-09T13:56:41Z
author avatar
Top 5LeaderboardReal User

Some vendors don't offer extra features for monitoring. Some come with Linux for default monitoring. Monitoring is very important. If something is not working properly, then our subscribers won't receive a notification. You then have to trace it back to Kafka and find the glitch or the messaging sequence that hasn't been racked up correctly. It should support Avro — which handles different data formats — as a default data format. It would be much more flexible if it did.

2020-10-22T16:39:00Z
author avatar
Top 20Real User

They need to have a proper portal to do everything because, at this moment, Kafka is lagging in this regard. It could be used to do the preprocessing or the configurations, instead of directly doing it on the queues or the topics. If you look at Solace, for example, they have come up with a portal where you don't need to touch these activities. You don't need to access the platform beyond the portal.

2020-09-27T04:09:51Z
author avatar
Top 5Real User

The model where you create the integration or the integration scenario needs improvement. It contains fewer developer words or maintaining words where someone prepared the topics, the connectors, or the streaming platforms. You would first need to have a control center from a third party for managing. If you would like to prepare something that is a more sophisticated integration scenario, where you use one microservice to provide the event or a second to several that consumed these microservices, then this needs to be modeled elsewhere. Also, when comparing to the traditional ESD for data mixing, you can create a scenario that could be deployed with inputs and some outputs. Most business like the topics, but for me, I think that it is a problem that messaging platforms have, there is no design tool with IDE for creating. It would be helpful to create a more complex solution for several types of styles, and not just for one provider or for one customer. That would be easier, but if you have more than one consumer then it could be a more complex scenario. It would be like events that go to several microservers to create orders, validate orders, and creating words. This would be helpful. In the next release, adding some IDE or developing tools, for creating better integration scenarios, even though it already a developer-oriented solution, would be helpful. It would also be helpful for the auto-deployment. Having a governance style would also be helpful to understand. It would be beneficial to have a repository of all of the topics, data types that exist, or data structures.

2020-08-19T07:57:37Z
author avatar
Top 20Consultant

Due to the fact that the solution is open source, it has a zookeeper dependency. If I could change anything about the solution, it would be that. The solution could always add a few more features to enhance its usage.

2020-06-28T08:51:00Z
author avatar
Top 10Real User

This solution could be made easier to manage. Compatibility with other solutions and integration with other tools can be improved. We cannot apply all of our security requirements because it is hard to upload them.

2020-04-19T07:40:31Z
author avatar
Top 20Real User

The manageability should be improved. There are lots of things we need to manage and it should have a function that enables us to manage them all cohesively. There should be a default property. It's really hard to manage all these things.

2020-04-05T09:13:00Z
author avatar
Top 20Real User

More adapters for connecting to different systems need to be available.

2020-03-30T07:58:09Z
author avatar
Top 20Real User

Kafka is complex and there is a little bit of a learning curve.

2020-03-30T07:58:07Z
author avatar
Top 10Real User

We're still going through the solution. Right now, I can't suggest any features that might be missing. I don't see where there can be an improvement in that regard. The speed isn't as fast as RabbitMQ, even though the solution touts itself as very quick. It could be faster. They should work to make it at least as fast as RabbitMQ. The UI is based on command line. It would be helpful if they could come up with a simpler user interface. They should make it easier to configure items on the solution. The solution would benefit from the addition of better monitoring tools.

2020-02-03T09:10:16Z
author avatar
Top 10Real User

In the next release, I would like for there to be some authorization features and HTL security. We also need bigger software and better monitoring.

2020-01-29T11:22:00Z
author avatar
Top 10Real User

If the graphical user interface was easier for the Kafka administration it would be much better. Right now, you need to use the program with a command-line interface. If the graphical user interface was easier, it could be a better product.

2020-01-19T06:38:00Z
author avatar
Top 20Real User

Kafka does not provide control over the message queue, so we do not know whether we are experiencing lost or duplicate messages. Better control over the message queue would be an improvement. Solutions such as ActiveMQ do afford better control. Because of this, there is sometimes a gap in the results where we have either lost messages, or there are duplicates. We have had problems when there was an imbalance because all of the messages were being sent back.

2020-01-12T07:22:00Z
author avatar
Top 20Real User

There is a feature that we're currently using called MirrorMaker. We use it to combine the information from different Kafka servers into another server. It's very wide and it gives a very generic scenario. I think it would be great if the possibility would exist out of the box and not as a third party. The third party is not very stable and sometimes you have problems with this component. There are some developments in newer versions and we're about to try them out, but I'm not sure if it closes the gap.

2019-12-30T06:53:00Z
author avatar
Real User

Kafka 2.0 has been released for over a month, and I wanted to try out the new features. However, the configuration is a little bit complicated: Kafka Broker, Kafka Manager, ZooKeeper Servers, etc.

2018-09-11T08:28:00Z
Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: October 2021.
540,984 professionals have used our research since 2012.