Apache Kafka Scalability

Harsha Ravnikar - PeerSpot reviewer
Senior Solutions Architect at Sysmex America, Inc.

Confluent is the cloud version of Apache Kafka and it is scalable.

View full review »
AT
Group Manager at a media company with 201-500 employees

I would rate the scalability of Apache Kafka somewhere around seven out of ten. I'm not going on the higher side because a lot of manual work is involved in upgrading Kafka. You have to estimate the overall capacity, not just in data but also in other use cases running on the same cluster. In the cloud, it is easier as you don't have to worry about the turnaround time of your cluster setup. But in an on-premise setup, you need to add more nodes, RAM resources, and storage based on the increasing data volume. 

Also, it would help if you ensured that the streaming use cases running on Kafka are not impacting other use cases like batch or archival use cases. Because of this manual activity of overall estimation, I will still keep it somewhere around six to seven. But regarding scalability in terms of horizontal or vertical scalability from the data perspective, I feel more comfortable with Kafka compared to other available streaming solutions.

View full review »
Amit Laddha - PeerSpot reviewer
Vice President (Information and Product Management) at Tradebulls Securities (P) Limited

Scalability-wise, I rate the solution an eight out of ten.

The users of the solution are not directly involved with it. Those who use Apache Kafka in our company use it to push orders on the frontend, and then the frontend calls for some microservice, after which the microservice pushes data to Apache Kafka. Around 10,000 to 15,000 people in my organization follow the aforementioned procedure.

View full review »
Buyer's Guide
Apache Kafka
April 2024
Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,578 professionals have used our research since 2012.
AbhishekGupta - PeerSpot reviewer
Engineering Leader at Walmart

The scalability of Apache Kafka could improve.

We have approximately 10,000 users using this solution.

View full review »
Jhon Rico - PeerSpot reviewer
Senior Solutions Architect at BVC

It depends on the configuration., but scalability is one of the best features of Kafka. I would rate it nine out of ten.

View full review »
Pratul Shukla - PeerSpot reviewer
Software Engineer at a financial services firm with 10,001+ employees

We use the solution in the distributed mode in multiple regions – the US, London, and Hong Kong. We have increased the number of nodes to ensure it is available to us at all times.

I give the scalability an eight out of ten.

We have around 600 people within my team using the solution.

View full review »
MB
Architect at Agence Française de Développement

A lot of my experience indicates that Apache Kafka is scalable. We can have ten or even fifty hundred users on the solution. So, it's possible because we are a big enterprise.

View full review »
Nor EL MALKI - PeerSpot reviewer
Project Manager at Leyton & Associés, SAS

the scalability of Apache Kafka is good. We have parts of the information we use in different geographical sites and it doesn't pose any problem.

View full review »
SP
CEO at a comms service provider with 11-50 employees

It is very scalable. It is easy to scale it. 

It doesn't matter how many users are using it. The licenses are calculated based on the number of nodes. It is not based on the number of users who are using it. We have between 10 to 20 nodes on average in an organization.

View full review »
Bharath-Reddy - PeerSpot reviewer
Architect at Tekgeminus

I rate Apache Kafka a seven out of ten for scalability.

View full review »
GT
Lead Architect at a financial services firm with 1,001-5,000 employees

The scalability is very good. It is scalable horizontally rather than vertically. 

It can scale up to any level horizontally. However, if the message, once used horizontally scalable, cannot be shrunk once the requirement is reduced, some process is actually taking place. That is one thing that is lacking.

I believe there are approximately 10 to 15 people who use it.

This is being used by the data migration, data team, data analytical team, and data engineer. It's being used by all application architects who are just looking into it, as well as middleware integrators and middleware application integrators.

We have big plans to increase the use of various other innovations and stuff like that. We are using it in relation to data activities. 

Also, we are only planning to use the financial part for publishing it, subscribing, and publishing a pop-up model for various use cases.

View full review »
Mukulit Bhati - PeerSpot reviewer
CTO at InsightGeeks Solutions Pvt.

The solution is very scalable, and its availability is brilliant. We have approximately 32,000 people on our customer base.

View full review »
Stuart-Cook - PeerSpot reviewer
CEO & Founder at a tech consulting company with 11-50 employees

I don't know how many people were using it on the client's side, but we had a four-person team doing the development work. 

View full review »
Rémy NOLLET - PeerSpot reviewer
Data Exchange Architect MQSeries at Decathlon International

It is easy to scale.

View full review »
Silvio Lucas Pereira Filho - PeerSpot reviewer
Senior Tech Lead at RecargaPay

The scalability of Apache Kafka is good. It can process many requests simultaneously.

We have approximately 600 people using this solution in my organization.

View full review »
Joaquin Marques - PeerSpot reviewer
CEO - Founder / Principal Data Scientist / Principal AI Architect at Kanayma LLC

My opinion is that Apache Kafka is a scalable solution. In our organization, there are hundreds of thousands of users using Kafka.

View full review »
TD
Head of Technology - Money Movement Platform at a financial services firm with 10,001+ employees

Apache Kafka is the most scalable solution in the market.

View full review »
AV
Technical Director at Metrofibre Networx

It is a scalable solution. We set up a category with different consumers balancing things, which works as I thought.

View full review »
RG
Senior Technology Architect at a tech services company with 10,001+ employees

Apache Kafka is Scalable. You can actually launch a server node or a broker. Three nodes and Zookeeper (the Kafka server management system) is optimal. If one of them goes down you can automatically launch another one. You can go three servers or brokers back — there's a repetition on each Kafka broker.

View full review »
it_user590451 - PeerSpot reviewer
Lead Engineer at a retailer with 10,001+ employees

We have clustered environments and we haven’t seen any scalability issues. We can provision a new node in as little as 45 minutes.

View full review »
Felipe Lopes - PeerSpot reviewer
Engineering Manager at Alice

The solution is scalable.

View full review »
NK
Director at Tibco

The solution is scalable.

View full review »
JA
Technical Lead at Interface Fintech Ltd

The solution is very scalable. We started with a cluster of three and then scaled it to seven. I would give the solution a five out of five for scalability. Currently, we have 20+ employees on the technical team that are using the solution. 

We provide outsource services for other institutions. There is a whole set queue management form, and we have about five institutions, with three technical teams that use the same cluster.

View full review »
SK
Barista Brewing Espresso at Linkedln

The solution scales horizontally and scales better than its competitors. We have around 400 to 500 microservices consuming this cluster and the company has around 600 employees. We have four different verticals, each with around 100 engineers with 100 to 150 microservices. 90% of the microservices have a touchpoint with Kafka.

View full review »
Salvatore Campana - PeerSpot reviewer
CEO & Founder at XAUTOMATA TECHNOLOGY GmbH

It's scalable.

View full review »
ME
CTO at Estrada & Consultores

Scaling is not really a problem with Kafka. We have used Kubernetes clusters and it is working very well. It scales up and down, almost automatically almost unnoticeable to the consumers, based upon our configuration. Kafka is just one pod inside of our cluster that scales horizontally.

We have a couple of customers that also have vertical scaling, meaning that, there's more CPU, more memory available to the Kafka pod.

View full review »
DZ
Enterprise Architect at Smals vzw

Apache Kafka is a scalable solution.

View full review »
Ravi Kuppusamy - PeerSpot reviewer
CEO and Founder at BAssure Solutions

I would recommend Apache Kafka for any enterprise.

The amount of people using the solution depends on the application. However, the starting point is from 6,000 to 7,000 concurrent users.

View full review »
KS
Solution Architect at a manufacturing company with 10,001+ employees

This is certainly a scalable product. There are currently 30 or more people using it but we expect to scale beyond this. It is going to be an enterprise tool within the company.

View full review »
Paul Adams - PeerSpot reviewer
Consultant Solution Architect at a tech services company with 51-200 employees

Apache Kafka is scalable.

I rate the scalability of Apache Kafka a nine out of ten.

View full review »
RP
Assistant Professor at CHAROTAR UNIVERSITY OF SCIENCE AND TECHNOLOGY

The solution's scalability is important for our ability to have more throughput from multiple receivers. If we need more throughput it can deliver.

View full review »
JB
Software Support & Development Engineer at a computer software company with 501-1,000 employees

Apache Kafka is scalable. It is easy to add brokers.

We have approximately 30 people using this solution in my organization. They use the solution daily.

View full review »
DP
Sr Technical Consultant at a tech services company with 1,001-5,000 employees

I have found the Apache Kafka to be highly scalable

View full review »
LP
Owner at Binarylogicworks.com.au

Kafka is a very scalable system. You can have multiple, scalable architectures.

View full review »
TM
Building Event-centric Data processing Architectures at a tech services company with 51-200 employees

Scalability is one of the main selling points of the tool. The additional nodes we add give us the additional storage capacity we need. I rate the scalability a ten out of ten. The solution is used across multiple domains in our organization. I use the product daily. It’s a continuously growing platform.

View full review »
TM
Assistant Student at a retailer with 5,001-10,000 employees

The solution is scalable. We have about 10 departments that use Kafka in various forms. Each department might have 5 or 10 people.

We use the solution all the time. We have consumers that consume messages that come every day because we have clients and customers for the main website. All of those messages go to KAF clients. Our backend departments consume messages from the actions of the final customers.

View full review »
it_user653562 - PeerSpot reviewer
Solutions Architect at a consultancy with 1,001-5,000 employees

We did not have any issues with scalability.

View full review »
Abdul-Samad - PeerSpot reviewer
Software Engineer at a tech services company with 201-500 employees

I rate Kafka eight out of 10 for scalability. 

View full review »
Guirino Ciliberti - PeerSpot reviewer
Data Governance & Lineage Product Manager at Primeur

The solution is scalable. Deployment is speedy, but we don't have many installations. We have over a thousand users using this solution and will most likely increase the number of users because we have tested 100,000 messages per second. The solution is impressive.

View full review »
DS
Principal Technology Architect at a computer software company with 5,001-10,000 employees

We did some benchmarking, but we are still looking further to scale up some of the benchmarking and performances. So far, it meets all our business requirements. We are just developers, so everything goes to the clients, who will deploy it at their scale and use it for their end customers. So were are looking at it from a developer's perspective. Those who are developing the products are working on this.

View full review »
MN
Project Engineer at Wipro Limited

The scaling of the solution is quite good.

View full review »
it_user660591 - PeerSpot reviewer
Senior Java Consultant at a tech services company with 501-1,000 employees

We have had no issues with scalability.

View full review »
it_user578787 - PeerSpot reviewer
Java Developer at a media company with 10,001+ employees

We have not had any scalability issues. The product is horizontally scalable, so adding extra hardware is all that is needed.

View full review »
KQ
Senior Technical Architect at a computer software company with 51-200 employees

Kafka is designed to be very easily scalable so I did not have any trouble here.

View full review »
ShoaibKhan - PeerSpot reviewer
Technical Specialist at APIZone

With the containerized version we have used, we have faced challenges with the scalability.

View full review »
YL
Vice President at a consultancy with 51-200 employees

We are a small team with a few people.

We might increase our usage in the future.

View full review »
RABBAHMahmoud - PeerSpot reviewer
Senior Technical Architect at RABBAH SOFT

Scalability is very good. 

View full review »
it_user642168 - PeerSpot reviewer
Big Data Lead at a marketing services firm with 51-200 employees

We have not encountered any scalability issues yet. We are growing and currently, we manage 1M events per second in Kafka.

View full review »
it_user660627 - PeerSpot reviewer
Senior Software Engineering Consultant at a tech services company with 51-200 employees

We have not yet had scalability issues!

View full review »
it_user650223 - PeerSpot reviewer
Principal Software Architect at a tech services company with 11-50 employees

We have to balance the nodes when topics partition across cluster nodes. As it assumes they are of equal sizes, sometimes some nodes may not be allocated similar resources. Reassignment moves all the partitions of specified topics which may be an issue when not planned for.

View full review »
Sreekar Nethagani - PeerSpot reviewer
Co-Founder at Attaika

The solution is scalable.

View full review »
NC
Chief Technology Officer at a tech services company with 1-10 employees

We are still in phase one so haven't yet tested scalability. That will come when we move to the second phase. We currently have around 15 users of this solution. 

View full review »
it_user650004 - PeerSpot reviewer
Team Lead at a financial services firm with 1,001-5,000 employees

We have not had any scalability issues.

View full review »
it_user647457 - PeerSpot reviewer
Head of Engineering

We did not encounter any issues with scalability.

View full review »
it_user644286 - PeerSpot reviewer
Deputy General Manager, DevOps Manager at a comms service provider with 10,001+ employees

No issues with scalability.

View full review »
Andrea Castorino - PeerSpot reviewer
Program Manager at SirfinPA

In this project, we used Kafka to synchronize 28 nodes spread out nationally and it seems scalable. We plan to consolidate the 28 nodes for national integration of nodes and schemas.

View full review »
AM
IBMi/MIMIX Administrator at Arab Bank

It is easy to scale. This is a new project so we only have about five users right now.

View full review »
Mukulit Bhati - PeerSpot reviewer
CTO at InsightGeeks Solutions Pvt.

It is scalable. 

I use it 24/7. We have to change both packets to Kafka and from Kafka it receives data. We retain the packet for only six seconds.

View full review »
it_user592356 - PeerSpot reviewer
Technical Lead/Project Manager(Consulting Apple Inc) at a tech services company with 1,001-5,000 employees

We have not had any scalability issues.

View full review »
it_user592338 - PeerSpot reviewer
Enterprise Architect at a logistics company with 1,001-5,000 employees

We have not encountered any scalability issues.

View full review »
OT
Senior Big Data Developer | Cloudera at Dilisim

The scalability of the solution is very good. You can analyze system events horizontally and the cluster can be brought over to the cloud side with the Kafka user's server.

We use the solution for both small and medium-sized organizations, but also larger enterprises. Some of our clients are in the banking and financial sector.

View full review »
DR
Founder, CEO at a tech vendor with 1-10 employees

Ah, scalability!!! We need to set up multiple servers again for handling the load, which makes Kafka not scalable, unless you subscribe to cloud services.

View full review »
MS
Senior Consultant at a tech services company with 51-200 employees

The solution is scalable, however, it's a 50/50 endeavor. It may require some management to build it out.

View full review »
RH
freelance at a tech services company with 11-50 employees

Kafka is very scalable, which is an important feature of it.

Our clients have approximately ten applications in their companies that communicate with Kafka.

View full review »
RH
freelance at a tech services company with 11-50 employees

Kafka is stable. Defining our user base is hard because Kafka influences the whole company, so you could say around 100 users. Kafka is a core system, so it affects all users we choose to link to the primary key.

View full review »
it_user998961 - PeerSpot reviewer
Enterprice Architect

This solution is scalable.

View full review »
it_user660630 - PeerSpot reviewer
SDET II at a tech services company with 5,001-10,000 employees

Kafka is a highly scalable product. We have not faced any scalability issues so far.

View full review »
it_user613983 - PeerSpot reviewer
Java Architect at a tech vendor with 51-200 employees

RESTful API implementation actually uses the Kafka Broker to publish the messages but I am not able to find it becoming scalable. Partially, the reason might be there is no load balancer for the RESTful API web server.

View full review »
WG
Solutions Architect at a tech services company with 201-500 employees

This is a scalable solution.

View full review »
Buyer's Guide
Apache Kafka
April 2024
Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,578 professionals have used our research since 2012.