StreamSets Scalability
It is a highly scalable solution. It can be scaled up as and when required. When our data velocity is high, we can scale it. We pay for the scalability charges, nothing else. The scalability is also a 10 out of 10.
Our number of end-users is between 80 and 100 in finance, sales, and primarily the data team.
View full review »It is a scalable solution. It automatically scales all the data and the data analytics. We have thousands of users concurrently using the data analytics software. StreamSets has been perfectly scalable.
View full review »StreamSets is scalable and we can add as many protocols as required to meet our needs.
View full review »Buyer's Guide
StreamSets
March 2024
Learn what your peers think about StreamSets. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.
It is pretty scalable, but it also depends on where it is installed, which is something a lot of developers misunderstand. Most of the time, the implementation is done on on-prem servers, which is not very scalable. If you install it on cloud-based servers, it is fast. So, the problem is not with StreamSets; the problem is with the underlying hardware. I have worked on both sides. Therefore, I'm aware of the scenarios, but if I were to work purely in the development team, I might not be aware that it is underlying hardware that is causing problems.
In terms of its usage, it is available enterprise-wide. I don't know the exact number of users now because I am not a part of the platform or admin team, but at one time, we had more than 200 users working on this platform. We had one implementation on AWS Cloud and one on GCP. We had Dev, QA, and prod environments. Even now, we have about four environments. We have SIT and NFT, and in prod, we have two environments.
We plan to increase its usage. We are rapidly increasing its usage in our projects. There is a lot of excitement around it. A lot of people want to explore this tool in our organization. A lot of people are trying to learn this technology or use it to migrate their data from legacy databases to the cloud. This will actually encourage more folks to join the data engineering or analytics team. There is a lot of curiosity around the product.
View full review »We are a medium enterprise. We only have three departments in our company, and only one of the departments is using it. Salespeople don't use it. The development people don't use it. We are the ones using it, and our job is to process the information, so only one department is using the solution. We have about 18 people in the department.
Up to medium enterprises, it's a good choice. You can scale between one million to ten million data files. I don't believe they offer the service for a hundred million or one billion datasets. It isn't too scalable for large enterprises, but for small and medium enterprises, it's good.
View full review »JA
Jai Agarwal
IT Project Manager at Orange España
It is as scalable as you require. If your data velocity and volume are high, you can scale up as needed, but it affects the cost.
StreamSets has helped us scale our data operations. We have a huge user base, about 7,200 people on our platform, and data is coming in regularly at velocity and in great volume. It collects, integrates, and transforms the data.
In our organization, it is being used in multiple locations because our team is diverse, with people in Asia-Pacific, EMEA, and the US.
MI
Mwase Isaranya
Software Engineer at Soft Hostings Limited
It's good enough. We don't use it at multiple locations. We use it at one location, and it's being used by the IT and development departments. We have five users who are using it.
View full review »I would like auto scaling for heavy load transfer. This applied particularly when we were our data migration project. The tables had more than 10 millions of records in them. When we utilized StreamSets, it took a huge amount of time. Though we were doing every schema generation, we were using ADLS as a destination, and it hung for a good amount of time. So, we considered PySpark processes for our tables, which have greater than 10 millions of records. Usually, it works pretty well with the source tables and the data size is close to five to six million records, but when it is closer to 10 million, I personally feel the auto scaling feature could be improved.
View full review »It's scalable enough. It integrates with AWS, Snowflake, Google Cloud, and Azure. It gives you a very good way to process and store your data.
We're using it in multiple departments in the same location. It's being used by the analytics team and our senior developers. There are about 10 people using this solution.
View full review »JM
Josua Mwesigwa
Software Engineer at ZIDIYO
The solution is scalable.
View full review »The solution is scalable. We currently have four people using StreamSets in our organization.
View full review »It's a scalable product. In our company, the platform is used across seven teams in our organization.
A couple of more teams are evaluating StreamSets in our organization. They're running things and asking for some feedback from our side as well. There are plans to expand our use of it.
View full review »SS
Srinivasan Sankar
Senior Data Engineer at a energy/utilities company with 1,001-5,000 employees
A core feature of the DataOps Platform is you can easily scale through engines when you have more pipelines running and data to process. So, if you would need to purchase more engines or cores, it is quite scalable. That is a major advantage that we are getting.
In the Control Hub Platform, the orchestration and load balancing are quite scalable. You don't need to fiddle with the existing solution. Everything is run on another engine that gets hooked up automatically to Control Hub, which makes it seamless.
There is sort of a developed template out of StreamSets, where you just have one template and can point it to any source system. You can just start ingesting, which has reduced a lot of time in building our new pipelines.
MB
reviewer2238417
Director Data Engineering, Governance, Operation and Analytics Platform at a financial services firm with 10,001+ employees
It's reasonably easy to scale. Around 25 to 30 end users are using this solution in our organization.
View full review »It seems pretty highly scalable to me. That's not going to be an issue. Just the administration of it could be an issue.
It's currently being used in a dev department for machine learning. It's being used by the business analyst team.
View full review »It's definitely scalable. We started with around 10 to 12 users, and now it has reached 35 to 40 users in our particular organization. We are now using it across four to five teams.
There are a lot of other teams in our company that are trying out the free version of the software. If it's suitable for them, they will obviously go for it as well.
View full review »It is a scalable solution for any company that needs to know about its data processing.
View full review »BR
reviewer1889181
Data Engineer at a consultancy with 11-50 employees
SR
reviewer2067186
Product Marketer at a media company with 1,001-5,000 employees
The platform is definitely scalable.
Maybe in the future we will increase our usage of StreamSets, but I don't see any immediate scalability requirements for us.
View full review »TH
reviewer2041068
Senior Network Administrator at a energy/utilities company with 201-500 employees
This software is very scalable.
View full review »AC
reviewer912129
Senior Technical Manager at a financial services firm with 501-1,000 employees
It's pretty scalable.
View full review »MP
reviewer1339230
Data Engineer at a energy/utilities company with 10,001+ employees
We haven't seen a problem with scaling it.
View full review »Buyer's Guide
StreamSets
March 2024
Learn what your peers think about StreamSets. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.