StreamSets Scalability

Reyansh Kumar - PeerSpot reviewer
Technical Specialist at Accenture

It is a highly scalable solution. It can be scaled up as and when required. When our data velocity is high, we can scale it. We pay for the scalability charges, nothing else. The scalability is also a 10 out of 10.

Our number of end-users is between 80 and 100 in finance, sales, and primarily the data team.

View full review »
Prateek Agarwal - PeerSpot reviewer
Manager at Indian Institute of Management Visakhapatnam

It is a scalable solution. It automatically scales all the data and the data analytics. We have thousands of users concurrently using the data analytics software. StreamSets has been perfectly scalable.

View full review »
Nantabo Jackie - PeerSpot reviewer
Sales Manager at Soft Hostings Limited

StreamSets is scalable and we can add as many protocols as required to meet our needs.

View full review »
Buyer's Guide
StreamSets
March 2024
Learn what your peers think about StreamSets. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.
Karthik Rajamani - PeerSpot reviewer
Principal Engineer at Tata Consultancy Services

It is pretty scalable, but it also depends on where it is installed, which is something a lot of developers misunderstand. Most of the time, the implementation is done on on-prem servers, which is not very scalable. If you install it on cloud-based servers, it is fast. So, the problem is not with StreamSets; the problem is with the underlying hardware. I have worked on both sides. Therefore, I'm aware of the scenarios, but if I were to work purely in the development team, I might not be aware that it is underlying hardware that is causing problems.

In terms of its usage, it is available enterprise-wide. I don't know the exact number of users now because I am not a part of the platform or admin team, but at one time, we had more than 200 users working on this platform. We had one implementation on AWS Cloud and one on GCP. We had Dev, QA, and prod environments. Even now, we have about four environments. We have SIT and NFT, and in prod, we have two environments.

We plan to increase its usage. We are rapidly increasing its usage in our projects. There is a lot of excitement around it. A lot of people want to explore this tool in our organization. A lot of people are trying to learn this technology or use it to migrate their data from legacy databases to the cloud. This will actually encourage more folks to join the data engineering or analytics team. There is a lot of curiosity around the product.

View full review »
Saket Pandey - PeerSpot reviewer
Product Manager at a hospitality company with 51-200 employees

We are a medium enterprise. We only have three departments in our company, and only one of the departments is using it. Salespeople don't use it. The development people don't use it. We are the ones using it, and our job is to process the information, so only one department is using the solution. We have about 18 people in the department.

Up to medium enterprises, it's a good choice. You can scale between one million to ten million data files. I don't believe they offer the service for a hundred million or one billion datasets. It isn't too scalable for large enterprises, but for small and medium enterprises, it's good.

View full review »
JA
IT Project Manager at Orange España

It is as scalable as you require. If your data velocity and volume are high, you can scale up as needed, but it affects the cost.

StreamSets has helped us scale our data operations. We have a huge user base, about 7,200 people on our platform, and data is coming in regularly at velocity and in great volume. It collects, integrates, and transforms the data.

In our organization, it is being used in multiple locations because our team is diverse, with people in Asia-Pacific, EMEA, and the US.

View full review »
MI
Software Engineer at Soft Hostings Limited

It's good enough. We don't use it at multiple locations. We use it at one location, and it's being used by the IT and development departments. We have five users who are using it.

View full review »
AbhishekKatara - PeerSpot reviewer
Technical Lead at Sopra Steria

I would like auto scaling for heavy load transfer. This applied particularly when we were our data migration project. The tables had more than 10 millions of records in them. When we utilized StreamSets, it took a huge amount of time. Though we were doing every schema generation, we were using ADLS as a destination, and it hung for a good amount of time. So, we considered PySpark processes for our tables, which have greater than 10 millions of records. Usually, it works pretty well with the source tables and the data size is close to five to six million records, but when it is closer to 10 million, I personally feel the auto scaling feature could be improved.

View full review »
Avinash Mukesh - PeerSpot reviewer
IT Specialists at Soft Hostings

It's scalable enough. It integrates with AWS, Snowflake, Google Cloud, and Azure. It gives you a very good way to process and store your data.

We're using it in multiple departments in the same location. It's being used by the analytics team and our senior developers. There are about 10 people using this solution.

View full review »
JM
Software Engineer at ZIDIYO

The solution is scalable.

View full review »
Kevin Kathiem Mutunga - PeerSpot reviewer
Chief software engineer at Appnomu Business Services

The solution is scalable. We currently have four people using StreamSets in our organization.

View full review »
Sumesh Gansar - PeerSpot reviewer
Product Marketing Manager at a tech vendor with 10,001+ employees

It's a scalable product. In our company, the platform is used across seven teams in our organization.

A couple of more teams are evaluating StreamSets in our organization. They're running things and asking for some feedback from our side as well. There are plans to expand our use of it.

View full review »
SS
Senior Data Engineer at a energy/utilities company with 1,001-5,000 employees

A core feature of the DataOps Platform is you can easily scale through engines when you have more pipelines running and data to process. So, if you would need to purchase more engines or cores, it is quite scalable. That is a major advantage that we are getting. 

In the Control Hub Platform, the orchestration and load balancing are quite scalable. You don't need to fiddle with the existing solution. Everything is run on another engine that gets hooked up automatically to Control Hub, which makes it seamless.

There is sort of a developed template out of StreamSets, where you just have one template and can point it to any source system. You can just start ingesting, which has reduced a lot of time in building our new pipelines.

View full review »
MB
Director Data Engineering, Governance, Operation and Analytics Platform at a financial services firm with 10,001+ employees

It's reasonably easy to scale. Around 25 to 30 end users are using this solution in our organization.

View full review »
Al Mercado - PeerSpot reviewer
AI Engineer at Techvanguard

It seems pretty highly scalable to me. That's not going to be an issue. Just the administration of it could be an issue.

It's currently being used in a dev department for machine learning. It's being used by the business analyst team.

View full review »
Ramesh Kuppuswamy - PeerSpot reviewer
Senior Software Developer at a tech vendor with 10,001+ employees

It's definitely scalable. We started with around 10 to 12 users, and now it has reached 35 to 40 users in our particular organization. We are now using it across four to five teams.

There are a lot of other teams in our company that are trying out the free version of the software. If it's suitable for them, they will obviously go for it as well.

View full review »
BahatiAsher Faith - PeerSpot reviewer
Software Developer at Appnomu Business Services

It is a scalable solution for any company that needs to know about its data processing.

View full review »
BR
Data Engineer at a consultancy with 11-50 employees

The scalability is good.

We do plan to increase usage. 

View full review »
SR
Product Marketer at a media company with 1,001-5,000 employees

The platform is definitely scalable.

Maybe in the future we will increase our usage of StreamSets, but I don't see any immediate scalability requirements for us.

View full review »
TH
Senior Network Administrator at a energy/utilities company with 201-500 employees

This software is very scalable.

View full review »
AC
Senior Technical Manager at a financial services firm with 501-1,000 employees

It's pretty scalable.

View full review »
MP
Data Engineer at a energy/utilities company with 10,001+ employees

We haven't seen a problem with scaling it.

View full review »
Buyer's Guide
StreamSets
March 2024
Learn what your peers think about StreamSets. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.