StreamSets Initial Setup

Reyansh Kumar - PeerSpot reviewer
Technical Specialist at Accenture

I was a part of the team that evaluated, onboarded, and implemented StreamSets for the organization. We had a team of four to five people involved. It is straightforward to deploy. All software these days uses the software as a service model, so implementation is quite easy.

On our side, there is no maintenance. It is a managed service that provides all the upgrades and updates. We don't have to deploy any patches or framework updates.

View full review »
Prateek Agarwal - PeerSpot reviewer
Manager at Indian Institute of Management Visakhapatnam

The deployment is quite easy because it is cloud-based. No external software or solution is required. You just start your work from day one, once your deployment is done. It takes 15 to 21 days for the initial setup. We have it deployed at a single location and used by 50 to 60 people who are mainly from product DevOps and DataOps.

Our data operations and DevOps teams checked and tested all the results for all the use cases that we have.

It is a fully managed cloud-based solution, so everything is managed by the StreamSets team. There is no maintenance on our end.

View full review »
Nantabo Jackie - PeerSpot reviewer
Sales Manager at Soft Hostings Limited

The initial setup was complex; we had to contact the technical team for assistance in order to begin the deployment and get synced. This was difficult because we lacked the necessary knowledge. However, we read the instructions and contacted support, the process became much simpler.

The deployment took around seven business days.

We used a team of three for the deployment.

View full review »
Buyer's Guide
StreamSets
March 2024
Learn what your peers think about StreamSets. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.
Karthik Rajamani - PeerSpot reviewer
Principal Engineer at Tata Consultancy Services

I was involved with AWS deployment. At that time, I was a part of the platform team. Now, I work with the application development team, and I'm not involved in that. It was complex at that time. About four years ago, when StreamSets was new, we had a tough time deploying because the documentation was not very clear at that time. A lot of the documents were very good and available on the web, but the documentation wasn't exhaustive or elaborate. We also had our own learning curve. We had someone from StreamSets to help us with the deployment. So, it went well. Now, it is better, but when we did it, it was very complex.

We implemented it in phases. We just implemented or installed the StreamSets platform in our company, and we let a couple of teams use it. We started with Data Collector, and we allowed teams to use and feel it. When they said that this is a good tool to use, we got the enterprise license, and we installed Control Hub and Data Collector. It was not implemented enterprise-wide at the same time. It was released to teams in phases.

View full review »
Namanya Brian - PeerSpot reviewer
CEO-founder at Tubayo

It took me three days to deploy it. I did it on my own. We use it in two departments in one location and there are four users.

There is no maintenance of the solution on our side.

View full review »
Saket Pandey - PeerSpot reviewer
Product Manager at a hospitality company with 51-200 employees

Its initial setup is a bit tough. You need to have the technical expertise to do that. The support team is good. They help you around, but if they could make it a bit easier, it would be better.

I believe it operates only from the cloud. We also received the data from our associations on the cloud. We processed it on the cloud, and everything happened on the cloud.

The initial setup was complex because we were not able to directly link the data we were receiving with the StreamSets solution. Linking it required us to fill in or enter some information in StreamSets, but we were not able to figure out what to enter. For that part, we needed their help.

We spent about a week. For the first three days, our team members were trying their best to do it, but then we had to schedule a meeting with them. In terms of the number of people, only one person was working with our team, and there were three people working with the product. I was also involved in the product as a product manager, but I was not directly operating that system.

It didn't require any maintenance as such. Any maintenance activities were related to our side of things. There were mistakes on our end. When we were entering different data, we had to do different configurations in the system.

View full review »
JA
IT Project Manager at Orange España

We have it deployed on the cloud, on both AWS and Azure. The initial deployment was straightforward. Their teams were very supportive. Internally, we had a team of four to six people involved from DevOps and networking.

No maintenance is required because it is managed by StreamSets, except that sometimes updates and upgrades have to be managed by us.

View full review »
MI
Software Engineer at Soft Hostings Limited

In the beginning, it's very hard, but after reading the documentation, you can set up things easily. The documentation is very good and helpful.

For me, deployment was initially very hard because it required a lot of technical skills that I didn't have at that time. I had to contact the team, and they helped me with how to deploy it. The following day, I was able to set up everything. So, deployment is initially very hard, but after you become familiar with StreamSets, you can deploy it more easily.

View full review »
AbhishekKatara - PeerSpot reviewer
Technical Lead at Sopra Steria

For our deployment model, we were following three environments: dev, QA and prod. Our team's main responsibility is to hydrate Azure Data Lake and GCP from the source system. Control Hub is hosted on GCP, and we were hitting the URL to log into StreamSets. All the data collector machines are created on Google Cloud Platform, and we use a dev environment. Whenever we create and do a PoC, we work in a dev environment. Once our pipeline and jobs are working fine, we move our pipelines to our QA environment, which is export and import. It is pretty easy to do via StreamSets Control Hub. We can simply select a job and export it, then log back into the QA environment and import the job. Once we import the job, the associated pipeline, and all the parameters, we have an option to import the whole bundle, like the pipeline, parameter, and instances. We can import everything. Once this is also working fine, we have another final environment, which is the production which is based on the source refresh frequencies. 

View full review »
Avinash Mukesh - PeerSpot reviewer
IT Specialists at Soft Hostings

Deploying StreamSets is not so complex. It's easy. It takes about three days.

It doesn't require any maintenance from our side.

View full review »
JM
Software Engineer at ZIDIYO

The initial setup is straightforward. I deployed the solution myself.

View full review »
Kevin Kathiem Mutunga - PeerSpot reviewer
Chief software engineer at Appnomu Business Services

The initial setup is a bit complex for first-time people. There is a lot of documentation that needs to be reviewed before deploying. The deployment takes around one month.

View full review »
SS
Senior Data Engineer at a energy/utilities company with 1,001-5,000 employees

I was there right from the start when they adopted an open-source version. Late last year, we moved to an enterprise version, i.e., the DataOps platform. So, I worked on the 3.2.2 version, and now I am working on the 5.0 version, which is the enterprise license version.

The implementation is straightforward, except for a few hiccups with known network, process, and firewall issues. Other than that, it was a very simple, lean implementation.

Because we had a lot of firewall issues and issues with our optimization, it took probably four weeks for us to get things running. However, if you exclude the issues, it took probably a week to a week and a half to get things up and running.

We are working, as a separate piece of the project, to migrate whatever is running in our existing custom platform to StreamSets. From a certain date, we started to work purely on StreamSets. For any future ingestion requirements, we are using StreamSets DataOps platform. However, the previous platform is inactive at the moment. We are only using it for existing pipelines, and the plan is to migrate them to the DataOps platform this year very soon.

View full review »
MB
Director Data Engineering, Governance, Operation and Analytics Platform at a financial services firm with 10,001+ employees

It's reasonably easy to deploy. However, since it is used at an enterprise level, it requires maintenance. So we had a maintenance contract. 

In the financial industry, we have very strict regulations around deploying something in the cloud. So, it requires a lot of permission and other processes.

Just one person is enough for the maintenance. 

View full review »
Al Mercado - PeerSpot reviewer
AI Engineer at Techvanguard

I was involved in its setup. I was the one who basically had to try to get it to run with whatever process or custom processor I developed. 

It was complex to set up. I had to go to the sessions. On a couple of occasions, I was doing it directly from the cloud platform, and apparently, that wasn't the way to do it. You have to go through their universal designer platform first. 

In terms of maintenance, once you're deployed from the cloud, that's all handled for you. It's managed for you directly from the cloud service. So, you don't have to worry about that. They maintain their design platform.

View full review »
BahatiAsher Faith - PeerSpot reviewer
Software Developer at Appnomu Business Services

Initially, the deployment could be very hard if you do not have a lot of technical skills, but as you get used to the software, within a day, the deployment becomes straightforward and becomes easy. It took two weeks to have everything configured in the right manner. I worked with one other colleague to set everything up.

It is hard, especially when you are a beginner, but when you read the documentation you can set things up quickly. The documentation helps out if you don't have good knowledge of the solution.

It doesn't require maintenance.

View full review »
BR
Data Engineer at a consultancy with 11-50 employees

The setup is straightforward, it's not complex and it is simple. 

We treat it like a pipeline. We are not writing code and putting things in. In the case of a pipeline, you can export it and input it, or you can make it a pipeline. It can be auto-deployed into a respective environment. That's what we did.

We have different destinations we need to send to. We aren't using a single destination. In that sense, we do have multiple computations. We set up, send the data and do the deployments. 

There is occasional maintenance needed. Sometimes, if something goes wrong, we'll have to correct the data. We just check here and there for the most part.

View full review »
SR
Product Marketer at a media company with 1,001-5,000 employees

I was not fully involved in the initial implementation, but we did the implementation in phases. We wanted to get it on board as soon as possible, so instead of doing a complete implementation, we did it in phases and it didn't take a lot of time. We were able to get on with the work as soon as possible with this model.

The initial setup was simple. We didn't require any additional training or third-party vendors. We were able to do it along with the StreamSets team, so it was smooth for us.

We have 15 people using StreamSets, all at one location. They are developers and users.

Because it is a cloud platform there isn't much maintenance required other than server updates, but that is expected with any cloud platform. No extensive maintenance is required. We have a team of two people who maintain it and handle updates and all the latest releases.

View full review »
TH
Senior Network Administrator at a energy/utilities company with 201-500 employees

The initial setup was somewhere between straightforward and complex. It was pretty straightforward to start with, but then it started ramping up to be more difficult as we wanted to add more stuff in.

The difficulty depends upon your data sources. If you have just one data source and you want to consume a lot of different types of data from that one source, it's pretty straightforward. But when you have 20 or 25 different data sources, and you need to pipeline all that data into a couple of data warehouses so that you can use advanced data analytics software to do reporting, analysis, and notifications, it's a lot more complicated. With every data source, it becomes exponentially more complicated to manage.

We spent a significant amount of time doing it, but otherwise, it was seamless because it was our own staff. We didn't have to worry about trying to find money or resource time or do any of the prep work needed to get external resources.

Ours is a single deployment, but it is used across our entire staff base of 200-plus people. We need three people for deployment and maintenance, whose responsibilities include software management, application management, and data analysis and management.

View full review »
AC
Senior Technical Manager at a financial services firm with 501-1,000 employees

The initial setup is pretty straightforward.

View full review »
MP
Data Engineer at a energy/utilities company with 10,001+ employees

This product was a lot easier to use than the one we had before it. It took us half an hour and we were set up and running it, the first time.

View full review »
Buyer's Guide
StreamSets
March 2024
Learn what your peers think about StreamSets. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.