Apache Hadoop Initial Setup

Syed Afroz Pasha - PeerSpot reviewer
Head Of Data Governance at Alibaba Group

The installation is a little difficult because it is an open-source tool. It is similar to Apache Spark. The product is not self-manageable. We will have to invest a little in the setup.

View full review »
Miodrag Milojevic - PeerSpot reviewer
Senior Data Archirect at Yettel

The setup depends on the data. Vast data can be hard to set up. You might have some issues with the setup, but it depends on the number of nodes. More nodes can cause issues and more time to resolve. The reshuffling is also complex and can cause problems.

The on-premise setup can be difficult as it requires the subsequent setup of nodes while expanding. Cloud deployment can be easier but only supports other software.
View full review »
Juliet Hoimonthi - PeerSpot reviewer
Manager at Robi Axiata Limited

I wasn't part of the team that set up Apache Hadoop, but using it after it was set up was very easy. The solution was ready immediately, and the GUI was smooth and fast, with no issues.

View full review »
Buyer's Guide
Apache Hadoop
April 2024
Learn what your peers think about Apache Hadoop. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,578 professionals have used our research since 2012.
GM
Data Architect at a computer software company with 51-200 employees

The initial setup is a hectic task. Configuring servers and nodes takes a long time. That's one of the big advantages of an Autonomous Data Warehouse. You can start implementing within half the time. 

With Apache Hadoop, you have to wait for the setup, architecture, and data evaluation. But with Autonomous, those things are automated. It scales as you use more data, so you can focus on the business rather than infrastructure.

View full review »
Anand Viswanath - PeerSpot reviewer
Project Manager at Unimity Solutions

The product's initial setup phase is complex.

I have not dealt with the setup phase straight away. I always like to rely on the infra person in my company who knows Apache Hadoop.

The solution is deployed on the cloud.

View full review »
AM
Credit & Fraud Risk Analyst at a financial services firm with 10,001+ employees

As it is proprietary software for the enterprise that I am currently working on, I had no trouble setting it up.

View full review »
Abhik Ray - PeerSpot reviewer
Co-Founder at Quantic

After the hardware is available, getting the environment and software up and running has taken us a minimum of a week or 10 days. Sometimes, it has taken us longer, but usually, this is what it takes at the minimum to get everything up. It includes the downloads and also setting it up and making things work together to start using it.

For the original deployment, because there are so many components and not everyone knows everything pretty well, we have seen that we had to deploy four or five people in various areas at the initial deployment stage. However, once it is running, one or two people are required for maintenance.

View full review »
RC
Senior Associate at a financial services firm with 10,001+ employees

Complex. Cloudera stack itself was insufficient. Integration with other tools like R and QlikView was required and in-house programs had to be built to create an automated data pipeline.

View full review »
Aria Amini - PeerSpot reviewer
Data Engineer at Behsazan Mellat

The initial setup is, to some extent, difficult because additional skills are required, specifically knowledge of the operating system at installation. We need someone with professional skills to install the Hadoop environment. With one engineer with those skills, Hadoop takes ten days to two weeks to deploy the solution.

Two or three people are needed to maintain the solution. At least two people are required to maintain the Hadoop stack, in case of unexpected situations, like when something gets corrupted, and they need to solve the problem as fast as possible. Hadoop is easy to maintain because of its governance feature, which helps maintain all the Hadoop stacks.

View full review »
YM
CEO at AM-BITS LLC

The setup is not easy for a financial or telecom company.


It takes around one month for basic development and around three to four months for enterprise. We require more than 50 engineers to do the engineering stuff and more than 20 If for the data engineering team.


In terms of production, the most significant aspects are security and staging, with a focus on either a one-month or three-month timeframe for security considerations.

View full review »
SF
Analytics Platform Manager at a consultancy with 10,001+ employees

There are capacities in which I have been responsible for setup, administration, and building the applications on those environments. Each of the components is relatively straightforward. The complexity comes from all the different components.

View full review »
it_user340983 - PeerSpot reviewer
Infrastructure Engineer at Zirous, Inc.

Initial setup was decently straightforward, especially when using Apache Ambari as a provisioning tool. (I highly recommend Ambari.)

View full review »
Lucas Dreyer - PeerSpot reviewer
Data Engineer at BBD

The initial setup is quite complex if you have to set it up yourself. Ambari makes it much easier, but on the cloud or local machines, it's quite a process.

It took at least a day to set it up.

View full review »
AM
CEO

We didn't have any major issues except for knowledge, so we hired the right person who had hands-on experience with this stack, and worked with the cloud provider to get the right mechanism for handling the stack.

General installation/dependency issues were there, but were not a major, complex issue. While migrating data from MySQL to Hive, things are a little challenging, but we were able to get through that with support from forums and a little trial and error. In addition, the old PoCs which were migrated had issues in directly connecting to Hive. We had to build some user functions to handle that.

View full review »
JP
Vice President - Finance & IT at a consumer goods company with 1-10 employees

The initial setup was a little complex the first time around. We were new to the system, and we didn't have any expertise at that time. Once we get some support and insights into how to work everything properly it went more smoothly.

First, we started with a POC - proof of concept. It takes a couple of days in terms of understanding and configuring everything, etc. When we went to production, it was a couple of hours for deployment and we put into practice everything we learned from the POC.

There's definitely a learning curve. It's stable for us now. 

We have a team of developers doing multiple tasks on the solution and few of them are taking care of Hadoop, so we do have a few people handling maintenance.

View full review »
it_user265830 - PeerSpot reviewer
Senior Hadoop Engineer with 1,001-5,000 employees

It's a bit complex in terms of build around for commodities, but soon it will ease up as the product matures.

View full review »
DD
Partner at a tech services company with 11-50 employees

The complexity of Hadoop's setup depends on the customer and their needs. However, most of my customers wind up using Hadoop as a service, which makes it very easy. It doesn't need much maintenance. My staff maintains multiple systems, so it's not like there would ever be somebody dedicated to one, and Hadoop is not a high-touch platform.

View full review »
MB
IT Expert at a comms service provider with 1,001-5,000 employees

The initial setup was straightforward. However, it was challenging to make it secure. We managed to do that and implement Kerberos because it's the only way to make Hadoop safe. But it was easy and worked for a few years without any problems. Three people implemented this solution over three months.

View full review »
YM
CEO at AM-BITS LLC

The initial setup might not be straightforward for our customers, but it's easy enough for us to handle. However, if we don't build a proof of concept for the company first it may take some time and be quite complex. Pilot projects take about three months to deploy and full spec projects take up to a year because we have to work in all requirements in data governance, security, etc.

View full review »
MB
IT Expert at a comms service provider with 1,001-5,000 employees

The initial setup wasn't very easy because of the incredible security, but we have managed to get by that. It's sort of simple, in my opinion, once you get past that part. I think, in all, it took about half of a year. But it wasn't a new deployment, it's an upgrade and the bigger challenge was moving the data. We pretty much just supported the existing product and moved to HDP.

View full review »
SS
Technical Lead at a government with 201-500 employees

The initial setup was pretty straightforward. It was not overly complex for our team.

View full review »
GA
Founder & CTO at a tech services company with 1-10 employees

It's a well-known fact that Hadoop's configuration is pretty hard. 

View full review »
it_user1093134 - PeerSpot reviewer
Technical Architect at RBSG Internet Operations

The initial setup was complex. There was a lot of data that we had to bring over from various sources and it was quite a long process.

View full review »
it_user693231 - PeerSpot reviewer
Big Data Engineer at a tech vendor with 5,001-10,000 employees

Initial setup of a few nodes was simple, but as we increased the node count it became complex, as we need to maintain rack topology, etc.

View full review »
Buyer's Guide
Apache Hadoop
April 2024
Learn what your peers think about Apache Hadoop. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,578 professionals have used our research since 2012.