2016-11-28T04:28:00Z

Apache Spark without Hadoop -- Is this recommended?

Padmanesh NC - PeerSpot reviewer
  • 4
  • 535
PeerSpot user
3

3 Answers

NK
Real User
Top 20
2021-09-03T05:02:26Z
Sep 3, 2021

I don't think using Apache Spark without Hadoop has any major drawbacks or issues. I have used Apache Spark quite successfully with AWS S3 on many projects which are batch based. Yes for very high performance system HDFS is a better option. 


The main problem with Apache Spark with object storage like S3 has been the consistency problem of these object storage systems. You can read this post which will help you understand the issue and how to avoid it. Hope this helps you.



https://arnon.me/2015/08/spark...


Product comparison that may be of interest to you
Padmanesh NC - PeerSpot reviewer
Reseller
Top 5Leaderboard
2017-01-04T13:58:25Z
Jan 4, 2017

I mean we can configure Spark without Hadoop as well like using WinUtils.exe . Is that recommended for Deployment ? Or would like to understand difference between Spark Hadoop Environment and Spark Without Hadoop?

it_user326337 - PeerSpot reviewer
Consultant
2016-12-06T14:25:42Z
Dec 6, 2016

Can you elaborate on the information you've been told about how using Apache Spark without Hadoop isn't good for deployment?

This insight would help many of our users.

Compare products

Software Configuration Management experts

Omar_Ismail - PeerSpot reviewer
Mustapha Sedki Ben Romdhane - PeerSpot reviewer
Efrén Yanez - PeerSpot reviewer
Daniel Antonio Jimenez Quintana - PeerSpot reviewer
Suvajit Chakraborty - PeerSpot reviewer
SB
JS
AK