Compare Apache Spark vs. Spark SQL

Apache Spark is ranked 1st in Hadoop with 9 reviews while Spark SQL is ranked 8th in Hadoop with 1 review. Apache Spark is rated 8.0, while Spark SQL is rated 8.0. The top reviewer of Apache Spark writes "Fast performance and has an easy initial setup". On the other hand, the top reviewer of Spark SQL writes "A good stable and scalable solution for processing big data". Apache Spark is most compared with Spring Boot, AWS Lambda and Azure Stream Analytics, whereas Spark SQL is most compared with Apache Spark, Informatica Big Data Parser and AtScale.
Cancel
You must select at least 2 products to compare!
Apache Spark Logo
11,326 views|9,310 comparisons
Spark SQL Logo
656 views|524 comparisons
Most Helpful Review
Use Spark SQL? Share your opinion.
Find out what your peers are saying about Apache, Cloudera, Hortonworks and others in Hadoop. Updated: August 2019.
366,756 professionals have used our research since 2012.
Quotes From Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:

Pros
I found the solution stable. We haven't had any problems with it.The scalability has been the most valuable aspect of the solution.Features include machine learning, real time streaming, and data processing.The fault tolerant feature is provided.It provides a scalable machine learning library.With Hadoop-related technologies, we can distribute the workload with multiple commodity hardware.DataFrame: Spark SQL gives the leverage to create applications more easily and with less coding effort.The most valuable feature is the Fault Tolerance and easy binding with other processes like Machine Learning, graph analytics.

Read more »

The stability was fine. It behaved as expected.

Read more »

Cons
It needs a new interface and a better way to get some data. In terms of writing our scripts, some processes could be faster.The management tools could use improvement. Some of the debugging tools need some work as well. They need to be more descriptive.It should support more programming languages.Needs to provide an internal schedule to schedule spark jobs with monitoring capability.Include more machine learning algorithms and the ability to handle streaming of data versus micro batch processing.Dynamic DataFrame options are not yet available.More ML based algorithms should be added to it, to make it algorithmic-rich for developers.Stability in terms of API (things were difficult, when transitioning from RDD to DataFrames, then to DataSet).

Read more »

In the next release, maybe the visualization of some command-line features could be added.

Read more »

report
Use our free recommendation engine to learn which Hadoop solutions are best for your needs.
366,756 professionals have used our research since 2012.
Ranking
1st
out of 24 in Hadoop
Views
11,326
Comparisons
9,310
Reviews
9
Average Words per Review
184
Avg. Rating
8.0
8th
out of 24 in Hadoop
Views
656
Comparisons
524
Reviews
1
Average Words per Review
226
Avg. Rating
8.0
Top Comparisons
Compared 29% of the time.
Compared 12% of the time.
Compared 27% of the time.
Compared 17% of the time.
Learn
Apache
Apache
Overview

Spark provides programmers with an application programming interface centered on a data structure called the resilient distributed dataset (RDD), a read-only multiset of data items distributed over a cluster of machines, that is maintained in a fault-tolerant way. It was developed in response to limitations in the MapReduce cluster computing paradigm, which forces a particular linear dataflowstructure on distributed programs: MapReduce programs read input data from disk, map a function across the data, reduce the results of the map, and store reduction results on disk. Spark's RDDs function as a working set for distributed programs that offers a (deliberately) restricted form of distributed shared memory

Spark SQL is a Spark module for structured data processing. Unlike the basic Spark RDD API, the interfaces provided by Spark SQL provide Spark with more information about the structure of both the data and the computation being performed. There are several ways to interact with Spark SQL including SQL and the Dataset API. When computing a result the same execution engine is used, independent of which API/language you are using to express the computation. This unification means that developers can easily switch back and forth between different APIs based on which provides the most natural way to express a given transformation.
Offer
Learn more about Apache Spark
Learn more about Spark SQL
Sample Customers
NASA JPL, UC Berkeley AMPLab, Amazon, eBay, Yahoo!, UC Santa Cruz, TripAdvisor, Taboola, Agile Lab, Art.com, Baidu, Alibaba Taobao, EURECOM, Hitachi Solutions
Information Not Available
Top Industries
REVIEWERS
Financial Services Firm29%
Software R&D Company29%
Non Profit14%
Marketing Services Firm14%
VISITORS READING REVIEWS
Software R&D Company23%
Comms Service Provider14%
Financial Services Firm12%
Media Company8%
No Data Available
Find out what your peers are saying about Apache, Cloudera, Hortonworks and others in Hadoop. Updated: August 2019.
366,756 professionals have used our research since 2012.
We monitor all Hadoop reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.
Sign Up with Email