We performed a comparison between Apache Spark and Spark SQL based on real PeerSpot user reviews.
Find out in this report how the two Hadoop solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI."Apache Spark provides a very high-quality implementation of distributed data processing."
"The product’s most valuable feature is the SQL tool. It enables us to create a database and publish it."
"With Spark, we parallelize our operations, efficiently accessing both historical and real-time data."
"The processing time is very much improved over the data warehouse solution that we were using."
"Its scalability and speed are very valuable. You can scale it a lot. It is a great technology for big data. It is definitely better than a lot of earlier warehouse or pipeline solutions, such as Informatica. Spark SQL is very compliant with normal SQL that we have been using over the years. This makes it easy to code in Spark. It is just like using normal SQL. You can use the APIs of Spark or you can directly write SQL code and run it. This is something that I feel is useful in Spark."
"The product’s most valuable features are lazy evaluation and workload distribution."
"The solution has been very stable."
"The solution is scalable."
"The performance is one of the most important features. It has an API to process the data in a functional manner."
"One of Spark SQL's most beautiful features is running parallel queries to go through enormous data."
"Spark SQL's efficiency in managing distributed data and its simplicity in expressing complex operations make it an essential part of our data pipeline."
"Offers a variety of methods to design queries and incorporates the regular SQL syntax within tasks."
"It is a stable solution."
"The speed of getting data."
"Overall the solution is excellent."
"This solution is useful to leverage within a distributed ecosystem."
"Apache Spark can improve the use case scenarios from the website. There is not any information on how you can use the solution across the relational databases toward multiple databases."
"I would like to see integration with data science platforms to optimize the processing capability for these tasks."
"Technical expertise from an engineer is required to deploy and run high-tech tools, like Informatica, on Apache Spark, making it an area where improvements are required to make the process easier for users."
"The solution must improve its performance."
"When using Spark, users may need to write their own parallelization logic, which requires additional effort and expertise."
"This solution currently cannot support or distribute neural network related models, or deep learning related algorithms. We would like this functionality to be developed."
"One limitation is that not all machine learning libraries and models support it."
"It needs a new interface and a better way to get some data. In terms of writing our scripts, some processes could be faster."
"In the next release, maybe the visualization of some command-line features could be added."
"In the next update, we'd like to see better performance for small points of data. It is possible but there are better tools that are faster and cheaper."
"There are many inconsistencies in syntax for the different querying tasks."
"It would be useful if Spark SQL integrated with some data visualization tools."
"I've experienced some incompatibilities when using the Delta Lake format."
"In terms of improvement, the only thing that could be enhanced is the stability aspect of Spark SQL."
"This solution could be improved by adding monitoring and integration for the EMR."
"It takes a bit of time to get used to using this solution versus Pandas as it has a steep learning curve."
Apache Spark is ranked 1st in Hadoop with 60 reviews while Spark SQL is ranked 4th in Hadoop with 14 reviews. Apache Spark is rated 8.4, while Spark SQL is rated 7.8. The top reviewer of Apache Spark writes "Reliable, able to expand, and handle large amounts of data well". On the other hand, the top reviewer of Spark SQL writes "Offers the flexibility to handle large-scale data processing". Apache Spark is most compared with Spring Boot, AWS Batch, SAP HANA, Cloudera Distribution for Hadoop and AWS Lambda, whereas Spark SQL is most compared with IBM Db2 Big SQL, SAP HANA, HPE Ezmeral Data Fabric and Netezza Analytics. See our Apache Spark vs. Spark SQL report.
See our list of best Hadoop vendors.
We monitor all Hadoop reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.
