We performed a comparison between Apache Spark and Spark SQL based on real PeerSpot user reviews.
Find out in this report how the two Hadoop solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI."The most valuable feature of Apache Spark is its ease of use."
"The features we find most valuable are the machine learning, data learning, and Spark Analytics."
"The product is useful for analytics."
"The good performance. The nice graphical management console. The long list of ML algorithms."
"The product's deployment phase is easy."
"The most valuable feature is the Fault Tolerance and easy binding with other processes like Machine Learning, graph analytics."
"ETL and streaming capabilities."
"It is highly scalable, allowing you to efficiently work with extensive datasets that might be problematic to handle using traditional tools that are memory-constrained."
"This solution is useful to leverage within a distributed ecosystem."
"The speed of getting data."
"Overall the solution is excellent."
"The stability was fine. It behaved as expected."
"It is a stable solution."
"Spark SQL's efficiency in managing distributed data and its simplicity in expressing complex operations make it an essential part of our data pipeline."
"I find the Thrift connection valuable."
"The performance is one of the most important features. It has an API to process the data in a functional manner."
"It would be beneficial to enhance Spark's capabilities by incorporating models that utilize features not traditionally present in its framework."
"Its UI can be better. Maintaining the history server is a little cumbersome, and it should be improved. I had issues while looking at the historical tags, which sometimes created problems. You have to separately create a history server and run it. Such things can be made easier. Instead of separately installing the history server, it can be made a part of the whole setup so that whenever you set it up, it becomes available."
"I would like to see integration with data science platforms to optimize the processing capability for these tasks."
"This solution currently cannot support or distribute neural network related models, or deep learning related algorithms. We would like this functionality to be developed."
"When you first start using this solution, it is common to run into memory errors when you are dealing with large amounts of data."
"The solution needs to optimize shuffling between workers."
"It requires overcoming a significant learning curve due to its robust and feature-rich nature."
"It should support more programming languages."
"There should be better integration with other solutions."
"It would be beneficial for aggregate functions to include a code block or toolbox that explains its calculations or supported conditional statements."
"SparkUI could have more advanced versions of the performance and the queries and all."
"In terms of improvement, the only thing that could be enhanced is the stability aspect of Spark SQL."
"I've experienced some incompatibilities when using the Delta Lake format."
"Anything to improve the GUI would be helpful."
"It takes a bit of time to get used to using this solution versus Pandas as it has a steep learning curve."
"Being a new user, I am not able to find out how to partition it correctly. I probably need more information or knowledge. In other database solutions, you can easily optimize all partitions. I haven't found a quicker way to do that in Spark SQL. It would be good if you don't need a partition here, and the system automatically partitions in the best way. They can also provide more educational resources for new users."
Apache Spark is ranked 1st in Hadoop with 60 reviews while Spark SQL is ranked 4th in Hadoop with 14 reviews. Apache Spark is rated 8.4, while Spark SQL is rated 7.8. The top reviewer of Apache Spark writes "Reliable, able to expand, and handle large amounts of data well". On the other hand, the top reviewer of Spark SQL writes "Offers the flexibility to handle large-scale data processing". Apache Spark is most compared with Spring Boot, AWS Batch, SAP HANA, Cloudera Distribution for Hadoop and Azure Stream Analytics, whereas Spark SQL is most compared with IBM Db2 Big SQL, Netezza Analytics, SAP HANA and HPE Ezmeral Data Fabric. See our Apache Spark vs. Spark SQL report.
See our list of best Hadoop vendors.
We monitor all Hadoop reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.
