We performed a comparison between Apache Spark and Spark SQL based on real PeerSpot user reviews.
Find out in this report how the two Hadoop solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI."The product’s most valuable features are lazy evaluation and workload distribution."
"ETL and streaming capabilities."
"The distribution of tasks, like the seamless map-reduce functionality, is quite impressive."
"The product's deployment phase is easy."
"It provides a scalable machine learning library."
"Apache Spark provides a very high-quality implementation of distributed data processing."
"It is useful for handling large amounts of data. It is very useful for scientific purposes."
"Features include machine learning, real time streaming, and data processing."
"The speed of getting data."
"It is a stable solution."
"Offers a variety of methods to design queries and incorporates the regular SQL syntax within tasks."
"Certain data sets that are very large are very difficult to process with Pandas and Python libraries. Spark SQL has helped us a lot with that."
"The team members don't have to learn a new language and can implement complex tasks very easily using only SQL."
"Spark SQL's efficiency in managing distributed data and its simplicity in expressing complex operations make it an essential part of our data pipeline."
"One of Spark SQL's most beautiful features is running parallel queries to go through enormous data."
"The performance is one of the most important features. It has an API to process the data in a functional manner."
"When you first start using this solution, it is common to run into memory errors when you are dealing with large amounts of data."
"Apache Spark provides very good performance The tuning phase is still tricky."
"Stream processing needs to be developed more in Spark. I have used Flink previously. Flink is better than Spark at stream processing."
"It needs a new interface and a better way to get some data. In terms of writing our scripts, some processes could be faster."
"Apart from the restrictions that come with its in-memory implementation. It has been improved significantly up to version 3.0, which is currently in use."
"It's not easy to install."
"There were some problems related to the product's compatibility with a few Python libraries."
"At the initial stage, the product provides no container logs to check the activity."
"Anything to improve the GUI would be helpful."
"It takes a bit of time to get used to using this solution versus Pandas as it has a steep learning curve."
"This solution could be improved by adding monitoring and integration for the EMR."
"The solution needs to include graphing capabilities. Including financial charts would help improve everything overall."
"SparkUI could have more advanced versions of the performance and the queries and all."
"There should be better integration with other solutions."
"Being a new user, I am not able to find out how to partition it correctly. I probably need more information or knowledge. In other database solutions, you can easily optimize all partitions. I haven't found a quicker way to do that in Spark SQL. It would be good if you don't need a partition here, and the system automatically partitions in the best way. They can also provide more educational resources for new users."
"In the next update, we'd like to see better performance for small points of data. It is possible but there are better tools that are faster and cheaper."
Apache Spark is ranked 1st in Hadoop with 60 reviews while Spark SQL is ranked 4th in Hadoop with 14 reviews. Apache Spark is rated 8.4, while Spark SQL is rated 7.8. The top reviewer of Apache Spark writes "Reliable, able to expand, and handle large amounts of data well". On the other hand, the top reviewer of Spark SQL writes "Offers the flexibility to handle large-scale data processing". Apache Spark is most compared with Spring Boot, AWS Batch, SAP HANA, Cloudera Distribution for Hadoop and AWS Lambda, whereas Spark SQL is most compared with IBM Db2 Big SQL, HPE Ezmeral Data Fabric, SAP HANA and Netezza Analytics. See our Apache Spark vs. Spark SQL report.
See our list of best Hadoop vendors.
We monitor all Hadoop reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.
