We performed a comparison between Apache Spark and AWS Lambda based on real PeerSpot user reviews.
Find out in this report how the two Compute Service solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI."We use Spark to process data from different data sources."
"The solution is scalable."
"The most valuable feature of Apache Spark is its memory processing because it processes data over RAM rather than disk, which is much more efficient and fast."
"It is highly scalable, allowing you to efficiently work with extensive datasets that might be problematic to handle using traditional tools that are memory-constrained."
"The processing time is very much improved over the data warehouse solution that we were using."
"Apache Spark provides a very high-quality implementation of distributed data processing."
"We use it for ETL purposes as well as for implementing the full transformation pipelines."
"Apache Spark can do large volume interactive data analysis."
"AWS Lambda's best features are log analysis and event triggering and actioning."
"We moved our users into the Amazon Cognito pool, so it helps us to standardize our security practices, approaches, etc. We can customize Lambda for authentication to integrate it with API Gateway and other services."
"We are building a Twitter-like application in the boot camp. I have used Lamda for the integration of the post-confirmation page in the application. This will help you get your one-time password via mail. You can log in with the help of a post-confirmation page. We didn’t want to setup an instance specifically for confirmation. We used the Lambda function so that it goes back to sleep after pushing up."
"The most valuable feature of AWS Lambda, from a conceptual point, is its functions. For example, it's mathematical templates into which you can write, and create your solution. You write small pieces of a solution under given parameters."
"The initial setup is pretty easy."
"We have no issues with the technical support."
"The tool scales automatically based on the number of incoming requests."
"It is serverless and scalable. It can scale infinitely. You don't have to worry about the size of the servers that you're pre-allocating. You don't have to build server scale-out models. Auto scale and other similar features are just inherent in Lambda. So, for atomic and fairly non-persistent transactional units of work, Lambda works very well."
"The initial setup was not easy."
"When you first start using this solution, it is common to run into memory errors when you are dealing with large amounts of data."
"When you are working with large, complex tasks, the garbage collection process is slow and affects performance."
"We use big data manager but we cannot use it as conditional data so whenever we're trying to fetch the data, it takes a bit of time."
"The solution needs to optimize shuffling between workers."
"Apart from the restrictions that come with its in-memory implementation. It has been improved significantly up to version 3.0, which is currently in use."
"At times during the deployment process, the tool goes down, making it look less robust. To take care of the issues in the deployment process, users need to do manual interventions occasionally."
"Dynamic DataFrame options are not yet available."
"The security needs to be improved."
"If it is a specific ETL process or a long-term one, then AWS Lambda is not a good option."
"My opinion is that the integration could be improved in this solution. We have had some difficulties integrating the EC2 module, but we found a solution for that by ourselves."
"We've had to revamp the way that it works due to that 15-minute timeout limitation."
"I would like the layers to have a bigger volume. I would like to be able to add more. I don't want to be limited by the layer."
"Lambda's dashboard could be more user-friendly and customizable. I want the dashboard to have more information to quickly identify what functions and events are running. Also, we want to be able to add more trigger points, push notifications, and events."
"AWS Lambda could be improved by increasing the size of the payload. Also, sometimes Lambda doesn't implement well for bigger solutions."
"If you're running a new application with a significant load, you need to be prepared for potential bottlenecks."
Apache Spark is ranked 5th in Compute Service with 60 reviews while AWS Lambda is ranked 1st in Compute Service with 70 reviews. Apache Spark is rated 8.4, while AWS Lambda is rated 8.6. The top reviewer of Apache Spark writes "Reliable, able to expand, and handle large amounts of data well". On the other hand, the top reviewer of AWS Lambda writes "An easily scalable solution with a variety of use cases and valuable event-based triggers". Apache Spark is most compared with Spring Boot, AWS Batch, Spark SQL, SAP HANA and Apache NiFi, whereas AWS Lambda is most compared with AWS Batch, Amazon EC2 Auto Scaling, Apache NiFi, AWS Fargate and Google Cloud Dataflow. See our AWS Lambda vs. Apache Spark report.
See our list of best Compute Service vendors.
We monitor all Compute Service reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.
