Compare AWS Lambda vs. Apache Spark

Cancel
You must select at least 2 products to compare!
Apache Spark Logo
11,289 views|9,168 comparisons
AWS Lambda Logo
5,315 views|4,721 comparisons
Most Helpful Review
Find out what your peers are saying about AWS Lambda vs. Apache Spark and other solutions. Updated: January 2021.
464,757 professionals have used our research since 2012.
Quotes From Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:

Pros
"I found the solution stable. We haven't had any problems with it.""The scalability has been the most valuable aspect of the solution.""The most valuable feature of this solution is its capacity for processing large amounts of data.""The solution is very stable.""I feel the streaming is its best feature.""The features we find most valuable are the machine learning, data learning, and Spark Analytics.""The main feature that we find valuable is that it is very fast.""The processing time is very much improved over the data warehouse solution that we were using."

More Apache Spark Pros »

"The ability to scale up and down very quickly helps because we can maintain our system performance and business at a low cost.""The most valuable feature of this solution is the API Gateway.""The most valuable feature is that it scans the cloud system and if they are any security anomalies it triggers an email.""The ease and speed of developing the services using any available language is the most valuable feature.""It's a serverless solution which is the best feature. It helps us because it offers free aspects. From the infrastructure perspective, it helps us manage costs. There is no overhead of estimating how much infrastructure we're going to need. We can focus on building the business functionality that we want to build.""The cool thing about AWS Lambda is that AWS does all the management. For compression, it is all about making the data small and then making it regular size again. We have an encode function and a decode function. AWS Lambda schedules each of those for us. It has a load balancer and all the fancy stuff, depending on the demand. The most valuable part of AWS Lambda is that I only need to write the software. I need to write two functions, and my cloud developer turns them into two AWS Lambda instances. That's it.""The solution is designed very well. You don't need to keep a server up. You just need some router to route your API request and Lambda provides a very well-designed feature to process the request.""Amazon takes care of the scalability. That's the right way. It's automatic and it's fully managed. That's one benefit of Lambda."

More AWS Lambda Pros »

Cons
"It needs a new interface and a better way to get some data. In terms of writing our scripts, some processes could be faster.""The management tools could use improvement. Some of the debugging tools need some work as well. They need to be more descriptive.""When you first start using this solution, it is common to run into memory errors when you are dealing with large amounts of data.""The solution needs to optimize shuffling between workers.""When you want to extract data from your HDFS and other sources then it is kind of tricky because you have to connect with those sources.""We've had problems using a Python process to try to access something in a large volume of data. It crashes if somebody gives me the wrong code because it cannot handle a large volume of data.""We use big data manager but we cannot use it as conditional data so whenever we're trying to fetch the data, it takes a bit of time.""I would like to see integration with data science platforms to optimize the processing capability for these tasks."

More Apache Spark Cons »

"Lamba functions have cold-starts that can cause some delay.""The security needs to be improved.""The running time of AWS Lambda runs fine. It takes around five minutes but it would be great if that time could be extended.""The product needs some updating as far as ease-of-customization and configuration opportunities to work with solutions outside of the cloud.""If you are setting it up on hybrid solution, there is a lot of work that needs to go in.""One area of improvement is to include support for more programming languages. AWS Lambda does not support a lot of programming languages. You have to write the Lambda functions in a certain programming language. We are using C++. My developer knows a couple of other languages. Python is his favorite language, but Python is not supported in AWS Lambda.""We need to better understand Lambda for different scenarios. We need some joint effort between Amazon and the users to have the users identify how they can really leverage Lambda. It's not about Lambda itself; it's about the practice, the guidance. There needs to be very good documentation. From the user perspective, what exists now is not always enough.""I think that perhaps Lambda could explore its functionality more."

More AWS Lambda Cons »

Pricing and Cost Advice
"Apache Spark is open-source. You have to pay only when you use any bundled product, such as Cloudera."

More Apache Spark Pricing and Cost Advice »

"AWS is slightly more expensive than Azure."

More AWS Lambda Pricing and Cost Advice »

report
Use our free recommendation engine to learn which Compute Service solutions are best for your needs.
464,757 professionals have used our research since 2012.
Questions from the Community
Top Answer: SQreamDB is a GPU DB. It is not suitable for real-time oltp of course. Cassandra is best suited for OLTP database use cases, when you need a scalable database (instead of SQL server, Postgres)… more »
Top Answer: I love every core functionality of Apache Spark Initially they have only provided RDD basic interface to process the data across distributed cluster. Then it evolved to dataframe and dataset interface… more »
Top Answer: Apache spark is available in cloud services like AWS cloud, Azure. We have to use the specific service for our use case. For example we can use AWS Glue which runs spark for ETL process, AWS EMR… more »
Top Answer: It's a serverless solution which is the best feature. It helps us because it offers free aspects. From the infrastructure perspective, it helps us manage costs. There is no overhead of estimating how… more »
Top Answer: AWS is slightly more expensive than Azure. But from a maturity perspective, AWS is way more ahead than Azure.
Top Answer: Lambda didn't use to support all the capabilities, if you wanted to initialize it before started serving the requests, it is not possible. In the most recent upgrades, they have added this feature… more »
Ranking
1st
out of 13 in Compute Service
Views
11,289
Comparisons
9,168
Reviews
13
Average Words per Review
400
Rating
8.2
2nd
out of 13 in Compute Service
Views
5,315
Comparisons
4,721
Reviews
7
Average Words per Review
663
Rating
8.4
Popular Comparisons
Learn More
Overview

Spark provides programmers with an application programming interface centered on a data structure called the resilient distributed dataset (RDD), a read-only multiset of data items distributed over a cluster of machines, that is maintained in a fault-tolerant way. It was developed in response to limitations in the MapReduce cluster computing paradigm, which forces a particular linear dataflowstructure on distributed programs: MapReduce programs read input data from disk, map a function across the data, reduce the results of the map, and store reduction results on disk. Spark's RDDs function as a working set for distributed programs that offers a (deliberately) restricted form of distributed shared memory

AWS Lambda is a compute service that lets you run code without provisioning or managing servers. AWS Lambda executes your code only when needed and scales automatically, from a few requests per day to thousands per second. You pay only for the compute time you consume - there is no charge when your code is not running. With AWS Lambda, you can run code for virtually any type of application or backend service - all with zero administration. AWS Lambda runs your code on a high-availability compute infrastructure and performs all of the administration of the compute resources, including server and operating system maintenance, capacity provisioning and automatic scaling, code monitoring and logging. All you need to do is supply your code in one of the languages that AWS Lambda supports (currently Node.js, Java, C# and Python).

You can use AWS Lambda to run your code in response to events, such as changes to data in an Amazon S3 bucket or an Amazon DynamoDB table; to run your code in response to HTTP requests using Amazon API Gateway; or invoke your code using API calls made using AWS SDKs. With these capabilities, you can use Lambda to easily build data processing triggers for AWS services like Amazon S3 and Amazon DynamoDB process streaming data stored in Amazon Kinesis, or create your own back end that operates at AWS scale, performance, and security.

Offer
Learn more about Apache Spark
Learn more about AWS Lambda
Sample Customers
NASA JPL, UC Berkeley AMPLab, Amazon, eBay, Yahoo!, UC Santa Cruz, TripAdvisor, Taboola, Agile Lab, Art.com, Baidu, Alibaba Taobao, EURECOM, Hitachi Solutions
Netflix
Top Industries
REVIEWERS
Financial Services Firm44%
Computer Software Company22%
Marketing Services Firm11%
Non Profit11%
VISITORS READING REVIEWS
Computer Software Company25%
Comms Service Provider19%
Media Company11%
Financial Services Firm10%
VISITORS READING REVIEWS
Computer Software Company28%
Media Company20%
Comms Service Provider17%
Financial Services Firm6%
Company Size
REVIEWERS
Small Business38%
Midsize Enterprise22%
Large Enterprise41%
REVIEWERS
Small Business36%
Midsize Enterprise9%
Large Enterprise55%
Find out what your peers are saying about AWS Lambda vs. Apache Spark and other solutions. Updated: January 2021.
464,757 professionals have used our research since 2012.

Apache Spark is ranked 1st in Compute Service with 13 reviews while AWS Lambda is ranked 2nd in Compute Service with 8 reviews. Apache Spark is rated 8.2, while AWS Lambda is rated 8.4. The top reviewer of Apache Spark writes "Good Streaming features enable to enter data and analysis within Spark Stream". On the other hand, the top reviewer of AWS Lambda writes "Programming is getting much easier and does not need a lot of configuration ". Apache Spark is most compared with Spring Boot, Azure Stream Analytics, AWS Batch, SAP HANA and Apache NiFi, whereas AWS Lambda is most compared with AWS Batch, Apache NiFi, Apache Storm, Google Cloud Dataflow and Azure Stream Analytics. See our AWS Lambda vs. Apache Spark report.

See our list of best Compute Service vendors.

We monitor all Compute Service reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.