Apache Spark Other Solutions Considered

Ilya Afanasyev - PeerSpot reviewer
Senior Software Development Engineer at Yahoo!

We have compared Flink and Spark as two possible options. 

View full review »
Lucas Dreyer - PeerSpot reviewer
Data Engineer at BBD

We evaluated Microsoft Synapse, which offers similar analytics functionality but not quite at the same scale as Apache Spark and Spark as a whole. While some tasks can be accomplished with Synapse on AWS, there are certain features and capabilities, such as micro-batching and scalability, that Spark excels at and remains unmatched.

View full review »
SB
CTO at Hammerknife Technologies d.o.o.

We are evaluating a few analytics engineering and DBT solutions. For now, Spark is in the secondary position.

View full review »
Buyer's Guide
Apache Spark
March 2024
Learn what your peers think about Apache Spark. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.
JK
Quantitative Developer at a marketing services firm with 11-50 employees

Currently, we extensively use pandas and Polaris. We are leveraging Docker and Kubernetes as a framework, along with AWS Batch for distribution. This is the closest substitute we have for Spark Distribution.

Both Docker and Kubernetes are more general-purpose solutions. If someone is already using Kubernetes and it's provided as a service, it can be used for special-purpose utilization, similar to Docker and Kubernetes.


In such cases, users may need to write the parallelization logic themselves, but it's relatively easy to onboard and start with a distributed load. Spark, on the other hand, is primarily used for special-purpose utilization. Users typically choose Spark when they have data-intensive tasks.

Another significant issue with Spark is its syntactics. For instance, if we have libraries like Panda or Polaris, we can run them single-threaded on a single core, or we can distribute them leveraging Kubernetes.

We don't need to rewrite that code base for Spark. However, if we are writing code specifically for Spark Executors, it will not be amenable to running it locally.

View full review »
it_user371832 - PeerSpot reviewer
Chief System Architect at a marketing services firm with 501-1,000 employees

Yes we've started to evaluate analytics databases : vertica, exasol, and other for all the them the price was an issue regarding the quantity of data we want to manipulate.

View full review »
it_user326142 - PeerSpot reviewer
Architect at a healthcare company with 51-200 employees

Yes, we considered other big data products in the Big Data Ecosystem.

View full review »
it_user374040 - PeerSpot reviewer
Systems Engineering Lead, Mid-Atlantic at a tech company with 10,001+ employees

I evaluated some other technologies such as Samza but community backing for Apache Spark stood out.

View full review »
it_user374028 - PeerSpot reviewer
Core Engine Engineer at a computer software company with 51-200 employees

We did not evaluate any other products.

View full review »
Buyer's Guide
Apache Spark
March 2024
Learn what your peers think about Apache Spark. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.