Apache Spark Streaming vs Databricks comparison

Cancel
You must select at least 2 products to compare!
Comparison Buyer's Guide
Executive Summary

We performed a comparison between Apache Spark Streaming and Databricks based on real PeerSpot user reviews.

Find out in this report how the two Streaming Analytics solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.
To learn more, read our detailed Apache Spark Streaming vs. Databricks Report (Updated: March 2024).
763,955 professionals have used our research since 2012.
Featured Review
Quotes From Members
We asked business professionals to review the solutions they use.
Here are some excerpts of what they said:
Pros
"As an open-source solution, using it is basically free.""The solution is better than average and some of the valuable features include efficiency and stability.""Apache Spark Streaming was straightforward in terms of maintenance. It was actively developed, and migrating from an older to a newer version was quite simple.""The solution is very stable and reliable.""Apache Spark Streaming is versatile. You can use it for competitive intelligence, gathering data from competitors, or for internal tasks like monitoring workflows.""Apache Spark Streaming has features like checkpointing and Streaming API that are useful.""Apache Spark Streaming's most valuable feature is near real-time analytics. The developers can build APIs easily for a code-steaming pipeline. The solutions have an ecosystem of integration with other stock services.""It's the fastest solution on the market with low latency data on data transformations."

More Apache Spark Streaming Pros →

"I like cloud scalability and data access for any type of user.""The solution is built from Spark and has integration with MLflow, which is important for our use case.""Databricks is a unified solution that we can use for streaming. It is supporting open source languages, which are cloud-agnostic. When I do database coding if any other tool has a similar language pack to Excel or SQL, I can use the same knowledge, limiting the need to learn new things. It supports a lot of Python libraries where I can use some very easily.""Databricks gives us the ability to build a lakehouse framework and do everything implicit to this type of database structure. We also like the ability to stream events. Databricks covers a broad spectrum, from reporting and machine learning to streaming events. It's important for us to have all these features in one platform.""I like the ability to use workspaces with other colleagues because you can work together even without seeing the other team's job.""We have the ability to scale, collaborate and do machine learning.""The most valuable feature of Databricks is the notebook, data factory, and ease of use.""Ability to work collaboratively without having to worry about the infrastructure."

More Databricks Pros →

Cons
"The service structure of Apache Spark Streaming can improve. There are a lot of issues with memory management and latency. There is no real-time analytics. We recommend it for the use cases where there is a five-second latency, but not for a millisecond, an IOT-based, or the detection anomaly-based. Flink as a service is much better.""The cost and load-related optimizations are areas where the tool lacks and needs improvement.""We would like to have the ability to do arbitrary stateful functions in Python.""In terms of improvement, the UI could be better.""There could be an improvement in the area of the user configuration section, it should be less developer-focused and more business user-focused.""The solution itself could be easier to use.""The initial setup is quite complex.""It was resource-intensive, even for small-scale applications."

More Apache Spark Streaming Cons →

"If I want to create a Databricks account, I need to have a prior cloud account such as an AWS account or an Azure account. Only then can I create a Databricks account on the cloud. However, if they can make it so that I can still try Databricks even if I don't have a cloud account on AWS and Azure, it would be great. That is, it would be nice if it were possible to create a pseudo account and be provided with a free trial. It is very essential to creating a workforce on Databricks. For example, students or corporate staff can then explore and learn Databricks.""It would be better if it were faster. It can be slow, and it can be super fast for big data. But for small data, sometimes there is a sub-second response, which can be considered slow. In the next release, I would like to have automatic creation of APIs because they don't have it at the moment, and I spend a lot of time building them.""The stability of the clusters or the instances of Databricks would be better if it was a much more stable environment. We've had issues with crashes.""Databricks can improve by making the documentation better.""I believe that this product could be improved by becoming more user-friendly.""I would like to see the integration between Databricks and MLflow improved. It is quite hard to train multiple models in parallel in the distributed fashions. You hit rate limits on the clients very fast.""A lot of people are required to manage this solution.""The solution could be improved by adding a feature that would make it more user-friendly for our team. The feature is simple, but it would be useful. Currently, our team is more familiar with the language R, but Databricks requires the use of Jupyter Notebooks which primarily supports Python. We have tried using RStudio, but it is not a fully integrated solution. To fully utilize Databricks, we have to use the Jupyter interface. One feature that would make it easier for our team to adopt the Jupyter interface would be the ability to select a specific variable or line of code and execute it within a cell. This feature is available in other Jupyter Notebooks outside of Databricks and in our own IDE, but it is not currently available within Databricks. If this feature were added, it would make the transition to using Databricks much smoother for our team."

More Databricks Cons →

Pricing and Cost Advice
  • "People pay for Apache Spark Streaming as a service."
  • "I was using the open-source community version, which was self-hosted."
  • "On a scale from one to ten, where one is expensive, or not cost-effective, and ten is cheap, I rate the price a seven."
  • More Apache Spark Streaming Pricing and Cost Advice →

  • "Whenever we want to find the actual costing, we have to send an email to Databricks, so having the information available on the internet would be helpful."
  • "I do not exactly know the costs, but one of our clients pays between $100 USD and $200 USD monthly."
  • "Licensing on site I would counsel against, as on-site hardware issues tend to really delay and slow down delivery."
  • "We find Databricks to be very expensive, although this improved when we found out how to shut it down at night."
  • "The pricing depends on the usage itself."
  • "I am based in South Africa, where it is expensive adapting to the cloud, and then there is the price for the tool itself."
  • "The price is okay. It's competitive."
  • "Databricks uses a price-per-use model, where you can use as much compute as you need."
  • More Databricks Pricing and Cost Advice →

    report
    Use our free recommendation engine to learn which Streaming Analytics solutions are best for your needs.
    763,955 professionals have used our research since 2012.
    Questions from the Community
    Top Answer:Apache Spark Streaming is versatile. You can use it for competitive intelligence, gathering data from competitors, or for internal tasks like monitoring workflows.
    Top Answer:In terms of improvement, the UI could be better. Additionally, Spark Streaming works well for various use cases, but improvements could be made for ultra-fast scenarios where seconds matter. While… more »
    Top Answer:As a data engineer, I use Apache Spark Streaming to process real-time data for web page analytics and integrate diverse data sources into centralized data warehouses.
    Top Answer:Databricks gives you the option of working with several different languages, such as SQL, R, Scala, Apache Spark, or Python. It offers many different cluster choices and excellent integration with… more »
    Top Answer:We researched AWS SageMaker, but in the end, we chose Databricks Databricks is a Unified Analytics Platform designed to accelerate innovation projects. It is based on Spark so it is very fast. It… more »
    Top Answer:Databricks is an easy-to-set-up and versatile tool for data management, analysis, and business analytics. For analytics teams that have to interpret data to further the business goals of their… more »
    Ranking
    8th
    out of 38 in Streaming Analytics
    Views
    4,523
    Comparisons
    3,689
    Reviews
    6
    Average Words per Review
    473
    Rating
    8.2
    1st
    out of 38 in Streaming Analytics
    Views
    9,698
    Comparisons
    6,228
    Reviews
    47
    Average Words per Review
    446
    Rating
    8.2
    Comparisons
    Also Known As
    Spark Streaming
    Databricks Unified Analytics, Databricks Unified Analytics Platform, Redash
    Learn More
    Overview

    Spark Streaming makes it easy to build scalable fault-tolerant streaming applications.

    Databricks is an industry-leading data analytics platform which is a one-stop product for all data requirements. Databricks is made by the creators of Apache Spark, Delta Lake, ML Flow, and Koalas. It builds on these technologies to deliver a true lakehouse data architecture, making it a robust platform that is reliable, scalable, and fast. Databricks speeds up innovations by synthesizing storage, engineering, business operations, security, and data science.

    Databricks is integrated with Microsoft Azure, Amazon Web Services, and Google Cloud Platform. This enables users to easily manage a colossal amount of data and to continuously train and deploy machine learning models for AI applications. The platform handles all analytic deployments, ranging from ETL to models training and deployment.

    Databricks deciphers the complexities of processing data to empower data scientists, engineers, and analysts with a simple collaborative environment to run interactive and scheduled data analysis workloads. The program takes advantage of AI’s cost-effectivity, flexibility, and cloud storage.

    Databricks Key Features

    Some of Databricks key features include:

    • Cloud-native: Works well on any prominent cloud provider.
    • Data storage: Stores a broad range of data, including structured, unstructured, and streaming.
    • Self-governance: Built-in governance and security controls.
    • Flexibility: Flexible for small-scale jobs as well as running large-scale jobs like Big Data processing because it’s built from Spark and is specifically optimized for Cloud environments.
    • Data science tools: Production-ready data tooling, from engineering to BI, AI, and ML.
    • Familiar languages: While Databricks is Spark-based, it allows commonly used programming languages like R, SQL, Scala, and Python to be used.
    • Team sharing workspaces: Creates an environment that provides interactive workspaces for collaboration, which allow multiple members to collaborate for data model creation, machine learning, and data extraction.
    • Data source: Performs limitless Big Data analytics by connecting to Cloud providers AWS, Azure, and Google, as well as on-premises SQL servers, JSON and CSV.

    Reviews from Real Users

    Databricks stands out from its competitors for several reasons. Two striking features are its collaborative ability and its ability to streamline multiple programming languages.

    PeerSpot users take note of the advantages of these features. A Chief Research Officer in consumer goods writes, “We work with multiple people on notebooks and it enables us to work collaboratively in an easy way without having to worry about the infrastructure. I think the solution is very intuitive, very easy to use. And that's what you pay for.”

    A business intelligence coordinator in construction notes, “The capacity of use of the different types of coding is valuable. Databricks also has good performance because it is running in spark extra storage, meaning the performance and the capacity use different kinds of codes.”

    An Associate Manager who works in consultancy mentions, “The technology that allows us to write scripts within the solution is extremely beneficial. If I was, for example, able to script in SQL, R, Scala, Apache Spark, or Python, I would be able to use my knowledge to make a script in this solution. It is very user-friendly and you can also process the records and validation point of view. The ability to migrate from one environment to another is useful.”

    Sample Customers
    UC Berkeley AMPLab, Amazon, Alibaba Taobao, Kenshoo, eBay Inc.
    Elsevier, MyFitnessPal, Sharethrough, Automatic Labs, Celtra, Radius Intelligence, Yesware
    Top Industries
    VISITORS READING REVIEWS
    Financial Services Firm19%
    Computer Software Company19%
    Comms Service Provider7%
    Manufacturing Company6%
    REVIEWERS
    Computer Software Company25%
    Financial Services Firm16%
    Retailer9%
    Manufacturing Company9%
    VISITORS READING REVIEWS
    Financial Services Firm14%
    Computer Software Company12%
    Manufacturing Company8%
    Healthcare Company6%
    Company Size
    REVIEWERS
    Small Business56%
    Midsize Enterprise11%
    Large Enterprise33%
    VISITORS READING REVIEWS
    Small Business21%
    Midsize Enterprise12%
    Large Enterprise67%
    REVIEWERS
    Small Business27%
    Midsize Enterprise14%
    Large Enterprise58%
    VISITORS READING REVIEWS
    Small Business17%
    Midsize Enterprise11%
    Large Enterprise71%
    Buyer's Guide
    Apache Spark Streaming vs. Databricks
    March 2024
    Find out what your peers are saying about Apache Spark Streaming vs. Databricks and other solutions. Updated: March 2024.
    763,955 professionals have used our research since 2012.

    Apache Spark Streaming is ranked 8th in Streaming Analytics with 6 reviews while Databricks is ranked 1st in Streaming Analytics with 47 reviews. Apache Spark Streaming is rated 8.0, while Databricks is rated 8.2. The top reviewer of Apache Spark Streaming writes "Easy deployment as a cluster and good documentation". On the other hand, the top reviewer of Databricks writes "Ahead of the competition in building data ecosystems, but needs to improve ease-of-use". Apache Spark Streaming is most compared with Amazon Kinesis, Azure Stream Analytics, Spring Cloud Data Flow, Confluent and SAS Event Stream Processing, whereas Databricks is most compared with Amazon SageMaker, Informatica PowerCenter, Microsoft Azure Machine Learning Studio, Dataiku Data Science Studio and Azure Stream Analytics. See our Apache Spark Streaming vs. Databricks report.

    See our list of best Streaming Analytics vendors.

    We monitor all Streaming Analytics reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.