Databricks Scalability

JH
Solution Architect at a insurance company with 10,001+ employees

The solution is scalable to the cluster size and Azure storage. 

Scalability is rated an eight out of ten. 

View full review »
SS
Business Architect at YASH Technologies

It is not really scalability but more about the combination of the structure, consolidation, and different formats we can split and merge. We do a lot of things while storing the target operational model. Snowflake is more flexible and scalable in that regard. 

View full review »
AbhishekGupta - PeerSpot reviewer
Engineering Leader at Walmart

The solution is scalable with no issues from a computer perspective.

View full review »
Buyer's Guide
Databricks
April 2024
Learn what your peers think about Databricks. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,857 professionals have used our research since 2012.
Sudhendra Umarji - PeerSpot reviewer
Technical Architect at Infosys

The scalability is fine. We had thousands of devices and were sending data infrequently, so that worked for us. If the amount increases, the windowing function and job schedule may not perform as expected.

View full review »
Nabil Fegaiere1 - PeerSpot reviewer
Chief Executive Officer at dotFIT, LLC

The solution is good for scalability.

View full review »
Karan  Sharma - PeerSpot reviewer
Data Analyst at Allianz

It is a scalable solution. Scalability-wise, I rate the solution an eight to nine out of ten.

My company has a team of 50 to 60 people who use the solution.

View full review »
Avadhut Sawant - PeerSpot reviewer
Consulting Architect at a computer software company with 10,001+ employees

The solution is scalable. In this particular ecosystem, there is no one else who can catch up with Databricks for now.

View full review »
Axel Richier - PeerSpot reviewer
Tech Lead Consultant | Manager Data Engineering at Ekimetrics

It's very scalable. I have a project that started as a proof of concept on connected cars. We had 100 cars to track at first - just for the proof of concept. Now we have millions of cars that are being tracked. It scales very well. We have terabytes of data every day and it doesn't even flinch.

View full review »
Alex Tsui - PeerSpot reviewer
Sr. Director at Omnicell

The product is scalable. We're a small organization with 12 users, and we don't currently have any plans to increase our usage.

View full review »
RichardXu - PeerSpot reviewer
Data Science Lead at a mining and metals company with 10,001+ employees

It's a scalable solution. Very few people are using this solution in our organization. Most don't have the skill.

View full review »
Sahil Taneja - PeerSpot reviewer
Principal Consultant/Manager at Tenzing

There are around 30 to 35 users in our organization. 

View full review »
AO
Lead Data Scientist at a manufacturing company with 10,001+ employees

The solution is scalable. There are five people using it in our organization.

View full review »
Anand Sharma - PeerSpot reviewer
Sr Data Engineer at PIMCO

This is a very scalable solution. We have twenty-five data engineers that use it, and we may grow our usage.

View full review »
Rupal Sharma - PeerSpot reviewer
Data Architect at Three Ireland (Hutchison) - Infrastructure

I would rate the scalability an eight out of ten.

View full review »
Kevin McAllister - PeerSpot reviewer
Executive Manager at Hexagon AB

Databricks is very scalable because it's very easy to spin up multiple clusters, but the cost of doing that is tremendous. I'd rate its scalability nine out of ten, but you'll pay for it.

View full review »
PankajKumar13 - PeerSpot reviewer
Computer Scientist at Adobe

I would rate the scalability of this solution a nine, on a scale from one to 10, with one being the worst and 10 being the best. I would say there are around 2,000 to 3,000 users of this solution in our organization.

View full review »
Shiva Prasad ELLUR - PeerSpot reviewer
Vice President - Data Engineering and Analytics at a financial services firm with 10,001+ employees

We have found that part of the beauty of this platform is that it is easy to scale and expand.

View full review »
RC
Sr. BigData Architect at ITC Infotech

The system is extremely scalable. It's one of its greatest features and a big selling point. If a company needs to scale or expand, they can do so very easily.

We require daily usage from the solution even though we don't directly work with Databricks on a day to day basis. Due to the fact that we schedule everything we need and it will trigger work that needs to be done, it's used often. Do you need to log into the database console every day? No. You just need to configure it one time and that's it. Then it will deliver everything needed in the time required.

View full review »
SA
Principal at a computer software company with 5,001-10,000 employees

Databricks has high scalability. Most of my work on the solution has been in the pharma space, which has massive data sets, so it's a nine out of ten, scalability-wise.

View full review »
PraveenS - PeerSpot reviewer
Design Engineer at Cyient Limited

The tool is scalable. We can make it available for a larger audience.

View full review »
Jeremy Salt - PeerSpot reviewer
Sr. Data Quality Analyst at Seek

Databricks is absolutely scalable, and I'd rate scalability at eight out of ten. We probably have between 60 and 100 users in our organization, and we hope to increase usage in the future.

View full review »
AB
STI Data Leader at grupo gtd

Scalability is no problem. At the beginning, we created a cluster, for example, and if we need more performance in the future, for example, or to accelerate the training, we can change the cluster. It's quite straightforward. 

We have five people using the solution. 

In one or two years, we'd like to promote the solution to clients and increase usage. Right now, the way it is used is limited. I know that some banks and aeronautics companies use it.

View full review »
Elizabeth Ho - PeerSpot reviewer
Manager, Customer Journey at a retailer with 10,001+ employees

Databricks is scalable.

View full review »
DevSmita Asthana - PeerSpot reviewer
Strategic Alliances& Ecosystems Manager at a outsourcing company with 501-1,000 employees

I rate the tool's scalability an eight out of ten. 

View full review »
RM
Head of Business Integration and Architecture at Jakala

This is a scalable solution. 

View full review »
Jithin James - PeerSpot reviewer
Financial Analyst 4 (Supply Chain & Financial Analytics) at Juniper Networks

The solution is hosted on the cloud. It supports high scalability features.

10-20 users are using this solution.

View full review »
GR
Head of Referential and Big Data at a financial services firm with 5,001-10,000 employees

Databricks is scalable. You can use the power of the cloud to scale your cluster size, either CPU or memory. The data doesn't work like a standard database, so you don't have it based on files, and you don't copy the data. It's super scalable. It's only the computing that you have to scale with the data.

We probably have 40 users with roles like developers, business analysts, and data scientists. We have big plans to increase the usage and have more departments using it.

View full review »
JH
Head of Credit Risk and Data at Cegid Invoice and Financing

Databricks is based on Spark, which is based on Scala. These languages aren't easy to handle, and it's challenging to find people who know them well. At the same time, a couple of other vendors that work on top of Databricks are low-code platforms. We have to work around Databrick's lack of scalability by using low-code platforms that work on top of Databricks to give us scalability.

View full review »
RC
Data Engineering Manager at a pharma/biotech company with 10,001+ employees

It is a scalable solution, both vertically and horizontally, which is good. My organization is big, and we have a lot of users. In my department, we have about 15 people using Databricks.

View full review »
RX
Machine Learning Engineer at a mining and metals company with 10,001+ employees

Databricks is a scalable solution. It is the largest advantage of the solution.

View full review »
MahalaxmanraoChappedi - PeerSpot reviewer
Associate Principal - Data Engineering at a tech services company with 10,001+ employees

Databricks relies on scalability and performance. Every cloud vendor prioritizes scalability, high availability, performance, and security. These are the most important reasons to move to the cloud.

View full review »
Oscar Estorach - PeerSpot reviewer
Chief Data-strategist and Director at Theworkshop.es

The scalability is quite good. A company that needs to expand it can do so with ease.

We only have four people on the solution at this time. The front-end users never use the product directly. The companies aren't that big here. If the economy improves, we'll likely have more of a need for the product.

View full review »
MA
Senior Data Engineer at TCS

The salability of Databricks is good. However, if I want to use the higher clusters and high concurrency clusters, you will need to wait more time to spin up the clusters. 

We have different teams. Among them, I'm part of the data analytics where, our team, almost 10 people are using it. But I'm not sure about the rest of the teams.

We are using Databricks extensively. We have a team of 10 using the solution.

View full review »
Sanjay Bheemasenarao - PeerSpot reviewer
Director - Data Engineering expert at Sankir Technologies

I have not used autoscaling because it consumes a lot of money and because my experience has been alright. In some cases, though, it is tied to the quota of the underlying infrastructure. I have not tested the scalability to its fullest extent, but with the workloads I run, it has been fine.

View full review »
MILTON FERREIRA - PeerSpot reviewer
Co-founder/Senior Data Scientist at Hence

Databricks is scalable.

I rate the scalability of Databricks a nine out of ten.

View full review »
Trond Jensen - PeerSpot reviewer
Data Analyst at Eviny

I would rate the scalability of this solution an eight out of 10, with one being not scalable and 10 being very scalable.

There are three people using this solution in our organization.

View full review »
Olubisi Akintunde - PeerSpot reviewer
Team Lead at a tech services company with 1,001-5,000 employees

The scalability of this solution is good. In our organization, users include analysts, data engineers, and data scientists.

View full review »
IshwarSukheja - PeerSpot reviewer
Sr Manager Data Scientist at Bizmetric

The solution is easy to scale horizontally and it has a useful auto-scaling feature. For vertical scaling, you need to bring the system down and make some adjustments.

On my current project I have a team of 30 members under me, including data engineers and data science people. Our data science, engineering, and MLOps projects are expanding, so we are planning to do some vertical scaling to increase the team size to over 100 members. In our company, we are trying to certify more and more people in Databricks because it's cloud-agnostic. 

View full review »
AJ
Lead Analytics at a manufacturing company with 10,001+ employees

The scalability of Databricks is good as long as you have a data lake, and it's easy to scale.

We have approximately 50 users using this solution in my company.

View full review »
JK
Lead Architect at Birlasoft IndiaLtd.

It is a scalable solution but this depends on the platform that is being used. If you use a cloud platform such as Azure, it offers scalability. However, some platforms will not support scalability using Databricks.

We have around 20 users in our development team using Databricks. 

View full review »
Jorge Alvarado - PeerSpot reviewer
Owner at a marketing services firm with 1-10 employees

It was simple to make Databricks scalable. We found that we could set up an alert to tell us if we needed more resources, money, or time from our team. We're alerted when the system detects some trigger for any use of the instance. If you have another alert from your side, that would be extremely useful because it takes a lot of time to develop that kind of trigger. 

View full review »
AK
Coordenador Financeiro at Icatu

The solution is scalable. 

View full review »
Tristan Bergh - PeerSpot reviewer
Data Scientist at a computer software company with 501-1,000 employees

Absolutely, seamlessly, and massively scalable, within only budgetary limits. Also, the product itself offers real-time efficiency and optimization recommendations. 

View full review »
Sarbani Maiti - PeerSpot reviewer
Vice President at a tech services company with 51-200 employees

Scalability is quite good because we just used 128 GB of resources. It's quite easy to scale.

View full review »
PD
Enterprise Data Architect at a financial services firm with 51-200 employees

The solution is scalable, and we are trying to implement more use cases with Databricks in our organization as we advance. I rate the scalability an eight out of ten.

View full review »
KG
Associate Manager at a consultancy with 501-1,000 employees

I have approximately 20 users using this solution in my organization. We have plans to increase our usage in the future.

View full review »
HA
Cloud Administrator at a retailer with 5,001-10,000 employees

I would rate the tool’s scalability a seven out of ten.

View full review »
Diego Henrique Da Silva Bastos - PeerSpot reviewer
Data Engineer Analyst at Metyis

Databricks is highly scalable.

We have 200 people using the solution in my organization.

View full review »
AM
Global Data Architecture and Data Science Director at FH

Databricks is scalable, regardless of which cloud provider is being used. It is supported on Microsoft Azure, AWS, and they have their own cloud as well.

For a small workload, Databricks may not be worth the costs. However, for larger workloads, Databricks is a very good solution.

In my previous organization, there were between 10 and 15 users.

View full review »
MM
Lead Data Architect at a government with 1,001-5,000 employees

Obviously running on top of Spark, ensures fast processing and elasticity for coping with big data volumes, up to 2 petabytes. You can spin up the cluster very quickly, and shut it down. It's elastic.

View full review »
YK
Pre-sale Leader, Big Data Enterprise Solutions at Ness Technologies

We have about 200 employees but it is only a small group using Databricks. We are at the beginning so scaling is not something we have had to do.

View full review »
OB
Cloud & Infra Security, Group Manager at a tech vendor with 10,001+ employees

Databricks is scalable. The only issue is how much money you have for it. For example, if you need to run 100 servers, there's an eight-course with 256 gigabytes of RAM. You run out of money easily. It's charged to your credit card or your account, and you'll have to pay for it if you don't plan for that in advance.

View full review »
OB
IT Manager: User Support at a financial services firm with 10,001+ employees

It's a scalable solution. 

View full review »
it_user1050483 - PeerSpot reviewer
CEO at Inosense

The product is quite easy to scale and increasing the number of users is quite simple. 

View full review »
VP
Data Scientist at a energy/utilities company with 10,001+ employees

We have not really gotten to the point of scaling and testing scalability at this point. We only have two people involved with the product. One is a data scientist and one is a data engineer.  

View full review »
ZH
Data engineer

The scalability depends on the project. At present, around 20 people use the solution in my company.

View full review »
Mullai Selvan - PeerSpot reviewer
Project Manager at MAQ Software

The scalability of Databricks is good.

We have approximately 500 users using this solution in my organization.

View full review »
Natalia  Raffo - PeerSpot reviewer
Co - Founder & Chief Data Officer -CDO at Data360

It's scalable.

View full review »
RB
Business Intelligence Coordinator Latam at a construction company with 5,001-10,000 employees

Scalability is really good.

View full review »
AP
Chief Research Officer at a consumer goods company with 1,001-5,000 employees

Scalability of the solution seems very easy to achieve. 

View full review »
PG
Data Science Developer at a tech services company with 501-1,000 employees

We started using Databricks with a small PoC application, and then we developed it into a larger one. It's scalable, and it's a simple process to scale.

We have eight people in our team who are using this solution. We do not plan to increase usage at this time.

View full review »
LV
Advanced Analytics Lead at a pharma/biotech company with 1,001-5,000 employees

Databricks is scalable.

View full review »
AD
Business Intelligence and Analytics Consultant at a tech services company with 201-500 employees

Scalability is one of the main features of Databricks. We have used datasets that are one hundred megabytes in size up to one terabyte, and we can manage, so it's easily scalable.

We have a large company with between 400 and 500 people using this solution.

View full review »
BG
Data Architect at a tech services company with 201-500 employees

Both the scalability and the stability of Databricks is good.

View full review »
NH
Director of Data (Engineering & Science) at a tech services company with 11-50 employees

The solution is scalable, and there are approximately seven developers and two DevOps employees utilizing the solution.

View full review »
RP
Big Data and Cloud Architect at a computer software company with 201-500 employees

Databricks is scalable.

View full review »
SN
Head of Data & Analytics at a tech services company with 11-50 employees

Our implementations have been small and we haven't needed to scale as of yet. 

Databricks can help you to build a data lake, and it's something that they need to help make more popular. People are slowly understanding it because if you look, there are lots of data lakes that people are trying to create. I'm not intimate with it, but the concept seems complicated. I think they need to write up something where videos can explain it better. What I have seen on YouTube is quite complicated for an end-user to understand.

View full review »
SV
Engineer at a tech services company with 10,001+ employees

It's hard to say how scalable the solution is. The scalability comes into play on the Spark side, not on the Databricks side.

We have about 20 people on the solution right now.

View full review »
SH
Data Science Consultant at Syniti

We are in the prototyping stage so we do not plan on increasing our usage yet.

View full review »
DW
Machine Learning Engineer at a tech vendor with 51-200 employees

The solution is scalable. However, it still needs us to manually set out the number of nodes in a cluster. It's really dependent on the application. Sometimes, when the tasks are bigger, and it gets a little difficult for us to define the number of nodes in a cluster. If the solution could allow users to set up the clusters, I think that'll be good.

Currently, we have three people using the solution. We may increase usage in the future.

View full review »
SC
Chief Data Scientist at a tech services company with 11-50 employees

Databricks is scalable. It operates three times faster than any of the other ecosystems which we have experimented on.

We have approximately five data scientists using this solution in my organization. We are a small company and as we grow, all our data scientists would be using this platform. We plan to increase usage.

View full review »
PC
Vice President, Business Intelligence and Analytics at a tech services company with 10,001+ employees

In terms of scalability, I think once we put it across for larger use-cases the scalability question will really arise. So we'll need detailed information. I assume that we will be able to scale up.

I think we do not have more than 10 people working on it now. Because we are in the earlier stages of implementation, it's more like a POC now. I really don't know whether it's been open for the larger audience yet.

View full review »
HL
Business Development Specialist at a tech services company with 51-200 employees

The scalability of Databricks is good.

We have 30 to 40 people are using this solution in my company.

View full review »
Buyer's Guide
Databricks
April 2024
Learn what your peers think about Databricks. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,857 professionals have used our research since 2012.