Azure Data Factory Scalability

AS
CTO at a construction company with 1,001-5,000 employees

The solution is scalable with no performance issues. We haven't yet reached our limit that would require scaling. Scalability is rated an eight out of ten. 

View full review »
Anirban Bhattacharya - PeerSpot reviewer
Practice Head, Data & Analytics at a tech vendor with 10,001+ employees

Azure Data Factory is scalable. The solution can move up and be aligned to resources or scaled down.

We have a lot of customers using the solution, approximately 100.

View full review »
KR
Data Governance/Data Engineering Manager at National Bank of Fujairah PJSC

When it comes to scalability and handling large datasets, it works well for datasets within the Azure environment because it's tightly integrated. 

However, for third-party integrations – things like SAP HANA, MongoDB, or handling semi-structured and unstructured data from logs – it's not as reliable. ADF excels with tight Azure cloud integration.

There are around seven end-users. Across the enterprise, Informatica is our main tool because it includes data governance (DG/DM), data quality (DQ), data cataloging, API integration, and streaming capabilities – like IDM and Informatica Cloud Services (ICS) as a SaaS platform hosted on either AWS or Azure. 

We are transitioning SSIS pipelines to ADF, but otherwise, Informatica is our central tool.

I would rate the scalability an eight out of ten.

View full review »
Buyer's Guide
Azure Data Factory
May 2024
Learn what your peers think about Azure Data Factory. Get advice and tips from experienced pros sharing their opinions. Updated: May 2024.
770,292 professionals have used our research since 2012.
Camilo Velasco - PeerSpot reviewer
CTO at Sosty

The solution is scalable.

View full review »
Thulani David Mngadi - PeerSpot reviewer
Data architect at Old Mutual

In terms of scalability, there are a few aspects of GitLab that I find disappointing. For instance, the limitation on self-hosted integration run time to just four VMs restricts scalability, especially for handling large volumes of data. Improvements are needed in this area to support more than four VMs for scalability. The documentation regarding bandwidth support is unclear, making it difficult to assess the full scalability potential. While GitLab performs well in cloud scalability in terms of compute power, the limitations on self-hosted integration run time are a concern for certain use cases. scalability in GitLab is highly dependent on specific use cases and could benefit from enhancements in self-hosted integration run time capabilities.

View full review »
Rama Subba Reddy Thavva - PeerSpot reviewer
Project Lead at Mercedes-Benz AG

Azure Data Factory is scalable, with clusters available on demand. There isn't any issue with scaling the solution.

View full review »
Rohit Sircar - PeerSpot reviewer
Integration Solutions Lead | Digital Core Transformation Service Line at Hexaware Technologies Limited

The solution is scalable and we intend to further increase its usage in the future.

View full review »
Davy Michiels - PeerSpot reviewer
Company Owner, Data Consultant at Telenet BVBA

I rate the solution’s scalability an eight out of ten.

View full review »
VT
Solution Architect at Giant Eagle

Azure Data Factory is a scalable solution. A team of 16 people from the data analytics team use the solution in our organization.

I rate the solution an eight out of ten for scalability.

View full review »
Anil Jha - PeerSpot reviewer
Director D&A at Iris Software Inc.

I have not seen any issues with respect to scalability, as it is all hosted within the cloud. We have approximately 20 users.

View full review »
Arpita-Mishra - PeerSpot reviewer
Specialist Software Engineer at a financial services firm with 10,001+ employees

Azure Data Factory is a scalable tool.

View full review »
GM
Data Architect at World Vision

So far, the performance of this solution is abysmal compared to SSIS. Especially with small tasks such as copying activity from one table to another within the same database. 

View full review »
Emad Afaq Khan - PeerSpot reviewer
Lead Architect & Scrum Master at a energy/utilities company with 10,001+ employees

In regards to scaling, when Azure Data Factory was introduced as your Databricks, it worked similarly to Hadoop or Spark, and it had some Spark clusters in the back end that could scale it as much as it could, and speed up the performance. So it is scalable, especially with Databricks, because a lot of data-related transformations can be performed.

On my team, there are approximately 20 people who work with Azure Data Factory.

View full review »
Dan_McCormick - PeerSpot reviewer
Chief Strategist & CTO at a consultancy with 11-50 employees

There is no limit to scalability.

We only have a few users. One is a data scientist, and the other is a data analyst.

We use it to push up various dashboards and reports, it's a transitional product for transferring, transforming, and transitioning data.

It is extensively used, and we intend to expand our use.

View full review »
Biswajith Gopinathan - PeerSpot reviewer
Data Analytics Specialist at GlaxoSmithKline

This solution is automatically scalable, since it's in the cloud. At my company, there were more than one thousand people using this solution because we were a big, media-based company. If there are many user requests in the front end application and the system is not responding much or has slow performance, the system will automatically scale up the performance hardware requirements. 

View full review »
Gyanendu Rai - PeerSpot reviewer
Senior Tech Consultant at Crowe

Scalability depends on the use case. 

View full review »
AS
Solution Architect at a computer software company with 1,001-5,000 employees

I rate the tool's scalability an eight out of ten. 

View full review »
reviewer826419 - PeerSpot reviewer
CIO, Director at Prosys Infotech Private Limited

The solution is scalable.

View full review »
Zubair_Ahmed - PeerSpot reviewer
Senior Consultant at Veraqor

It provides impressive scalability. There are a total of eight switches currently in use. I would rate it nine out of ten.

View full review »
PK
BI Technical Development Lead at a energy/utilities company with 10,001+ employees

Scalability-wise, I rate the solution a seven or eight out of ten. So, scalability can be improved. Also, there are around 150 people in my company using the solution. Moreover, we use the solution daily in our company.

View full review »
PedroNavarro - PeerSpot reviewer
BI Development & Validation Manager at JT International SA

The solution is scalable.

View full review »
AM
Senior Devops Consultant (CPE India Delivery Lead) at a computer software company with 201-500 employees

The scalability of the product is impressive. Scalability-wise, I rate the solution an eight out of ten.

Most of the people in my company work on Azure, and those who want to use the native ETL capabilities provided by the product opt for Azure Data Factory.

The product is useful in medium to large-sized businesses. Smaller businesses can opt for other options other than Azure Data Factory, considering the amount of money they are ready to spend. There are better options available in the market than Azure Data Factory.

View full review »
Kevin McAllister - PeerSpot reviewer
Executive Manager at Hexagon AB

I would rate Data Factory's scalability eight out of ten.

View full review »
Aurora Calderon - PeerSpot reviewer
Experienced Consultant at Bluetab

It isn't that expensive to scale Data Factory up. My client can ask for more resources on the tool, and paying more is never an issue. 

View full review »
Charles Nordine - PeerSpot reviewer
Senior Partner at Collective Intelligence

It is very scalable. It is a cloud product. It is being used by business analysts, business managers, and Azure cloud architects. We have just one developer/integrator for deployment and maintenance purposes.

We have plans to increase its usage. We'll be rolling it out for other clients.

View full review »
Pavan Yogender - PeerSpot reviewer
Founder and CEO at Zertain

Azure Data Factory is a scalable solution. We have around 15 to 20 customers, of which about 8 to 10 use Azure Data Factory.

View full review »
Mano Senaratne - PeerSpot reviewer
Management Consultant at a consultancy with 201-500 employees

Scalability-wise, Azure Data Factory is a four out of five because Microsoft is still developing certain tiers, which means you can't upgrade an older skill or tier. In contrast, the more modern, newer tiers could be upgraded easily. Rarely will you get stuck in one platform where you have completely destroyed that container and then fit a new container. Most of the time, Azure Data Factory is pretty easy to scale.

View full review »
Vishnu Derkar - PeerSpot reviewer
Sr. Big Data Consultant at a tech services company with 11-50 employees

There is a team of 30 people working on the solution. 

View full review »
DD
PRESIDENT at a computer software company with 51-200 employees

The solution is highly scalable.

View full review »
TD
Data engineer at Target

This solution is scalable.

View full review »
AS
Senior Manager at a tech services company with 51-200 employees

The solution is pretty easy to scale on Azure. I have found it to be very efficient and it is pretty fast. You just need to get the order done properly, and then you will be able to scale up.

We have about five to seven people using it at this time.

View full review »
Joaquin Marques - PeerSpot reviewer
CEO - Founder / Principal Data Scientist / Principal AI Architect at Kanayma LLC

There are thousands of users so the solution is scalable. 

View full review »
Katarzyna Palikowska - PeerSpot reviewer
ETL Developer at Det Norske Veritas

We've had no problems with Data Factory's scalability.

View full review »
PD
Works with 5,001-10,000 employees

Azure Data Factory is a scalable product.

In my current company, I have a team of five people, but in my previous organization, there were 20.

View full review »
MM
Senior Software Developer at a insurance company with 10,001+ employees

It is easy to scale. 

View full review »
AT
Senior Director/ Advisory Architect at a tech vendor with 10,001+ employees

Data Factory is scalable.

View full review »
MG
Lead BI&A Consultant at a computer software company with 10,001+ employees

Yes, Azure Data Factory is scalable.

View full review »
AN
.NET Architect at a computer software company with 10,001+ employees

It's scalable. We're doing a lot of different integrations with a lot of data, and scalability is great.

View full review »
MR
Sr. Technology Architect at a tech services company with 10,001+ employees

The solution allows you to create reusable components, so it can be scaled pretty easily.

View full review »
Sarath Boppudi - PeerSpot reviewer
Data Strategist, Cloud Solutions Architect at BiTQ

The way we've used the product is around streaming data, and that doesn't work well with Data Factory. As loads increase, some of the underlying infrastructure that gets used to process data seems to slow down. This is basically a development product, so in terms of scalability it doesn't have a wide user base, it's only meant for developers and analysts. The number of users will vary from anywhere between one to five people. 

View full review »
BS
Chief Analytics Officer at Idiro Analytics

It's a cloud solution, so it's inherently scalable. I don't know If we have to raise the limits on resources like clusters and processing power or if it will just automatically scale up. I can't remember offhand. 

View full review »
HF
Data engineer at Inicon S.r.l.

For Azure Data Factory, scalability doesn't mean really too much. However, in some scenarios, you can play with it a little bit.

Azure Data Factory is not for users. Is for engineers, for developers. The end user does not interact with Azure Data Factory. There might be 20 developers on the solution currently.

View full review »
AS
Enterprise Architect at TechnipEnergies

Scalability is one of the points that we were looking for because we are hosting approximately two terabytes of data and we expect that it will grow at least five times over the next two years. This is one of the reasons that we adopted this solution.

In perhaps a year, we will increase our usage.

View full review »
Richard Griffin - PeerSpot reviewer
Manager Data & Analytics at Fletcher Building

I haven't had to scale this solution as of yet.

We have six people in our company who use this solution.

Increasing the usage is not on our strategy pathway.

View full review »
RD
Chief Technology Officer at cornerstone defense

It is decent for most things. I'm not sure if it is necessarily intended for large volume and high-speed streams of data. By large, I mean really big, but for pretty much anything that most users would want to do, including ourselves, it is fine. Our clients are large government organizations.

It scales fine within its environment. You can literally throw another Data Factory in or replicate one and do things pretty quickly. So, it is not at all hard to increase your processing footprint, but you have to pay for it. It doesn't end up being quite expensive. Although I haven't really done it, I would suspect that if I did the equivalent in AWS, Azure would be more expensive than AWS because of the way they price data.

View full review »
CarlosAraque - PeerSpot reviewer
Data Warehouse Analyst at ACSO Australia

Data Factory is scalable.

View full review »
Abdelmonem Metwally - PeerSpot reviewer
ETL/BI Senior Consultant at Qrious

This is an easily scalable product, due to it being cloud-based.

View full review »
SS
Principal at a tech services company with 51-200 employees

The solution is scalable. 

For multi-tenant applications connected to multiple databases, Microsoft recommends a share box and a cell post integration run time. But a run time connecting to multiple sources has limitations and requires multiple shares connecting to your data if you are ingesting it from on-premises. 

View full review »
BM
IT Analyst at a tech vendor with 10,001+ employees

It's scalable.

View full review »
TF
General Manager Data & Analytics at a tech services company with 1,001-5,000 employees

We work with medium to enterprise-level organizations. Customers have anywhere from 300 employees up to 160,000 employees.

View full review »
BM
Azure Technical Architect at a computer software company with 10,001+ employees

I haven't had much experience with scalability. I know we do have scalability options though. It's used daily. 

There are around 1,000 plus users using this solution in my company. 

It requires two people for maintenance. The administrators are the ones who maintain it and give access to the engineers. They regulate who has privileges. 

View full review »
AK
Director Technology at a computer software company with 10,001+ employees

Data Factory's scalability is good.

View full review »
Jacques Du Preez - PeerSpot reviewer
Chief Executive Officer at Intellinexus

6 developers are using the solution at present. 

View full review »
GV
Head of IT at a logistics company with 10,001+ employees

The scalability seems okay. As we have only been using it for a short time, it is hard to say more. We are not currently planning to scale usage dramatically at this point but of course we would like to grow. On a scale from one to ten and from what I know, I would say scalability is an eight-out-of-ten. I can't be sure exactly how many people are using the system, but we have hundreds of thousands of users currently. Internally, I would say we use the product often.  

View full review »
BM
Azure Technical Architect at a computer software company with 10,001+ employees

I haven't put much thought into scalability.

View full review »
MK
Technical Director, Senior Cloud Solutions Architect (Big Data Engineering & Data Science) at NorthBay Solutions

The solution is easy to scale keeping in mind that Data Factory doesn't do any computations. We use it mainly to push the computations to Databricks or Synapse. Projects with our clients generally last a few months and only until they go into production. I believe the ability to increase is always there.

View full review »
LB
IT Functional Analyst at a energy/utilities company with 1,001-5,000 employees

Azure Data Factory is a very scalable solution. Including internal developers and external consultants working for us, we have about 10-15 people using this solution.

View full review »
Soumyojit Sen - PeerSpot reviewer
Director of Product Management at EIM solutions

The scalability is good. We have about five users for this solution and they are all developers.

View full review »
LG
Principal Consultant at a tech services company with 11-50 employees

It's easy to scale.

View full review »
SW
Senior Data Engineer at a real estate/law firm with 201-500 employees

Azure Data Factory is extremely simple to scale.

This solution is used by a dozen people in our company.

View full review »
GB
Principal Engineer at a computer software company with 501-1,000 employees

Being a cloud product, it can scale as much as we need.

In my project, there are ten people using this product.

View full review »
NT
Senior Systems Analyst at a non-profit with 201-500 employees

When it comes to scalability, the solution is similar to the previous Azure Synapse Analytics. They are both very easy.

View full review »
Sarath Boppudi - PeerSpot reviewer
Data Strategist, Cloud Solutions Architect at BiTQ

I haven't had the ability to scale any of my projects personally. I also wouldn't need to scale too high if I did. I'm not sure if I can speak to aspects of scalability as I've never dealt with it.

View full review »
KS
Director at a tech services company with 1-10 employees

We have not tried it scaling up. But, Azure promises the stability and scalability should not be an issue.

From a development perspective, I think there were four developers who use Azure Data Factory. From a warehouse perspective, once we roll out the reports out, it should be used by at least 40 or 50 people minimum.

View full review »
NK
Delivery Manager at a tech services company with 1,001-5,000 employees

This solution is 100% scalable.

We have two clients working with this solution.

View full review »
AT
Team Leader at a insurance company with 201-500 employees
LS
Enterprise Data Architect at a financial services firm with 201-500 employees

It is scalable. From the security perspective, it depends on how we implement it. It depends on the organization, but I don't see a challenge. 

In my previous organization, we had close to a hundred people.

View full review »
RK
Business Unit Manager Data Migration and Integration at a tech services company with 201-500 employees

We don't have any complaints regarding scalability or stability.

View full review »
DD
Principal Data Architect at Predica

The solution is scalable.

View full review »
LG
Principal Consultant at a tech services company with 11-50 employees

The scalability of the connected engines makes this solution very scalable.

View full review »
AK
Microsoft Consultant at a tech services company with 201-500 employees

The solution is scalable. Right now, we have only three or four people on it. We may increase usage int eh future.

View full review »
Buyer's Guide
Azure Data Factory
May 2024
Learn what your peers think about Azure Data Factory. Get advice and tips from experienced pros sharing their opinions. Updated: May 2024.
770,292 professionals have used our research since 2012.