What is our primary use case?
Used Azure Data Factory, Data Flow (private preview) and Databricks to develop data integration processes from multiple and varied external software sources to an OLTP application Azure SQL database. The tools are impressively well-integrated, allowing quick development of ETL, big data, data warehousing and machine learning solutions with the flexibility to grow and adapt to changing or enhanced requirements. I can't recommend it highly enough.
How has it helped my organization?
This solution will allow the organisation to improve its existing data offerings over time by adding predictive analytics, data sharing via APIs and other enhancements readily. It's as simple as extending the data pipelines with new modules and components. The solution is improving the organisation by offering something the organisation can grow with.
What is most valuable?
Data Flow and Databricks are going to be extremely valuable services, allowing data solutions to scale as the business grows and new data sources are added.
What needs improvement?
Data Flow is in the early stages — currently public preview — and it is growing into a tool that will offer everything other ETL tools offer. There are a few features still to come. The thing we missed most was data update, but this is now available as of two weeks ago. A feature that is confirmed as coming soon is the ability to pass in a parameter and filter, etc.
For how long have I used the solution?
Less than one year.