2019-09-02T05:33:00Z

What advice do you have for others considering Apache Airflow?

3

If you were talking to someone whose organization is considering Apache Airflow, what would you say?

How would you rate it and why? Any other tips or advice?

ITCS user
Guest
77 Answers

author avatar
Top 10Real User

I usually create my own custom operators every time. We upgraded to 2.0, but I am not using any of the new features. I haven't yet used DAG of DAGs or the new way of using Python functions in the Python operator yet. But we might use DAG of DAGs eventually. I Love this solution and I would rate it a nine out of ten.

2021-03-26T23:33:18Z
author avatar
Top 10Real User

I can recommend Apache Airflow, especially if there are serious data engineers on your team. If, on the other hand, you're looking to enable business users, then it's not suitable. I would rate Apache Airflow an eight out of ten.

2021-02-11T12:31:46Z
author avatar
Top 10Real User

We are unsure of which solution we will end up with, we are testing them currently. We are trying to get into new business types and new industries. We are looking into how well the solutions can be used in production facilities. I rate Apache Airflow an eight out of ten.

2021-01-15T22:07:12Z
author avatar
Top 20Real User

This is a good product and I definitely recommend it. I would rate this solution a seven out of ten.

2020-12-23T23:10:23Z
author avatar
Top 5LeaderboardReal User

We're just customers and end-users. We don't have a special business relationship with Apache. I'm not sure of which version of the solution we're using. It's likely the most up-to-date, or at the very most back two or three versions as we are not using any of the older versions. I'd advise others considering the solution to first understand what exactly you're trying to achieve. You either select a non-cloud native Apache workflow manager or select something that is way too big for what you are actually trying to achieve. Understand what is exactly what you need and the volumes that you need, and what exactly are the use cases. After that, in terms of deployment, that depends on what you exactly are trying to do. If all of your solutions are cloud-native, try to do it with a cloud-native tools solution. Specifically, go to the CMCS site and look into the solutions that there. Those have been tested at least for the cloud-native solutions that exist. Then, just make sure that the components you have will match and will be available to whatever you're trying to build. For example, the user management is something that is important for us and for this specific setup. Probably for some others, it's not going to be. Take into consideration, what are the different connection points and make sure that they are either supported or that you can support the integration of such items. You need to have a proper developer that can help you build your connector or your API. In general, I would rate the solution at a seven out of ten. If they fix the APIs and the price on LTK, I'd rate it closer to a nine.

2020-12-22T20:05:08Z
author avatar
Real User

We have a team of people, four to five team members, who initially evaluated Airflow and wanted to implement it. We have customers onboarded on our legacy systems. I cannot disrupt the service and bring everything into Airflow. I have to onboard Airflow seamlessly, while I protect my current, ongoing business systems. So I'm trying to balance things here. We have only been able to onboard a couple of workflows. Eventually, we want to do it more fully, but there were a few challenges as I told you: There is no pipeline to take information, which is forcing me to retain my state in a separate state repository. That would be the next big area where I would like to see improvement.

2020-04-13T06:27:00Z
author avatar
Vendor

My advice would be to use this solution for simple tasks. They should have a Python expert for features that are not available out of the box, as it is not enough. It could be a good solution for enterprise workflow automation and solutions like Control-M within the next two to three years. We are happy and satisfied with this solution, but not fully satisfied, as this solution has some positive and negative aspects. I would rate this solution a seven out of ten.

2019-09-02T05:33:00Z
Learn what your peers think about Apache Airflow. Get advice and tips from experienced pros sharing their opinions. Updated: June 2021.
512,221 professionals have used our research since 2012.