What is our primary use case?
The primary use case is to collect data from different source systems. This includes different soft types of files, such as text files, bin files, and CSV files, to name a few. It is also used for API training.
It works for a large amount of data. It is oriented for endpoint solutions and high and low frequency in small packets of data, for example, files. It can also work well when integrated with Spark, they are complementary in some use cases. At times we work only with Nifi, at times only with Spark and other times when they are integrated.
What is most valuable?
The most valuable features of this solution are ease of use and implementation. It is very visual. We can orchestrate all of the data flow right on the user interface.
We can connect each processor that we design and we can make it quicker with faster development. This is an advantage in terms of development.
Regarding the use, it is very powerful because it scales easily based on the number of workers and nodes that you have. Also, the expiration based on the computation is very easy to configure.
What needs improvement?
There should be a better way to integrate a development environment with local tools. Most of the development is done on the console. For example, in Spark, we can develop on our local desktop and then deploy it to another environment. The integration using this solution is not simple.
Eventually, we can implement a local environment on our machines that is web-oriented and we have a browsing console to do it.
At times, it is not easy to integrate with other components that could be a part of the entire solution in terms of development.
There are issues with stability due to memory. It would be good to include a lock or an alarm to detect or alert you that it needs more resources.
If the solution could be integrated more it would repair every part of the flow. Eventually, if we are integrated with other technology, we need to have a notion that we need to plan and have the correct sizing. We could implement an ecosystem that could scale with the requirements.
In the next release, I would like to see the support of monitorization with the interface. Also an integrated development environment.
For how long have I used the solution?
I have been working with Apache NiFi for a year and a half.
What do I think about the stability of the solution?
It's very stable if used properly. If we use it the way it is intended then it should work fine, it's a good solution.
There can be some issues with memory. If there were mechanisms in place to detect issues beforehand, that there was a need for available resources, it would be much easier.
What do I think about the scalability of the solution?
This solution is very scalable, depending on how the solution is implemented.
We have four or five administrators who manage this solution.
It depends on the frequency of the data source and the use case on often it is being used, but generally, it is used intensively.
How are customer service and technical support?
Because it is an open-source solution, there are forums and a community that will exchange information.
Which solution did I use previously and why did I switch?
We used other solutions previously, but this is the best one. It was more stable, it is easier to use and it deploys quickly.
How was the initial setup?
In terms of the setup, the most complex part is if you want to put it on the cloud. On a private cloud, it's tricky. If you set up on-premises, on the other hand, it's easier.
It can take a week or less to deploy the environment.
What about the implementation team?
We are part of the multinational department and they have their team in place. In my company, we do end to end. If you understand the plan it is easy to install a local environment to develop.
On the other end, if you are a customer then you need other departments to install it. Mainly departments for the infrastructure are required because there are certain requirements that the development team may not have. For example, contacts, security, and setting up hardware.
What's my experience with pricing, setup cost, and licensing?
It's an open-source solution.
What other advice do I have?
The architect needs to evaluate the entire architecture with this platform so eventually, we are left with our architects and we need to get approval from them to do that.
I would rate this solution an eight out of ten.
Which deployment model are you using for this solution?