Apache NiFi Valuable Features

Arjun Pandey - PeerSpot reviewer
Engineering Lead- Cloud and Platform Architecture at a financial services firm with 1,001-5,000 employees

The good thing about Apache NiFi is that it has a concept called a flow file, and there's something called a flow file processor. The processor is the building block of your entire job. They have close to 500 processors for each purpose. 

For example, for reading from Kafka, Ni-Fi has a processor called "consumer Kafka". To write to S3, they have a processor called "put S3". Now, if I read from Kafka and write my own application, I'd need to ensure the library I'm using tracks my messages. I'd also need to handle any failures by rereading messages and ensuring acknowledgment. But all this complexity is already handled by Apache processor. 

They have around 500 processors, with a community investing significant effort into developing them. I can design your processor with a single click, export the entire workflow, and import it. The format is actionable, so NiFi is immediately set up. 

It's also distributed in nature so that I can scale it across nodes based on the workload. These nodes share their state. If one node goes down during processing, that data might be lost, but any subsequent data is safe. Such occurrences are rare. 

In essence, if you want a quick solution, Apache NiFi is a strong contender. There are other solutions like AirFlow and some paid pipeline options. 

AirFlow is open-source but can be complicated. For ETL or ERT solutions, there are pricier options. But if I need a pipeline that I can monitor step by step, Apache NiFi is a good choice. It integrates with Prometheus metrics, allowing me to embed them in my workflow. 

There's also a processor for integration with Slack, and I can receive notifications when the workflow is completed or fails. 

Another feature I appreciate is "back pressure," which NiFi handles automatically. It maintains its own queue and addresses back-pressure issues. If, for instance, an upstream entity isn't fast enough, items get stored in a queue, managed internally by NiFi's back pressure algorithm.

View full review »
SabinaZeynalova - PeerSpot reviewer
Data Engineer Team Lead at Unibank

I use custom functions for specific features in Apache NiFi. I also use the processes available in NiFi. I can write custom functions to achieve the desired functionality, even if it is not explicitly available as a built-in NiFi feature.

View full review »
Bruno_Silva - PeerSpot reviewer
Partner at Timestamp, SA

The user interface is good and makes it easy to design very popular workflows. 

There are nice parameters for migration.

It is very easy to schedule jobs that realize improvements and monetize.

View full review »
Buyer's Guide
Apache NiFi
April 2024
Learn what your peers think about Apache NiFi. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,415 professionals have used our research since 2012.
RG
Senior Technology Architect at a tech services company with 10,001+ employees

It's very handy.

I like building the independent flow. 

It's an automated flow, where you can build a flow from source to destination, then do the transformation in between.

View full review »
MN
Project Engineer at Wipro Limited

We can integrate the tool with other applications easily.

View full review »
it_user340983 - PeerSpot reviewer
Infrastructure Engineer at Zirous, Inc.

The web based UI is phenomenal in allowing an easier usage of configurations than Apache Flume. We will see Hortonworks begin to slowly move away from Flume and bring Apache NiFi into their platform. NiFi allows the user to track the data flow from source to destination with a visual representation and statistics at every connection and process along the way.

The installation and configuration of a simple process that moves a file from a local server, copies it into HDFS, and indexes the file in Solr. It took all of 30 minutes at most and each process is started, stopped, deleted with a few clicks.

View full review »
Bruno_Silva - PeerSpot reviewer
Partner at Timestamp, SA

The most valuable features of this solution are ease of use and implementation. It is very visual. We can orchestrate all of the data flow right on the user interface.

We can connect each processor that we design and we can make it quicker with faster development. This is an advantage in terms of development.

Regarding the use, it is very powerful because it scales easily based on the number of workers and nodes that you have. Also, the expiration based on the computation is very easy to configure.

View full review »
it_user370506 - PeerSpot reviewer
Software Engineer at a consultancy with 1,001-5,000 employees

We are a research institution and use NiFi for its easy Java extensibility, built-in provenance capturing, and graphical web interface.

View full review »
Aria Amini - PeerSpot reviewer
Data Engineer at Behsazan Mellat

The most valuable feature has been the range of clients and the range of connectors that we could use. The processing ability of the solution is also really good because it allows high volumes of data to flow in real time. 

Overall, this solution is reliable. 

View full review »
JP
Senior Solutions Architect/ Software Architect at a comms service provider with 51-200 employees

It is really good when it comes to dealing with pipelines.

Visually, this is a good product.

View full review »
Buyer's Guide
Apache NiFi
April 2024
Learn what your peers think about Apache NiFi. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,415 professionals have used our research since 2012.