The Apache NiFi user interface is…. well, not great. So it takes some time to fumble around & find the components that you need. So, in this post, we’ll make a simple ETL process, which simply lifts and shifts files from its source to somewhere else. To do that, start by dragging a processor block onto the canvas. The block looks like the below & can be found in the top toolbar.
You now have a choice of hundreds of predefined connectors. We will use the GetFile connector in this instance.
If we look at the configuration options for the connector, we can see quite a few options that we can play around with. To keep this tutorial simple, let’s leave them as they are. In future posts, we’ll deep dive into some of these. All we need to do for now is update the input directory parameter to be the current file location.
Now, repeat the above steps again. We need to add another processor, this time a PutFile block. Again, we only need to configure a single option for this block and that is ‘Directory’. We can then drag a line between the two blocks to create a process. This automatically creates the queue block.
You can right click on each block & start the process at this point. Start creating files in one directory & watch how they flow through the queue and move to the next.
If you double click on your connection, you can change the prioritisation rules, back pressure and more options.
