Understanding how your data came to be in its current form is critical for any kind of data flow. When you’ve got a simple flow, you can eyeball it & understand what happened & why […]
Category: NiFi
Analyse the flow file at each step of the flow in Apache NiFi
Flow files, as we discussed in a previous article are basically the data that flows through your process. The files have attributed, which are metadata about the file (for example: created data; size) and content, […]
Using ExtractText Regex to select only matched records from input data
In many cases, we will want to route data based on its content. For example, it may be that we process data for UK citizens slightly differently to US citizens. Hence, we can create a […]
Box up your NiFi flows with process groups for easier management
When you’re developing several NiFi flows; it may be tempting to keep them all on the same top-level canvas. Doing this is not necessarily wrong, but it’s best practice to group each flow. On your […]
Generating dummy FlowFiles to test your NiFi flow
It is best practice to test that our flow works, before unleashing it (even in a test environment). The GenerateFlowFile processor “creates FlowFiles with random data or custom content. GenerateFlowFile is useful for load testing, […]
Nifi Connectors- What Can I do?
There are so many connectors available through Apache NiFi. I won’t discuss them all here; however I do want to give an idea of the capabilities that Apache NiFi has. As we have discussed before; […]
Using the GetFile & PutFile blocks to create a simple flow
The Apache NiFi user interface is…. well, not great. So it takes some time to fumble around & find the components that you need. So, in this post, we’ll make a simple ETL process, which […]
Introduction to Apache NiFi & installation
Apache NiFi is the tool to use if you need to automate the flow of data between systems. It has a simple drag and drop interface, which enables us to create pipelines to shift data […]
