Project 1 (Print file): This is as simple use case, we are simply going to print the contents of a txt file. To do that, we simply use: awk ‘{print}’ filename.csv Here, the single quotes […]
Category: Data
What’s all this about Flume + Kafka (Flafka)?
There is a concept called ‘Flafka’ through which we exploit the strengths of both Kafka and Flume to create a more robust ingestion process. By deploying Flafka, we take advantage of the out of the […]
An introduction to Kafka
Kafka is a low-latency messaging system. It takes data from one location, for example application log files and makes it available to other systems with very little delay (latency). That may be useful in the […]
An introduction to Flume
Flume is similar to Kafka in many ways. It takes data from its source and distributes it to its destination. The differentiation here is that Flume is designed for high-throughput log streaming into Hadoop (HDFS […]
