Logging network traffic offers valuable insights into data flow, enabling the proactive analysis and troubleshooting of issues as they arise. Moreover, it provides a means to access and examine the exchanged information among network users that would otherwise be inaccessible. To enhance network traffic analysis, the integration of innovative technologies that facilitate real-time querying and pattern matching proves indispensable. This research paper presents a system that exemplifies such advancements—an innovative network traffic logging tool. The tool specifically focuses on performing real-time network packet transfer to Apache Kafka and ksqlDB, leveraging their capabilities to ensure swift and dependable storage of network packets in Apache Kafka topics. By showcasing this solution, the paper demonstrates the benefits and effectiveness of employing modern technologies for network traffic analysis and management.

Capturing traffic and processing its contents is a valuable skill that when put in the right hands makes diagnosing and troubleshooting network issues an approachable task. Apart from aiding in fixing common problems, packet capture can also be used for any application that requires getting a deeper understanding of how things work under the hood. Many tools have been developed in order to allow the user to study the flow of data inside of a network. This paper focuses on documenting the process of creating such tools and showcasing their use in different contexts. This is achieved by leveraging the power of the C++ programming language and of the libtins library in order to create custom extensible sniffing tools, which are then used in VoIP (Voice over IP) and IDS (Intrusion Detection System) applications.