Persistent Infrequent Flows (PIFs) refer to the packet flows that last for a long time but always at low frequencies in network traffic. Accurate identification of the PIFs plays a vital role in intrusion detection, attack prevention, traffic engineering, and other network fields. However, existing methods often require to save all flows for finding out the PIFs due to their infrequency feature, which brings about the problem of low identification accuracy and high memory overhead. To solve this problem, this paper proposes an accurate PIF identification method with low overhead called PIF-Identifier, composed of a new-flow discriminator and a PIF tracker. Specifically, we first design a compact new-flow discriminator by applying probabilistic data structures, to quickly determine whether a packet flow arrives for the first time within current time window. Then we design a PIF tracker to accurately identify and report persistent infrequent flows. In the PIF tracker, we configure a small-size frequency counter for each tracked flow in accordance with the frequency threshold of the PIF, without sacrificing the accuracy of PIF identification. Furthermore, we design a probabilistic replacement strategy based on the number of time windows of flow persistence, to accommodate newly arrived potential PIFs when there is no vacancy in their mapped buckets of the PIF tracker. Finally, we evaluate the performance of our proposed PIF-Identifier by theoretical analysis and experimental verification with real network traffic traces. Experimental results indicate that the PIF-Identifier achieves the precision of 100%, much higher recall rate and F1 score, as well as lower average relative error than the state-of-the-art methods, significantly promoting the identification performance of persistent infrequent flows.

Persistent and infrequent (PI) flows refer to packet flows that last for a long time at low packet arrival rates in network traffic. It is crucial to accurately find these flows for uncovering stealthy network threats, such as advanced persistent threats (APTs) and low-rate denial-of-service attacks (LDoS). To this end, we propose a low-overhead and accurate method for detecting persistent and infrequent flows called PIF-Detector, which separates the monitoring and identification functions of PI flows. Specifically, we design a monitoring table composed of bucket arrays to store and monitor potential PI flows, and an identifier composed of compact bit vectors to recognize potential PI flows. For the monitoring table, we design an adjacent-bucket relocation strategy to improve bucket utilization and thereby enhance PI-flow detection accuracy. For the identifier, we track flow arrivals across consecutive time windows using compact bit vectors to detect PI flows with low memory overhead. Finally, we perform theoretical derivations on the false positive rate and the false negative rate of PIF-Detector, and evaluate its performance through experiments on three datasets. Experimental results show that PIF-Detector achieves up to 99.57% precision and 99.93% recall, outperforming state-of-the-art solutions under the same memory overhead.