Enhancing the generalization performance of CNN-based deep learning models for network traffic prediction requires a substantial amount of data. However, collecting large-scale network traffic data is costly, making data augmentation a practical alternative for improving model performance without additional data acquisition costs. Network traffic data inherently possesses both spatial and temporal characteristics, necessitating data augmentation methods that effectively incorporate both aspects. Despite this need, existing studies have largely overlooked data augmentation techniques that simultaneously address spatial and temporal features. Moreover, network traffic data often exhibits localized and granular patterns, meaning that augmented data with significant spatial deviations from the original distribution can undermine structural consistency, leading to severe performance degradation in prediction models. To address these challenges, we propose a novel spatio-temporal data augmentation framework based on a UNet architecture. The proposed method treats UNet as an inpainting model to generate augmented data by masking and reconstructing partial regions of input samples. A spatial attention module is embedded within the UNet structure to better capture localized features, while the temporal aspect is modeled by providing a sequence of n consecutive time steps as conditional input. This enables the model to generate spatially and temporally coherent augmented samples. By leveraging this inpainting-based approach, we are able to generate subtle augmentations that maintain structural consistency and temporal context, ultimately leading to improved prediction performance. All experiments were conducted on the Telecom Italia Milano dataset, a widely used real-world network traffic dataset. Experimental results demonstrate that the proposed method consistently outperforms baseline models and conventional augmentation techniques, achieving performance improvements in MSE of 15.07% in SMS, 32.7% in CALL, and 10.15% in INTERNET traffic prediction.
Recently, STDenseNet (SpatioTemporal Densely connected convolutional Network) showed remarkable performance in predicting network traffic by leveraging the inductive bias of convolution layers. However, it is known that such convolution layers can only barely capture long-term spatial and temporal dependencies. To solve this problem, we propose Attention-DenseNet (ADNet), which effectively incorporates an attention module into STDenseNet to learn representations for long-term spatio-temporal patterns. Specifically, we explored the optimal positions and the types of attention modules in combination with STDenseNet. Our key findings are as follows: i) attention modules are very effective when positioned between the last dense module and the final feature fusion module, meaning that the attention module plays a key role in aggregating low-level local features with long-term dependency. Hence, the final feature fusion module can easily exploit both global and local information; ii) the best attention module is different depending on the spatio-temporal characteristics of the dataset. To verify the effectiveness of the proposed ADNet, we performed experiments on the Telecom Italia dataset, a well-known benchmark dataset for network traffic prediction. The experimental results show that, compared to STDenseNet, our ADNet improved RMSE performance by 3.72%, 2.84%, and 5.87% in call service (Call), short message service (SMS), and Internet access (Internet) sub-datasets, respectively.
By effectively forecasting network traffic patterns, future cellular networks have the potential to manage themselves and incorporate intelligent, streamlined operations autonomously. Existing methods have relied on handcrafted architectures to predict network traffic, which still have two main limitations. First, human-designed models may have sub-optimality in modeling. Second, existing handcrafted designs did not take spatio-temporal long-term dependencies into account explicitly even though they play crucial roles in network traffic prediction. To address these issues, we propose a method using NAS (Neural Architecture Search) to find more optimized architectures than handcrafted ones for network traffic prediction. Specifically, we introduce a new search space that integrates attention modules enabling better extraction and reflection of the unique characteristics of network traffic data. These attention modules are seamlessly incorporated into the architecture from an optimization perspective. To verify the effectiveness of the proposed method, we validate the performance of the proposed framework using the Italia Telecom dataset. Experimental results show that our framework significantly improves the performance compared to the baseline in terms of root mean squared error between predicted value and ground-truth value. Specifically, our method improves the performance by 9.5%, 9.8%, and 10.2% on SMS, CALL, and INTERNET traffic datasets, respectively.
Knowledge distillation (KD) is a method of model compression. It involves extracting valuable knowledge from a high-performance and high-capacity teacher model and transferring this knowledge to a target student model having relatively small capacity. However, we discover that naively applying mixed-augmentation to KD impacts the student model’s learning through knowledge distillation negatively. We analyze this side-effect of mix-augmentation when using knowledge distillation and propose a new method that addresses this problem. That is, we found that mixed images tend to make the teacher generate unstable and poor-quality logits which hinder transferring the knowledge. To solve this problem, we decouple an input mixed image into two original images and feed them into the teacher model individually. After then we interpolate the two individual logits to generate a logit for KD. For the student, the mixed image is still used as input of the student. This decoupling strategy allows the stability of logit distributions of the teacher, thus resulting in higher KD performance with mixed augmentation. To verify the effectiveness of the proposed method, we experiment on various datasets and mixed augmentation methods, demonstrating that the proposed method showed 0.31%-0.69% improvement in top-1 accuracy compared to the original KD method on theImageNet dataset.