Convolutional neural networks (CNNs) have achieved promising results in many tasks, and evaluating the model’s generalization ability based on the trained model and training data is paramount for practical applications. Although many measures for evaluating the generalization of CNN models have been proposed, the existing works are limited to small-scale or simplified model sets, which would result in poor accuracy and applicability of the derived methods. This study addresses these limitations by leveraging ResNet models as a case study to evaluate the model’s generalization ability. We utilized Intersection over Union (IoU) as a method to quantify the ratio of task-relevant features to assess model generalization. Class activation maps (CAMs) were used as a representation of the distribution of features learned by the model. To systematically investigate the generalization ability, we constructed a diverse model set based on the ResNet architecture. A total of 2000 CNN models were trained on the ImageNet subset by systematically changing commonly used hyperparameters. The results of our experiments revealed a strong correlation between the IoU-based evaluation method and the model’s generalization performance (Pearson correlation coefficient more than 0.8). We also performed extensive experiments to demonstrate the feasibility and robustness of the evaluation methods.
To enhance the attack resistance of the Controller Area Network (CAN) system and optimize the communication software design, a comprehensive model that combines a variable attacker with the CAN bus (VACB) is proposed to evaluate the bus communication risk. The VACB model consists of a variable attacker and the CAN bus model. A variable attacker is a visualized generation of the attack traffic based on a recurrent neural network (RNN), which is used to evaluate the anti-attack performance of the CAN bus; the CAN bus model combines the data link layer and the application layer to analyze the anomalies in CAN bus data transmission after the attack message. The simulation results indicate that the transmission accuracy and successful response rate decreased by 1.8% and 4.3% under the constructed variable attacker. The CAN bus’s authenticity was promoted after the developers adopted this model to analyze and optimize the software design. The transmission accuracy and the successful response rate were improved by 2.5% and 5.1%, respectively. Moreover, the model can quantify the risk of potential attacks on the CAN bus, prompting developers to avoid it in early development to reduce the loss caused by system crashes. The comprehensive model can provide theoretical guidance for the timing design of embedded software.
1 January 2026 in Chinese Journal of Space Science