31 March 2022 in IEEE Access
The overall structure of a convolutional neural network classifier includes multiple convolutional layers and one or more linear layers. Due to the fully connected characteristics of linear layer networks, there are usually many parameters, which may easily lead to local optimization and overfitting of the model. Therefore, its indispensability in Convolutional Neural Networks (CNNs) classifier is questionable. At the same time, the excessive number of output features of the convolutional layer will also lead to the curse of dimensionality. After verifying the redundancy characteristics of the final activation function, linear layer, and the number of channels by eigen-values analysis of latent features and experiments, we propose a Fully Convolutional Neural Network (FCNN) classifier architecture, which removes the linear layers and the corresponding activation functions from the conventional CNN classifiers. By modifying the number of output channels of the last convolutional layer, the network can be trained directly through Softmax Loss. Furthermore, a softmax-free loss (POD Loss) based on Predefined Optimal-Distribution of latent features is adopted instead of Softmax Loss to obtain better recognition performance. Experiments on multiple commonly used datasets and typical networks have proved that the network structure not only reduces the amount of parameters and calculation, but also improves the recognition rate. In the meanwhile, the adoption of POD Loss further improves the classification accuracy and robustness of the model, making them play a better synergy. Code is available in https://github.com/TianYuZu/Fully-Convolutional-Network.
5 October 2022 in IEEE Transactions on Circuits and Systems for Video Technology
In the field of pattern classification, the training of deep learning classifiers is mostly end-to-end learning, and the loss function is the constraint on the final output (posterior probability) of the network, so the existence of Softmax is essential. In the case of end-to-end learning, there is usually no effective loss function that completely relies on the features of the middle layer to restrict learning, resulting in the distribution of sample latent features is not optimal, so there is still room for improvement in classification accuracy. Based on the concept of Predefined Evenly-Distributed Class Centroids (PEDCC), this article proposes a Softmax-free loss function based on predefined optimal-distribution of latent features—POD Loss. The loss function only restricts the latent features of the samples, including the norm-adaptive Cosine distance between the latent feature vector of the sample and the center of the predefined evenly-distributed class, and the correlation between the latent features of the samples. Finally, Cosine distance is used for classification. Compared with the commonly used Softmax Loss, some typical Softmax related loss functions and PEDCC-Loss, experiments on several commonly used datasets on several typical deep learning classification networks show that the classification performance of POD Loss is always significant better and easier to converge. Code is available in https://github.com/TianYuZu/POD-Loss.