Eunseop Shin

Eunseop Shin

Auto-generated profile, based on publicly available information

Eunseop Shin

Department of Computer Science and Engineering, Kyung Hee University, Yongin-si, Republic of Korea

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 November 2021 in IEEE Access
Verified
Journal Article
Open access
ICIncheon Cho
Incheon Cho
ESEunseop Shin
Eunseop Shin
MAMuhammad Salman Ali
Muhammad Salman Ali
SBSung-Ho Bae
Sung-Ho Bae
Various pruning methods have been proposed to solve the overparameterized problem in deep neural networks. Most of the structured pruning methods have used magnitude-based filter importance to remove unnecessary filters. Usually, Convolutional Neural Networks (CNN) consist of blocks that proceed with batch normalization (BN) operations after convolutional (Conv) operations. Each element is calculated through cooperation between Conv and BN, so both Conv and BN parameters must be considered together in pruning. However, previous pruning methods independently used the norm of parameters, either Conv weights or BN scales, as filter importance, ignoring these CNN structures. With this intuition, we propose a new magnitude-based filter importance method that considers both Conv weights and BN scales, and provide evidence of importance through experimental analysis on the rank of the feature map as well as mathematical analysis on the feature distortion. Furthermore, in recent works, dynamically applying a mask to recover weights has enabled more accurate pruning. However, when the weights are multiplied by zero mask values, the gradients become zero, which prevents updating the weights. We name this problem the zero gradient transferring. To solve this problem, we propose a leaky masking method that replaces the zero value of the mask with a positive constant. As a result, we solve the zero gradient transferring through the leaky masking method, enabling more accurate dynamic structured pruning. We denote our proposed method as CoBaL that incorporates Conv and BN based filter importance and Leaky mask into dynamic structured pruning. Experimental results show that our CoBaL compresses 50.09% parameters and 32.51% FLOPs on ResNet 56 with the CIFAR-10 with slightly improved accuracy. Also, we possess comparable results on ImageNet with 50.74% and 29.38% reduction in the number of parameters and FLOPs at ResNet18, respectively.

8 August 2022 in IEEE Access
Verified
Journal Article
Open access
JJJoonhyun Jeong
Joonhyun Jeong
ICIncheon Cho
Incheon Cho
ESEunseop Shin
Eunseop Shin
SBSung-Ho Bae
Sung-Ho Bae
Some conventional transforms such as Discrete Walsh-Hadamard Transform (DWHT) and Discrete Cosine Transform (DCT) have been widely used as feature extractors in image processing but rarely applied in neural networks. However, we found that these conventional transforms can serve as a powerful feature extractor in channel dimension without any learnable parameters in deep neural networks. This paper firstly proposes to apply conventional transforms on pointwise convolution, showing that such transforms can significantly reduce the computational complexity of neural networks without accuracy degradation on various classification tasks and even on face detection task. Our comprehensive experiments show that the proposed DWHT-based model gained 1.49% accuracy increase with 79.4% reduced parameters and 49.4% reduced FLOPs compared with its baseline model on the CIFAR 100 dataset while achieving comparable accuracy under the condition that 81.4% of parameters and 49.4% of FLOPs reduced on SVHN dataset. Additionally, our DWHT-based model showed comparable accuracy with 89.2% reduced parameters and 26.5% reduced FLOPs compared to the baseline models on WIDER FACE and FDDB datasets.