Deep neural network (DNN) has demonstrated astounding performance in large-scale image recognition task, and pre-trained DNN models trained for one task have also been applied to domains different from their original purposes. Following such an idea, a novel hand-dorsa vein recognition model is constructed by adopting DNN pre-trained on a large-scale database as a universal feature descriptor. Unlike most of these studies which adopt activations of the fully connected layer of DNN as the image representation, we adopt convolutional activations as the region representation. However, not local features of all regions are equally important for final classification. Thus, to solve this issue, a novel selective convolutional feature model based on spatial weighting is proposed to acquire more robust and discriminative feature representation. In specific, a spatial weighting scheme is applied on the convolutional activations to weigh the importance of local features at different regions for classification, which further enhances the discriminability of feature representation. Besides, to take full advantage of the spatial information of the convolutional activations, spatial pyramid pooling is introduced to obtain feature representation with rich spatial information. The final image representation is formed by concatenating the local features of different level of the spatial pyramid. Series rigorous experiments on the lab-made database are conducted to evidence the effectiveness and feasibility of the proposed model. What is more, an additional experiment with subset of PolyU database illustrates its generalization ability and robustness.

The recent success of Deep Convolutional Neural Network (DCNN) for various computer vision tasks such as image recognition has already demonstrated its robust feature representation ability. However, the limitation of training database on small scale vein recognition tasks restricts its performance because the recognition result of DCNN depends heavily on the number of trainsets. This motivates the design of a Multi-Scale Deep Representation Aggregation (MSDRA) model based on a pre-trained DCNN for vein recognition. First, the multi-scale feature maps are extracted by a pre-trained DCNN model. Second, a local mean threshold approach is designed to preliminarily remove the noisy information of multi-scale feature maps and generate the selected feature maps. Third, we propose an Unsupervised Vein Information Mining (UVIM) method to localize vein information of selected feature maps for generating a binary vein information mask, and then the vein information mask is utilized to keep useful deep representation and discard the background information. Finally, the discriminative multi-scale deep representations, which are generated by using the vein information mask to aggregate multi-scale feature maps, are concatenated into the final compact feature vectors, and then a Support Vector Machine (SVM) is introduced for final recognition. Our proposed model outperforms the state-of-the-art methods on two benchmark vein databases. Moreover, an additional experiment using the subset of PolyU Palmprint database illustrates the system’s generalization ability and robustness.

Pre-trained DCNN trained on a large-scale image database can be used as a universal feature representation for image classification, which has achieved significant progress in some image recognition tasks. Compared with other image recognition tasks, directly utilizing a single convolutional feature as feature representation for vein recognition task cannot achieve the impressive result due to the sparse distribution of vein information. Therefore, to obtain more representative and discriminative convolutional feature for vein recognition, a novel multi-layer convolutional features' concatenation with semantic feature selector is proposed in this paper. In the pre-trained DCNN, different convolutional layers can encode different-level feature information. High-level convolutional features with vein information cover more semantic information and low-level convolutional features with vein information cover more detail information. However, low-level convolutional features also contain some background information. Therefore, in order to remove the background information of low-level convolutional features, a novel semantic feature selector is presented. First, the proposed local max-pooling of preserving spatial position (LMP-PSP) information is applied on activation map obtained by adding up all feature maps of the high-level convolutional layer to generate the semantic weighting map, which reflects key vein information of high-level convolutional features. Then, semantic weighting map is regarded as a feature selector to discard the background information of the low-level convolutional features and preserve the detail information of low level convolutional features. Finally, low-level convolutional features with vein information are selectively linked to high-level convolutional features with vein information based on the proposed semantic feature selector. A series of rigorous experiments on two lab-made vein databases named CUMT-Hand-Dorsa Vein database and CUMT-Palm Vein database is conducted to verify the effectiveness and feasibility of the proposed model. Besides, additional experiments with PUT Palm Vein database and the subset of PolyU database illustrate its generalization ability and robustness.

Recently, vein recognition has been paid more considerable attention in biometric recognition fields. In the process of vein image acquisition, due to the influence of external factors such as illumination change, the texture information of vein images with the same identity information may change, which enhances the difference of intra-class and extremely degrades the performance of vein recognition systems. To address this problem, we proposed a Disentangled Representation and Enhancement Network for vein recognition called DRE-Net. First, robust vein shape masks are obtained by the designed vein segmentation algorithms, which are utilized as label information in DRE-Net to acquire the shape features of vein images. Second, a disentangled representation network which contains two encoders and two decoders is designed to disentangle the texture features and shape features of vein images. Besides, a Multi-Scale Attention Residual Block (MSARB) is presented to better mine the vein network information and enhance the representation ability of DRE-Net for vein images. Finally, a Weight-Guided Feature Enhancement Module (WGFEM) is presented to obtain more discriminative representations for vein recognition by reducing the importance of texture features and increasing the importance of shape features, which decreases the difference of intra-class caused by illumination change. Extensive experiments have been carried out on three benchmark vein databases, and the experimental results demonstrate that our proposed model outperforms the state-of-the-art methods.

Deep convolutional neural network (DCNN) has achieved an outstanding performance in large-scale image recognition task because of its discriminative feature representation ability, and pre-trained DCNN models trained for one task have also been applied to domains that are different from their original purposes. Inspired by this idea, a novel hand-dorsa vein recognition model is constructed by adopting DCNN pre-trained on a large-scale database as a universal feature descriptor. However, due to the sparse distribution property of vein information, it is difficult to employ pre-trained DCNN model to extract discriminative deep convolutional features. Therefore, to obtain useful and discriminative deep convolutional features, a novel minutiae-based weighting aggregation (MWA) method is proposed. In specific, the proposed global max-pooling of preserving spatial position information is applied on the feature maps of convolutional layer to localize the minutiae of vein information, and then the minutiae feature of vein image is regarded as the mask that is named as minutiae feature mask, to select deep convolutional features that contain minutiae feature information of vein image. The final feature representation is formed by concatenating each selected deep convolutional feature that is generated by each minutiae feature mask. Series rigorous experiments on the lab-made database are conducted to evidence the effectiveness and feasibility of the proposed MWA for vein recognition. What's more, an additional experiment with subset of PolyU database illustrates its generalization ability and robustness.

Multimodal biometric recognition has shown great potential in identity authentication tasks and has attracted increasing interest recently. Currently, most existing multimodal biometric recognition algorithms require test samples with complete multimodal data. However, it often encounters the problem of missing modality data and thus suffers severe performance degradation in practical scenarios. To this end, we proposed a hierarchical cross-modal image generation for palmprint and palmvein based multimodal biometric recognition with missing modality. First, a hierarchical cross-modal image generation model is designed to achieve the pixel alignment of different modalities and reconstruct the image information of missing modality. Specifically, a cross-modal texture transfer network is utilized to implement the texture style transformation between different modalities, and then a cross-modal structure generation network is proposed to establish the correlation mapping of structural information between different modalities. Second, multimodal dynamic sparse feature fusion model is presented to obtain more discriminative and reliable representations, which can also enhance the robustness of our proposed model to dynamic changes in image quality of different modalities. The proposed model is evaluated on three multimodal biometric benchmark datasets, and experimental results demonstrate that our proposed model outperforms recent mainstream incomplete multimodal learning models.