27 February 2023 in IEEE Transactions on Industrial Informatics
Segmentation of left and right ventricles from 3D echocardiographic images is premised and key for quantitative analysis of cardiac function, which is important for pediatric cardiac diagnosis. Compared with 2D echocardiography, 3D echocardiography can fully represent ventricular structure without geometric inference. However, it usually takes experts several hours to obtain a 3D segmentation mask of the left and right ventricles. Therefore, a fast and automatic segmentation method is highly desired. Unfortunately, 3D echocardiography suffers from low contrast, unclear left and right ventricles borders, and blind zone. Existing segmentation methods usually have poor performance at ventricular boundaries. To deal with these problems, we propose a novel dual-branch TransV-Net (DBTV). DBTV comprises two parallel, interleaved, and relatively independent V-shaped encoder-decoder branches. The main branch acts on the original data to extract image features, and the auxiliary branch acts on edge maps to extract the additional edge features. To suppress noise and enhance the edge information, extra concatenations are added to bridge the features from the main and auxiliary branches. To reduce object missing caused by blind zone, a 3D transformer-based module is proposed in the bottom layer of the dual-branch structure to extract the global contexts. We do experiments on a self-collected dataset with 120 3D echocardiographic images from 60 cardiac sequences, and the dice score of 0.913 and 0.880 are obtained in the left and right ventricle segments, respectively. Each inference takes about two-thirds of a second for a single 3D frame.
19 January 2024 in IEEE Access
Few-shot image classification, whose goal is to generalize to unseen tasks with scarce labeled data, has developed rapidly over the years. However, in traditional few-shot learning methods with CNNs, non-local features and long-rang dependencies of the image may be lost, and this leads to a poor generalization of the trained model. With the advantage of the self-attention mechanism of Transformer, researchers have tried to use vision transformer to improve few-shot learning recently. However, these methods are more complicated and take up a lot of computing resources, and there is no baseline to measure their effectiveness. We propose a new method called ViTFSL-baseline. We take advantage of vision transformer and train our model on all train set without episodic training. Meanwhile, we design a new nearest-neighbor classifier to used for few-shot image classification. Furthermore, in order to narrow the gap between difference of same class, we introduce centroid calibration in classifier after the feature extraction of backbone. We run the experiments on popular benchmarks to show that our method is a simple and effective for few-shot image classification. Our approach could be taken as the baseline upon vision transformer for few-shot learning.
22 May 2024 in Biomedical Signal Processing and Control
22 January 2024 in Engineering Applications of Artificial Intelligence
1 March 2022 in Signal Processing: Image Communication
25 March 2022 in IEEE Internet of Things Journal
The progress of the deep neural network and visual sensors promote the facial landmark detection. However, faces are easily collected from Internet of Things (IoT) devices worldwide but are hard-labeled facial landmarks in consistent style. Though many weak supervision (WS) algorithms and theories have been proposed to handle the labeling problem, most of them are tailored for classification tasks and fail in regression tasks, especially in facial landmark detection. To tackle this WS regression task, first, we propose a regressive pseudo-labeling method by analyzing the cluster assumption of facial landmark detection, where overlaps are reduced and interval areas are increased among clusters of facial parts for unlabeled faces. Moreover, auxiliary information and domain loss are utilized to adapt model to samples with different styles. Second, we design a generator–regressor network to first estimate facial boundary attentions and then to locate facial landmarks. However, when generating pseudo labels based on predictive models automatically, there are two major issues. One is that the results of the multistage network highly depend on the former-stage accuracy, and another is that jointing different annotation styles always produces ambiguity feature representation. Thus, we propose two ideas. During training, the generator and regressor are decoupled to alleviate the inner dependence of the multistage network, and the heatmap discriminator is introduced to improve the quality of the predicted facial boundary. We design a transformer structure to fuse face image and boundary attentions, so that it can further complement useful features. Based on these ideas, our methods can automatically annotate accurate pseudo facial landmarks for unlabeled faces. Extensive experiments show that our model achieves good performance on different benchmark data sets, both in accuracy and efficiency.