23 September 2021 in IEEE Transactions on Instrumentation and Measurement
Semantic segmentation is of great importance and a challenge in computer vision. One of its main problems is how to efficiently obtain rich information (geometric structure) and identify useful features from higher dimensions. A light field camera, due to its special microlens array structure, can completely record the angular-spatial information of scenes, which is attractive and has great potential to improve the performance of semantic segmentation tasks. Inspired by this, we propose an end-to-end semantic segmentation network that can process light field macropixel images robustly and extract their features. In addition, this network can flexibly and efficiently load the different popular deep learning backbones. Furthermore, we propose an efficient angular model, which, to learn the angular features between the different viewpoints of the macropixel image, improves the nonlinearity of angular-spatial features and enhances multichannel semantic correlations. To evaluate the network, we construct a new real scene light field dataset comprising 800 high-quality samples. The quantitative and qualitative results show that the highest mean intersection over union (mIoU) based on our algorithm is greater than 57%. Our algorithm achieves a 10.30% increase compared with state-of-the-art semantic segmentation algorithms. In combination with different backbones or multiscale light field macropixel images, the network can also achieve comparable results. This preliminary work demonstrates that the combination of light field imaging and deep learning technology has potential applications in the future study of semantic segmentation.
25 July 2022 in Frontiers of Information Technology & Electronic Engineering
Light field (LF) imaging has attracted attention because of its ability to solve computer vision problems. In this paper we briefly review the research pro
10 December 2021 in IEEE Sensors Journal
The application of THz technique into detecting myocardial amyloidosis exists some challenges: the irregularity of tissues, the overlap of spectra, the few sampling points and low signal-to-noise ratio (SNR) of spectra. In this paper, we propose an efficient method to construct the THz dataset of myocardial amyloidosis, which includes 4319 samples. Besides, we propose a convolutional neural networks (CNNs) based framework to detect myocardial amyloidosis. Benefiting from both the THz technique and CNNs, an accuracy of 98.37%, a precision of 97.94%, a recall of 98.85% and a F1-score of 98.39% are achieved. In 10 independent repeated tests, the extreme deviations of the four indicators obtained are within 0.65%, which proves that the proposed model has prominent stability. Furthermore, we have compared it with the existing sensors, which shows that our method provides innovation and improvement in the simplicity of operation, the efficiency of detection and the reliability of discrimination. To verify the comparability of CNNs, we compare it with the state-of-the-art (SOTA) algorithms and has achieved all the best performance under different number of samples. Not only that, under the circumstances of fewest sampling points and lowest SNR, the CNNs can still achieve a F1-score of 93.74% and 94.68%, respectively. This preliminary work indicates that the combination of THz-TDS and CNNs is of great potential in accurately and efficiently detecting myocardial amyloidosis.
16 July 2023 in Multimedia Systems
16 August 2022 in IEEE Transactions on Intelligent Transportation Systems
Multi-object tracking is of great importance in autonomous driving. However, with the outbreak of COVID-19, multi-object tracking faces new challenges in areas gripped by epidemics because of complex motion blur, frequent occlusions, and appearance deformations. To reliably improve object trajectory association in epidemic-plagued areas, we propose a temporal-spatial aggregation embedding network (TSAEN) for multi-object tracking. Our embedding network contains a temporal-aware correlation module (TACM) and spatial-aggregate embedding module (SAEM) that can fully obtain and aggregate appearance clues related to moving objects in previous frames. The TACM learns the temporal homogeneity features of the current and previous frames to perceive features with correlated appearance cues. Then, the SAEM adjusts the spatial deformation for each perceived temporal homogeneity feature and aggregates them for re-ID embedding learning. The experimental results demonstrate that our proposed method is able to achieve excellent overall performance.