Automated retinal vessel segmentation is among the most significant application and research topics in ophthalmologic image analysis. Deep learning based retinal vessel segmentation models have attracted much attention in the recent years. However, current deep network designs tend to predominantly focus on vessels which are easy to segment, while overlooking vessels which are more difficult to segment, such as thin vessels or those with uncertain boundaries. To address this critical gap, we propose a new end-to-end deep learning architecture for retinal vessel segmentation: hard attention net (HAnet). Our design is composed of three decoder networks: the first of which dynamically locates which image regions are “hard” or “easy” to analyze, while the other two aim to segment retinal vessels in these “hard” and “easy” regions independently. We introduce attention mechanisms in the network to reinforce focus on image features in the “hard” regions. Finally, a final vessel segmentation map is generated by fusing all decoder outputs. To quantify the network's performance, we evaluate our model on four public fundus photography datasets (DRIVE, STARE, CHASE_DB1, HRF), two recent published color scanning laser ophthalmoscopy image datasets (IOSTAR, RC-SLO), and a self-collected indocyanine green angiography dataset. Compared to existing state-of-the-art models, the proposed architecture achieves better/comparable performances in segmentation accuracy, area under the receiver operating characteristic curve (AUC), and f1-score. To further gauge the ability to generalize our model, cross-dataset and cross-modality evaluations are conducted, and demonstrate promising extendibility of our proposed network architecture.

Tomato spotted wilt virus is a wide-spread plant disease in the world. It can threaten thousands of plants with a persistent and propagative manner. Early disease detection is expected to be able to control the disease spread, to facilitate management practice, and further to guarantee accompanying economic benefits. Hyperspectral imaging, a powerful remote sensing tool, has been widely applied in different science fields, especially in plant science domain. Rich spectral information makes disease detection possible before visible disease symptoms showing up. In the paper, a new hyperspectral analysis proximal sensing method based on generative adversarial nets (GAN) is proposed, named as outlier removal auxiliary classifier generative adversarial nets (OR-AC-GAN). It is an all-in-one method, which integrates the tasks of plant segmentation, spectrum classification and image classification. The model focuses on image pixels, which can effectively visualize potential plant disease positions, and keep experts’ attention on these diseased pixels. Meanwhile, this new model can improve the performances of classic spectrum band selection methods, including the maximum variance principle component analysis (MVPCA), fast density-peak-based clustering, and similarity-based unsupervised band selection. Selecting spectrum wavebands reasonably is an important preprocessing step in spectroscopy/hyperspectral analysis applications, which can reduce the computation time for potential in-field applications, affect the prediction results and make the hyperspectral analysis results explainable. In the experiment, the hyperspectral reflectance imaging system covers the spectral range from 395 nm to 1005 nm. The proprosed model makes use of 83 bands to do the analysis. The plant level classification accuracy gets 96.25% before visible symptoms shows up. The pixel prediction false positive rate in healthy plants gets as low as 1.47%. Combining the OR-AC-GAN with three existing band selection algorithms, the performance of these band selection models can be significantly improved. Among them, MVPCA can leverage only 8 spectrum bands to get the same plant level classification accuracy as OR-AC-GAN, and the pixel prediction false positive rate in healthy plants is 1.57%, which is also comparable to OR-AC-GAN. This new model can be potentially transferred to other plant diseases detection applications. Its property to boost the performance of existing band selection methods can also accelerate the in-field applications of hyperspectral imaging technology.

End-to-end learning in optical communication systems is a promising technique to solve difficult communication problems, especially for peak to average power ratio (PAPR) reduction in orthogonal frequency division multiplexing (OFDM) systems. The less complex, highly adaptive hardware and advantages in the analysis of unknown or complex channels make deep learning a valid tool to improve system performance. In this paper, we propose an autoencoder network combined with extended selected mapping methods (ESLM-AE) to reduce the PAPR for the DC-biased optical OFDM system and to minimize the bit error rate (BER). The constellation mapping/de-mapping of the transmitted symbols and the phase factor of each subcarrier are acquired and optimized adaptively by training the autoencoder with a combined loss function. In the loss function, both the PAPR and BER performance are taken into account. The simulation results show that a significant PAPR reduction of more than 10 dB has been achieved by using the ESLM-AE scheme in terms of the complementary cumulative distribution function. Furthermore, the proposed scheme exhibits better BER performance compared to the standard PAPR reduction methods.

The measurement of retinal blood flow (RBF) in capillaries can provide a powerful biomarker for the early diagnosis and treatment of ocular diseases. However, no single modality can determine capillary flowrates with high precision. Combining erythrocyte-mediated angiography (EMA) with optical coherence tomography angiography (OCTA) has the potential to achieve this goal, as EMA can measure the absolute RBF of retinal microvasculature and OCTA can provide the structural images of capillaries. However, multimodal retinal image registration between these two modalities remains largely unexplored. To fill this gap, we establish MEMO, the first public multimodal EMA and OCTA retinal image dataset. A unique challenge in multimodal retinal image registration between these modalities is the relatively large difference in vessel density (VD). To address this challenge, we propose a segmentation-based deep-learning framework (VDD-Reg), which provides robust results despite differences in vessel density. VDD-Reg consists of a vessel segmentation module and a registration module. To train the vessel segmentation module, we further designed a two-stage semi-supervised learning framework (LVD-Seg) combining supervised and unsupervised losses. We demonstrate that VDD-Reg outperforms existing methods quantitatively and qualitatively for cases of both small VD differences (using the CF-FA dataset) and large VD differences (using our MEMO dataset). Moreover, VDD-Reg requires as few as three annotated vessel segmentation masks to maintain its accuracy, demonstrating its feasibility.