Transformer-based approaches have shown good results in image captioning tasks. However, current approaches have a limitation in generating text from global features of an entire image. Therefore, we propose novel methods for generating better image captioning as follows: (1) The Global-Local Visual Extractor (GLVE) to capture both global features and local features. (2) The Cross Encoder-Decoder Transformer (CEDT) for injecting multiple-level encoder features into the decoding process. GLVE extracts not only global visual features that can be obtained from an entire image, such as size of organ or bone structure, but also local visual features that can be generated from a local region, such as lesion area. Given an image, CEDT can create a detailed description of the overall features by injecting both low-level and high-level encoder outputs into the decoder. Each method contributes to performance improvement and generates a description such as organ size and bone structure. The proposed model was evaluated on the IU X-ray dataset and achieved better performance than the transformer-based baseline results, by 5.6% in BLEU score, by 0.56% in METEOR, and by 1.98% in ROUGE-L.

A significant amount of research has investigated automating medical diagnosis using deep learning. However, because medical data are collected through diagnostic tests, deep learning methods used in existing studies have had a disadvantage in that the number of training samples is insufficient and the labeling cost is high. Training approaches considering the common characteristics of medical images are needed. Therefore, in this study, we investigated approaches to overcome the lack of data for representative medical imaging tasks using transfer learning technologies. The tasks were divided into image classification, object detection, and segmentation, commonly needed functions in medical image analyses. We proposed transfer learning approaches suitable for each task that can be applied when there are little medical image data available. These approaches were experimentally validated in the following applications that share similar issues of lacking data: cervical cancer classification (image classification), skin lesion detection and classification (object detection and classification), and pressure ulcer segmentation (segmentation). We also proposed multi-task learning and ensemble learning that can be applied to these applications. Finally, the approaches were compared with state-of-the-art results. In cervical cancer analysis, the performance was improved by 5.4% in sensitivity. Skin lesion classification showed improvement in accuracy of 8.7%, precision of 28.3%, and sensitivity of 39.7%. Finally, pressure ulcer segmentation improved in accuracy by 1.2%, intersection over union by 16.9%, and Dice similarity coefficient by 3.5%.

Increasing numbers of patients with disabilities or elderly people with mobility issues often suffer from a pressure ulcer. The affected areas need regular checks, but they have a difficulty in accessing a hospital. Some remote diagnosis systems are being used for them, but there are limitations in checking a patient's status regularly. In this paper, we present a remote medical assistant that can help pressure ulcer management with image processing techniques. The proposed system includes a mobile application with a deep learning model for wound segmentation and analysis. As there are not enough data to train the deep learning model, we make use of a pretrained model from a relevant domain and data augmentation that is appropriate for this task. First of all, an image preprocessing method using bilinear interpolation is used to resize images and normalize the images. Second, for data augmentation, we use rotation, reflection, and a watershed algorithm. Third, we use a pretrained deep learning model generated from skin wound images similar to pressure ulcer images. Finally, we added an attention module that can provide hints on the pressure ulcer image features. The resulting model provides an accuracy of 99.0%, an intersection over union (IoU) of 99.99%, and a dice similarity coefficient (DSC) of 93.4% for pressure ulcer segmentation, which is better than existing results.