There has been a substantial amount of research involving computer methods and technology for the detection and recognition of diabetic foot ulcers (DFUs), but there is a lack of systematic comparisons of state-of-the-art deep learning object detection frameworks applied to this problem. DFUC2020 provided participants with a comprehensive dataset consisting of 2,000 images for training and 2,000 images for testing. This paper summarizes the results of DFUC2020 by comparing the deep learning-based algorithms proposed by the winning teams: Faster R-CNN, three variants of Faster R-CNN and an ensemble method; YOLOv3; YOLOv5; EfficientDet; and a new Cascade Attention Network. For each deep learning method, we provide a detailed description of model architecture, parameter settings for training and additional stages including pre-processing, data augmentation and post-processing. We provide a comprehensive evaluation for each method. All the methods required a data augmentation stage to increase the number of images available for training and a post-processing stage to remove false positives. The best performance was obtained from Deformable Convolution, a variant of Faster R-CNN, with a mean average precision (mAP) of 0.6940 and an F1-Score of 0.7434. Finally, we demonstrate that the ensemble method based on different deep learning methods can enhance the F1-Score but not the mAP.

Event cameras are bio-inspired vision sensors that mimic retinas to asynchronously report per-pixel intensity changes rather than outputting an actual intensity image at regular intervals. This new paradigm of image sensor offers significant potential advantages; namely, sparse and non-redundant data representation. Unfortunately, however, most of the existing artificial neural network architectures, such as a CNN, require dense synchronous input data, and therefore, cannot make use of the sparseness of the data. We propose EventNet, a neural network designed for real-time processing of asynchronous event streams in a recursive and event-wise manner. EventNet models dependence of the output on tens of thousands of causal events recursively using a novel temporal coding scheme. As a result, at inference time, our network operates in an event-wise manner that is realized with very few sum-of-the-product operations---look-up table and temporal feature aggregation---which enables processing of 1 mega or more events per second on standard CPU. In experiments using real data, we demonstrated the real-time performance and robustness of our framework.

Layered Depth Video (LDV) is recognized as a promising 3D video data representation for supporting advanced 3D video services required in Multiview Video (MVV) systems such as Three-Dimensional Television (3DTV). This representation consists of one full or central view in the form of a video-plus-depth sequence as the main layer, and additional enhancement layers including residual texture and depth data that represent side views. LDV is thus both a derivative of and an alternative to Multiview Video-plus-Depth (MVD) representation by only transmit one full view with associated residual data over the channel. There is a risk, however, of residual data information rapidly increasing as the distance between the center view and side views increases. This occurs when parts of the central view are not visible in the side views, leaving blank spots called disocclusions. These disocclusions may grow larger, which increases the amount of residual data that needs to be sent with the main layer. In this paper, we address the residual layer generation problem. We propose an inpainting-based LDV generation method to reduce the amount of residual data to send by retrieving the missing pixels from the main layer. In the proposed method, we take into account the depth information by distinguishing between foreground and background parts of the scene, at low complexity, during the texture and structure propagation stage of the inpainting process. Experimental results demonstrated the effectiveness of the proposed method.

Point cloud registration is a key problem for robotics and computer vision communities. This represents estimating a rigid transform which aligns one point cloud to another. Iterative closest point (ICP) is a well-known classical method for this problem, yet it generally achieves high alignment only when the source and template point cloud are mostly pre-aligned. If each point cloud is far away or contains a repeating structure, the registration often fails because of being fallen into a local minimum. Recently, inspired by PointNet, several deep learning-based methods have been developed. PointNetLK is a representative approach, which directly optimizes the distance of aggregated features using gradient method by Jacobian. In this paper, we propose a point cloud registration system based on deep learning: CorsNet. Since CorsNet concatenates the local features with the global features and regresses correspondences between point clouds, not directly pose or aggregated features, more useful information is integrated than the conventional approaches. For comparison, we also developed a novel deep learning approach (DirectNet) that directly regresses the pose between point clouds. Through our experiments, we show that CorsNet achieves higher accuracy than not only the classic ICP method, but also the recently proposed learning-based proposal PointNetLK and DirectNet, including on seen and unseen categories.

Learning to play the guitar is difficult. We proposed a system that assists people learning to play the guitar using augmented reality. This system shows a learner how to correctly hold the strings by overlaying a virtual hand model and lines onto a real guitar. The player learning to play the guitar can easily understand the required position by overlapping their hand on a visual guide. An important issue for this system to address is the accurate registration between the visual guide and the guitar, therefore we need to track the pose and the position of the guitar. We also proposed a method to track the guitar with a visual marker and natural features of the guitar. Since we used marker information and edge information as natural features, the system could continually track the guitar. Accordingly, our system can constantly display visual guides at the required position to enable a player to learn to play the guitar in a natural manner.

The drowsiness affects the driver’s sensory, cognitive, and psychomotor abilities, which are necessary for safe driving. Drowsiness detection is a critical technique to avoid traffic accidents. The federated learning (FL) can solve the problem of insufficient driver facial data by utilizing different industrial entities’ data. However, in the FL system, the privacy information of the drivers might be leaked. In addition, reducing the communication costs and maintaining the model performance is also a challenge in industrial scenarios. In this work, we propose a federated transfer learning method with privacy-preserving protocol for driver drowsiness detection, named PFTL-DDD. We use fine-tuning transfer learning on the initial model of the drowsiness detection FL system. Furthermore, a CKKS-based privacy-preserving protocol is applied to preserve the drivers’ privacy data by encrypting the exchanged parameters. The experimental results show that the PFTL-DDD method is superior in terms of accuracy and efficiency as compared to the conventional federated learning on the NTHU-DDD dataset and YAWDD dataset. The theoretical analysis demonstrates that the proposed transfer learning method can reduce the communication cost of the system, and the CKKS-based security protocol can protect the personal privacy.