This study designed a finger-worn device—named the Chinese FingerReader—that can be practically applied by visually impaired users for recognizing traditional Chinese characters on the micro IOT processor. The device is portable, easy to operate, and designed to be worn on the index finger. The Chinese FingerReader on the index finger contains a small camera and buttons. The small camera captures images by identifying the relative position of the index finger to the printed text, and the buttons are applied to operate the software functions and capture an image by visually impaired users. To recognize Chinese characters, English letters, and numbers, a robust Chinese optical character recognition (OCR) system was developed according to the training strategy of an augmented convolution neural network algorithm. The proposed Chinese OCR system can segment a single character from the captured image, and the system can accurately recognize rotated Chinese characters. The experimental results revealed that compared with the OCR application programming interfaces of Google and Microsoft, the proposed OCR system obtains 95% accuracy rate in dealing with rotated character images where the Google and Microsoft OCR APIs only obtain 65% and 34% accuracy rates. This results illustrate that the proposed OCR system was more suitable to the needs of visually impaired people in actual use. Finally, three usage scenarios were simulated, and the accuracy and operational performance of the system were tested. Field tests of this system were conducted for visually impaired users to verify its feasibility.

In this study, we designed a four-dimensional (4D) audiovisual entertainment system called Sense. This system comprises a scene recognition system and hardware modules that provide haptic sensations for users when they watch movies and animations at home. In the scene recognition system, we used Google Cloud Vision to detect common scene elements in a video, such as fire, explosions, wind, and rain, and further determine whether the scene depicts hot weather, rain, or snow. Additionally, for animated videos, we applied deep learning with a single shot multibox detector to detect whether the animated video contained scenes of fire-related objects. The hardware module was designed to provide six types of haptic sensations set as line-symmetry to provide a better user experience. After the system considers the results of object detection via the scene recognition system, the system generates corresponding haptic sensations. The system integrates deep learning, auditory signals, and haptic sensations to provide an enhanced viewing experience.

The development of a portable communication aid, which allows deaf-blind people to communicate with others without the help of an assistant, is reviewed. The system comprises two major units: one for deaf-blind people and the other for sighted people. A deaf-blind person can send messages by typing on a Braille terminal and the messages will be converted to Mandarin phonetic symbols, which are then displayed on an LCD display to be read by a sighted partner. Then the sighted partner can send messages back by typing on a simple keyboard and the messages will be displayed on a Braille display to be `read' by the blind-deaf person. The aid has been designed to be effective, reliable and inexpensive. Some experimental results are reported to demonstrate the applicability of the aid.

Previous works have shown the potential of immersive technologies to make physical activities a more engaging experience. With encountered-type haptic feedback, users can perceive a more realistic sensation for exertion interaction in substitutions reality. Although substitutional reality has utilized physical environments, props, and devices to provide encountered-type haptic feedback, these cannot withstand the fierce force of humans and do not give feedback when users move around simultaneously, such as in combat sports. In this work, we present Hit Around, a substitutional moving robot for immersive and exertion interaction, in which the user can move and punch the virtual opponent and perceive encountered-type haptic feedback anywhere. We gathered insight into immersive exertion interaction from three exhibitions with iterative prototypes, then designed and implemented the hardware system and application. To understand the ability of mobility and weight loading, we conducted two technical evaluations and a laboratory experiment to validate the feasibility. Finally, a field deployment study explored the limitations and challenges of developing immersive exertion interaction with encountered-type haptics.

This study proposes a percussion learning device that combines tablet computers and robots. This device comprises two systems: a rhythm teaching system, in which users can compose and practice rhythms by using a tablet computer, and a robot performance system. First, teachers compose the rhythm training contents on the tablet computer. Then, the learners practice these percussion exercises by using the tablet computer and a small drum set. The teaching system provides a new and user-friendly score editing interface for composing a rhythm exercise. It also provides a rhythm rating function to facilitate percussion training for children and improve the stability of rhythmic beating. To encourage children to practice percussion exercises, a robotic performance system is used to interact with the children; this system can perform percussion exercises for students to listen to and then help them practice the exercise. This interaction enhances children’s interest and motivation to learn and practice rhythm exercises. The results of experimental course and field trials reveal that the proposed system not only increases students’ interest and efficiency in learning but also helps them in understanding musical rhythms through interaction and composing simple rhythms.

In the age of Industry 4.0, the techniques of artificial intelligence and pattern recognition play a critical role to develop the smart factories. In this paper, a block recognition system, named e-Block, was developed by using a novel projection algorithm and the convolution neural networks. The developed system displays a picture of the target object (e.g., a car or a house), and the children follow the instructions provided by the system and use the various blocks to build the object. Subsequently, this system compares the assembled block with the target object and determines whether the shape is identical. To identify the assembled block, this system applies Kinect to obtain information on the depth of the object and a new projection algorithm is proposed for converting the depth information into three feature images. By integrating three feature images, the convolution neural networks (CNN) are employed to construct the classifier to identify the assembled block. In the experiments conducted in this paper, the CNN classifier was compared with three classification algorithms. The experimental results show that the CNN classifier can accurately recognizes whether the assembled object is identical to the target object and outperform the compared classification algorithms. In additions, the experimental results also reveal that the proposed recognition algorithm can be a useful technique for applying in various applications of Industry 4.0.