This paper describes the application of a Convolutional Neural Network (CNN) in the context of a predator/prey scenario. The CNN is trained and run on data from a Dynamic and Active Pixel Sensor (DAVIS) mounted on a Summit XL robot (the predator), which follows another one (the prey). The CNN is driven by both conventional image frames and dynamic vision sensor “frames” that consist of a constant number of DAVIS ON and OFF events. The network is thus “data driven” at a sample rate proportional to the scene activity, so the effective sample rate varies from 15 Hz to 240 Hz depending on the robot speeds. The network generates four outputs: steer right, left, center and non-visible. After off-line training on labeled data, the network is imported on the on-board Summit XL robot which runs jAER and receives steering directions in real time. Successful results on closed-loop trials, with accuracies up to 87% or 92% (depending on evaluation criteria) are reported. Although the proposed approach discards the precise DAVIS event timing, it offers the significant advantage of compatibility with conventional deep learning technology without giving up the advantage of data-driven computing.
The surface properties of an object and the environment in which it is located are important for robot grasping and manipulation. Physical contact with an object using tactile sensors can enable the retrieval of detailed information about the object, i.e. compressibility, surface texture and thermal properties. This paper describes a system that classifies materials based on their thermal properties alone, minimising the amount of manipulation required. Following acquisition of data from a sophisticated tactile sensor, the system uses an Artificial Neural Network (ANN) to classify materials based on representations of their thermal properties. The system was compared with human performance in the task of classifying materials and was found to perform better.
The ability to quickly and accurately triage a person’s medical condition in an emergency situation or other critical scenarios could mean the difference between life and death. Endowing a robotic system with vision and tactile capabilities, similar to those of medical professionals, and thus enabling robots to assess a patient’s status in an emergency is a highly sought after characteristic in healthcare robotics. This paper presents a novel fuzzy triage system exploiting visual and tactile sensing, to equip a robot with the skills to accurately determine key vital signs in humans. There are three key signs of human health: respiratory rate, pulse rate (Beats Per Minute (BPM)) and capillary refill time. Using ground truth from a medical professional, the fuzzy triage system is trained and validated initially with informed synthetic data and then further evaluated using vital signs data collected from subjects in a pilot study. Results from this pilot study indicate that the fuzzy triage system is capable of classifying a patient’s health using the novel approaches for collecting BPM, Respiratory Rate (RR) and Capillary Refill Time (CRT) which replicate, to some extent, the approaches used by medical professionals for measuring vital signs. Furthermore, the intelligent system proved capable of determining whether a pulse was regular or arrhythmic, whether respiratory rate was regular or irregular, and determining the subject’s capillary refill time. Such results imply that this system could ultimately be used, for example, in a home assistance robot for elderly or disabled persons, or as a first responder robot. Ultimately the aim would be that these methods could be utilised by robotic systems in emergency scenarios or disaster zones.