Deep learning has provided new ways of manipulating, processing and analyzing data. It sometimes may achieve results comparable to, or surpassing human expert performance, and has become a source of inspiration in the era of artificial intelligence. Another subfield of machine learning named reinforcement learning, tries to find an optimal behavior strategy through interactions with the environment. Combining deep learning and reinforcement learning permits resolving critical issues relative to the dimensionality and scalability of data in tasks with sparse reward signals, such as robotic manipulation and control tasks, that neither method permits resolving when applied on its own. In this paper, we present recent significant progress of deep reinforcement learning algorithms, which try to tackle the problems for the application in the domain of robotic manipulation control, such as sample efficiency and generalization. Despite these continuous improvements, currently, the challenges of learning robust and versatile manipulation skills for robots with deep reinforcement learning are still far from being resolved for real-world applications.

The Codebook model is one of the popular real-time models for background subtraction. In this paper, we first extend it from traditional Red-Green-Blue (RGB) color model to multispectral sequences. A self-adaptive mechanism is then designed based on the statistical information extracted from the data themselves, with which the performance has been improved, in addition to saving time and effort to search for the appropriate parameters. Furthermore, the Spectral Information Divergence is introduced to evaluate the spectral distance between the current and reference vectors, together with the Brightness and Spectral Distortion. Experiments on five multispectral sequences with different challenges have shown that the multispectral self-adaptive Codebook model is more capable of detecting moving objects than the corresponding RGB sequences. The proposed research framework opens a door for future works for applying multispectral sequences in moving object detection.

New technologies for monitoring grip forces during hand and finger movements in non-standard task contexts have provided unprecedented functional insights into somatosensory cognition. Somatosensory cognition is the basis of our ability to manipulate and transform objects of the physical world and to grasp them with the right amount of force. In previous work, the wireless tracking of grip-force signals recorded from biosensors in the palm of the human hand has permitted us to unravel some of the functional synergies that underlie perceptual and motor learning under conditions of non-standard and essentially unreliable sensory input. This paper builds on this previous work and discusses further, functionally motivated, analyses of individual grip-force data in manual robot control. Grip forces were recorded from various loci in the dominant and non-dominant hands of individuals with wearable wireless sensor technology. Statistical analyses bring to the fore skill-specific temporal variations in thousands of grip forces of a complete novice and a highly proficient expert in manual robot control. A brain-inspired neural network model that uses the output metric of a self-organizing pap with unsupervised winner-take-all learning was run on the sensor output from both hands of each user. The neural network metric expresses the difference between an input representation and its model representation at any given moment in time and reliably captures the differences between novice and expert performance in terms of grip-force variability.Functionally motivated spatiotemporal analysis of individual average grip forces, computed for time windows of constant size in the output of a restricted amount of task-relevant sensors in the dominant (preferred) hand, reveal finger-specific synergies reflecting robotic task skill. The analyses lead the way towards grip-force monitoring in real time. This will permit tracking task skill evolution in trainees, or identify individual proficiency levels in human robot-interaction, which represents unprecedented challenges for perceptual and motor adaptation in environmental contexts of high sensory uncertainty. Cross-disciplinary insights from systems neuroscience and cognitive behavioral science, and the predictive modeling of operator skills using parsimonious Artificial Intelligence (AI), will contribute towards improving the outcome of new types of surgery, in particular the single-port approaches such as NOTES (Natural Orifice Transluminal Endoscopic Surgery) and SILS (Single-Incision Laparoscopic Surgery).

Aerial drone technology is currently being investigated worldwide for the delivery of blood components. Although it has been demonstrated to be safe, the delivered medical substances still need to be analyzed at the end of the flight mission to assess the level of haemolysis and pH prior to the use in a patient. This process can last up to 30 min and prevent the time saved using drone delivery. Our study aims to integrating an innovative sensor for the haemolysis and pH detection into the Smart Capsule, an already demonstrated technology capable of managing transfusion transport through drones. In the proposed scenario, the haemolysis is evaluated optically by a minilysis device using LED–photodetector combination. The preliminary validation has been demonstrated for both the thermal stability of the Smart Capsule and the haemolysis detection of the minilysis device prototype. Firstly, the onboard temperature test has shown that the delivery system is capable of maintaining proper temperature, even though the samples have been manipulated to reach a higher temperature before inserting into the Smart Capsule. Then, in the laboratory haemolysis test, the trend of linear regression between the outputs from the spectrophotometer and the minilysis prototype confirmed the concept design of the minilysis device.

The temporal evolution of individual grip force profiles of a novice using a robotic system for minimally invasive endoscopic surgery is analyzed on the basis of thousands of individual sensor data recorded in real time through a wearable wireless sensor glove system. The spatio-temporal grip force profiles from specific sensor locations in the dominant hand performing a four-step pick-and-drop simulator task reveal skill-relevant differences in force deployment by the small finger (fine grip force control) and the middle finger (gross grip force contribution) by comparison with the profiles of a highly proficient expert. Cross-disciplinary insights from systems neuroscience, cognitive behavioral science, and robotics, with implications for biologically inspired AI for human–robot interactions, highlight the functional significance of spatio-temporal grip force profiling.