Long-horizon dexterous robot manipulation of deformable objects, such as banana peeling, is a problematic task because of the difficulties in object modeling and a lack of knowledge about stable and dexterous manipulation skills. This paper presents a goal-conditioned dual-action (GC-DA) deep imitation learning (DIL) approach that can learn dexterous manipulation skills using human demonstration data. Previous DIL methods map the current sensory input and reactive action, which often fails because of compounding errors in imitation learning caused by the recurrent computation of actions. The method predicts reactive action only when the precise manipulation of the target object is required (local action) and generates the entire trajectory when precise manipulation is not required (global action). This dual-action formulation effectively prevents compounding error in the imitation learning using the trajectory-based global action while responding to unexpected changes in the target object during the reactive local action. The proposed method was tested in a real dual-arm robot and successfully accomplished the banana-peeling task.

Deep imitation learning enables the learning of complex visuomotor skills from raw pixel inputs. However, this approach suffers from the problem of overfitting to the training images. The neural network can easily be distracted by task-irrelevant objects. In this letter, we use the human gaze measured by a head-mounted eye tracking device to discard task-irrelevant visual distractions. We propose a mixture density network-based behavior cloning method that learns to imitate the human gaze. The model predicts gaze positions from raw pixel images and crops images around the predicted gazes. Only these cropped images are used to compute the output action. This cropping procedure can remove visual distractions because the gaze is rarely fixated on task-irrelevant objects. This robustness against irrelevant objects can improve the manipulation performance of robots in scenarios where task-irrelevant objects are present. We evaluated our model on four manipulation tasks designed to test the robustness of the model to irrelevant objects. The results indicate that the proposed model can predict the locations of task-relevant objects from gaze positions, is robust to task-irrelevant objects, and exhibits impressive manipulation performance especially in multi-object handling.

Deep imitation learning is promising for robot manipulation because it only requires demonstration samples. In this study, deep imitation learning is applied to tasks that require force feedback. However, existing demonstration methods have deficiencies; bilateral teleoperation requires a complex control scheme and is expensive, and kinesthetic teaching suffers from visual distractions from human intervention. This research proposes a new master-to-robot (M2R) policy transfer system that does not require robots for teaching force feedback-based manipulation tasks. The human directly demonstrates a task using a controller. This controller resembles the kinematic parameters of the robot arm and uses the same end-effector with force/torque (F/T) sensors to measure the force feedback. Using this controller, the operator can feel force feedback without a bilateral system. The proposed method can overcome domain gaps between the master and robot using gaze-based imitation learning and a simple calibration method. Furthermore, a Transformer is applied to infer policy from F/T sensory input. The proposed system was evaluated on a bottle-cap-opening task that requires force feedback.

A high-precision manipulation task, such as needle threading, is challenging. Physiological studies have proposed connecting low-resolution peripheral vision and fast movement to transport the hand into the vicinity of an object, and using high-resolution foveated vision to achieve the accurate homing of the hand to the object. The results of this study demonstrate that a deep imitation learning based method, inspired by the gaze-based dual resolution visuomotor control system in humans, can solve the needle threading task. First, we recorded the gaze movements of a human operator who was teleoperating a robot. Then, we used only a high-resolution image around the gaze to precisely control the thread position when it was close to the target. We used a low-resolution peripheral image to reach the vicinity of the target. The experimental results obtained in this study demonstrate that the proposed method enables precise manipulation tasks using a general-purpose robot manipulator and improves computational efficiency.

To understand the prenatal origin of developmental and psychiatric disorders, studies in laboratory animals are imperative. However, the developmental pace differs between humans and animals; hence, corresponding human ages must be estimated to infer the most vulnerable developmental timings in humans. Because rats and mice are extensively used as models in developmental research, a correspondence between human foetal ages and rodents’ ages must be precisely determined; thus, developing a translational model is of utmost importance. Optimizing a translational model involves classifying the brain regions according to developmental paces, but previous studies have conducted this classification arbitrarily. Here we used a clustering method and showed that the brain regions can be classified into two groups. To quantify the developmental pace, we gathered data for a range of development events in humans and rodents and created a linear mixed model that translates human developmental timings into the corresponding rat timings. We conducted an automatic classification of brain regions using an EM algorithm and obtained a model to translate human foetal age to rat age. Our model could predict rat developmental timings within 2.5 days of root mean squared error. This result provides useful information for designing animal studies and clinical tests.

People with autism spectrum disorder (ASD) frequently show the symptoms of oversensitivity to sound (hyperacusis). Although the previous studies have investigated methods for quantifying hyperacusis in ASD, appropriate physiological signs for quantifying hyperacusis in ASD remain poorly understood. Here, we investigated the relationship of loudness tolerance with the threshold of the stapedial reflex and with contralateral suppression of the distortion product otoacoustic emissions, which has been suggested to be related to hyperacusis in people without ASD. We tested an ASD group and a neurotypical group. The results revealed that only the stapedial reflex threshold was significantly correlated with loudness tolerance in both groups. In addition to reduced loudness tolerance, people with lower stapedial reflex thresholds also exhibited higher scores on the Social Responsiveness Scale-2.