Augmented reality (AR) is used to enhance the perception of the real world by integrating virtual objects to an image sequence acquired from various camera technologies. Numerous AR applications in robotics have been developed in recent years. The aim of this paper is to provide an overview of AR research in robotics during the five year period from 2015 to 2019. We classified these works in terms of application areas into four categories: (1) Medical robotics: Robot-Assisted surgery (RAS), prosthetics, rehabilitation, and training systems; (2) Motion planning and control: trajectory generation, robot programming, simulation, and manipulation; (3) Human-robot interaction (HRI): teleoperation, collaborative interfaces, wearable robots, haptic interfaces, brain-computer interfaces (BCIs), and gaming; (4) Multi-agent systems: use of visual feedback to remotely control drones, robot swarms, and robots with shared workspace. Recent developments in AR technology are discussed followed by the challenges met in AR due to issues of camera localization, environment mapping, and registration. We explore AR applications in terms of how AR was integrated and which improvements it introduced to corresponding fields of robotics. In addition, we summarize the major limitations of the presented applications in each category. Finally, we conclude our review with future directions of AR research in robotics. The survey covers over 100 research works published over the last five years.

This paper presents the design and preliminary experimental validation of a multigrasp myoelectric controller. The described method enables direct and proportional control of multigrasp prosthetic hand motion among nine characteristic postures using two surface electromyography electrodes. To assess the efficacy of the control method, five nonamputee subjects utilized the multigrasp myoelectric controller to command the motion of a virtual prosthesis between random sequences of target hand postures in a series of experimental trials. For comparison, the same subjects also utilized a data glove, worn on their native hand, to command the motion of the virtual prosthesis for similar sequences of target postures during each trial. The time required to transition from posture to posture and the percentage of correctly completed transitions were evaluated to characterize the ability to control the virtual prosthesis using each method. The average overall transition times across all subjects were found to be 1.49 and 0.81 s for the multigrasp myoelectric controller and the native hand, respectively. The average transition completion rates for both were found to be the same (99.2%). Supplemental videos demonstrate the virtual prosthesis experiments, as well as a preliminary hardware implementation.

We present SpeakingFaces as a publicly-available large-scale multimodal dataset developed to support machine learning research in contexts that utilize a combination of thermal, visual, and audio data streams; examples include human–computer interaction, biometric authentication, recognition systems, domain transfer, and speech recognition. SpeakingFaces is comprised of aligned high-resolution thermal and visual spectra image streams of fully-framed faces synchronized with audio recordings of each subject speaking approximately 100 imperative phrases. Data were collected from 142 subjects, yielding over 13,000 instances of synchronized data (∼3.8 TB). For technical validation, we demonstrate two baseline examples. The first baseline shows classification by gender, utilizing different combinations of the three data streams in both clean and noisy environments. The second example consists of thermal-to-visual facial image translation, as an instance of domain transfer.

Objective: The intent recognizers of advanced lower limb prostheses utilize mechanical sensors on the prosthesis and/or electromyographic measurements from the residual limb. Besides the delay caused by these signals, such systems require user-specific databases to train the recognizers. In this work, our objective is the development and validation of a user-independent intent recognition framework utilizing depth sensing. Methods: We collected a depth image dataset from 12 healthy subjects engaging in a variety of routine activities. After filtering the depth images, we extracted simple features employing a recursive strategy. The feature vectors were classified using a support vector machine. For robust activity mode switching, we implemented a voting filter scheme. Results: The model selection showed that the support vector machine classifier with no dimension reduction has the highest classification accuracy. Specifically, it reached 94.1% accuracy on the testing data from four subjects. We also observed a positive trend in the accuracy of classifiers trained with data from increasing the number of subjects. Activity mode switching using a voting filter detected 732 out of 778 activity mode transitions of the four users while initiating 70 erroneous transitions during steady-state activities. Conclusion: The intent recognizer trained on multiple subjects can be used for any other subject, providing a promising solution for supervisory control of powered lower limb prostheses. Significance: A user-independent intent recognition framework has the potential to decrease or eliminate the time required for extensive data collection regiments for intent recognizer training. This could accelerate the introduction of robotic lower limb prostheses to the market.

In this work, we present an open-source stochastic epidemic simulator calibrated with extant epidemic experience of COVID-19. The simulator models a country as a network representing each node as an administrative region. The transportation connections between the nodes are modeled as the edges of this network. Each node runs a Susceptible-Exposed-Infected-Recovered (SEIR) model and population transfer between the nodes is considered using the transportation networks which allows modeling of the geographic spread of the disease. The simulator incorporates information ranging from population demographics and mobility data to health care resource capacity, by region, with interactive controls of system variables to allow dynamic and interactive modeling of events. The single-node simulator was validated using the thoroughly reported data from Lombardy, Italy. Then, the epidemic situation in Kazakhstan as of 31 May 2020 was accurately recreated. Afterward, we simulated a number of scenarios for Kazakhstan with different sets of policies. We also demonstrate the effects of region-based policies such as transportation limitations between administrative units and the application of different policies for different regions based on the epidemic intensity and geographic location. The results show that the simulator can be used to estimate outcomes of policy options to inform deliberations on governmental interdiction policies.

Negative stiffness honeycombs (NSHs) have multiple advantages compared to traditional honeycomb structures. These advantages include recoverable elastic buckling, shock absorption, and vibration isolation. As a result, NSHs have great potential in applications such as acoustic wave guiding, shape morphing, design of impact-protection devices and robotic grippers. In this paper, we present a design methodology for metamaterial consisting of negative stiffness beams assembled in a honeycomb structure. Based on analytical results, our methodology allows designing NSH mesostructures with predetermined buckling sequence. An NSH prototype was designed based on our algorithm and fabricated using a 3D printer with a nylon filament. The validity of our approach was experimentally verified by performing displacement controlled compression force measurements. Our methodology gives the ability to design NSH mesostructures with desired force–displacement characteristics.