Digital technology has evolved towards a new way of processing information: web searches, social platforms, internet forums, and video games have substituted reading books and writing essays. Trainers and educators currently face the challenge of providing natural training and learning environments for digital-natives. In addition to physical spaces, effective training and education require virtual spaces. Digital twins enable trainees to interact with real hardware in virtual training environments. Interactive real-world elements are essential in the training of robot operators. A natural environment for the trainee supports an interesting learning experience while including enough professional substances. We present examples of how virtual environments utilizing digital twins and extended reality can be applied to enable natural and effective robotic training scenarios. Scenarios are validated using cross-platform client devices for extended reality implementations and safety training applications.

Digitalization shapes the ways of learning, working, and entertainment. The Internet, which enables us to connect and socialize is evolving to become the metaverse, a post-reality universe, enabling virtual life parallel to reality. In addition to gaming and entertainment, industry and academia have noticed the metaverse’s benefits and possibilities. For industry, the metaverse is the enabler of the future digital workplace, and for academia, digital learning spaces enable realistic virtual training environments. A connection bridging the virtual world with physical production systems is required to enable digital workplaces and digital learning spaces. In this publication, extended reality–digital twin to real use cases are presented. The presented use cases utilize extended reality as high-level user interfaces and digital twins to create a bridge between virtual environments and robotic systems in industry, academia, and underwater exploration.

This article examines the problems of multi-robot coverage and persistent monitoring of regions of interest with limited sensing robots. A group of robots, each equipped with only contact sensors and a clock, execute a simple trajectory by repeatedly moving straight and then bouncing at perimeter boundaries by rotating in place. We introduce an approach by finding a joint trajectory for multiple robots to cover a given environment and generating cycles for the robots to persistently monitor the target regions in the environment. From a given initial configuration, our approach iteratively finds the joint trajectory of all the robots that covers the entire environment. Our approach also computes periodic trajectories of all the robots for monitoring of some regions, where trajectories overlap but do not involve robot-robot collisions. We present experimental results from multiple simulations and physical experiments demonstrating the practical utility of our approach.

We present an incremental scalable motion planning algorithm for finding maximally informative trajectories for decentralized mobile robots. These robots are deployed to observe an unknown spatial field, where the informativeness of observations is specified as a density function. Existing works that are typically restricted to discrete domains and synchronous planning often scale poorly depending on the size of the problem. Our goal is to design a distributed control law in continuous domains and an asynchronous communication strategy to guide a team of cooperative robots to visit the most informative locations within a limited mission duration. Our proposed Asynchronous Information Gathering with Bayesian Optimization (AsyncIGBO) algorithm extends ideas from asynchronous Bayesian Optimization (BO) to efficiently sample from a density function. It then combines them with decentralized reactive motion planning techniques to achieve efficient multi-robot information gathering activities. We provide a theoretical justification for our algorithm by deriving an asymptotic no-regret analysis with respect to a known spatial field. Our proposed algorithm is extensively validated through simulation and real-world experiment results with multiple robots.

There has been significant interest in recent years in the utility and implementation of autonomous underwater and surface vehicles (AUVs and ASVs) for persistent surveillance of the ocean. Example studies include the dynamics of physical phenomena, e.g., ocean fronts, temperature and salinity profiles, and the onset of harmful algae blooms. For these studies, AUVs are presented with a complex planning and navigation problem to achieve autonomy lasting days and weeks under uncertainties while dealing with resource constraints. We address these issues by adopting motion, sensing, and environment uncertainties via a Partially Observable Markov Decision Process (POMDP) framework. We propose a methodology with a novel extension of POMDPs to incorporate spatiotemporally-varying ocean currents as energy and dynamic obstacles as environment uncertainty. Existing POMDP solutions such as the Cost-Constrained Partially Observable Monte-Carlo Planner (POMCP) do not account for energy efficiency. Therefore, we present a scalable Energy Cost-Constrained POMCP algorithm utilizing the predicted ocean dynamics that optimizes energy and environment costs along with goal-driven rewards. A theoretical analysis, along with simulation and real-world experiment results is presented to validate the proposed methodology.

Occupant behavior is a significant contributor to energy waste in buildings. This research introduces an advanced smartphone application, developed based on the theoretical underpinnings of situational awareness theory, to effectively implement multi-method and personalized intervention to encourage energy conservation behaviors of building occupants. The new smart application provides several innovative features, such as energy saving points, customized feedback, and visualized user interface, which are implemented in the application to support multi-method interventions. The application was created using the Java language for Android devices. With the use of the Android platform, the app takes advantage of hardware technology from the user’s mobile device. Measurement of occupancy behavior is accomplished by making use of the device’s positional sensors. Orientation and geomagnetic field sensors serve to provide an accurate location of an occupant inside the building. The application can determine energy waste in a zone by using occupancy behavior. Moreover, the application offers real-time and projected future energy consumption based on occupants’ behaviors. This novel feature can significantly improve communication that can lead to prompt action for building energy reduction. Results show how the app can compile raw data on energy behavior and make it easy to understand for the user through the use of visuals and statistical algorithms.