In this work, we propose MAGES 4.0, a novel Software Development Kit (SDK) to accelerate the creation of collaborative medical training applications in VR/AR. Our solution is essentially a low-code metaverse authoring platform for developers to rapidly prototype high-fidelity and high-complexity medical simulations. MAGES breaks the authoring boundaries across extended reality, since networked participants can also collaborate using different virtual/augmented reality as well as mobile and desktop devices, in the same metaverse world. With MAGES we propose an upgrade to the outdated 150-year-old master-apprentice medical training model. Our platform incorporates, in a nutsell, the following novelties: a) 5G edge-cloud remote rendering and physics dissection layer, b) realistic real-time simulation of organic tissues as soft-bodies under 10ms, c) a highly realistic cutting and tearing algorithm, d) neural network assessment for user profiling and, e) a VR recorder to record and replay or debrief the training simulation from any perspective.

We present a new approach to Extended Reality (XR), denoted as iCOPYWAVES, which seeks to offer naturally low-latency operation and cost effectiveness, overcoming the critical scalability issues faced by existing solutions. iCOPYWAVES is enabled by emerging PWEs, a recently proposed technology in wireless communications. Empowered by intelligent (meta)surfaces, PWEs transform the wave propagation phenomenon into a software-defined process. We leverage PWEs to: i) create, and then ii) selectively copy the scattered RF wavefront of an object from one location in space to another, where a machine learning module, accelerated by FPGAs, translates it to visual input for an XR headset using PWE-driven, RF imaging principles (XR-RF). This makes for an XR system whose operation is bounded in the physical-layer and, hence, has the prospects for minimal end-to-end latency. Over large distances, RF-to-fiber/fiber-to-RF is employed to provide intermediate connectivity. The paper provides a tutorial on the iCOPYWAVES system architecture and workflow. A proof-of-concept implementation via simulations is provided, demonstrating the reconstruction of challenging objects in iCOPYWAVES-produced computer graphics.

The detection and prevention of workers’ body straining postures and other stressing conditions within the work environment, supports establishing occupational safety and promoting well being and sustainability at work. Developed methods towards this aim typically rely on combining highly ergonomic workplaces and expensive monitoring mechanisms including wearable devices. In this work, we demonstrate how the input from low-cost sensors, specifically, passive camera sensors installed in a real manufacturing workplace, and smartwatches used by the workers can provide useful feedback on the workers’ conditions and can yield key indicators for the prevention of work-related musculo-skeletal disorders (WMSD) and physical fatigue. To this end, we study the ability to assess the risk for physical strain of workers online during work activities based on the classification of ergonomically sub-optimal working postures using visual information, the correlation and fusion of these estimations with synchronous worker heart rate data, as well as the prediction of near-future heart rate using deep learning-based techniques. Moreover, a new multi-modal dataset of video and heart rate data captured in a real manufacturing workplace during car door assembly activities is introduced. The experimental results show the efficiency of the proposed approach that exceeds 70% of classification rate based on the F1 score measure using a set of over 300 annotated video clips of real line workers during work activities. In addition a time lagging correlation between the estimated ergonomic risks for physical strain and high heart rate was assessed using a larger dataset of synchronous visual and heart rate data sequences. The statistical analysis revealed that imposing increased strain to body parts will results in an increase to the heart rate after 100–120 s. This finding is used to improve the short term forecasting of worker’s cardiovascular activity for the next 10 to 30 s by fusing the heart rate data with the estimated ergonomic risks for physical strain and ultimately to train better predictive models for worker fatigue.

In this work, a visual object detection and localization workflow integrated into a robotic platform is presented for the 6D pose estimation of objects with challenging characteristics in terms of weak texture, surface properties and symmetries. The workflow is used as part of a module for object pose estimation deployed to a mobile robotic platform that exploits the Robot Operating System (ROS) as middleware. The objects of interest aim to support robot grasping in the context of human–robot collaboration during car door assembly in industrial manufacturing environments. In addition to the special object properties, these environments are inherently characterised by cluttered background and unfavorable illumination conditions. For the purpose of this specific application, two different datasets were collected and annotated for training a learning-based method that extracts the object pose from a single frame. The first dataset was acquired in controlled laboratory conditions and the second in the actual indoor industrial environment. Different models were trained based on the individual datasets and a combination of them were further evaluated in a number of test sequences from the actual industrial environment. The qualitative and quantitative results demonstrate the potential of the presented method in relevant industrial applications.

Edge detection is supported by extensive research and is part of different photogrammetric and computer vision tasks across numerous application areas. While 2D edge detection may achieve high accuracy results from several automated methods, the automation of edge detection in 3D space remains a challenge. Existing methods are often computationally demanding and heavily parameterized, leading to a lack of adaptability. In real-world applications 3D edges, representing the object boundaries and break lines, are crucial, particularly in fields such as computer vision, robotics and architecture. In this context, we present a method that automates 3D edge detection in 3D point clouds exploiting the normal vectors’ direction differences to detect finite edges, which are further pruned and grouped to edge segments and fitted to indicate the presence of a 3D edge.