One of the main challenges robots need to overcome is crowd analysis. Crowd analysis deals with the detection of individuals and interaction groups as well as the recognition of their activities. This paper focuses on the detection of conversational groups, where there have been a number of approaches addressing this problem in both supervised and unsupervised ways. Supervised bottom-up approaches primarily relied on pairwise affinity matrices and were limited to static, third-person views. In this work, we present our approach based on Graph Neural Networks (GNNs) to the problem of interaction group detection, called improved Group Detection With Link Prediction (iGROWL). iGROWL utilises the fact that interaction groups exist in certain inherent spatial configurations and improves its predecessor, GROWL, by introducing an ensemble learning-based sample balancing technique to the algorithm. Our results show that iGROWL outperforms other state-of-the-art methods by 16.7% and 26.4% in terms of F1-score when evaluated on the Salsa Poster Session and Cocktail Party datasets, respectively. Moreover, we show that sample balancing with GNNs is not trivial, but consistent results can be achieved by employing ensemble learning.

Although robots are able to navigate both static and dynamic environments, they lack the desired level of social awareness to move among humans. Previous work has explored solutions combining social force models and reinforcement learning to address this problem, but socially aware navigation has not been truly achieved. Furthermore, these works are hardly comparable due to the wide variety of metrics used for evaluation. Our work, Socially Aware Navigation Between Groups (SANG), aims to improve socially aware navigation by introducing a real-to-sim simulation for the training and testing of a reinforcement learning algorithm that has a more realistic group-based social reward calculation compared to previous works. To provide a thorough and unified evaluation practice, we propose a wide range of objective and subjective metrics including the use of a Navigation Turing Test. Based on our evaluation, the SANG algorithm achieves human-level smoothness scores, and better Social Information Processing scores as compared with the state-of-the-art. On average it was judged to be more human-like 43% of the time in a Navigation Turing Test, significantly improving the state of the art’s 38% chance (t-score = 2.64, p = 0.0086).

The revolutionary opportunities presented by eXtended Reality (XR) technologies will only materialize if modeling and analysis activities, undertaken during the engineering process of XR systems, are directed towards ensuring their social acceptance. By this, we mean integrating human and technical aspects during system development to ensure that the system guarantees communication security and data privacy, and is trusted by end users. One approach to achieve these guarantees is through rigorous, formal specification and verification during system modeling and analysis, explicitly considering the human factor. Accordingly, in this survey, we systematically investigate 6 mainstream formalisms for modeling and analyzing socio-technical security concerns, encompassing privacy and trust, in XR systems. We consider both desired concerns (e.g., requirements, properties) and undesired ones (e.g., threats, attacks). Our investigation incorporates 34 state-of-the-art approaches comprising languages, techniques, frameworks, and tools, leveraging these formalisms, which we compare against a diverse set of criteria: (1) expressivity, (2) modeling and analysis complexity, (3) modeling and analysis constructs, (4) power of inference, (5) user-friendliness, (6) applicability. Based on our findings, we identify the current gaps and considerable challenges and suggest an agenda for future research. To guide our investigation from a more practical perspective, we also present two real-world pilot studies that illustrate the potential application of formal methods in specific XR applications. This work thus aims to provide insights from a twofold perspective: for formal methods researchers seeking to learn more about socio-technical security in XR systems, and for security practitioners focused on socio-technical aspects in XR who are interested in formal approaches.