Extensive research conducted in controlled laboratory settings has prompted an inquiry into how results can be generalized to real-world situations influenced by the subjects' actions. Virtual reality lends itself ideally to investigating complex situations but requires accurate classification of eye movements, especially when combining it with time-sensitive data such as EEG. We recorded eye-tracking data in virtual reality and classified it into gazes and saccades using a velocity-based classification algorithm, and we cut the continuous data into smaller segments to deal with varying noise levels, as introduced in the REMoDNav algorithm. Furthermore, we corrected for participants' translational movement in virtual reality. Various measures, including visual inspection, event durations, and the velocity and dispersion distributions before and after gaze onset, indicate that we can accurately classify the continuous, free-exploration data. Combining the classified eye-tracking with the EEG data, we generated fixation-onset event-related potentials (ERPs) and event-related spectral perturbations (ERSPs), providing further evidence for the quality of the eye-movement classification and timing of the onset of events. Finally, investigating the correlation between single trials and the average ERP and ERSP identified that fixation-onset ERSPs are less time sensitive, require fewer repetitions of the same behavior, and are potentially better suited to study EEG signatures in naturalistic settings. We modified, designed, and tested an algorithm that allows the combination of EEG and eye-tracking data recorded in virtual reality.

The proxemics theory explains the consistent social boundaries surrounding individuals as reported (Hall in The Hidden Dimension, Doubleday, Garden City, 1966), yet little is known about the social boundaries surrounding pairs or groups of people. The current study explored interpersonal proxemics behavior in a virtual environment, focusing on distances maintained towards individual pedestrians, pairs, and groups. Using virtual reality to simulate a city center, participants freely navigated it while their movements and gazes were captured. Importantly, the city was populated by pedestrians in different social configurations. Eye movements identified interactions defined by gaze-onsets towards a pedestrian’s head. Our results indicate that participants approached individuals with a median distance of 3.18 unity units aligned with the social space boundary as reported (Hall in The Hidden Dimension, Doubleday, Garden City, 1966). Distances kept from pairs and groups were similarly centered within the social space, revealing no significant difference in approaching behavior across different social configurations. The consistency in approaching distances suggests that personal and social spaces are not substantially altered, irrespective of the social context.

As research on human behavior, such as spatial navigation, increasingly adopts naturalistic settings, establishing best practices for such experiments becomes essential. Although virtual reality offers a bridge between laboratory control and real-world complexity, it does not fully capture the experiential richness of real-world environments. Here, we present a demonstration of a mobile eye-tracking study conducted in a large-scale, outdoor urban environment, featuring unconstrained, long-duration free exploration and outside-pointing tasks. Using the city of Limassol, Cyprus, as our testbed, we showcase the feasibility of collecting high-quality mobile eye-tracking, head orientation, and GPS data "in the wild," capturing a wide range of natural behavior with minimal experimental constraints. Based on this experience, we provide a set of best practices tailored to the logistical and methodological challenges posed by complex, real-world urban settings, challenges unlikely to arise in traditional indoor or highly controlled environments. Although these recommendations have general relevance, we exemplify them in the context of spatial navigation research. By establishing methodological standards for studies at this scale, we aimed to encourage and inform future research into naturalistic human behavior outside the laboratory.

Current research strives to investigate cognitive processes under natural conditions. Virtual reality and EEG are promising techniques combining naturalistic settings with close experimental control. However, many questions and technical challenges remain, e.g., are saccade onsets a suitable replacement of fixation onsets as key events in continuous gaze trajectories ( Amme et al., 2024), and consequently, can VR capture differences across different stimulus categories associated with varying saccade durations? To address both questions, we investigate the N170 face effect in humans (14 males, 19 females, zero diverse) using a free-viewing and free-movement immersive VR study that contained houses, various background stimuli, and, notably, static and moving pedestrians to study face perception under naturalistic conditions. Our results show that aligning trials to saccade onsets leads to more well-defined ERPs than fixation onsets, especially for the P100 component, demonstrating that saccade-onset ERPs are a better-suited analysis method for this type of experiment. Furthermore, we observe an evolution of category-based differences, i.e., face versus background saccade-onset ERPs, compatible with previous reports but extending in a large temporal window and including all electrode sites at different points in time. In summary, employing VR, EEG, and eye-tracking to investigate differences across fixation categories provides insights into the relevance of saccadic onsets as event triggers and enhances our understanding of cognitive processes in naturalistic settings.

Current research strives to investigate cognitive processes under natural conditions. Virtual reality and EEG are promising techniques combining naturalistic settings with close experimental control. However, many questions and technical challenges remain, e.g., are fixation or saccade onsets a suitable replacement as key events in continuous gaze trajectories (Amme et al., 2024), and can VR effectively capture differences across experimental conditions (Rossion & Jacques, 2008)? To address both questions, we investigate the N170 face effect in a free-viewing immersive VR study that contained houses, various background stimuli, and, notably, static and moving pedestrians to study face perception under naturalistic conditions. Our results show that aligning trials to saccade-onset leads to more well-defined ERPs, especially for the P100 component, and support that saccade-onset ERPs are the better-suited analysis method than fixation-onset ERPs for this type of experiment. Further, we observe an evolution of condition-based differences, i.e., face vs. background fixations, compatible with previous reports but extending in a large temporal window and including all electrode sites at different points in time. In summary, experiments combining VR, EEG, and eye-tracking provide further insights into the processing of faces and the relevance of saccadic onsets as event triggers under natural conditions. Significance Statement: With the effort of investigating and understanding cognitive processes under naturalistic conditions, combining virtual reality and EEG can be fruitful to implement free-viewing studies. The current work combines these technologies to explore key challenges in the context of face perception in an immersive virtual environment. Our results show that saccade-onset ERPs yield more precise measurements when analyzing continuous eye-tracking data than fixation onsets. Furthermore, when processing face compared to background stimuli, distinct temporal patterns encompassing all electrode sites can be observed, offering new insights into face perception. Overall, this work highlights the potential of integrating VR and EEG to advance our understanding of cognitive processes in naturalistic settings.

Extensive research conducted in controlled laboratory settings has prompted an inquiry into how results can be generalized to real-world situations influenced by the subjects’ actions. Virtual reality lends itself ideally to investigating complex situations but requires accurate classification of eye movements, especially when combining it with time-sensitive data such as EEG. We recorded eye-tracking data in virtual reality and classified it into gazes and saccades using a velocity-based classification algorithm, and we cut the continuous data into smaller segments to deal with varying noise levels, as introduced in the REMoDNav algorithm. Furthermore, we corrected for participants’ translational movement in virtual reality. Various measures, including visual inspection, event durations, and the velocity and dispersion distributions before and after gaze onset, indicate that we can accurately classify the continuous, free-exploration data. Combining the classified eye-tracking with the EEG data, we generated fixation-onset ERPs and ERSPs, providing further evidence for the quality of the eye movement classification and timing of the onset of events. Finally, investigating the correlation between single trials and the average ERP and ERSP identified that fixation-onset ERSPs are less time-sensitive, require fewer repetitions of the same behavior, and are potentially better suited to study EEG signatures in naturalistic settings. We modified, designed, and tested an algorithm that allows the combination of EEG and eye-tracking data recorded in virtual reality.