The Think Aloud Protocol (TAP) is a verbalisation technique widely employed in HCI user studies to give insight into user experience, yet little work has explored the impact that TAPs have on participants during user studies. This paper utilises a brain sensing technique, fNIRS, to observe the effect that TAPs have on participants. Functional Near-Infrared Spectroscopy (fNIRS) is a brain sensing technology that offers the potential to provide continuous, detailed insight into brain activity, enabling an objective view of cognitive processes during complex tasks. Participants were asked to perform a mathematical task under 4 conditions: nonsense verbalisations, passive concurrent think aloud protocol, invasive concurrent think aloud protocol, and a baseline of silence. Subjective ratings and performance measures were collected during the study. Our results provide a novel view into the effect that different forms of verbalisation have on workload during tasks. Further, the results provide a means for estimating the effect of spoken artefacts when measuring workload, which is another step towards our goal of proactively involving fNIRS analysis in ecologically valid user studies

As computer interfaces spread across and even into our bodies, so we find ourselves engaged in ever more deeply connected bodily interactions. From the mouse and keyboard, to wearables and gestural...

This paper explores the design space of bio-responsive entertainment, in this case using a film that responds to the brain and blink data of users. A film was created with four parallel channels of footage, where blinking and levels of attention and meditation, as recorded by a commercially available EEG device, affected which footage participants saw. As a performance-led piece of research in the wild, this experience, named #Scanners, was presented at a week long national exhibition in the UK. We examined the experiences of 35 viewers, and found that these forms of partially-involuntary control created engaging and enjoyable, but sometimes distracting, experiences. We translate our findings into a two-dimensional design space between the extent of voluntary control that a physiological measure can provide against the level of conscious awareness that the user has of that control. This highlights that novel design opportunities exist when deviating from these two-dimensions - when giving up conscious control and when abstracting the affect of control. Reflection on of how viewers negotiated this space during an experience reveals novel design tactics.

Purpose: It is possible for civil engineering (CE) students to graduate from a degree programme without gaining experience on a construction site. The implementation of virtual field trips using virtual reality (VR) in CE education is a development that can address this phenomenon and help facilitate the consolidation of abstract theories into tangible competences. This project aims to solve a fundamental CE education problem: once a structure has been completed, it is often impossible to see how it was built; hence, how can you demonstrate the construction process to a student?Design/methodology/approach: This research used the opportunity of a new campus library development to record its construction sequence. This was achieved by visiting the site eight times to take panoramic stereoscopic photos of the construction process. By its nature, using VR as a didactic tool facilitates experiential learning, but this project also incorporates discovery learning and situated cognition to develop students’ understanding of the construction process.Findings: The use of VR in education is becoming increasingly common, but the explicit pedagogy used in these environments is rarely obvious or stated. This project draws upon current VR education discussions and explores the development of a VR environment with a pedagogical context.Originality/value: The development of the VR resource draws upon the pedagogical frameworks of discovery learning (Bruner, 1961) and situated cognition (Lave and Wenger, 1991). A further unique aspect of this research is the use of stereoscopic cameras to capture the library’s construction over time.

Future airspace is expected to become more congested with additional in-service cargo and commercial flights. Pilots will face additional burdens in such an environment, given the increasing number of factors that they must simultaneously consider while completing their work activities. Therefore, care and attention must be paid to the mental workload (MWL) experienced by operating pilots. If left unaddressed, a state of mental overload could affect the pilot’s ability to complete his or her work activities in a safe and correct manner. This study examines the impact of two different cockpit display interfaces (CDIs), the Steam Gauge panel and the G1000 Glass panel, on novice pilots’ MWL and situational awareness (SA) in a flight simulator-based setting. A combination of objective (EEG and HRV) and subjective (NASA-TLX) assessments is used to assess novice pilots’ cognitive states during this study. Our results indicate that the gauge design of the CDI affects novice pilots’ SA and MWL, with the G1000 Glass panel being more effective in reducing the MWL and improving SA compared with the Steam Gauge panel. The results of this study have implications for the design of future flight deck interfaces and the training of future pilots.

Smart pervasive sensor networks are becoming an important part of our daily lives. Low-power, high-availability and high-throughput 5G mobile networks provide the necessary communication means for highly pervasive sensor networks, introducing a technological disruption to health monitoring. The meaningful use of large concurrent sensor networks in healthcare requires multi-level health knowledge integration with sensor data streams. In this paper, we highlight some software engineering and data-processing issues that can be addressed by metamorphic testing. The proposed solution combines data streaming with filtering and cross-calibration, use of medical knowledge for system operation and data interpretation, and IoT-based calibration using certified linked diagnostic devices.