Experiments testing sound for augmented reality can involve real and virtual sound sources. Paradigms are either based on rating various acoustic attributes or testing whether a virtual sound source is believed to be real (i.e., evokes an auditory illusion). This study compares four experimental designs indicating such illusions. The first is an ABX task suitable for evaluation under the authenticity paradigm. The second is a Yes/No task, as proposed to evaluate plausibility. The third is a three-alternative-forced-choice (3AFC) task using different source signals for real and virtual, proposed to evaluate transfer-plausibility. Finally, a 2AFC task was tested. The renderings compared in the tests encompassed mismatches between real and virtual room acoustics. Results confirm that authenticity is hard to achieve under nonideal conditions, and ceiling effects occur because differences are always detected. Thus, the other paradigms are better suited for evaluating practical augmented reality audio systems. Detection analysis further shows that the 3AFC transfer-plausibility test is more sensitive than the 2AFC task. Moreover, participants are more sensitive to differences between real and virtual sources in the Yes/No task than theory predicts. This contribution aims to aid in selecting experimental paradigms in future experiments regarding perceptual and technical requirements for sound in augmented reality.
5 September 2024 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Smart glasses are increasingly recognized as a key medium for augmented reality, offering a hands-free platform with integrated microphones and non-ear-occluding loudspeakers to seamlessly mix virtual sound sources into the real-world acoustic scene. To convincingly integrate virtual sound sources, the room acoustic rendering of the virtual sources must match the real-world acoustics. Information about a user's acoustic environment however is typically not available. This work uses a microphone array in a pair of smart glasses to blindly identify binaural room impulse responses (BRIRs) from a few seconds of speech in the real-world environment. The proposed method uses dereverberation and beamforming to generate a pseudo reference signal that is used by a multichannel Wiener filter to estimate room impulse responses which are then converted to BRIRs. The multichannel room impulse responses can be used to estimate room acoustic parameters which is shown to outperform baseline algorithms in the estimation of reverberation time and direct-to-reverberant energy ratio. Results from a listening experiment further indicate that the estimated BRIRs often reproduce the real-world room acoustics perceptually more convincingly than measured BRIRs from other rooms of similar size.
Ambisonics is a scene-based spatial audio format that has several useful features compared to object-based formats, such as efficient whole scene rotation and versatility. However, it does not provide direct access to the individual source signals, so that these have to be separated from the mixture when required. Typically, this is done with linear spherical harmonics (SH) beamforming. In this paper, we explore deep-learning-based source separation on static Ambisonics mixtures. In contrast to most source separation approaches, which separate a fixed number of sources of specific sound types, we focus on separating arbitrary sound from specific directions. Specifically, we propose three operating modes that combine a source separation neural network with SH beamforming: refinement, implicit, and mixed mode. We show that a neural network can implicitly associate conditioning directions with the spatial information contained in the Ambisonics scene to extract specific sources. We evaluate the performance of the three proposed approaches and compare them to SH beamforming on musical mixtures generated with the musdb18 dataset, as well as with mixtures generated with the FUSS dataset for universal source separation, under both anechoic and room conditions. Results show that the proposed approaches offer improved separation performance and spatial selectivity compared to conventional SH beamforming.
1 November 2021 in The Journal of the Acoustical Society of America
Multichannel auralizations based on spatial room impulse responses often employ sample-wise assignment of an omnidirectional response to form loudspeaker responses. This leads to sparse impulse responses in each reproduction loudspeaker and the auralization of transient signals can sound rough. Based on this observation, we conducted a listening test to examine the general phenomenon of roughness due to spatial assignment. First, participants assessed the roughness of both Gaussian noise and velvet noise, assigned sample-wise to up to 36 loudspeakers by two algorithms. The first algorithm assigns channels merely by selecting random indices, while the second one constrains the time between two peaks on each channel. The results show that roughness already occurs when few channels are used and that the assignment algorithm influences it. In a second experiment, virtualizations of the test were used to examine the factors contributing to increased roughness. We systematically show the effect of spatial assignment on noise and conclude that besides time-differences, level-differences caused by head-shadowing are the principal cause for the perceived roughness. The results have significance in spatial room impulse response rendering and spatial reverberator design.
Measured spatial room impulse responses have been used to compare acoustic spaces. One way to analyze and render such responses is to apply parametric methods, yet those methods have been bound to single measurement locations. This paper introduces a method that locates image sources from spatial room impulse responses measured at multiple source and receiver positions. The method aligns the measurements to a common coordinate frame and groups stable direction-of-arrival estimates to find image source positions. The performance of the method is validated with three case studies—one small room and two concert halls. The studies show that the method is able to locate the most prominent image sources even in complex spaces, providing new insights into available Spatial Room Impulse Response (SRIR) data and a starting point for six degrees of freedom (6DoF) acoustic rendering.
1 July 2023 in The Journal of the Acoustical Society of America
This study investigates the room acoustics of seven chamber music halls of various modern and historical architecture by means of objective room acoustic measures and a subjective listening experiment. The acoustic measurements were performed with heavy cloth covering the audience areas to simulate occupancy in the halls. A loudspeaker quartet was used for auralizations, which were reproduced in a surrounding loudspeaker array. The perceptual differences between the halls were evaluated in terms of envelopment, warmth, clarity, proximity, and preference by using a paired comparison paradigm. The subjective evaluations were conducted in two different laboratories and latent class analysis was used to study the agreement between laboratories and the emergence of different listener groups in the ratings of each attribute. Concerning preference, the emergence of two groups found in the study of large symphony halls was confirmed, where one group prefers rich, enveloping sound and one group prefers high clarity. The perceptual ratings were not clearly associated with a specific hall shape, but rather depended on the distribution of early and late sound energy. Thus, the distinction between rectangular and non-rectangular floor plans previously found for large symphony halls was not observed with these smaller halls.