1 April 2006 in Current Biology
1 May 2005 in Perception & Psychophysics
19 February 2023 in Computer Graphics Forum
We present ZeroEGGS, a neural network framework for speech‐driven gesture generation with zero‐shot style control by example. This means style can be controlled via only a short example motion clip, even for motion styles unseen during training. Our model uses a Variational framework to learn a style embedding, making it easy to modify style through latent space manipulation or blending and scaling of style embeddings. The probabilistic nature of our framework further enables the generation of a variety of outputs given the input, addressing the stochastic nature of gesture motion. In a series of experiments, we first demonstrate the flexibility and generalizability of our model to new speakers and styles. In a user study, we then show that our model outperforms previous state‐of‐the‐art techniques in naturalness of motion, appropriateness for speech, and style portrayal. Finally, we release a high‐quality dataset of full‐body gesture motion including fingers, with speech, spanning across 19 different styles. Our code and data are publicly available at https://github.com/ubisoft/ubisoft‐laforge‐ZeroEGGS.
3 June 2006 in Journal of Vision
Human visual perception is highly adaptive. While this has been known and studied for a long time in domains such as color vision, motion perception, or the processing of spatial frequency, a number of more recent studies have shown that adaptation and adaptation aftereffects also occur in high-level visual domains like shape perception and face recognition. Here, we present data that demonstrate a pronounced aftereffect in response to adaptation to the perceived gender of biological motion point-light walkers. A walker that is perceived to be ambiguous in gender under neutral adaptation appears to be male after adaptation with an exaggerated female walker and female after adaptation with an exaggerated male walker. We discuss this adaptation aftereffect as a tool to characterize and probe the mechanisms underlying biological motion perception.
1 January 2009 in Journal of Vision
The ability to derive the facing direction of a spatially scrambled point-light walker relies on the motions of the feet and is impaired if they are inverted. We exploited this local inversion effect in three experiments that employed novel stimuli derived from only fragments of full foot trajectories. In Experiment 1, observers were presented with stimuli derived from a single fragment or a pair of counterphase fragments of the foot trajectory of a human walker in a direction discrimination task. We show that direction can be retrieved for displays as short as 100 ms and is retrieved in an orientation-dependent manner only for stimuli derived from the paired fragments. In Experiment 2, we investigated direction retrieval from stimuli derived from paired fragments of other foot motions. We show that the inversion effect is correlated with the difference in vertical acceleration between the constituent fragments of each stimulus. In Experiment 3, we compared direction retrieval from the veridical human walker stimuli with stimuli that were identical but had accelerations removed. We show that the inversion effect disappears for the stimuli containing no accelerations. The results suggest that the local inversion effect is carried by accelerations contained in the foot motions.
1 July 2007 in Journal of Vision
Biological motion perception, having both evolutionary and social importance, is performed by the human visual system with a high degree of sensitivity. It is unclear whether peripheral vision has access to the specialized neural systems underlying biological motion perception; however, given the motion component, one would expect peripheral vision to be, if not specialized, at least highly accurate in perceiving biological motion. Here we show that the periphery can indeed perceive biological motion. However, the periphery suffers from an inability to detect biological motion signals when they are embedded in dynamic visual noise. We suggest that this peripheral deficit is not due to biological motion perception per se, but to signal/noise segregation.