4 April 2022 in Nature Methods
The desire to understand how the brain generates and patterns behavior has driven rapid methodological innovation in tools to quantify natural animal behavior. While advances in deep learning and computer vision have enabled markerless pose estimation in individual animals, extending these to multiple animals presents unique challenges for studies of social behaviors or animals in their natural environments. Here we present Social LEAP Estimates Animal Poses (SLEAP), a machine learning system for multi-animal pose tracking. This system enables versatile workflows for data labeling, model training and inference on previously unseen data. SLEAP features an accessible graphical user interface, a standardized data model, a reproducible configuration system, over 30 model architectures, two approaches to part grouping and two approaches to identity tracking. We applied SLEAP to seven datasets across flies, bees, mice and gerbils to systematically evaluate each approach and architecture, and we compare it with other existing approaches. SLEAP achieves greater accuracy and speeds of more than 800 frames per second, with latencies of less than 3.5 ms at full 1,024 × 1,024 image resolution. This makes SLEAP usable for real-time applications, which we demonstrate by controlling the behavior of one animal on the basis of the tracking and detection of social interactions with another animal.
20 December 2018 in Nature Methods
6 October 2014 in Journal of The Royal Society Interface
A frequent assumption in behavioural science is that most of an animal's activities can be described in terms of a small set of stereotyped motifs. Here, we introduce a method for mapping an animal's actions, relying only upon the underlying structure of postural movement data to organize and classify behaviours. Applying this method to the ground-based behaviour of the fruit fly,Drosophila melanogaster, we find that flies perform stereotyped actions roughly 50% of the time, discovering over 100 distinguishable, stereotyped behavioural states. These include multiple modes of locomotion and grooming. We use the resulting measurements as the basis for identifying subtle sex-specific behavioural differences and revealing the low-dimensional nature of animal motions.
9 November 2020 in Nature Neuroscience
7 September 2022 in Nature
23 November 2020 in Nature Physics