19 October 2020 in Nature Nanotechnology
15 June 2026 in International Journal of Computer Vision
Machine learning and computer vision methods have a major impact on the study of natural animal behavior, as they enable the (semi-) automatic analysis of vast amounts of video data. Such large-scale analysis is essential not only to mere behavioral research but its applicability spans across a large range of disciplines. Mice are the standard mammalian model system in most behavioral research fields, but the datasets available today to refine such methods focus either on isolated or social behaviors. In contrast, datasets in which animals interact with a physical apparatus, which is highly relevant across disciplines that study learning, are as of yet unavailable. In this work, we present a video dataset of individual mice solving (multi-step) mechanical puzzles, so-called lockboxes. The more than 110 hours of total playtime show their behavior recorded from three different perspectives. As a benchmark for frame-level action classification methods, we provide human-annotated labels for all videos of two different mice, equaling 13% of our dataset. Our action labels provide two levels of complexity, requiring the classification of both what the mouse is doing as well as which object is targeted. Additionally, as an initial comparison against the human-annotated labels, we used two different keypoint (pose) tracking-based action classification frameworks as well as an autoencoder-based framework, which illustrates the challenges of automated labeling of fine-grained behaviors, such as the manipulation of objects. We hope that our work will help accelerate the advancement of automated action and behavior classification in a task-driven environment. Our dataset, including videos and human-annotated labels, is publicly available at https://doi.org/10.14279/depositonce-23850 .
21 November 2025 in Nature Communications
HEHans-Christian Ruiz Euler
Hans-Christian Ruiz Euler
+2
Deep learning, a multilayered neural-network approach inspired by the brain, has revolutionized machine learning. Its success relies on backpropagation, which computes gradients of a loss function for use in gradient descent. However, digital implementations are energy hungry, with power demands limiting many applications. This has motivated specialized hardware, from neuromorphic CMOS and photonic tensor cores to unconventional material-based systems. Learning in such systems, for example via artificial evolution, equilibrium propagation, or surrogate modelling, is typically complicated and slow. Here, we demonstrate a simple gradient-extraction method based on homodyne detection, enabling gradient descent directly in physical systems without the need for an analytical description. By perturbing parameters with sinusoidal waveforms at distinct frequencies, we robustly obtain gradient information in a scalable manner. We illustrate the method in reconfigurable nonlinear-processing units and argue for broad applicability. Homodyne gradient extraction can in principle be fully implemented in materia, facilitating autonomously learning material systems.