Taotao Zhou

Taotao Zhou

Auto-generated profile, based on publicly available information

Taotao Zhou

ShanghaiTech University & LumiAni Technology, Shanghai, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 April 2025 in bioRxiv
Verified
Preprint
Open access
PDF file
TXTeng Xu
Teng Xu
TZTaotao Zhou
Taotao Zhou
YWYoujia Wang
Youjia Wang
PYPeng Yang
Peng Yang
STSimin Tang
Simin Tang
JYJingyi Yu
Jingyi Yu
Analyzing animal behavior is crucial in neuroscience, yet deciphering its intricate dynamics remains challenging. Traditional machine vision approaches detect spontaneous behaviors but are constrained by limited interpretability and reliance on manual labeling, restricting exploration of the full behavioral spectrum. Here, we introduce MouseGPT, an integrated vision-language model and analytical framework that translates raw video of animal activity into natural language to automate open-vocabulary behavior analysis. Built upon a comprehensive dataset of over 42 million frames spanning diverse psychiatric conditions, MouseGPT maps multi-view or single-camera recordings directly into computable semantic representations. An agentic framework leverages these representations for detailed behavior profiling, cross-view stable clustering, and zero-shot novel behavior discovery without requiring labor-intensive annotation. Surpassing conventional kinematics-based pipelines and generalist vision-language models in behavioral precision and adaptability, MouseGPT establishes continuous semantic representations as a powerful tool for deciphering complex ethological dynamics in animal models.

28 October 2024
Verified
Conference Paper
Open access
TZTaotao Zhou
Taotao Zhou
TXTeng Xu
Teng Xu
DZDong Zhang
Dong Zhang
YJYuyang Jiao
Yuyang Jiao
PXPeijun Xu
Peijun Xu
JYJingyi Yu
Jingyi Yu
We present Sophia-in-Audition (SiA), a new frontier in virtual production, by employing the humanoid robot Sophia within an UltraStage environment composed of a controllable lighting dome coupled with multiple cameras. We demonstrate Sophia's capability to replicate iconic film segments, follow real performers, and perform a variety of motions and expressions, showcasing her versatility as a virtual actor. Key to this process is the integration of facial motion transfer algorithms and the UltraStage's controllable lighting and multi-camera setup, enabling dynamic performances that align with the director's vision. Our comprehensive user studies indicate positive audience reception towards Sophia's performances, highlighting her potential to reduce the uncanny valley effect in virtual acting. Additionally, the immersive lighting in dynamic clips was highly rated for its naturalness and its ability to mirror professional film standards. The paper presents a first-of-its-kind multi-view robot performance video dataset with dynamic lighting, offering valuable insights for future enhancements in humanoid robotic performers and virtual production techniques. This research contributes significantly to the field by presenting a unique virtual production setup, developing tools for sophisticated performance control, and providing a comprehensive dataset and user study analysis for diverse applications.