Tadas Baltrusaitis

Tadas Baltrusaitis

Auto-generated profile, based on publicly available information

Tadas Baltrusaitis

Microsoft, Cambridge, United Kingdom

Research Focus & Metrics

Academic Career Length

17Years (2010-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 January 2018 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
TBTadas Baltrusaitis
Tadas Baltrusaitis
CAChaitanya Ahuja
Chaitanya Ahuja
LMLouis-Philippe Morency
Louis-Philippe Morency
Our experience of the world is multimodal - we see objects, hear sounds, feel texture, smell odors, and taste flavors. Modality refers to the way in which something happens or is experienced and a research problem is characterized as multimodal when it includes multiple such modalities. In order for Artificial Intelligence to make progress in understanding the world around us, it needs to be able to interpret such multimodal signals together. Multimodal machine learning} aims to build models that can process and relate information from multiple modalities. It is a vibrant multi-disciplinary field of increasing importance and with extraordinary potential. Instead of focusing on specific multimodal applications, this paper surveys the recent advances in multimodal machine learning itself and presents them in a common taxonomy. We go beyond the typical early and late fusion categorization and identify broader challenges that are faced by multimodal machine learning, namely: representation, translation, alignment, fusion, and co-learning. This new taxonomy will enable researchers to better understand the state of the field and identify directions for future research.

1 January 2011
Verified
Book Chapter
Closed access
PDF file
GRGeovany A. Ramirez
Geovany A. Ramirez
TBTadas Baltrušaitis
Tadas Baltrušaitis
LMLouis-Philippe Morency
Louis-Philippe Morency

1 March 2011
Verified
Conference Paper
Open access
TBTadas Baltrusaitis
Tadas Baltrusaitis
DMDaniel McDuff
Daniel McDuff
NBNtombikayise Banda
Ntombikayise Banda
MMMarwa Mahmoud
Marwa Mahmoud
RKRana el Kaliouby
Rana el Kaliouby
RPRosalind Picard
Rosalind Picard
We present a real-time system for detecting facial action units and inferring emotional states from head and shoulder gestures and facial expressions. The dynamic system uses three levels of inference on progressively longer time scales. Firstly, facial action units and head orientation are identified from 22 feature points and Gabor filters. Secondly, Hidden Markov Models are used to classify sequences of actions into head and shoulder gestures. Finally, a multi level Dynamic Bayesian Network is used to model the unfolding emotional state based on probabilities of different gestures. The most probable state over a given video clip is chosen as the label for that clip. The average F1 score for 12 action units (AUs 1, 2, 4, 6, 7, 10, 12, 15, 17, 18, 25, 26), labelled on a frame by frame basis, was 0.461. The average classification rate for five emotional states (anger, fear, joy, relief, sadness) was 0.440. Sadness had the greatest rate, 0.64, anger the smallest, 0.11.

1 January 2011
Verified
Book Chapter
Closed access
PDF file
MMMarwa Mahmoud
Marwa Mahmoud
TBTadas Baltrušaitis
Tadas Baltrušaitis
PRPeter Robinson
Peter Robinson
LRLaurel D. Riek
Laurel D. Riek

19 November 2024 in ACM Transactions on Graphics
Verified
Journal Article
Open access
CHCharlie Hewitt
Charlie Hewitt
FSFatemeh Saleh
Fatemeh Saleh
SASadegh Aliakbarian
Sadegh Aliakbarian
LPLohit Petikam
Lohit Petikam
SRShideh Rezaeifar
Shideh Rezaeifar
TBTadas Baltrusaitis
Tadas Baltrusaitis
We tackle the problem of highly-accurate, holistic performance capture for the face, body and hands simultaneously. Motion-capture technologies used in film and game production typically focus only on face, body or hand capture independently, involve complex and expensive hardware and a high degree of manual intervention from skilled operators. While machine-learning-based approaches exist to overcome these problems, they usually only support a single camera, often operate on a single part of the body, do not produce precise world-space results, and rarely generalize outside specific contexts. In this work, we introduce the first technique for markerfree, high-quality reconstruction of the complete human body, including eyes and tongue, without requiring any calibration, manual intervention or custom hardware. Our approach produces stable world-space results from arbitrary camera rigs as well as supporting varied capture environments and clothing. We achieve this through a hybrid approach that leverages machine learning models trained exclusively on synthetic data and powerful parametric models of human shape and motion. We evaluate our method on a number of body, face and hand reconstruction benchmarks and demonstrate state-of-the-art results that generalize on diverse datasets.

16 February 2022 in IEEE Transactions on Biomedical Engineering
Verified
Journal Article
Closed access
DMDaniel McDuff
Daniel McDuff
JHJavier Hernandez
Javier Hernandez
XLXin Liu
Xin Liu
EWErroll Wood
Erroll Wood
TBTadas Baltrusaitis
Tadas Baltrusaitis
Non-contact physiological measurement has the potential to provide low-cost, non-invasive health monitoring. However, machine vision approaches are often limited by the availability and diversity of annotated video datasets resulting in poor generalization to complex real-life conditions. To address these challenges, this work proposes the use of synthetic avatars that display facial blood flow changes and allow for systematic generation of samples under a wide variety of conditions. Our results show that training on both simulated and real video data can lead to performance gains under challenging conditions. We show strong performance on three large benchmark datasets and improved robustness to skin type and motion. These results highlight the promise of synthetic data for training camera-based pulse measurement; however, further research and validation is needed to establish whether synthetic data alone could be sufficient for training models.