Dima Damen

Dima Damen

Auto-generated profile, based on publicly available information

Dima Damen

University of Bristol, Bristol, UK

Research Focus & Metrics

Academic Career Length

20Years (2007-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
20 October 2021 in International Journal of Computer Vision
Verified
Journal Article
Open access
PDF file
DDDima Damen
Dima Damen
HDHazel Doughty
Hazel Doughty
GFGiovanni Maria Farinella
Giovanni Maria Farinella
AFAntonino Furnari
Antonino Furnari
EKEvangelos Kazakos
Evangelos Kazakos
MWMichael Wray
Michael Wray
This paper introduces the pipeline to extend the largest dataset in egocentric vision, EPIC-KITCHENS. The effort culminates in EPIC-KITCHENS-100, a collection of 100 hours, 20M frames, 90K actions in 700 variable-length videos, capturing long-term unscripted activities in 45 environments, using head-mounted cameras. Compared to its previous version (Damen in Scaling egocentric vision: ECCV, 2018), EPIC-KITCHENS-100 has been annotated using a novel pipeline that allows denser (54% more actions per minute) and more complete annotations of fine-grained actions (+128% more action segments). This collection enables new challenges such as action detection and evaluating the “test of time”—i.e. whether models trained on data collected in 2018 can generalise to new footage collected two years later. The dataset is aligned with 6 challenges: action recognition (full and weak supervision), action detection, action anticipation, cross-modal retrieval (from captions), as well as unsupervised domain adaptation for action recognition. For each challenge, we define the task, provide baselines and evaluation metrics.

6 May 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
DDDima Damen
Dima Damen
HDHazel Doughty
Hazel Doughty
GFGiovanni Maria Farinella
Giovanni Maria Farinella
SFSanja Fidler
Sanja Fidler
AFAntonino Furnari
Antonino Furnari
MWMichael Wray
Michael Wray
Since its introduction in 2018, EPIC-KITCHENS has attracted attention as the largest egocentric video benchmark, offering a unique viewpoint on people’s interaction with objects, their attention, and even intention. In this paper, we detail how this large-scale dataset was captured by 32 participants in their native kitchen environments, and densely annotated with actions and object interactions. Our videos depict nonscripted daily activities, as recording is started every time a participant entered their kitchen. Recording took place in four countries by participants belonging to ten different nationalities, resulting in highly diverse kitchen habits and cooking styles. Our dataset features 55 hours of video consisting of 11.5M frames, which we densely labelled for a total of 39.6K action segments and 454.2K object bounding boxes. Our annotation is unique in that we had the participants narrate their own videos (after recording), thus reflecting true intention, and we crowd-sourced ground-truths based on these. We describe our object, action and anticipation challenges, and evaluate several baselines over two test splits, seen and unseen kitchens. We introduce new baselines that highlight the multimodal nature of the dataset and the importance of explicit temporal modelling to discriminate fine-grained actions (e.g., ‘closing a tap’ from ‘opening’ it up).

1 October 2019
Verified
Conference Paper
Closed access
EKEvangelos Kazakos
Evangelos Kazakos
ANArsha Nagrani
Arsha Nagrani
AZAndrew Zisserman
Andrew Zisserman
DDDima Damen
Dima Damen
We focus on multi-modal fusion for egocentric action recognition, and propose a novel architecture for multi-modal temporal-binding, i.e. the combination of modalities within a range of temporal offsets. We train the architecture with three modalities -- RGB, Flow and Audio -- and combine them with mid-level fusion alongside sparse temporal sampling of fused representations. In contrast with previous works, modalities are fused before temporal aggregation, with shared modality fusion weights over time. Our proposed architecture is trained end-to-end, outperforming individual modalities as well as late-fusion of modalities. We demonstrate the importance of audio in egocentric vision, on per-class basis, for identifying actions as well as interacting objects. Our method achieves state of the art results on both the seen and unseen test sets of the largest egocentric dataset: EPIC-Kitchens, on all metrics using the public leaderboard.

6 October 2018
Verified
Book Chapter
Closed access
PDF file
DDDima Damen
Dima Damen
HDHazel Doughty
Hazel Doughty
GFGiovanni Maria Farinella
Giovanni Maria Farinella
SFSanja Fidler
Sanja Fidler
AFAntonino Furnari
Antonino Furnari
MWMichael Wray
Michael Wray

26 July 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Open access
KGKristen Grauman
Kristen Grauman
AWAndrew Westbury
Andrew Westbury
EBEugene Byrne
Eugene Byrne
VCVincent Cartillier
Vincent Cartillier
ZCZachary Chavis
Zachary Chavis
JMJitendra Malik
Jitendra Malik
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,670 hours of daily-life activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 931 unique camera wearers from 74 worldwide locations and 9 different countries. The approach to collection is designed to uphold rigorous privacy and ethics standards, with consenting participants and robust de-identification procedures where relevant. Ego4D dramatically expands the volume of diverse egocentric video footage publicly available to the research community. Portions of the video are accompanied by audio, 3D meshes of the environment, eye gaze, stereo, and/or synchronized videos from multiple egocentric cameras at the same event. Furthermore, we present a host of new benchmark challenges centered around understanding the first-person visual experience in the past (querying an episodic memory), present (analyzing hand-object manipulation, audio-visual conversation, and social interactions), and future (forecasting activities). By publicly sharing this massive annotated dataset and benchmark suite, we aim to push the frontier of first-person perception.

1 June 2020
Verified
Conference Paper
Closed access
JMJonathan Munro
Jonathan Munro
DDDima Damen
Dima Damen
Fine-grained action recognition datasets exhibit environmental bias, where multiple video sequences are captured from a limited number of environments. Training a model in one environment and deploying in another results in a drop in performance due to an unavoidable domain shift. Unsupervised Domain Adaptation (UDA) approaches have frequently utilised adversarial training between the source and target domains. However, these approaches have not explored the multi-modal nature of video within each domain. In this work we exploit the correspondence of modalities as a self-supervised alignment approach for UDA in addition to adversarial alignment (Fig. 1). We test our approach on three kitchens from the large-scale EPIC-Kitchens dataset, using two modalities commonly employed for action recognition: RGB and Optical Flow. We show that multi-modal self-supervision alone improves the performance over source-only training by 2.4% on average. We then combine adversarial training with multi-modal self-supervision, showing that our approach outperforms other UDA methods by 3%.