Yvette Graham

Yvette Graham

Auto-generated profile, based on publicly available information

Yvette Graham

School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
12 June 2023
Verified
Conference Paper
Open access
NANaushad Alam
Naushad Alam
YGYvette Graham
Yvette Graham
Cathal GurrinCathal Gurrin
Cathal Gurrin
In this work, we present our system Memento 3.0 for participation in the Lifelog Search Challenge 2023, which is a successor to the previous 2 iterations of our system called Memento 1.0 [1] and Memento 2.0 [2]. Memento 3.0 employs image-text embeddings derived from OpenAI CLIP models as well as larger OpenCLIP models trained on ∼ 5x more data. Our system also significantly reduces the query processing time by almost 75% when compared to its predecessor systems by employing a cluster-based search technique. We additionally make important updates to the system’s user interface to offer more flexibility to the user and at the same time be better suited to efficiently handle new query types introduced in the Lifelog Search Challenge.

10 June 2024
Verified
Conference Paper
Open access
NANaushad Alam
Naushad Alam
YGYvette Graham
Yvette Graham
Cathal GurrinCathal Gurrin
Cathal Gurrin
The practice of lifelogging, capturing one's daily experiences through wearable devices, has evolved significantly over the last decade, presenting both challenges and opportunities in information retrieval. This paper presents an early prototype of a conversational lifelog retrieval system designed to address the open challenges in this domain. Our system integrates a hierarchical event segmentation approach to automatically organize lifelog data into meaningful events, facilitating event-based retrieval over traditional image retrieval. Moreover, we incorporate a question-answering pipeline, leveraging large language models such as GPT-3.5 Turbo and Mistral7B, to enable free-form natural language interaction with the lifelog dataset. Moreover, we enhance our system's user interface by building on previous versions to streamline event-based retrieval and question-answering functionalities.

26 October 2023
Verified
Conference Paper
Open access
CLChenyang Lyu
Chenyang Lyu
WLWenxi Li
Wenxi Li
TJTianbo Ji
Tianbo Ji
LWLongyue Wang
Longyue Wang
LZLiting Zhou
Liting Zhou
JFJennifer Foster
Jennifer Foster
Video-language learning has attracted significant attention in the fields of multimedia, computer vision and natural language processing in recent years. One of the key challenges in this area is how to effectively integrate visual and linguistic information to enable machines to understand video content and query information. In this work, we leverage graph-based representations and multi-grained audio-visual alignment to address this challenge. First, our approach starts by transforming video and query inputs into visual-scene graphs and semantic role graphs using a visual-scene parser and semantic role labeler respectively. These graphs are then encoded using graph neural networks to obtain enriched representations and combined to obtain a video-query joint representation that enhances the semantic expressivity of the inputs. Second, to achieve accurate matching of relevant parts of audio and visual features, we propose a multi-grained alignment module that aligns the audio and visual features at multiple scales. This enables us to effectively fuse the audio and visual information in a way that is consistent with the semantic-level information captured by the graph-based representations. Experiments on five representative datasets collected for Video Retrieval and Video Question Answering tasks show that our approach outperforms the literature on several metrics. Our extensive ablation studies demonstrate the effectiveness of graph-based representation and multi-grained audio-visual alignment.

1 January 2023
Verified
Conference Paper
Open access
PDF file
CLChenyang Lyu
Chenyang Lyu
TJTianbo Ji
Tianbo Ji
YGYvette Graham
Yvette Graham
JFJennifer Foster
Jennifer Foster
Chenyang Lyu, Tianbo Ji, Yvette Graham, Jennifer Foster. Proceedings of The Fourth Workshop on Simple and Efficient Natural Language Processing (SustaiNLP). 2023.

15 July 2026 in Information
Verified
Journal Article
Open access
PDF file
SHSami Ul Haq
Sami Ul Haq
Sheila CastilhoSheila Castilho
Sheila Castilho
YGYvette Graham
Yvette Graham
Multimodal machine translation (MMT) aims to integrate visual context with textual data to improve the translation of ambiguous source text, such as the inclusion of an image as additional context. However, the evaluation of systems largely still relies on automatic metrics designed to evaluate text alone, and do not account for additional modalities during evaluation. The lack of dedicated MMT evaluation methods often results in inconsistent findings and creates uncertainty regarding the actual contribution of visual context in translation. In this work, we examine the performance of state-of-the-art trained and untrained evaluation metrics, particularly when comparing multimodal and text-only systems. Our evaluation focuses on the degree to which existing metrics are sensitive enough to distinguish between multimodal and text-only machine translation systems. We further investigate the potential for automatically generated image descriptions to serve as effective contextual signals for improving metric sensitivity to multimodal tasks. Our results show that incorporating such visual information into supervised metrics yields better alignment with human judgment. While all metrics successfully distinguished image-aware from image-agnostic systems on general test sets, both n-gram-based and embedding-based metrics struggled with respect to contrastive evaluation designed to capture context-dependent errors. Furthermore, we discuss how the presence of visual context may influence human evaluator judgment, observing that given the opportunity, human ratings are often substantially revised, further emphasizing the critical role of context in the evaluation of MMT.

9 October 2022 in arXiv
Verified
Preprint
Open access
PDF file
TJTianbo Ji
Tianbo Ji
CLChenyang Lyu
Chenyang Lyu
GJGareth Jones
Gareth Jones
Liting ZhouLiting Zhou
Liting Zhou
YGYvette Graham
Yvette Graham
Question Generation (QG) aims to automate the task of composing questions for a passage with a set of chosen answers found within the passage. In recent years, the introduction of neural generation models has resulted in substantial improvements of automatically generated questions in terms of quality, especially compared to traditional approaches that employ manually crafted heuristics. However, the metrics commonly applied in QG evaluations have been criticized for their low agreement with human judgement. We therefore propose a new reference-free evaluation metric that has the potential to provide a better mechanism for evaluating QG systems, called QAScore. Instead of fine-tuning a language model to maximize its correlation with human judgements, QAScore evaluates a question by computing the cross entropy according to the probability that the language model can correctly generate the masked words in the answer to that question. Furthermore, we conduct a new crowd-sourcing human evaluation experiment for the QG evaluation to investigate how QAScore and other metrics can correlate with human judgements. Experiments show that QAScore obtains a stronger correlation with the results of our proposed human evaluation method compared to existing traditional word-overlap-based metrics such as BLEU and ROUGE, as well as the existing pretrained-model-based metric BERTScore.