Lifelogging is an activity of recording all events that happen in the daily life of an individual. The events can contain images, audio, health index, etc which are collected through various devices such as wearable cameras, smartwatches, and other digital services. Exploiting lifelog data can bring significant benefits for lifeloggers from creating personalized healthcare plans to retrieving events in the past. In recent years, there has been a growing development of interactive lifelog retrieval systems, such as competitors at the annual Lifelog Search Challenge (LSC), to assist lifeloggers in finding events from the past. This paper introduces an interactive lifelog image retrieval called MemoriEase for the LSC’23 challenge. This system combines concept-based and embedding-based retrieval approaches to answer accurate images for LSC’23 queries. This system uses BLIP for the embedding-based retrieval approach to reduce the semantic gap between images and text queries. The concept-based retrieval approach uses full-text search in Elasticsearch to retrieve images having visual concepts similar to keywords in the query. Regarding the user interface, we make it as simple as possible to make novices users can use it with only a small effort. This is the first version of MemoriEase and we expect this can help users perform well in the LSC’23 competition.

Lifelog retrieval plays an important role in memory support for lifeloggers. It helps the lifeloggers to browse, search and navigate their life moments from the lifelog data. However, the volume and variety of lifelog data are enormous and range in multiple modalities so they impose a big challenge to retrieve accurate lifelog moments. The Lifelog Search Challenges (LSCs) are a benchmark challenge for evaluating lifelog retrieval systems in different tasks. In this paper, we introduce the MemoriEase 2.0 lifelog retrieval system that participates in LSC'24. This system not only inherits core functions from the precedent system but also incorporates new components such as conversational search, visual similarity search and retrieval-augmented generation for question-answering tasks. The new functions are expected to help expert and novice users solve all topics in three tasks of LSC'24. We evaluate MemoriEase 2.0 in KIS topics in LSC'23 and the system achieves promising results with Recall@1 is 40% at the first hint and it solves 8 over 10 topics.

The advent of large language models (LLMs) has energised research in Question-Answering (QA) tasks, enabling responses across varied domains like economics and mathematics. Despite their capabilities, LLMs often lack explainability due to their complex parameter embeddings. Additionally, integrating multimedia data into QA systems introduces challenges in processing and interpreting diverse data types such as text, images, audio, and video. This necessitates sophisticated algorithms for accurate information retrieval across media while ensuring the reliability of the data and responses remains a significant challenge. The AIQAM workshop aims to bring together researchers and practitioners to address these challenges and enhance QA systems with multimedia data. The focus is on promoting innovations that improve the accuracy, explainability, and trustworthiness of QA systems, contributing to the development of the field.

Lifelogging can be referred to as the process of passively collecting data on an individual's daily life. Lifelog data provides a large amount of information which can be used to understand the lifelogger's lifestyle and preferences. This data can also support the lifeloggers in saving their memories and important moments. Question-answering (QA) is a common task in natural language processing (NLP) and can be extended to multi-modal such as the visual question-answering task. QA for lifelog data can be described as the task of answering questions about a lifelogger's past using lifelog data, which can significantly help lifeloggers understand their life by asking questions about their lifelog. QA for lifelogs can also provide useful insights into lifelogger's life for those exploring their lifelog. This paper presents the MemoriQA lifelog dataset designed to explore the question-answering task for lifelogs. This dataset provides 61-day lifelog images and other lifelog data such as internet activity, health metrics, music listening history and GPS. A comprehensive annotation process is performed to create the description as well as question-answer pairs. We propose some methods to address the QA in lifelog problem in this paper.