In this paper, we introduce LifeInsight – an interactive lifelog retrieval system developed for the sixth annual Lifelog Search Challenge (LSC’23). LifeInsight incorporates semantic search mechanisms from state-of-the-art lifelog retrieval systems while focusing on providing insights into the lifelogger’s routine using spatial information to support question-answering tasks. The system employs the Bootstrapping Language-Image Pre-training (BLIP) model for zero-shot image-text retrieval, which has been shown to achieve higher recall scores than the CLIP model on the Flickr30K dataset. In addition, the Elastic Search filtering mechanism is utilized to remove irrelevant images. Apart from semantic search mechanisms, the system also supports visual similarity search by comparing the inner product distance between the vectors in the lifelog image corpus and the query image. Furthermore, the system includes an explicit relevance feedback function, AI-based query description rewriting, and visual-example-generating features to re-phrase the query to describe it better and support end-users envisioning the targeted image for retrieval.
We introduce the LifeInsight 2.0 system - an enhanced version of LifeInsight, built specifically for the sixth annual Lifelog Search Challenge (LSC'23). LifeInsight 2.0 leverages the core functionalities of LifeInsight while incorporating significant improvements to address performance bottlenecks. This refined architecture aims to deliver superior search capabilities within the LSC'24. LifeInsight 2.0 employs an ensemble approach combining two powerful foundation models: CLIP (Contrastive Language-Image Pretraining) and BLIP2 (Bootstrapping Language-Image Pretraining) model. In addition, the system incorporates a temporal query mechanism and an automatic query parser. The former enables LifeInsight 2.0 to interpret queries that include time-based information, while the latter specifically handles tasks involving question answering.
In lifelog data search, despite automatic supports for identifying relevant pieces of information, the processes of inputting queries and filtering information from the generated results still heavily rely on human searchers. With the rapid increase in volume of such data, these tasks could become both mentally and physically tedious for an individual to perform. In this paper, we present CollaXRSearch, a collaborative virtual reality (VR) information retrieval system, which we aim to use for participating at the Lifelog Search Challenge 2024. This is a collaborative virtual reality system based on a heterogeneous setup of VR headsets, mobile devices and public displays. Its purpose is to facilitate coordination between teammates in terms of inputting and exploration operations in lifelog search. For efficiently providing textual query input and filtering, this system utilizes an interface which can be operated on a personal computer or a mobile device. Search results returned by the engine will be displayed in a VR environment where a user wearing a VR headset can explore to identify suitable items. To reduce the workload for the VR user during the searching process, we employs collaborative VR interface designs on a large physical display which enable he/she to communicate findings on the search results to the rest of the team. In this paper, we describe the conceptual interface and interaction designs of aforementioned setup.
User interfaces are essential in digital lifelogging systems, enabling fast and precise retrieval of images and memory data. Despite lifelogging's established presence in information retrieval, user interfaces have been notably underexplored. This research evaluates common UI and UX issues in existing lifelog systems and applies established design principles to improve our system's interface. Introducing VitaChronicle, we implement these principles to enhance user interaction and accessibility with an adaptable and consistent design. VitaChronicle addresses key interface design challenges, providing a user-friendly platform that simplifies navigation. The system is intuitively designed for both novice and expert users, ensuring ease of use for all. Through VitaChronicle, we aim to set new standards for lifelog interfaces, creating a blueprint for future systems that prioritize simplicity, usability, and engagement.
The amplified advancements of lifelog retrieval systems has led to increased demands for user-friendly systems tailored to both expert and novice users. At the heart of this evolution is the Lifelog Search Challenge (LSC), an annual forum facilitating collaborative research in lifelog retrieval systems. While systems such as Lifeinsight 2.0 exhibit advanced features, their optimization primarily caters to expert users, leaving novice users underserved. In response, this paper introduces Retrospect, an enhanced user interface (UI) developed for the seventh annual LSC'24. Building upon the innovative functionalities of its predecessor, Retrospect prioritizes user experience (UX) and UI enhancements to address the specific needs of novice users. Through iterative user testing and interface refinements, Retrospect offers an intuitive and accessible platform designed to excel in the diverse tasks presented within the Lifelog Search Challenge.