Lifelogging is the practice of continuously capturing and recording personal data and experiences through wearable technology, digital devices, and applications. Some of the most common lifelogging technologies today include social media platforms and wearable sensors. In recent years, the adoption and normalization of lifelogging have surged, driven by advancements in wearable devices, smartphone capabilities, and data analytics. These technologies not only allow individuals to track their fitness and health metrics but also enable them to document their daily activities, moods, and social interactions. The integration of artificial intelligence and machine learning has significantly enhanced the analysis of this data, providing personalized insights and recommendations. Consequently, lifelogging has evolved into a mainstream practice, empowering individuals to make informed decisions about their health and lifestyle.
The Annual ACM International Conference on Multimedia Retrieval (ICMR) offers an opportunity for exchanging leading-edge multimedia retrieval ideas among researchers, practitioners, and other potential users of multimedia retrieval systems. During this conference, the Lifelog Search Challenge (LSC) takes place. In this challenge, participants are tasked with finding either specific images, as many images as possible that fit a specific query, or specific answers to questions. The challenge consists of one expert round, where the creators of the systems participate, and one novice session where novice users who have not used the system before compete and try to solve the challenges. The final score is calculated by combining scores from both the expert and novice sessions. The goal is to successfully complete as many correct queries as possible.
During the conference we also got the opportunity to assist by volunteering and help with some technical issues to help ensure everything ran smoothly. This gave us valuable insights into the behind-the-scenes workings of such big conferences. We got to participate in a variety of insightful talks and workshops, gaining new perspectives and knowledge from esteemed researchers and practitioners within this research field.
Together with students from the University of Science at the University of Ho Chi Minh City and Dublin City University, we created Retrospect, a lifelog retrieval system with an improved focus on novice users. The increase in new lifelogging technologies creates the need for more user-friendly systems customized for different types of users, ranging from experts to novices. Retrospect is based on the previous LifeInsight, a lifelog retrieval system developed for LSC '23. By performing various design iterations with novice users, the system is focused on presenting the data in an understandable and accessible way, maximizing both clarity and accessibility for both expert and novice users.
From testing this and other systems created for LSC, we found especially one common issue. The systems are all created for expert users, making several systems difficult for novice users to understand and use. During the process of designing Retrospect, we chose to focus on novice users like we were at the beginning of the process. By focusing on creating a good user experience for novice users, we believe it will lead to effective retrieval of the desired data for everyone, not just expert users. The process started by identifying common challenges with previous retrieval systems, such as information overload, lack of visual hierarchy, and lack of understanding of the system's complexity. By doing this, it was easier to know what to focus on solving during the design process.
The interface in Retrospect is designed to minimize distractions and maximize clarity, employing a clean design with clear visual hierarchies and contrasting colors. The strategic use of whitespace and logically placed buttons helps users navigate the system without feeling overwhelmed. This design ensures that even those new to lifelog retrieval can efficiently find relevant images without unnecessary complexity. A new clustering function, accessible via a "Sort by" button, allows users to organize images by location, time, or relevance, further simplifying the retrieval process. The presentation of images includes highlights such as location and date, with both horizontal and vertical scroll options to manage the display effectively. The addition of a progress bar keeps users informed of their location within the search process. To design for all users, the system has been universally designed to be accessible to different types of users. The system complies with the WCAG guidelines for accessible web design, by, for example, fulfilling the color contrast rules and ensuring all font sizes are above the minimum size guidelines.
One important aspect of this system is participation in the Lifelog Search Challenge, and some of the design elements have been created especially for this. For example, the buttons that are connected to each other have been placed together, which is important for the lifelog search challenge. By providing easy access to the buttons used to submit answers and making it easy for the user to mark images, it enables fast and easy submission of answers and images during the challenge.
An interactive map has also been implemented, allowing users to view examples of images taken at different places, making it easy to find images connected to a specific location. This feature enables users to easily locate personal landmarks such as “home” or “work,” facilitating quicker and more accurate responses to specific queries.
More information about Retrospect can be read in the published article here!
Participating and competing in both the ICMR conference and the Lifelog Search Challenge has been an incredibly educational and fun experience. After months of user testing and iterations of our prototype, we were very proud to present and compete with our system, Retrospect. The competition also allowed us to gather comprehensive feedback from users with varying levels of expertise. We learned a lot about improving the UI and UX of our system for next year's competition. Particularly from novice users, we received valuable insights on enhancing the user experience to maximize the system's effectiveness. The final results led to an 18th place finish, and we are eager to return with an improved system and achieve an even better placement in the lifelog retrieval search challenge.
