Where am I? This is one of the most critical questions that any intelligent system should answer to decide whether it navigates to a previously visited area. This problem has long been acknowledged for its challenging nature in simultaneous localization and mapping (SLAM), wherein the robot needs to correctly associate the incoming sensory data to the database allowing consistent map generation. The significant advances in computer vision achieved over the last 20 years, the increased computational power, and the growing demand for long-term exploration contributed to efficiently performing such a complex task with inexpensive perception sensors. In this article, visual loop closure detection, which formulates a solution based solely on appearance input data, is surveyed. We start by briefly introducing place recognition and SLAM concepts in robotics. Then, we describe a loop closure detection system's structure, covering an extensive collection of topics, including the feature extraction, the environment representation, the decision-making step, and the evaluation process. We conclude by discussing open and new research challenges, particularly concerning the robustness in dynamic environments, the computational complexity, and scalability in long-term operations. The article aims to serve as a tutorial and a position paper for newcomers to visual loop closure detection.

Place recognition of pre-visited areas, widely known as Loop Closure Detection (LCD), constitutes one of the most important components in robotic applications, where the robot needs to estimate its pose while navigating through the field (e.g., simultaneous localization and mapping). In this paper, we present a novel approach for LCD based on the assignment of Visual Words (VWs) to particular places of the traversed path. The system operates in real time and does not require any pre-training procedure, such as visual vocabulary construction or descriptor-space dimensionality reduction. A place is defined through a dynamic segmentation of the incoming image stream and is assigned with VWs through the usage of an on-line clustering algorithm. At query time, image descriptors are converted into VWs on the map accumulating votes to the corresponding places. By means of a probability function, the mechanism is capable of identifying a loop closing candidate place. A nearest neighbor voting scheme on the descriptors' space allows the system to select the most appropriate image match at the chosen place. Geometrical and temporal consistency checks are applied on the proposed loop closing pair increasing the system's performance. Evaluation took place on several publicly available and challenging datasets offering high precision and recall scores as compared to other state-of-the-art approaches.

A key feature in robotics applications is to recognize whether the current environment observation corresponds to a location visited hitherto. Should the place be recognized by the robot, a Loop Closure Detection (LCD) has occurred. The paper in hand deploys a novel low complexity LCD method based on the representation of the route by unique Visual Features (VFs). Each of these VFs, referred to as “Tracked Word” (TW), is generated on-line through a tracking technique coupled with a guided-feature-detection mechanism and belongs to a group of successive images. During the robot's navigation, new TWs are added to the database forming a Bag of Tracked Words (BoTW). When querying the database seeking for loop closures, the new local-feature-descriptors are associated to the nearest neighboring TWs in the map casting votes to the corresponding instances. The system relies on a probabilistic method to select the most suitable loop closing pair, based on the number of votes each location polls. The proposed system depends solely on the appearance information of the scenes on the trajectory, without requiring any pre-training phase. The evaluation of the method is provided via a variety of tests with several community datasets, proving its capability of achieving high recall rates for perfect precision.

One’s internal state is mainly communicated through nonverbal cues, such as facial expressions, gestures and tone of voice, which in turn shape the corresponding emotional state. Hence, emotions can be effectively used, in the long term, to form an opinion of an individual’s overall personality. The latter can be capitalized on in many human–robot interaction (HRI) scenarios, such as in the case of an assisted-living robotic platform, where a human’s mood may entail the adaptation of a robot’s actions. To that end, we introduce a novel approach that gradually maps and learns the personality of a human, by conceiving and tracking the individual’s emotional variations throughout their interaction. The proposed system extracts the facial landmarks of the subject, which are used to train a suitably designed deep recurrent neural network architecture. The above architecture is responsible for estimating the two continuous coefficients of emotion, i.e., arousal and valence, following the broadly known Russell’s model. Finally, a user-friendly dashboard is created, presenting both the momentary and the long-term fluctuations of a subject’s emotional state. Therefore, we propose a handy tool for HRI scenarios, where robot’s activity adaptation is needed for enhanced interaction performance and safety.