Visual Place Recognition (VPR) is vital for robot localization. To date, the most performant VPR approaches are environment- and task-specific: while they exhibit strong performance in structured environments (predominantly urban driving), their performance degrades severely in unstructured environments, rendering most approaches brittle to robust real-world deployment. In this work, we develop a universal solution to VPR – a technique that works across a broad range of structured and unstructured environments (urban, outdoors, indoors, aerial, underwater, and subterranean environments) without any re-training or finetuning. We demonstrate that general-purpose feature representations derived from off-the-shelf self-supervised models with no VPR-specific training are the right substrate upon which to build such a universal VPR solution. Combining these derived features with unsupervised feature aggregation enables our suite of methods, AnyLoc , to achieve up to $4\times$ significantly higher performance than existing approaches. We further obtain a 6% improvement in performance by characterizing the semantic properties of these features, uncovering unique domains which encapsulate datasets from similar environments. Our detailed experiments and analysis lay a foundation for building VPR solutions that may be deployed anywhere , anytime , and across anyview .
14 February 2024 in IEEE Robotics and Automation Letters
Tracking and following objects of interest is critical to several robotics use cases, ranging from industrial automation to logistics and warehousing, to healthcare and security. In this letter, we present a robotic system to detect, track, and follow any object in real-time. Our approach, dubbed follow anything ( FAn ), is an open-vocabulary and multimodal model – it is not restricted to concepts seen at training time and can be applied to novel classes at inference time using text, images, or click queries. Leveraging rich visual descriptors from large-scale pre-trained models ( foundation models ), FAn can detect and segment objects by matching multimodal queries (text, images, clicks) against an input image sequence. These detected and segmented objects are tracked across image frames, all while accounting for occlusion and object re-emergence. We demonstrate FAn on a real-world robotic system (a micro aerial vehicle), and report its ability to seamlessly follow the objects of interest in a real-time control loop. FAn can be deployed on a laptop with a lightweight (6–8 GB) graphics card, achieving a throughput of 6–20 frames per second. To enable rapid adoption, deployment, and extensibility, we open-source our code on our project webpage. We also encourage the reader to watch our 5-minute explainer video.
1 August 2019 in IEEE Robotics and Automation Letters
Active localization consists of generating robot actions that allow it to maximally disambiguate its pose within a reference map. Traditional approaches use an information-theoretic criterion for action selection and hand-crafted perceptual models. In this work we propose an end-to-end differentiable method for learning to take informative actions that is trainable entirely in simulation and then transferable to real robot hardware with zero refinement. The system is composed of two learned modules: a convolutional neural network for perception, and a deep reinforcement learned planning module. We leverage a multi-scale approach in the perceptual model since the accuracy needed to take actions using reinforcement learning is much less than the accuracy needed for robot control. We demonstrate that the resulting system outperforms traditional approach for either perception or planning. We also demonstrate our approach's robustness to different map configurations and other nuisance parameters through the use of domain randomization in training. The code has been released: https://github.com/montrealrobotics/dal and is compatible with the OpenAI gym framework, as well as the Gazebo simulator.
19 December 2019 in IEEE Robotics and Automation Letters
In this work, we present MapLite: a one-click autonomous navigation system capable of piloting a vehicle to an arbitrary desired destination point given only a sparse publicly available topometric map (from OpenStreetMap). The onboard sensors are used to segment the road region and register the topometric map in order to fuse the high-level navigation goals with a variational path planner in the vehicle frame. This enables the system to plan trajectories that correctly navigate road intersections without the use of an external localization system such as GPS or a detailed prior map. Since the topometric maps already exist for the vast majority of roads, this solution greatly increases the geographical scope for autonomous mobility solutions. We implement MapLite on a full-scale autonomous vehicle and exhaustively test it on over 15km of road including over 100 autonomous intersection traversals. We further extend these results through simulated testing to validate the system on complex road junction topologies such as traffic circles.
This article presents an algorithm for online multirobot coverage that proceeds with minimal knowledge of the already explored region and the frontier cells. It creates clusters of frontier cells which are designated to robots using an optimal assignment scheme. Coverage is then performed using a novel path planning technique. Many approaches that use clustering for multi-robot coverage do not specify strict time criteria for re-clustering. Moreover, the motion plans they use result in redundant coverage. To overcome these limitations, an appropriate motion plan for the robots is chosen based on the context of already covered frontiers. Dispersion of robots is vital for efficient coverage and is an emergent behavior in our approach. The efficacy of the proposed approach is tested in simulation and on a multi-robot test-bed. The algorithm performs better than some state of the art approaches.
Strength training exercises are an important part of fitness training. Although several fitness devices exist to track and analyze cardio-training activities like walking and running, devices to track repetitive movements during strength training exercises are few and they have technical limitations. We introduce Maxxyt - an autonomous wearable device that can count repetitive movements during strength training in real-time. Maxxyt functions without any prior exercise-specific training and can self-adapt to any generic exercise involving repetitive movements. It has been designed with the motive of optimizing the experience of a typical user training in a gym. It uses an Arduino-compatible microcontroller, with primary memory of 4 KB. Maxxyt provides hap tic and visual feedback to the user within 2 seconds from starting an exercise. The proposed method to count repetitions involves three phases: i) pre-processing, ii) peak-detection using adaptive techniques, and iii) aggregation of results from various axes. We present results on test data collected in a gym consisting of 1171 repetitions from 100 sets of 10 distinct exercises. The repetition count accuracy of the device is 2 repetitions 98% of the time. These results suggest a new paradigm of responsive, wearable strength training fitness devices.