Many of the robust visual tracking techniques utilized by augmented reality applications rely on 3D models and information extracted from images. Models enhanced with image information make it possible to initialize tracking and detect poor registration. Unfortunately, generating 3D CAD models and registering them to image information can be a time consuming operation. Regularly the process requires multiple trips between the site being modeled and the workstation used to create the model. The system presented in this work eliminates the need for a separately generated 3D model by utilizing modern structure-from-motion techniques to extract the model and associated image information directly from an image sequence. The technique can be implemented on any handheld device instrumented with a camera and network connection. The process of creating the model requires minimal user interaction in the form of a few cues to identify planar regions on the object of interest. In addition the system selects a set of keyframes for each region to capture viewpoint based appearance changes. This work also presents a robust tracking framework to take advantage of these new edge models. Performance of both the modeling technique and the tracking system are verified on several different objects.

Research in the field of autonomous Unmanned Aerial Vehicles (UAVs) has significantly advanced in recent years, mainly due to their relevance in a large variety of commercial, industrial, and military applications. However, UAV navigation in GPS-denied environments continues to be a challenging problem that has been tackled in recent research through sensor-based approaches. This paper presents a novel offline, portable, real-time in-door UAV localization technique that relies on macro-feature detection and matching. The proposed system leverages the support of machine learning, traditional computer vision techniques, and pre-existing knowledge of the environment. The main contribution of this work is the real-time creation of a macro-feature description vector from the UAV captured images which are simultaneously matched with an offline pre-existing vector from a Computer-Aided Design (CAD) model. This results in a quick UAV localization within the CAD model. The effectiveness and accuracy of the proposed system were evaluated through simulations and experimental prototype implementation. Final results reveal the algorithm’s low computational burden as well as its ease of deployment in GPS-denied environments.

This paper presents a novel pipeline for development of an efficient set of tools for extracting information from the video of a structure, captured by an Unmanned Aircraft System (UAS) to produce as-built documentation to aid inspection of large multi-storied building during construction. Our system uses the output from a Simultaneous Localization and Mapping system and a 3D CAD model of the structure in order to construct a spatial database to store images into the 3D CAD model space. This allows the user to perform a spatial query for images through spatial indexing into the 3D CAD model space. The image returned by the spatial query is used to extract metric information. The spatial database is also used to generate a 3D textured model which provides a visual as-built documentation.

This paper presents a visual tracking system that is capable or running real time on-board a small UAV (Unmanned Aerial Vehicle). The tracking system is computationally efficient and invariant to lighting changes and rotation of the object or the camera. Detection and tracking is autonomously carried out on the payload computer and there are two different methods for creation of the image patches. The first method starts detecting and tracking using a stored image patch created prior to flight with previous flight data. The second method allows the operator on the ground to select the interest object for the UAV to track. The tracking system is capable of re-detecting the object of interest in the events of tracking failure. Performance of the tracking system was verified both in the lab and during actual flights of the UAV. Results show that the system can run on-board and track a diverse set of objects in real time.