Real-time simultaneous localization and dense mapping is very helpful for providing virtual reality and augmented reality for surgeons or even surgical robots. In this letter, we propose MIS-SLAM: A complete real-time large-scale dense deformable SLAM system with stereoscope in minimal invasive surgery (MIS) based on heterogeneous computing by making full use of CPU and GPU. Idled CPU is used to perform ORB-SLAM for providing robust global pose. Strategies are taken to integrate modules from CPU and GPU. We solved the key problem raised in the previous work, that is, fast movement of scope and blurry images make the scope tracking fail. Benefiting from improved localization, MIS-SLAM can achieve large-scale scope localizing and dense mapping in real time. It transforms and deforms current model and incrementally fuses new observation while keeping vivid texture. In-vivo experiments conducted on publicly available datasets presented in the form of videos demonstrate the feasibility and practicality of MIS-SLAM for potential clinical purpose.

Airborne light detection and ranging (LiDAR) data are widely applied in building reconstruction, with studies reporting success in typical buildings. However, the reconstruction of curved buildings remains an open research problem. To this end, we propose a new framework for curved building reconstruction via assembling and deforming geometric primitives. The input LiDAR point clouds are first converted into contours where individual buildings are identified. After recognizing geometric units (primitives) from building contours, we get initial models by matching the basic geometric primitives to these primitives. To polish assembly models, we employ a warping field for model refinements. Specifically, an embedded deformation (ED) graph is constructed via downsampling the initial model. Then, the point to model displacements is minimized by adjusting node parameters in the ED graph based on our objective function. The presented framework is validated on several highly curved buildings collected by various LiDAR in different cities. The experimental results, as well as accuracy comparison, demonstrate the advantage and effectiveness of our method. The new insight attributes to an efficient reconstruction manner. Moreover, we prove that the primitive-based framework significantly reduces the data storage to 10%-20% of classical mesh models.

This letter proposes a new image-based localization framework that explicitly localizes the camera/robot by fusing Convolutional Neural Network (CNN) and sequential images’ geometric constraints. The camera is localized using a single or few observed images and training images with 6-degree-of-freedom pose labels. A Siamese network structure is adopted to train an image descriptor network, and the visually similar candidate image in the training set is retrieved to localize the testing image geometrically. Meanwhile, a probabilistic motion model predicts the pose based on a constant velocity assumption. The two estimated poses are finally fused using their uncertainties to yield an accurate pose prediction. This method leverages the geometric uncertainty and is applicable in indoor scenarios predominated by diffuse illumination. Experiments on simulation and real data sets demonstrate the efficiency of our proposed method. The results further show that combining the CNN-based framework with geometric constraint achieves better accuracy when compared with CNN-only methods, especially when the training data size is small.

In stereoscope-based minimally invasive surgeries (MISs), dense stereo matching plays an indispensable role in 3-D shape recovery, AR, VR, and navigation tasks. Although numerous deep neural network (DNN) approaches are proposed, the conventional prior-free approaches are still popular in the industry because of the lack of open-source annotated dataset and the limitation of the task-specific pretrained DNNs. Among the prior-free stereo matching algorithms, there is no successful real-time algorithm in none GPU environment for MIS. This article proposes the first CPU-level real-time prior-free stereo matching algorithm for general MIS tasks. We achieve an average $14-17$ Hz on $640 \times 480$ images with a single-core CPU (i5-9400) for surgical images. Meanwhile, it achieves slightly better accuracy than the popular efficient large-scale stereo matching (ELAS) method. The patch-based fast disparity searching algorithm is adopted for the rectified stereo images. A coarse-to-fine Bayesian probability and a spatial Gaussian mixed model were proposed to evaluate the patch probability at different scales. An optional probability density function estimation algorithm was adopted to quantify the prediction variance. Extensive experiments demonstrated the proposed method's capability to handle ambiguities introduced by the textureless surfaces and the photometric inconsistency from the non-Lambertian reflectance and dark illumination. The estimated probability managed to balance the confidences of the patches for stereo images at different scales. It has similar or higher accuracy and fewer outliers than the baseline ELAS in MIS, while it is 4–5 times faster.

The accelerating impact of climate change on giant panda (Ailuropoda melanoleuca) habitats have become an international research topic. Recently, many studies have also focused on medium-sized mountain ranges or entire giant panda habitats to predict how habitats will change as the climate warms, but few say in detail what to do or where to focus efforts. To fill this gap, this paper presents a new method to take comprehensive, fine-scale evaluations incorporating climate change, human disturbance, and current conservation networks and translate them into practical countermeasures in order to help decision-makers set priority regions for conservation. This study looked at the core area of the Sichuan Giant Panda Sanctuaries United Nations Educational, Scientific and Cultural Organisation (UNESCO) World Natural Heritage site, namely Ya’an Prefecture, as a case study. The research employs the Maximum Entropy (MaxEnt) modeling algorithm to analyze how climate change will affect the habitats by 2050 under two scenarios: only considering the influence of climate change, and thinking about the coupled influence of climate change and human disturbance together. The results showed the following: (1) only considering climate change, the overall habitat that can be used by giant pandas in this region will increase, which differs from most of the previous results showing a decrease; (2) the new suitable habitat will shift westward, northward and eastward in this region; (3) conversely, the suitable habitat will be significantly reduced (about 58.56%) and fragmentized when taking into account human disturbance factors; (4) at present, the three small nature reserves are far from each other and cannot cover the present habitat well nor protect the potentially suitable habitats. Based on the comprehensive analysis of habitat shifts and our two field investigations, we suggest two regions that can be expanded into the conservation network to contain more potentially suitable habitats in the future. Furthermore, we used a geographical information system to incorporate high-resolution remote-sensing images from the GF-1 satellite, land-cover maps, and a digital elevation model (DEM) to verify the possibility of our two suggested regions.