2 February 2021 in IEEE Robotics and Automation Letters
Pedestrian trajectory prediction is an essential task in robotic applications such as autonomous driving and robot navigation. State-of-the-art trajectory predictors use a conditional variational autoencoder (CVAE) with recurrent neural networks (RNNs) to encode observed trajectories and decode multi-modal future trajectories. This process can suffer from accumulated errors over long prediction horizons (≥2 seconds). This letter presents BiTraP, a goal-conditioned bi-directional multi-modal trajectory prediction method based on the CVAE. BiTraP estimates the goal (end-point) of trajectories and introduces a novel bidirectional decoder to improve longer-term trajectory prediction accuracy. Extensive experiments show that BiTraP generalizes to both first-person view (FPV) and bird's-eye view (BEV) scenarios and outperforms state-of-the-art results by ~10-50%. We also show that different choices of non-parametric versus parametric target models in the CVAE directly influence the predicted multimodal trajectory distributions. These results provide guidance on trajectory predictor design for robotic applications such as collision avoidance and navigation systems. Our code is available at: https://github.com/umautobots/bidireaction-trajectory-prediction.
10 July 2014 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
This paper introduces the beta compositional model (BCM) for hyperspectral unmixing and four algorithms for unmixing given the BCM. Hyperspectral unmixing estimates the proportion of each endmember at every pixel of a hyperspectral image. Under the BCM, each endmember is a random variable distributed according to a beta distribution. By using a beta distribution, spectral variability is accounted for during unmixing, the reflectance values of each endmember are constrained to a physically realistic range, and skew can be accounted for in the distribution. Spectral variability is incorporated to increase hyperspectral unmixing accuracy. Two BCM-based spectral unmixing approaches are presented: BCM-spectral and BCM-spatial. For each approach, two algorithms, one based on quadratic programming (QP) and one using a Metropolis-Hastings (MH) sampler, are developed. Results indicate that the proposed BCM unmixing algorithms are able to successfully perform unmixing on simulated data and real hyperspectral imagery while incorporating endmember spectral variability and spatial information.
25 January 2019 in IEEE Robotics and Automation Letters
12 December 2019 in IEEE Transactions on Geoscience and Remote Sensing
In remote sensing, each sensor can provide complementary or reinforcing information. It is valuable to fuse outputs from multiple sensors to boost overall performance. Previous supervised fusion methods often require accurate labels for each pixel in the training data. However, in many remote-sensing applications, pixel-level labels are difficult or infeasible to obtain. In addition, outputs from multiple sensors often have different resolutions or modalities. For example, rasterized hyperspectral imagery (HSI) presents data in a pixel grid while airborne light detection and ranging (LiDAR) generates dense 3-D point clouds. It is often difficult to directly fuse such multimodal, multiresolution data. To address these challenges, we present a novel multiple instance multiresolution fusion (MIMRF) framework that can fuse multiresolution and multimodal sensor outputs while learning from automatically generated, imprecisely labeled data. Experiments were conducted on the MUUFL Gulfport HSI and LiDAR data set and a remotely sensed soybean and weed data set. Results show improved, consistent performance on scene understanding and agricultural applications when compared to traditional fusion methods.
9 November 2018 in IEEE Transactions on Geoscience and Remote Sensing
In classifier (or regression) fusion, the aim is to combine the outputs of several algorithms to boost overall performance. Standard supervised fusion algorithms often require accurate and precise training labels. However, accurate labels may be difficult to obtain in many remote sensing applications. This paper proposes novel classification and regression fusion models that can be trained given ambiguously and imprecisely labeled training data in which the training labels are associated with sets of data points (i.e., ``bags'') instead of individual data points (i.e., ``instances'') following a multiple-instance learning framework. Experiments were conducted based on the proposed algorithms on both synthetic data and applications such as target detection and crop yield prediction given remote sensing data. The proposed algorithms show effective classification and regression performance.
25 December 2019 in IEEE Robotics and Automation Letters
Pedestrian detection is an important task for human-robot interaction and autonomous driving applications. Most previous pedestrian detection methods rely on data collected from three-dimensional (3D) Light Detection and Ranging (LiDAR) sensors in addition to camera imagery, which can be expensive to deploy. In this paper, we propose a novel Pedestrian Planar LiDAR Pose Network (PPLP Net) based on two-dimensional (2D) LiDAR data and monocular camera imagery, which offers a far more affordable solution to the oriented pedestrian detection problem. The proposed PPLP Net consists of three sub-networks: an orientation detection network (OrientNet), a Region Proposal Network (RPN), and a PredictorNet. The OrientNet leverages state-of-the-art neural-network-based 2D pedestrian detection algorithms, including Mask R-CNN and ResNet, to detect the Bird's Eye View (BEV) orientation of each pedestrian. The RPN transfers 2D LiDAR point clouds into occupancy grid map and uses a frustum-based matching strategy for estimating non-oriented 3D pedestrian bounding boxes. Outputs from both OrientNet and RPN are passed through the PredictorNet for a final regression. The overall outputs of our proposed network are 3D bounding box locations and orientation values for all pedestrians in the scene. We present oriented pedestrian detection results on two datasets, the CMU Panoptic Dataset and a newly collected FCAV M-Air Pedestrian (FMP) Dataset, and show that our proposed PPLP network based on 2D LiDAR and monocular camera achieves similar or better performance to previous state-of-the-art 3D-LiDAR-based pedestrian detection methods in both indoor and outdoor environments.