Depth completion involves estimating a dense depth image from sparse depth measurements, often guided by a color image. While linear upsampling is straight forward, it results in depth pixels being interpolated in empty space across discontinuities between objects. Current methods use deep networks to maintain gaps between objects. Nevertheless depth smearing remains a challenge. We propose a new representation for depth called Depth Coefficients (DC) to address this problem. It enables convolutions to more easily avoid inter-object depth mixing. We also show that the standard Mean Squared Error (MSE) loss function can promote depth mixing, and so we propose instead to use cross-entropy loss for DC. Both quantitative and qualitative evaluation are conducted on benchmarks, and we show that switching out sparse depth input and MSE loss functions with our DC representation and loss is a simple way to improve performance, reduce pixel depth mixing and can improve object detection.

We analyze the use of kinematic constraints for articulated object tracking. Conditions for the occurrence of singularities in 3-D models are presented and their effects on tracking are characterized. We describe a novel 2-D Scaled Prismatic Model (SPM) for figure registration. In contrast to 3-D kinematic models. the SPM has fewer singularity problems and does not require detailed knowledge of the 3-D kinematics. We fully characterize the singularities in the SPM and illustrate tracking through singularities using synthetic and real examples with 3-D and 2-D models. Our results demonstrate the significant benefits of the SPM in tracking with a single source of video.

For an autonomous vehicle, detecting and tracking other vehicles is a critical task. Determining the orientation of a detected vehicle is necessary for assessing whether the vehicle is a potential hazard. If a detected vehicle is moving, the orientation can be inferred from its trajectory, but if the vehicle is stationary, the orientation must be determined directly. In this paper, we focus on vision-based algorithms for determining vehicle orientation of vehicles in images. We train a set of Histogram of Oriented Gradients (HOG) classifiers to recognize different orientations of vehicles detected in imagery. We find that these orientation-specific classifiers perform well, achieving a 88% classification accuracy on a test database of 284 images. We also investigate how combinations of orientation-specific classifiers can be employed to distinguish subsets of orientations, such as driver's side versus passenger's side views. Finally, we compare a vehicle detector formed from orientation-specific classifiers to an orientation-independent classifier and find that, counter-intuitively, the orientation-independent classifier outperforms the set of orientation-specific classifiers.

As a direct depth sensor, radar holds promise as a tool to improve monocular 3D object detection, which suffers from depth errors, due in part to the depth-scale ambiguity. On the other hand, leveraging radar depths is hampered by difficulties in precisely associating radar returns with 3D estimates from monocular methods, effectively erasing its benefits. This paper proposes a fusion network that addresses this radar-camera association challenge. We train our network to predict the 3D offsets between radar returns and object centers, enabling radar depths to enhance the accuracy of 3D monocular detection. By using parallel radar and camera backbones, our network fuses information at both the feature level and detection level, while at the same time leveraging a state-of-the-art monocular detection technique without retraining it. Experimental results show significant improvement in mean average precision and translation error on the nuScenes dataset over monocular counterparts. Our source code is available at https://github.com/longyunf/radiant.