Human–Robot interaction represents a cornerstone of mobile robotics, especially within the field of social robots. In this context, user localization becomes of crucial importance for the interaction. This work investigates the capabilities of wide field-of-view RGB cameras to estimate the 3D position and orientation (i.e., the pose) of a user in the environment. For that, we employ a social robot endowed with a fish-eye camera hosted in a tilting head and develop two complementary approaches: (1) a fast method relying on a single image that estimates the user pose from the detection of their feet and does not require either the robot or the user to remain static during the reconstruction; and (2) a method that takes some views of the scene while the camera is being tilted and does not need the feet to be visible. Due to the particular setup of the tilting camera, special equations for 3D reconstruction have been developed. In both approaches, a CNN-based skeleton detector (OpenPose) is employed to identify humans within the image. A set of experiments with real data validate our two proposed methods, yielding similar results than commercial RGB-D cameras while surpassing them in terms of coverage of the scene (wider FoV and longer range) and robustness to light conditions.
This paper tackles the resolution of the Relative Pose problem with optimality guarantees by stating it as an optimization problem over the set of essential matrices that minimizes the squared epipolar error. We relax this non-convex problem with its Shor’s relaxation, a convex program that can be solved by off-the-shelf tools. We follow the empirical observation that redundant but independent constraints tighten the relaxation. For that, we leverage equivalent definitions of the set of essential matrices based on the translation vectors between the cameras. Overconstrained characterizations of the set of essential matrices are derived by the combination of these definitions. Through extensive experiments on synthetic and real data, our proposal is empirically proved to remain tight and to require only 7 milliseconds to be solved even for the overconstrained formulations, finding the optimal solution under a wide variety of configurations, including highly noisy data and outliers. The solver cannot certify the solution only in very extreme cases, e.g.noise $$100~{\texttt {pix}} $$ and number of pair-wise correspondences under 15. The proposal is thus faster than other overconstrained formulations while being faster than the minimal ones, making it suitable for real-world applications that require optimality certification.
The Relative Pose problem (RPp) seeks for the relative rotation and translation between two central, calibrated cameras given a set of pairwise feature correspondences. The RPp is a fundamental block for many 3D computer vision tasks, and hence the quality of the estimated relative pose is of key importance for the correct performance of these applications. Nonetheless, the RPp is a nonconvex problem that presents multiple local minima. Recent nonminimal solvers provide relatively fast certifiable solutions, usually relying on a convex relaxation of the problem; however, there is no guarantee a priori that these relaxations return the optimal solution, i.e., are tight. This work presents a sufficient condition to guarantee that a given solution of the RPp is the global optimum in a faster way than evaluating a certifiable algorithm (up to four times faster). We state the RPp as an optimization problem that minimizes the squared normalized epipolar error over the set of normalized essential matrices. The proposed condition is derived through spectral analysis and builds up on the recently proposed certifiable algorithm in [M. Garcia-Salguero, J. Briales, and J. Gonzalez-Jimenez, Image Vis. Comput., 109 (2021), 104142]. The results of extensive experiments, with both synthetic and real data, support that by using the proposed conditions we can detect a large number of optimal solutions for most common problem instances.