The direct use of the entire photometric image information as dense feature for visual servoing brings several advantages. First, it does not require any feature detection, matching, or tracking process. Thanks to the redundancy of visual information, the precision at convergence is really accurate. However, the corresponding highly nonlinear cost function reduces the convergence domain. In this paper, we propose a visual servoing based on the analytical formulation of Gaussian mixtures to enlarge the convergence domain. Pixels are represented by two-dimensional Gaussian functions that denotes a “power of attraction.” In addition to the control of the camera velocities during the servoing, we also optimize the Gaussian spreads allowing the camera to precisely converge to a desired pose even from a far initial one. Simulations show that our approach outperforms the state of the art and real experiments show the effectiveness, robustness, and accuracy of our approach.
8 March 2024 in IEEE Robotics and Automation Letters
It is well established that wide field of view cameras, as well as using the whole photometric information contained in images, offer many advantages for visual servoing. Therefore, we propose to extend the photometric visual servoing to full spherical cameras. More precisely, we are dealing with 360-degree optical rigs composed of two wide-angle lenses oriented in opposite directions that capture everything around the device in one acquisition. The photometric visual feature coupled to dual-hemispherical acquisitions that contain the whole surrounding scene provide useful complementary information, showing large convergence domains, straighter camera trajectories than with a single hemispherical camera, and high accuracy. We report thorough simulations and several challenging real experiments using a 6 degrees-of-freedom robotic arm controlled from dual-hemispherical acquisitions.
12 March 2025 in IEEE Robotics and Automation Letters
Integrating robots into real-world applications requires effective consideration of various agents, including other robots. Multi-agent reinforcement learning (MARL) is an established field that addresses multi-agent systems problems by leveraging reinforcement learning techniques. Despite its potential, the study of multi-robot systems, particularly in vision-based robotics, remains in its early stages. In this context, this article tackles the PointGoal navigation problem for multi-robot systems, extending the traditional single agent focus to a multi-agent context. To this end, we introduce a training environment designed to address vision-based multi-robot challenges. In addition, we propose a method based on the centralized training-decentralized execution paradigm within MARL to explore three PointGoal navigation scenarios: the SpecificGoal scenario, where each agent has a distinct target; the CommonGoal scenario, where all agents share the same target; and the Ad-hoCoop scenario, which requires agents to adapt to varying team sizes. Our results contribute to lay the groundwork for adopting MARL approaches to address vision-based tasks for multi-robot systems. The code for the approach and the training environment are available at: https://github.com/TakieddineSOUALHI/MARLPointNAV
9 July 2018 in IEEE Robotics and Automation Letters
A light-field camera can obtain richer information of a scene than a usual camera. This property offers a lot of potential for robot vision. In this paper, we present a method for pose estimation of planar object with a light-field camera. The light-field camera can be regarded as a set of sub-aperture cameras. Although any combination of them can theoretically be used for the pose estimation, the accuracy depends on the combination. We show that the estimated pose error can be reduced by selecting the best pair of sub-aperture cameras. We have evaluated the accuracy of our approach by real experiments using a light-field camera in front of planar targets held by an industrial manipulator for ground truth comparison.