Cooperative perception, or collective perception (CP), is an emerging and promising technology for intelligent transportation systems (ITS). It enables an ITS station (ITS-S) to share its local perception information with others by means of vehicle-to-X (V2X) communication, thereby achieving improved efficiency and safety in road transportation. In this paper, we present our recent progress on the development of a connected and automated vehicle (CAV) and intelligent roadside unit (IRSU). The main contribution of the work lies in investigating and demonstrating the use of CP service within intelligent infrastructure to improve awareness of vulnerable road users (VRU) and thus safety for CAVs in various traffic scenarios. We demonstrate in experiments that a connected vehicle (CV) can “see” a pedestrian around the corners. More importantly, we demonstrate how CAVs can autonomously and safely interact with walking and running pedestrians, relying only on the CP information from the IRSU through vehicle-to-infrastructure (V2I) communication. This is one of the first demonstrations of urban vehicle automation using only CP information. We also address in the paper the handling of collective perception messages (CPMs) received from the IRSU, and passing them through a pipeline of CP information coordinate transformation with uncertainty, multiple road user tracking, and eventually path planning/decision-making within the CAV. The experimental results were obtained with manually driven CV, fully autonomous CAV, and an IRSU retrofitted with vision and laser sensors and a road user tracking system.

Understanding and predicting the intention of pedestrians is essential to enable autonomous vehicles and mobile robots to navigate crowds. This problem becomes increasingly complex when we consider the uncertainty and multimodality of pedestrian motion, as well as the implicit interactions between members of a crowd, including any response to a vehicle. Our approach, Probabilistic Crowd GAN, extends recent work in trajectory prediction, combining Recurrent Neural Networks (RNNs) with Mixture Density Networks (MDNs) to output probabilistic multimodal predictions, from which likely modal paths are found and used for adversarial training. We also propose the use of Graph Vehicle-Pedestrian Attention Network (GVAT), which models social interactions and allows input of a shared vehicle feature, showing that inclusion of this module leads to improved trajectory prediction both with and without the presence of a vehicle. Through evaluation on various datasets, we demonstrate improvements on the existing state of the art methods for trajectory prediction and illustrate how the true multimodal and uncertain nature of crowd interactions can be directly modelled.

An automated vehicle operating in an urban environment must be able to perceive and recognise objects and obstacles in a three-dimensional world for navigation and path planning. In order to plan and execute accurate and sophisticated driving maneuvers, a high-level contextual understanding of the surroundings is essential. Due to the recent progress in image processing, it is now possible to obtain high definition semantic information in 2D from monocular cameras, though cameras cannot reliably provide the high accuracy 3D information provided by lasers. The fusion of these two sensor modalities can overcome the shortcomings of each individual sensor, though there are a number of important challenges that need to be addressed in a probabilistic manner. In this paper we address the common, yet challenging, LIDAR/camera/semantic fusion problems which are seldom approached in a wholly probabilistic manner. Our approach is capable of using a multi-sensor platform to build a three-dimensional semantic voxelized map that considers the uncertainty of all of the processes involved. We present a probabilistic pipeline that incorporates uncertainty from the sensor readings (cameras, LIDAR, IMU and wheel encoders), compensation for the motion of the vehicle, and heuristic label probabilities for the semantic images depicted in Fig. 1. We also present a novel and efficient viewpoint validation algorithm to check for occlusions within the camera frame. A probabilistic projection is performed from the camera images to the LIDAR point cloud. Each labelled LIDAR scan then feeds into an octree map-building algorithm that updates the class probabilities of the map voxels every time a new observation is available. We validate our approach using a set of qualitative and quantitative experiments using the USyd Campus Dataset.1 These tests demonstrate the usefulness of a probabilistic sensor fusion approach by evaluating the performance of the perception system in a typical autonomous vehicle application.

This brief investigates the flexible leader-follower formation tracking problem for a group of nonholonomic mobile robots, while most of the formation control related work in the literature focuses on the rigid formation. The flexible formation discussed in this brief is defined in curvilinear coordinates in terms of longitudinal separations between robots along the reference trajectory and lateral deviations with respect to this trajectory. Unlike the previous studies on flexible formation control, this brief is under a more challenging assumption that the global position and orientation measurements are not available. To obtain the relative pose relationships amongst robots, a stereo camera is mounted on each follower. In consideration of the fact that visual observations are noise-corrupted and intermittently available, a particle filter-based relative pose estimation approach is employed to estimate the position and orientation of the leader in the local reference frame of the follower using the polluted and discontinuous information. Also, to form a flexible formation, the leader historical trajectory is reconstructed with respect to the current local frame attached on the follower, based on which a reference point is generated. In addition, this brief considers the situation where robots operate in unknown obstacle environments. To ensure robot safety in such environments, a multiobjective control law is proposed to balance reference tracking and collision avoidance in different situations. Simulation and real-robot experiment have been performed to demonstrate the efficacy of the proposed method.

This article addresses the dynamic formation reconstruction and trajectory replanning problem in the air patrol task using multiple fixed-wing unmanned aerial vehicle formations. Unlike most of the formation flying related work, this article considers a more practical issue that some of the vehicles may break down during operation. In this case, a more reasonable coping strategy is proposed which is to reconstruct the formation such that the task objectives can be achieved optimally. To perform the patrol task, a virtual target is introduced which moves along the patrol path with a predetermined speed. Considering the fact that not all the vehicles have access to the patrol path information, a decentralized estimator is designed for each vehicle to estimate the virtual target state respectively based on which the individual reference trajectories can be generated. As these reference trajectories do not satisfy relevant physical constraints, including system model, control input limits, no-fly zone avoidance, and intervehicle collision avoidance, a novel model predictive trajectory replanning algorithm is developed to generate feasible reference trajectories for each vehicle in real time, which is computationally attractive by incorporating a situation-dependent mechanism. Simulations have been conducted to validate the effectiveness of our proposed algorithm.

This paper analyzes the motion capability of multiple fixed-wing unmanned aerial vehicle (UAV) formations by taking speed and heading rate constraints of individual UAVs into account. Unlike most of the existing literature investigating multi-UAV formation flying problems, which mainly focus on planner/controller design in order to achieve desired configurations, this work aims at providing a quantitative description of the UAV formation maneuverability, which can serve as a metric for evaluating formation trajectory plans. Three common formations, namely perfect rigid formation, semi-perfect rigid formation and flexible formation, are considered in this paper. Motion capabilities of these formations are analyzed in terms of turning performance. In particular, the feasible set of the formation heading rate is employed as the evaluation metric, based on which the relationships among these three formations are discussed. It has been proven that perfect rigid formation suffers from a rather limited mobility, while flexible formation has the largest feasible set of the formation heading rate in some situations. An interesting finding of this paper is that flexible form