Unlike popular modularized framework, end-to-end autonomous driving seeks to solve the perception, decision and control problems in an integrated way, which can be more adapting to new scenarios and easier to generalize at scale. However, existing end-to-end approaches are often lack of interpretability, and can only deal with simple driving tasks like lane keeping. In this article, we propose an interpretable deep reinforcement learning method for end-to-end autonomous driving, which is able to handle complex urban scenarios. A sequential latent environment model is introduced and learned jointly with the reinforcement learning process. With this latent model, a semantic birdeye mask can be generated, which is enforced to connect with certain intermediate properties in today’s modularized framework for the purpose of explaining the behaviors of learned policy. The latent space also significantly reduces the sample complexity of reinforcement learning. Comparison tests in a realistic driving simulator show that the performance of our method in urban scenarios with crowded surrounding vehicles dominates many baselines including DQN, DDPG, TD3 and SAC. Moreover, through masked outputs, the learned model is able to provide a better explanation of how the car reasons about the driving environment.

Precise and large torque generation, back drivability, low output impedance, and compactness of hardware are important requirements for human assistive robots. In this paper, a compact rotary series elastic actuator (cRSEA) is designed considering these requirements. To magnify the torque generated by an electric motor in the limited space of the compact device, a worm gear is utilized. However, the actual torque amplification ratio provided by the worm gear is different from the nominal speed reduction ratio due to friction, which makes the controller design challenging. In this paper, the friction effect is considered in the model of cRSEA, and a robust control algorithm is designed to precisely control the torque output in the presence of nonlinearities such as the friction. The mechanical design and dynamic model of the proposed device and the design of a robust control algorithm are discussed, and actuation performance is verified by experiments. Experimental results with a human subject are also presented to show the performance of the cRSEA while interacting with humans.

In the past decades, we have witnessed significant progress in the domain of autonomous driving. Advanced techniques based on optimization and reinforcement learning become increasingly powerful when solving the forward problem: given designed reward/cost functions, how we should optimize them and obtain driving policies that interact with the environment safely and efficiently. Such progress has raised another equally important question: what should we optimize ? Instead of manually specifying the reward functions, it is desired that we can extract what human drivers try to optimize from real traffic data and assign that to autonomous vehicles to enable more naturalistic and transparent interaction between humans and intelligent agents. To address this issue, we present an efficient sampling-based maximum-entropy inverse reinforcement learning (IRL) algorithm in this letter. Different from existing IRL algorithms, by introducing an efficient continuous-domain trajectory sampler, the proposed algorithm can directly learn the reward functions in the continuous domain while considering the uncertainties in demonstrated trajectories from human drivers. We evaluate the proposed algorithm via real-world driving data, including both non-interactive and interactive scenarios. The experimental results show that the proposed algorithm achieves more accurate prediction performance with faster convergence speed and better generalization compared to other baseline IRL algorithms.

This paper proposes a novel communication-enabled distributed conflict resolution mechanism in order for a group of connected autonomous vehicles (CAVs) to navigate safely and efficiently in intersections without any traffic manager. The conflict resolution strategy for individual vehicle is decoupled temporally. In a decision maker, the vehicle computes the desired time slots to pass the conflict zones by solving a conflict graph locally based on the broadcasted information from other vehicles. In a motion planner, the vehicle computes the desired speed profile by solving a temporal optimization problem constrained in the desired time slot. The estimated time to occupy the conflict zones given the new speed profile is then broadcasted again. It is proved that the aggregation of these local decisions solves the conflicts globally. Theoretically, this method provides an efficient parallel mechanism to obtain local optimal solutions of a large-scale optimization problem (e.g. multi-vehicle navigation). Application-wise, as demonstrated by extensive simulation, this mechanism increases the efficiency of autonomous vehicles in terms of smaller delay time as well as the efficiency of the traffic in terms of larger throughput when there is no traffic manager to mediate the conflicts.

Object detection serves as one of most fundamental computer vision tasks. Existing works on object detection heavily rely on dense object candidates, such as $k$ anchor boxes pre-defined on all grids of an image feature map of size $H\times W$ . In this paper, we present Sparse R-CNN, a very simple and sparse method for object detection in images. In our method, a fixed sparse set of learned object proposals ( $ N$ in total) are provided to the object recognition head to perform classification and localization. By replacing $HWk$ (up to hundreds of thousands) hand-designed object candidates with $N$ (e.g., 100) learnable proposals, Sparse R-CNN makes all efforts related to object candidates design and one-to-many label assignment completely obsolete. More importantly, Sparse R-CNN directly outputs predictions without the non-maximum suppression (NMS) post-processing procedure. Thus, it establishes an end-to-end object detection framework. Sparse R-CNN demonstrates highly competitive accuracy, run-time and training convergence performance with the well-established detector baselines on the challenging COCO dataset and CrowdHuman dataset. We hope that our work can inspire re-thinking the convention of dense prior in object detectors and designing new high-performance detectors.