Collaborative perception is essential to address occlusion and sensor failure issues in autonomous driving. In recent years, theoretical and experimental investigations of novel works for collaborative perception have increased tremendously. So far, however, few reviews have focused on systematical collaboration modules and large-scale collaborative perception datasets. This article reviews recent achievements in this field to bridge this gap and motivate future research. We start with a brief overview of collaboration schemes. After that, we systematically summarize the collaborative perception methods for ideal scenarios and real-world issues. The former focuses on collaboration modules and efficiency, and the latter is devoted to addressing the problems in actual application. Furthermore, we present large-scale public datasets and summarize quantitative results on these benchmarks. Finally, we highlight gaps and overlooked challenges between current academic research and real-world applications.

This letter reports on a TIV DHW(decentralized and hybrid workshop) that explores the prospective influence of ChatGPT on research and development in intelligent vehicles. To assess the update capabilities of ChatGPT, we conducted tests involving both basic and technically relevant questions. Our preliminary testing revealed that ChatGPT's information can be updated and corrected at one time, but it may take some time for the changes to be reflected in ChatGPT's responses, so it may not always possess the latest knowledge regarding specific topics. We further discuss the prospective influence of ChatGPT on the field of intelligent vehicles, particularly possible applications of ChatGPT in areas like autonomous driving, human-vehicle interaction, and intelligent transportation systems, highlighting challenges and opportunities associated with these applications. Additionally, we address technical questions, such as the feasibility of training intelligent vehicles using the same methods as ChatGPT and the reflection of the intelligence of intelligent vehicles in the context of human-machine shared control. In conclusion, this letter presents a preliminary exploration of the potential of ChatGPT for intelligent vehicle research, from an IEEE TIV perspective, acknowledging the limitations and uncertainties of this emerging technology.

Autonomous driving vehicle (ADV) that is ready to transform our society and economy, is in desperate need of precise positioning over itself as well as surrounding environments. However, it is still a challenging issue for ADV to retrieve real-time positioning knowledge over road participants and dynamic surrounding environments, due to unsatisfied perception accuracy caused by sparse observations and limited perception range. Cooperative perception, which advocates cooperatively disseminating perception data among vehicles, has the potential to overcome the above limitations. To this end, this article proposes a novel edge-assisted multi-vehicle perception system to enhance vehicles’ awareness over surrounding environments, which is termed as EdgeCooper. EdgeCooper first schedules vehicles to share complementarity-enhanced and redundancy-minimized raw sensor data with an edge server, using multi-hop cooperative 5G V2X communications. Then, EdgeCooper merges vehicles’ individual views to form a holistic view with a higher resolution, thus enhancing perception robustness and enlarging perception range. We formulate multi-vehicle multi-hop cooperative data sharing as a minimum cost flow problem with conflict, and further prove that there exists no polynomial-time approximation algorithm with a constant performance ratio unless P = NP. Furthermore, a two-dimension graph coloring algorithm with guaranteed performance is proposed to eliminate conflict. We evaluate EdgeCooper by building a comprehensive simulation platform through a joint manipulation of SUMO, CARLA, NS3, and PyTorch. The experiment results show that, compared to a single vehicle’s perception, EdgeCooper performs effective and efficient in enhancing vehicular awareness, e.g., extending up to 3.6 times detection range and improving perception accuracy by 20%.

Diverse transport demands have resulted in the wide existence of heterogeneous vehicle automation systems. While these systems have demonstrated effectiveness, they also pose challenges in terms of the share of technological advancements among different organizations and lead to poor generalization ability of individual systems. This paper proposes a Transformer-based unified framework, VistaGPT, to address these challenges. VistaGPT, composed of Modular Federations of Vehicular Transformers (M-FoV) and Automated Composing of Autonomous Driving Systems (AutoAuto), aims to overcome the information barriers due to system-level and module-level heterogeneity. M-FoV collects and organizes Transformer-based models in a modular fashion to facilitate system integration by providing diversity and versatility. AutoAuto utilizes large language models (LLMs) to automatically compose end-to-end autonomous driving systems with a “Dividing and Recombining” strategy. Besides, we deploy Scenario Engineering systems to evaluate the composed systems and provide systematic feedback for the optimization of AutoAuto, and Federated intelligence to contribute to diverse training samples and applications. With its capacity, scalability, and diversity, VistaGPT provides a new paradigm of LLM-aided system development for transport automation, which promotes virtual-real interactive parallel driving and advances progress toward “6S” objectives.

Metaverse and intelligent transportation system (ITS) are disruptive technologies that have the potential to transform the current transportation system by decreasing traffic accidents and improving driving safety. The integration of Metaverse and transportation technology, called metaverse transportation system (MTS), can greatly improve the intelligence of real transportation system. The digital models built in MTS help to simulate the full life cycle of physical entities, which equip the virtual space with controllability and flexibility. In this article, we concentrate on the field of environment perception, which is the basic function of intelligent vehicles in MTS. To overcome the poor scalability of traditional environment perception methods, we develop the framework of parallel vision for ITS in metaverse (PVITS), consisting of construction of virtual transportation space, model learning based on computational experiments, and feedback optimization based on parallel execution. This article highlights opportunities brought by PVITS in terms of model precision and generalization improvement. Then, the challenges of PVITS are discussed, i.e., distribution difference between virtual and real transportation space, structure design and theoretical interpretation of vision models, and data security and privacy in virtual transportation space. After that, we present several solutions to tackle the application challenges and fully exploit the superior characteristics of PVITS while attenuating their negative side effects. Some potential applications are also given to represent the effectiveness and reliability of PVITS.

Recent years have witnessed the growing popularity of connectionist temporal classification (CTC) and attention mechanism in scene text recognition (STR). CTC-based methods consume less time with few computational burdens, while they are not as effective as attention-based methods. To retain computational efficiency and effectiveness, we propose the global–local attention-augmented light Transformer (GLaLT), which adopts a Transformer-based encoder–decoder structure to orchestrate CTC and attention mechanism. The encoder integrates the self-attention module with the convolution module to augment the attention, where the self-attention module pays more attention to capturing long-term global dependencies and the convolution module focuses on local context modeling. The decoder consists of two parallel modules: one is the Transformer-decoder-based attention module and the other is the CTC module. The first one is removed in the testing phase and can guide the second one to extract robust features in the training phase. Extensive experiments on standard benchmarks demonstrate that GLaLT achieves state-of-the-art performance for both regular and irregular STR. In terms of tradeoffs, the proposed GLaLT is at or near the frontiers for maximizing speed, accuracy, and computational efficiency at the same time.