Foundation models are used to train a broad system of general data to build adaptations to new bottlenecks. Typically, they contain hundreds of billions of hyperparameters that have been trained with hundreds of gigabytes of data. However, this type of black-box vulnerability places foundation models at risk of data poisoning attacks that are designed to pass on misinformation or purposely introduce machine bias. Moreover, ordinary researchers have not been able to completely participate due to the rise in deployment standards. This study introduces the theoretical framework of scenarios engineering (SE) for building accessible and reliable foundation models in metaverse, namely, “SE-enabled foundation models in metaverse.” Particularly, the research framework comprises a six-layer architecture (infrastructure layer, operation layer, knowledge layer, intelligence layer, management layer, and interaction layer), which can provide controllability, trustworthiness, and interactivity for the foundation models in metaverse. This creates closed-loop, virtual–real, and human–machine environments that provides the best indices and goals for the foundation models, which allows us to fully validate and calibrate the corresponding models. Then, examples of use cases from the automotive industry are listed to provide transparency on the possible use and benefits of our approach. Finally, the open research topics of related frameworks are discussed.

Long-tail effect, characterized by highly frequent occurrence of normal scenarios and the scarce appearance of extreme “long-tail” scenarios, ubiquitously exists in the vision-related problems in the real-world applications. Though many computer vision methods to date have already achieved feasible performance for most of the normal scenarios, it is still challenging for existing vision systems to accurately perceive the long-tail scenarios. This deficiency largely hinders the practical application of computer vision systems, since long-tail problems may incur fatal consequences, such as traffic accidents, taking the vision systems of autonomous vehicles as an example. In this paper, we firstly propose a theoretical framework named Long-tail Regularization (LoTR), for analyzing and tackling the long-tail problems in the vision perception of autonomous driving. LoTR is able to regularize the scarcely occurred long-tail scenarios to be frequently encountered. Then we present a Parallel Vision Actualization System (PVAS), which consists of closed-loop optimization and virtual-real interaction, to search for challenging long-tail scenarios and produce large-scale long-tail driving scenarios for autonomous vehicles. In addition, we introduce how to perform PVAS in Intelligent Vehicle Future Challenge of China (IVFC), the most durable autonomous driving competition around the world. Results over the past decade demonstrate that PVAS can effectively guide the collection of long-tail data to diminish the cost in the real world, and thus promote the capability of vision systems to adapt to complex environments, alleviating the impact of long-tail effect.

Diverse transport demands have resulted in the wide existence of heterogeneous vehicle automation systems. While these systems have demonstrated effectiveness, they also pose challenges in terms of the share of technological advancements among different organizations and lead to poor generalization ability of individual systems. This paper proposes a Transformer-based unified framework, VistaGPT, to address these challenges. VistaGPT, composed of Modular Federations of Vehicular Transformers (M-FoV) and Automated Composing of Autonomous Driving Systems (AutoAuto), aims to overcome the information barriers due to system-level and module-level heterogeneity. M-FoV collects and organizes Transformer-based models in a modular fashion to facilitate system integration by providing diversity and versatility. AutoAuto utilizes large language models (LLMs) to automatically compose end-to-end autonomous driving systems with a “Dividing and Recombining” strategy. Besides, we deploy Scenario Engineering systems to evaluate the composed systems and provide systematic feedback for the optimization of AutoAuto, and Federated intelligence to contribute to diverse training samples and applications. With its capacity, scalability, and diversity, VistaGPT provides a new paradigm of LLM-aided system development for transport automation, which promotes virtual-real interactive parallel driving and advances progress toward “6S” objectives.

As a foundation of large language models, fine-tuning drives rapid progress, broad applicability, and profound impacts on human–AI collaboration, surpassing earlier technological advancements. This paper provides a comprehensive overview of large language model (LLM) fine-tuning by integrating hermeneutic theories of human comprehension, with a focus on the essential cognitive conditions that underpin this process. Drawing on Gadamer’s concepts of Vorverständnis, Distanciation, and the Hermeneutic Circle, the paper explores how LLM fine-tuning evolves from initial learning to deeper comprehension, ultimately advancing toward self-awareness. It examines the core principles, development, and applications of fine-tuning techniques, emphasizing its growing significance across diverse field and industries. The paper introduces a new term, “Tutorial Fine-Tuning (TFT)”, which annotates a process of intensive tuition given by a “tutor” to a small number of “students”, to define the latest round of LLM fine-tuning advancements. By addressing key challenges associated with fine-tuning, including ensuring adaptability, precision, credibility and reliability, this paper explores potential future directions for the co-evolution of humans and AI. By bridging theoretical perspectives with practical implications, this work provides valuable insights into the ongoing development of LLMs, emphasizing their potential to achieve higher levels of cognitive and operational intelligence.

Accurate and timely traffic flow prediction is crucial for intelligent transportation systems (ITS). Recent advances in graph-based neural networks have achieved promising prediction results. However, some challenges remain, especially regarding graph construction and the time complexity of models. In this paper, we propose a multi-stream feature fusion approach to extract and integrate rich features from traffic data and leverage a data-driven adjacent matrix instead of the distance-based matrix to construct graphs. We calculate the Spearman rank correlation coefficient between monitor stations to obtain the initial adjacent matrix and fine-tune it while training. As to the model, we construct a multi-stream feature fusion block (MFFB) module, which includes a three-channel network and the soft-attention mechanism. The three-channel networks are graph convolutional neural network (GCN), gated recurrent unit (GRU) and fully connected neural network (FNN), which are used to extract spatial, temporal and other features, respectively. The soft-attention mechanism is utilized to integrate the obtained features. The MFFB modules are stacked, and a fully connected layer and a convolutional layer are used to make predictions. We conduct experiments on two real-world traffic prediction tasks and verify that our proposed approach outperforms the state-of-the-art methods within an acceptable time complexity.