Dataset plays an essential role in the training and testing of traffic vision algorithms. However, the collection and annotation of images from the real world is time-consuming, labor-intensive, and error-prone. Therefore, more and more researchers have begun to explore the virtual dataset, to overcome the disadvantages of real datasets. In this paper, we propose a systematic method to construct large-scale artificial scenes and collect a new virtual dataset (named ``ParallelEye'') for the traffic vision research. The Unity3D rendering software is used to simulate environmental changes in the artificial scenes and generate ground-truth labels automatically, including semantic/instance segmentation, object bounding boxes, and so on. In addition, we utilize ParallelEye in combination with real datasets to conduct experiments. The experimental results show the inclusion of virtual data helps to enhance the per-class accuracy in object detection and semantic segmentation. Meanwhile, it is also illustrated that the virtual data with controllable imaging conditions can be used to design evaluation experiments flexibly.

Object detection plays an important role in intelligent transportation systems and intelligent vehicles. Although the topic of object detection has been studied for decades, it is still challenging to accurately detect objects under complex scenarios. The contributing factors for challenges include diversified object and background appearance, motion blur, adverse weather conditions, and complex interactions among objects. In this paper, we propose a new convolutional neural network (CNN) model for traffic object detection, by using multi-scale local and global feature representation (MFR). The proposed model consists of two components: a region proposal network that generates candidate object regions and an object detection network that incorporates multi-scale features and global information, namely MFR-CNN. These two components are jointly optimized. Once the system is trained, it can detect real-world traffic objects accurately, especially small objects and heavily occluded objects. We evaluate the proposed method on four benchmark datasets, achieving consistent improvements over the state of the art.

Virtual simulation testing is becoming indispensable for the intelligence testing of intelligent vehicles. However, even the most advanced simulation software provides rather limited test conditions. In the long run, intelligent vehicles are expected to work at SAE (Society of Automotive Engineers) level 4 or level 5. Researchers should make full use of virtual simulation scenarios to test the visual intelligence algorithms of intelligent vehicles under various imaging conditions. In this paper, we create realistic artificial scenes to simulate the self-driving scenarios, and collect a dataset of synthetic images from the virtual driving scenes, named “ParallelEye-CS”. In the artificial scenes, we can flexibly change environmental conditions and automatically acquire accurate and diverse ground-truth labels. As a result, ParallelEye-CS has six ground-truth labels and includes twenty types of tests, which are divided into normal, environmental, and difficult tasks. Furthermore, we utilize ParallelEye-CS in combination with other publicly available datasets to conduct experiments for visual object detection. The experimental results indicate that: 1) object detection algorithms of intelligent vehicles can be tested under various scenario challenges; 2) mixed dataset can improve the accuracy of object detection algorithms, but domain shift is a serious issue worthy of attention.

Covalent organic frameworks (COFs) have the advantages of high thermal stability and large specific surface and have great application prospects in the fields of gas storage and catalysis. This article mainly focuses on COFs’ working capacity of methane (CH₄). Due to the vast number of possible COF structures, it is time-consuming to use traditional calculation methods to find suitable materials, so it is important to apply appropriate machine learning (ML) algorithms to build accurate prediction models. A major obstacle for the use of ML algorithms is that the performance of an algorithm may be affected by many design decisions. Finding appropriate algorithm and model parameters is quite a challenge for nonprofessionals. In this work, we use automated machine learning (AutoML) to analyze the working capacity of CH₄ based on 403,959 COFs. We explore the relationship between 23 features such as the structure, chemical characteristics, atom types of COFs, and the working capacity. Then, the tree-based pipeline optimization tool (TPOT) in AutoML and the traditional ML methods including multiple linear regression, support vector machine, decision tree, and random forest that manually set model parameters are compared. It is found that the TPOT can not only save complex data preprocessing and model parameter tuning but also show higher performance than traditional ML models. Compared with traditional grand canonical Monte Carlo simulations, it can save a lot of time. AutoML has broken through the limitations of professionals so that researchers in nonprofessional fields can realize automatic parameter configuration for experiments to obtain highly accurate and easy-to-understand results, which is of great significance for material screening.

Object instance segmentation in traffic scenes is an important research topic. For training instance segmentation models, synthetic data can potentially complement real data, alleviating manual effort on annotating real images. However, the data distribution discrepancy between synthetic data and real data hampers the wide applications of synthetic data. In light of that, we propose a virtual-real interaction method for object instance segmentation. This method works over synthetic images with accurate annotations and real images without any labels. The virtual-real interaction guides the model to learn useful information from synthetic data while keeping consistent with real data. We first analyze the data distribution discrepancy from a probabilistic perspective, and divide it into image-level and instance-level discrepancies. Then, we design two components to align these discrepancies, i.e., global-level alignment and local-level alignment. Furthermore, a consistency alignment component is proposed to encourage the consistency between the global-level and the local-level alignment components. We evaluate the proposed approach on the real Cityscapes dataset by adapting from virtual SYNTHIA, Virtual KITTI, and VIPER datasets. The experimental results demonstrate that it achieves significantly better performance than state-of-the-art methods.

Virtual simulated scenes are becoming a critical part of autonomous driving. In the context of knowledge automation and machine learning, simulated images are widely used for visual environmental perception. However, even the most inspirational applications have not fully exploited the potential of simulated images in solving real-world problems. In this article, we propose a novel framework “ParallelEye Pipeline,” which uses image-to-image translation and simulated images to automatically generate realistic synthetic images with multiple ground-truth annotations. Specifically, this method has three steps: first, we use Unity3D software to simulate driving scenarios and generate simulated image pairs (including raw images and six ground-truth labels) from the simulated scenes; second, advanced image-to-image translation algorithms can generate realistic and high-resolution synthetic images from simulated image pairs; third, we exploit publicly datasets, simulated images, and synthetic images to conduct experiments for visual perception. The experimental results suggest: 1) synthetic images and simulated images can improve the performance of detectors in real autonomous driving scenarios and 2) image-to-image translation algorithms can be affected by occlusion condition.