Artificial intelligence and big data analytics enable autonomous vehicles (AVs) to dramatically change future intelligent transportation in smart cities. AVs are envisaged to evolve to a service rather than a product in the future. To provide best user experience of such services, three primary factors, namely, waiting time, travel time, and supply of AV services, are taken into consideration in a multi-objective optimization. Conventional optimization of services relies on traffic flow analysis over a queuing network model. However, due to the mobility of vehicles and the transfer uncertainty of road networks, the queuing network analysis is too complicated and practically intractable. For accuracy and convenient processing, network calculus (NC) is extended to model the queueing problem in this paper. The optimal number of available AVs can be identified by guaranteeing the waiting time of customers. The satisfaction of AV services can be viewed as a supply and demand problem, and optimized by bipartite graph matching. In order to reduce the average travel time, especially for rush hours with heavy traffic, we further propose a new online AVs fleet management scheme with congestion control for smart cities. It is shown that the intelligent management of AV fleet can be efficiently achieved, outperforming the cases of traditional vehicles. NC-assisted AI enables an efficient intelligent transportation paradigm in smart cities, while achieving substantial energy saving.

Autonomous driving and intelligent transportation demand ultra-low latency and high reliability communication in future vehicular networks. Proactive wireless communication can facilitate minimal latency by open-loop communication, which discards traditional feedback control mechanisms. However, appropriate radio resource allocation in such proactive mobile networks has not been fully studied due to lacking channel state information (CSI) and the alleviation of multiple access interference (MAI) in multiple virtual cells. This paper aims to ensure the reliability of downlink communication by a novel radio resource allocation scheme in proactive vehicular networks with ultra-low latency. We regard data transmission success rate as the reliability indicator and propose a joint radio resource allocation model based on the “generalized closed-loop”, where anchor node (AN) uses the radio resource utilization information (RRUI) from the vehicle in the immediate past uplink as a guide to assist resource allocation. Subsequently, we study the radio resource allocation model solution on the vehicle side and the network side respectively. On the vehicle side, vehicles use the local or global data transmission experience to select the radio resource with the best quality as the RRUI. On the network side, according to the latest RRUI of vehicle and resource occupancy information, deep reinforcement learning is proposed to make appropriate radio resource allocation decisions. Simulations demonstrate the effectiveness of the intelligent joint radio resource allocation scheme under the cooperation between vehicles and AN. When the resource load rate reaches 40%, the joint radio resource allocation scheme can achieve a data transmission success rate of more than 98%.

In this article, a novel joint energy and Age of Information (AoI) optimization framework for Internet of Things (IoT) devices in a nonstationary environment is presented. In particular, IoT devices that are distributed in the real-world are required to efficiently utilize their computing resources so as to balance the freshness of their data and their energy consumption. To optimize the performance of IoT devices in such a dynamic setting, a novel lifelong reinforcement learning (RL) solution that enables IoT devices to continuously adapt their policies to each newly encountered environment is proposed. Given that IoT devices have limited energy and computing resources, an unmanned aerial vehicle (UAV) is leveraged to visit the IoT devices and update the policy of each device sequentially. As such, the UAV is exploited as a mobile learning agent that can learn a shared knowledge base with a feature base in its training phase, and feature sets of a zero-shot learning method in its testing phase, to generalize between the environments. To optimize the trajectory and flying velocity of the UAV, an actor-critic network is leveraged so as to minimize the UAV energy consumption. Simulation results show that the proposed lifelong RL solution can outperform the state-of-art benchmarks by enhancing the balanced cost of IoT devices by 8.3% when incorporating warm-start policies for unseen environments. In addition, our solution achieves up to 49.38% reduction in terms of energy consumption by the UAV in comparison to the random flying strategy.

Resilient operation of a wireless networked multirobot system (MRS) in a smart factory relies on the effective detection of physical anomalies from robots and cyber anomalies from wireless transmission errors or imprecise artificial intelligence decisions, which leads to a new technological frontier in data-driven industrial informatics: cyber-physical anomaly detection (AD). Furthermore, data patterns in a single smart factory are unlikely enough to train high-quality learning models for this new cyber-physical AD, which suggests the necessity to utilize operating data from multiple smart factories while keeping the privacy of each factory's data. To overcome the aforementioned technical challenges for cyber-physical AD in smart factories, this article proposes an integral mechanism of generative adversarial networks, federated learning, and fuzzy clustering acceleration. Generative adversarial networks facilitate data imputation to regenerate complete datasets alleviating anomalies caused by wireless communications. Federated learning enables rich privacy-preserving datasets to be jointly used among multiple collaborative factories. Furthermore, fuzzy clustering acceleration is embedded to speed up the factory selection algorithm such that efficient training and real-time physical AD in the large-scale operation of multiple smart factories can be achieved. Extensive computational experiments based on the KDD-99 dataset demonstrate the effective and efficient cyber-physical AD of wireless networked MRS in collaborative multiple smart factories.

Proactive Mobile Network (PMN) has been proposed to support extremely low latency communications with multi-tier computing architectures, machine-centricity, and data-driven operation features. Nevertheless, the communication reliability of the PMN introduces new substantial technological challenges. As PMN employs unique proactive open-loop communication, any feedback-based control is avoided to enhance end-to-end latency. This paper focuses on machine-initiated uplink transmission in PMN and proposes a reliability-guaranteed resource management scheme. Without requiring feedback control information, our scheme uniquely decomposes conventional resource management into two collaborative decision processes: predictive resource allocation suggested by network anchor nodes (ANs) and proactive smart resource utilization by smart equipment (SE). These two decision-making processes are constructed as independent reinforcement learning (RL) problems, but implicitly share the states of radio resources according to operating environments. Different algorithms for different operating scenarios have been investigated for this dual-decision solution. Simulation results in various scenarios show that our scheme enables PMN’s reliability close to the theoretical optimal value and successfully serves radio resource utilization in the PMNs.

A proactive mobile network (PMN) is a novel architecture enabling extremely low-latency communication. This architecture employs an open-loop transmission mode that prohibits all real-time control feedback processes and employs virtual cell technology to allocate resources non-exclusively to users. However, such a design also results in significant potential user interference and worsens the communication’s reliability. In this paper, we propose introducing multi-reconfigurable intelligent surface (RIS) technology into the downlink process of the PMN to increase the network’s capacity against interference. Since the PMN environment is complex and time varying and accurate channel state information cannot be acquired in real time, it is challenging to manage RISs to service the PMN effectively. We begin by formulating an optimization problem for RIS phase shifts and reflection coefficients. Furthermore, motivated by recent developments in deep reinforcement learning (DRL), we propose an asynchronous advantage actor–critic (A3C)-based method for solving the problem by appropriately designing the action space, state space, and reward function. Simulation results indicate that deploying RISs within a region can significantly facilitate interference suppression. The proposed A3C-based scheme can achieve a higher capacity than baseline schemes and approach the upper limit as the number of RISs increases.