As urban traffic condition is diverse and complicated, applying reinforcement learning to reduce traffic congestion becomes one of the hot and promising topics. Especially, how to coordinate the traffic light controllers of multiple intersections is a key challenge for multi-agent reinforcement learning (MARL). Most existing MARL studies are based on traditional Q-learning, but unstable environment leads to poor learning in the complicated and dynamic traffic scenarios. In this paper, we propose a novel multi-agent recurrent deep deterministic policy gradient (MARDDPG) algorithm based on Deterministic Policy Gradient algorithm for traffic light control (TLC) in vehiclar networks. Specifically, the centralized learning in each critic network enables each agent to estimate the policies of other agents in the decision-making process and each agent can coordinate with each other, alleviating the problem of poor learning performance caused by environmental instability. The decentralized execution enables each agent to make decisions independently. We share parameter in actor networks to speed up training process and reduce the memory footprint. The addition of LSTM is beneficial to alleviate the instability of the environment caused by partial observable state. We utilize surveillance cameras and vehicular networks to collect status information for each intersection. Unlike previous work, we have not only considered the vehicle but also considered the pedestrians waiting to pass through the intersection. Moreover, we also set different priorities for buses and ordinary vehicles. The experimental results in a vehicular network show that our method can run stably in various scenarios and coordinate multiple intersections, which significantly reduces vehicle congestion and pedestrian congestion.

Federated Learning (FL) has recently attracted considerable attention in internet of things, due to its capability of enabling mobile clients to collaboratively learn a global prediction model without sharing their privacy-sensitive data to the server. Despite its great potential, a main challenge of FL is that the training data are usually non-Independent, Identically Distributed (non-IID) on the clients, which may bring the biases in the model training and cause possible accuracy degradation. To address this issue, this paper aims to propose a novel FL algorithm to alleviate the accuracy degradation caused by non-IID data at clients. Firstly, we observe that the clients with different degrees of non-IID data present heterogeneous weight divergence with the clients owning IID data. Inspired by this, we utilize weight divergence to recognize the non-IID degrees of clients. Then, we propose an efficient FL algorithm, named CSFedAvg, in which the clients with lower degree of non-IID data will be chosen to train the models with higher frequency. Finally, we conduct simulations using publicly-available datasets to train deep neural networks. Simulation results show that the proposed FL algorithm improves the training performance compared with existing FL protocol.

Nowadays, vehicles have been increasingly adopted in mobile crowdsensing applications. Due to their predictable mobility trajectories, vehicles as participants bring new insight in improving the crowdsensing quality. The predictable mobility of vehicles provides not only the current locations of the vehicles, but also their future mobility trajectory. In this context, the existing participant recruitment solutions, which are mainly based on the current locations of the participants, cannot be directly used in vehicle-based mobile crowdsensing. Utilizing the predicted mobility trajectory of vehicles, this paper aims to propose efficient vehicle recruitment algorithms for mobile crowdsensing, so as to minimize the overall recruitment cost. Specifically, we study two mobility trajectory models of the vehicles, named deterministic and probabilistic models. We first prove that the vehicle recruitment problem under both models is NP-hard. Then, for the deterministic trajectory model, an efficient LP-relaxation-based heuristic algorithm is proposed, and an approximation ratio is analyzed. For the probabilistic trajectory model, we propose a greedy algorithm and analyze its performance with a guaranteed approximation ratio. Finally, we evaluate the performance of the proposed schemes through simulations.

Internet-of-Things-enabled E-health system, which could monitor and collect the personal health information (PHI), has gradually transformed the clinical treatment to a more personalized way with in-home monitoring smart devices. Then, with the collected PHI, clinical decision support systems (CDSSs), which are based on data mining techniques and historical electronic medical records (EMRs) to help clinicians make proper treatment decisions, have attracted considerable attention. To address issues, such as network congestion and low rate of responsiveness for traditional methods when implementing CDSSs, we integrate the technologies mobile-edge computing (MEC) and software-defined networking for exploiting the computation resources and storage capacities among edge nodes (ENs) (i.e., MEC servers) in our model. Based on this integrated system, each edge node will deploy a double deep Q-network (DDQN) to obtain a stable and sequential clinical treatment policy. It is enabled by a novel fully decentralized federated framework (FDFF) for aggregating models of DDQN and extracting the knowledge from EMRs across all ENs. Furthermore, we discuss the convergence of FDFF in resource-constrained environments. However, since most EMRs are faced with stringent privacy concerns, we adopt two additively homomorphic encryption schemes to prevent leakage of EMRs' privacy during the training process of FDFF. Finally, we measure the time cost of our additively homomorphic encryption schemes and validate DDQN with experiments on large data sets based on FDFF, which shows promising performance on clinician treatment.

The rapid development of internet of vehicles (IoV) has recently led to the emergence of diverse intelligent vehicular applications such as automatic driving, auto navigation, and advanced driver assistance, etc. However, the current vehicular communication framework, such as vehicle-to-vehicle (V2V), vehicle-to-cloud (V2C), and vehicle-to-roadside infrastructure (V2I), still remain challenging in supporting these intelligent and delay-sensitive applications, due to its long communication latency or low computational capability. Besides that, the traditional vehicle network is prone to be unavailable because of the mobility with high-speed of the vehicles. To address these issues, this paper proposes a vehicle-to-multi-edges (V2Es) communication framework in vehicular networks. By utilizing the resource of edge nodes in the proximity, the emergency information or services of vehicles can be timely processed and completed, which improves the service quality of vehicles. Furthermore, we define a joint task offloading and edge caching problem, targeting optimizing both the latency of services and energy consumption of vehicles. Based on this, we propose a multi-agent reinforcement learning (RL) method to learn the dynamic communication status between vehicles and edge nodes, and make decisions on task offloading and edge caching. Finally, results of the simulation show that our proposal is able to learn the scheduling policy more quickly and effectively, and reduce the service latency by more than 10% on average.

In mobile crowdsensing, it is a challenge to assign tasks to appropriate smartphones. Existing task allocation mechanisms mainly aim at optimizing the global system performance, while ignoring the personal preferences of individual crowdsensing tasks and smartphone users. Nevertheless, in an open crowdsensing system, a task assignment is prone to be unstable if smartphone users or tasks have incentives to deviate from the global assignment, and seek for alternative choices to improve their own utilities. Besides that, during task competition, the rational smartphone users might choose to adjust their payments after the first few failures, which however, brings new challenges in achieving the stability. To address these issues, this paper constructs a distributed many-to-many matching model to capture the interaction between crowdsensing tasks and smartphone users, taking into account the budget constraints of tasks. Then, we design a stable matching algorithm to allocate the tasks to the users, and determine their payments. We prove that the proposed algorithm achieves several desirable properties including individual rationality, stability, and convergency. It is also proved that the proposed scheme achieves at least half of the optimal system efficiency when each smartphone provides homogeneous service quality. Finally, simulation results confirm the effectiveness of the proposed scheme.