Dynamic topology, fast-changing channels and the time sensitivity of safety-related services present challenges to the status quo of resource allocation for cellular-underlaying vehicle-to-vehicle (V2V) communications. In this paper, we investigate a novel federated multi-agent deep reinforcement learning (FedMARL) approach for the decentralized joint optimization of channel selection and power control for V2V communication. The approach takes advantage of both deep reinforcement learning (DRL) and federated learning (FL), satisfying the reliability and delay requirements of V2V communication while maximizing the transmit rates of cellular links. Specifically, we elaborately construct individual V2V agent implement by the dueling double deep Q-network (D3QN), and design the reward function to train V2V agents collaboratively. As a result, each agent individually optimizes channel selection and power level based on its local observations, including the instantaneous channel state information (CSI) of corresponding V2V link, the instantaneous co-channel interference from the cellular link, the previous channels selections of nearby V2V pairs, and the queue backlog at the V2V transmitter. Another important aspect is that we incorporate FL to alleviate the training instability problem induced by cooperative multi-agent environment. The local DRL models of different V2V agents are federated periodically, addressing the limitations of partial observability on the entire network status for individual agent, and accelerating the training process of multi-agent learning. Validated via simulations, the proposed FedMARL scheme shows superiority to the baselines in terms of the cellular sum-rate and the V2V packet delivery rate.

A vehicular network is expected to empower all aspects of the intelligent transportation system (ITS) and aim at improving road safety and traffic efficiency. In view of the fact that spectrum scarcity becomes more severe owing to the increasing number of connected vehicles, implying spectrum sensing technology in vehicular network, i.e., cognitive vehicular network, has emerged as a promising solution to provide opportunistic usage of licensed spectrum. However, some unique features of vehicular networks, such as high movement and dynamic topology, take on high challenges for spectrum sensing. Recently, machine learning-based approaches, especially deep learning, for spectrum sensing have attracted sufficient interest. In this work, we investigate a learning-based cooperative spectrum sensing (CSS) approach for multiband spectrum sensing in the cognitive vehicular network. Specifically, we integrate two powerful deep learning models, i.e., the convolutional neural network (CNN) to exploit the features from sensing data, and the long-short-term memory (LSTM) network is then utilized to extract temporal correlations given input as the generated features by the CNN structure. Instead of the predefined decision threshold typically set in conventional approaches, our proposed approach could eliminate the impact of impertinent threshold value setting. Extensive simulations have been conducted to evaluate the effectiveness of the proposed method in achieving satisfactory spectrum sensing performance, particularly in terms of higher detection accuracy, robustness in low signal-to-noise ratio (SNR) environments, and a significant reduction in spectrum sensing time compared to other methods.