27 March 2020 in IEEE Internet of Things Journal
Cooperative Multi-Access Edge Computing (MEC) is a promising paradigm for the next generation mobile networks. However, when the number of users explodes, the computational complexity of existing optimization or learning based task placement approaches in the cooperative MEC can increase significantly, which leads to intolerable MEC decision making delay. In this paper, we propose a mean field game (MFG) guided deep reinforcement learning (DRL) approach for the task placement in the cooperative MEC, which can help servers make timely task placement decisions, and significantly reduce average service delay. Instead of applying MFG or DRL separately, we jointly leverage MFG and DRL for task placement, and let the equilibrium of MFG guide the learning directions of DRL. We also ensure that the MFG and DRL approaches are consistent with the same goal. Specifically, we novelly define a mean field guided Q-value (MFG-Q), which is an estimation of Q-value with the Nash equilibrium gained by MFG. We evaluate the proposed method’s performance using real-world user distribution. Through extensive simulations, we show that the proposed scheme is effective in making timely decisions and reducing the average service delay. Besides, the convergence rates of our proposed method outperform the pure DRL based approaches.
26 April 2024 in IEEE Transactions on Vehicular Technology
Unmanned Aerial Vehicle (UAV) swarms play an effective role in timely data collection from ground sensors in remote and hostile areas. Optimizing the collective behavior of swarms can improve data collection performance. This paper puts forth a new mean field flight resource allocation optimization to minimize age of information (AoI) of sensory data, where balancing the trade-off between the UAVs' movements and AoI is formulated as a mean field game (MFG). The MFG optimization yields an expansive solution space encompassing continuous state and action, resulting in significant computational complexity. To address practical situations, we propose, a new mean field hybrid proximal policy optimization (MF-HPPO) scheme to minimize the average AoI by optimizing the UAV's trajectories and data collection scheduling of the ground sensors given mixed continuous and discrete actions. Furthermore, a long short term memory (LSTM) is leveraged in MF-HPPO to predict the time-varying network state and stabilize the training. Numerical results demonstrate that the proposed MF-HPPO reduces the average AoI by up to 45% and 57% in the considered simulation setting, as compared to multi-agent deep Q-learning (MADQN) method and non-learning random algorithm, respectively.
15 March 2022 in IEEE Transactions on Vehicular Technology
In this paper, we propose an energy-efficient velocity control algorithm for a large number of UAVs based on mean field games. In particular, we first formulate the velocity control problem for a large number of UAVs as a differential game, where we jointly consider the energy consumption, channel capacity, and obstacle avoidance in the cost function. Meanwhile, the state dynamics are used to describe the motion of UAVs under the influence of the wind. Then we derive the corresponding mean field game for a large number of UAVs and solve it with the G-prox primal dual hybrid gradient (PDHG) method using its underlying variational primal dual structure. Scalability analysis shows that the computational complexity of the proposed method is unrelated to the number of UAVs. Based on the PDHG method, we conduct a comprehensive experiment where we analytically show the fast convergence of our energy-efficient velocity control algorithm by the convergence of the residual errors of the Hamilton-Jacobi-Bellman equation and the Fokker-Planck-Kolmogorov equation. The experiment also shows that a large number of UAVs can avoid obstacles and provide communication services for the search and rescue team while minimizing their energy consumption.
21 April 2023 in IEEE Transactions on Vehicular Technology
Reconfigurable intelligent surfaces (RISs), as a promising technology for 6 G communications, have received considerable attention. Herein, we investigate the downlink precoding in the RIS-aided millimeter-wave (mmWave) multi-user multiple-input single-output (MU-MISO) system, where the access point (AP) uses hybrid digital and analog precoders. Due to the constraints of practical hardware and the difficulty of acquiring channel state information, we consider codebook-based passive beamforming and analog beamforming, and then formulate the downlink precoding problem as a mixed-integer nonlinear programming (MINLP) problem. To solve this NP-hard MINLP problem with low channel estimation overheads, we investigate beam training for the codebook-based passive beamforming and analog beamforming, and the MINLP problem is then reduced to a traditional MU-MISO precoding problem. To further reduce the beam training overheads, we propose a two-timescale-based beam training (TT-BT) algorithm. We prove that the TT-BT algorithm achieves the maximum codebook-based passive and analog beamforming gain. To avoid the limitations imposed the use of the TT property, we design the analog precoder by aligning the analog beamforming to RISs. Moreover, we characterize the rate loss due to codebook-based quantization. Compared with the empirical BT (EBT) algorithm, the proposed TT-BT algorithm achieves better performance with significantly lower training and feedback overheads.
8 April 2022 in IEEE Transactions on Signal and Information Processing over Networks
The dominanceof social networks has advanced immensely as many users become more dependent on these networks to be able to engage on social discussions and activities. The behavior of these users about a specific topic or issue can be extracted from their own belief or opinion as well as that of their connections. In order to derive meaningful and important behavioral information, these users can be modeled and analyzed together according to similarities in attributes such as political orientation, race, gender, and age. In this research work, the opinion dynamics of a multiple-population social network is investigated through the application of multiple-population mean field game (MPMFG) for behavior modeling and analysis. As a consequence of the proposed MPMFG model, information can be gained on the behavior of social network users belonging to different populations or groups. Specifically, the proposed MPFMG model can be utilized to estimate and predict the behavior of a social network group as well as their effect on the belief and opinion of other groups. Simulations are provided to demonstrate the belief and opinion dynamics of social network users in multiple-population settings. Moreover, theoretical and experimental results as well as comprehensive performance analysis are presented to demonstrate the effectiveness and validity of the proposed MPMFG approach in modeling and analyzing the evolution of opinions in multiple-population social networks.
21 August 2020 in IEEE Transactions on Wireless Communications
Multi-access edge computing (MEC) can use the distributed computing resources to serve the large numbers of mobile users in the next generation of communication systems. In this new architecture, a limited number of mobile edge servers will serve a relatively large number of mobile users. Heterogeneous servers can provide either single resource or multiple different resources to the massive number of selfish mobile users. To achieve high quality of service (QoS) and low latency under these two cases, we construct two system models and formulate our problems as two non-cooperative population games. Then we apply our proposed mean field evolutionary approach with two different strategy graphs to solve the load balancing problems under those two cases. Finally, to evaluate the performance of our algorithms, we employ the following performance indicators: overall response time (average response time of the whole system), individual response time (response time of each server), and fairness index (equality of users’ response time).