Heterogeneous cellular networks with hybrid energy supplies can relieve traffic pressure and reduce grid energy consumption. In heterogeneous cellular networks, rational resource management can help improve system performances. In general, more than one performance is expected to do well, but there can exist a trade-off among different performance metrics, thus making resource management a multi-objective problem. The existing solution usually transforms a multi-objective problem into another single-objective problem by assigning weights for various objectives. However, it is difficult to know the exact weights in advance, and different systems call for different requirements for objectives. Hence, a multi-objective optimization approach based on the gravitational search algorithm (GSA) is proposed to find a series of Pareto optimal solutions. The decision-makers can select an appropriate solution according to the system requirement. In this work, three different multi-objective GSA-based algorithms are proposed to determine user association and power control, with the goal to optimize the traffic load balancing among small base stations and grid energy consumption per unit throughput simultaneously. The complexity of the proposed algorithms is analyzed, and simulations compare the performances of the proposed algorithms and the benchmark algorithm. Experimental results reveal the feasibility and effectiveness of this approach.

This paper investigates a long-term average total energy cost minimization problem via resource management, including admission control, power allocation, and Energy Sharing (ES) of renewable energy in Heterogeneous Cellular Networks powered by Grid-connected Smart Micro Grids (GSMG-HCNs). In GSMG-HCNs, both renewable and grid energy power the base stations. Unlike existing works, we consider the cost of both renewable and grid energy and formulate the power line loss process caused by ES into our model. To solve the proposed problem, we transform it into a real-time issue by the Lyapunov technique. The proposed Cost-Aware Online Resource Management (CAORM) algorithm decouples the real-time issue into two sub-problems, one of which is linear and the other is addressed based on the successive convex approximation approach. We theoretically prove the asymptotic optimality of the CAORM algorithm and a tradeoff between the average total energy cost and the average queue length. Simulation results reveal that the CAORM algorithm outperforms benchmarks in reducing total energy cost and can make appropriate decisions according to different unit costs of renewable energy. Besides, the designed distance-related ES loss rate can help obtain better solutions with lower ES losses.

Vehicular edge computing is an emerging enabler in strengthening driving efficiency and traffic safety. However, both performance bottlenecks and low resource efficiency of heterogeneous edge servers arise concurrently because of the inhomogeneous load distribution among the servers. Further, the unsaturated infrastructure coverage situation can deteriorate the concurrent issues. Although transmitting raw task data with large sizes among heterogeneous edge servers can relieve the concurrent issues, it distinctly degrades the core network’s efficiency, especially during rush hours. Meanwhile, it cannot settle the unsaturated coverage situation. To relieve the concurrent issues without degrading the core network’s efficiency, we introduce an aerial relay station that can flexibly relay vehicular tasks to nearby heterogeneous edge servers. The long-term task scheduling problem without any prior environment knowledge for the considered vehicular edge system is crucial but still up in the air. We formulate the system latency minimization problem as a partially observable stochastic game. Then a model-free multi-agent reinforcement learning algorithm is developed to search the real-time load-aware scheduling policy. Besides, we design a practical factor named offloading latency gain to assist the training process of the learning algorithm. Simulation experiments show that our proposed algorithm can better exploit idle computation resources of heterogeneous edge infrastructures and significantly reduce the average system latency up to 15-20% over existing algorithms.

High energy costs and carbon-neutral targets emphasize the economics and sustainability of mobile communications. This paper studies a long-term average energy transaction expenditure minimization problem for smart-grid powered heterogeneous cellular networks (SG-HCNs) where renewable energy is introduced. Renewable energy and wireless channel dynamics evolve over different timescales. Thus, we seek a two-timescale dynamic resource management solution in SG-HCNs, where the real-time joint issue of flow control, power allocation, and energy sharing of renewable energy, and the ahead-of-time two-way energy trading are considered. Based on a two-layer Lyapunov framework, a two-timescale dynamic optimization (TTDO) algorithm is developed for the proposed problem. Specifically, the real-time joint issue is decoupled into two subproblems, addressed by linear programming and successive convex approximation methods. An approximate solution for ahead-of-time two-way energy trading is achieved via the stochastic subgradient approach, where past data of related random events is referred to as prior knowledge that is required but difficult to acquire. Theoretically, the proposed TTDO algorithm can attain an asymptotic optimum and ensure queue stability. Simulation results verify the theoretical analysis and reveal that the proposed TTDO algorithm can obtain lower energy transaction expenditure than benchmarks. Besides, the proposed TTDO algorithm presents an energy-saving property.

To meet the ever-increasing demand for mobile data traffic, mobile operators are seeking to utilize unlicensed spectrum as a supplement to the licensed spectrum. The harmonious and spectrum-efficient coexistence scheme between LTE and incumbent users on the unlicensed spectrum is thus mandatory. Currently, advanced intelligent technologies are being expected to play the crucial role in the future communication system. We thus introduce the Q-learning (QL) framework into LTE licensed assisted access (LAA) scheme in the paper, thereby forming a QL based LAA scheme. We first redefine the fairness in the sharing of unlicensed spectrum and then divide the state space into six states based on the predefined throughput and fairness thresholds, followed by the definition of the action set and reward function. In the proposed scheme, based on the convergent Q table, where each element is used to evaluate the pros and cons of taking an action, the agent can repeatedly interact with the environment until it reaches the terminal state, i.e selects the optimal action (i.e. contention window size). Additionally, the chaotic motion with ergodicity, regularity and randomness is first introduced into the action-decision strategy to accelerate the training velocity with the balance consideration of exploration and exploitation. The simulation results prove that the proposed $\epsilon$ -chaotic greedy selection strategy has faster convergence velocity compared with other methods such as $\epsilon$ -greedy, pure greedy, Bolzmann and random selection strategy, and that the proposed chaotic QL LAA scheme outperforms the other LAA schemes such as the 3GPP, linear, fixed LAA and Listen Before Talk (LBT) adaptive schemes in terms of throughput, collision probability, fairness and delay

Given the problems of uneven distribution, strong time variability of ground service demands, and low utilization rate of on-board resources in Low-Earth-Orbit (LEO) satellite communication systems, how to efficiently utilize limited beam resources to flexibly and dynamically serve ground users has become a research hotspot. This paper studies the dynamic resource allocation and interference suppression strategies for beam hopping satellite communication systems. Specifically, in the full-frequency-reuse scenario, we adopt spatial isolation techniques to avoid co-channel interference between beams and construct a multi-objective optimization problem by introducing weight coefficients, aiming to maximize user satisfaction and minimize transmission delay simultaneously. We model this optimization problem as a Markov decision process and apply a value decomposition network (VDN) algorithm based on cooperative multi-agent reinforcement learning (MARL-VDN) to reduce computational complexity. In this algorithm framework, each beam acts as an agent, making independent decisions on hopping patterns and power allocation strategies, while achieving multi-agent cooperative optimization through sharing global states and joint reward mechanisms. Simulation results show that the applied algorithm can effectively enhance user satisfaction, reduce delay, and maintain high resource utilization in dynamic service demand scenarios. Additionally, the offline-trained MARL-VDN model can be deployed on LEO satellites in a distributed mode to achieve real-time on-board resource allocation on demand.