Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond. However, its uncertainty, referred to as dynamic and randomness, from the mobile device, wireless channel, and edge network sides, results in high-dimensional, nonconvex, nonlinear, and NP-hard optimization problems. Thanks to the evolved reinforcement learning (RL), upon iteratively interacting with the dynamic and random environment, its trained agent can intelligently obtain the optimal policy in MEC. Furthermore, its evolved versions, such as deep reinforcement learning (DRL), can achieve higher convergence speed efficiency and learning accuracy based on the parametric approximation for the large-scale state-action space. This paper provides a comprehensive research review on RL-enabled MEC and offers insight for development in this area. More importantly, associated with free mobility, dynamic channels, and distributed services, the MEC challenges that can be solved by different kinds of RL algorithms are identified, followed by how they can be solved by RL solutions in diverse mobile applications. Finally, the open challenges are discussed to provide helpful guidance for future research in RL training and learning MEC.

Orthogonal time frequency space (OTFS) system constitutes an effective structure conceived for efficiently utilizing the channel information, which is capable of achieving a promising transmission performance in high-mobility environment. To extract enough channel diversity, a two-dimensional Fourier transformation combined with a pulse shape is designed at the OTFS transmitter. Consequently, the amplitude of OTFS signals may fluctuate drastically, owing to the combined dependency of the OTFS transformation and the pulse shape. To quantify the amplitude fluctuation, we investigate the peak-to-average power ratio (PAPR) of OTFS signals, for a large amount of data in the delay-Doppler domain. We first reveal that when the number of data points approaches to infinity, based on central limit theorems for dependent variables, the complex-valued OTFS signals weakly converge to a Gaussian distribution. Then, according to the extremal theory of the Chi-squared process for stationary OTFS signals, an accurate expression of the PAPR distribution is derived, depending on the transmit pulse and the number of data points. It is also demonstrated that upon modifying the exponential factor, the analytical PAPR expression is applicable for the non-stationary Gaussian distribution caused by the bandlimited pulse with a large roll-off factor. Simulation results confirm the accuracy of the analytical PAPR probability for practical conditions.

Benefiting from the fusion of communication and intelligent technologies, network-enabled robots have become important to support future machine-assisted and unmanned applications. To provide high-quality services for robots in wide areas, hybrid satellite-terrestrial networks are a key technology. Through hybrid networks, computation-intensive and latency-sensitive tasks can be offloaded to mobile edge computing (MEC) servers. However, due to the mobility of mobile robots and unreliable wireless network environments, excessive local computations and frequent service migrations may significantly increase the service delay. To address this issue, this paper aims to minimize the average task completion time for MEC-based offloading initiated by satellite-terrestrial-network-enabled robots. Different from conventional mobility-aware schemes, the proposed scheme makes the offloading decision by jointly considering the mobility control of robots. A joint optimization problem of task offloading and velocity control is formulated. Using Lyapunov optimization, the original optimization is decomposed into a velocity control subproblem and a task offloading subproblem. Then, based on the Markov decision process (MDP), a dual-agent reinforcement learning (RL) algorithm is proposed. The convergence and complexity of the improved RL algorithm are theoretically analyzed, and the simulation results show that the proposed scheme can effectively reduce the offloading delay.

Networked robots have become crucial for unmanned applications since they can collaborate to complete complex tasks in remote/hazardous/depopulated areas. Due to the cost inefficiency of deploying cellular network infrastructure in these areas, hybrid satellite-UAV networks emerge as a promising solution. These networks provide seamless and on-demand connectivity for multiple robots with various task requirements, and support computation-intensive and latency-sensitive services through mobile edge computing (MEC)-based offloading. However, to complete tasks in limited times, the rapid collective movement of mobile robots may cause frequent service migration, and a large number of gathered robots may compete for limited bandwidth resources in satellite and UAV communications. As a result, offloading latency may increase significantly. To address this issue, the average completion time of multi-robot offloading in task-oriented satellite-UAV networks with MEC is formulated as an optimization problem. Unlike conventional mobility-aware MEC-based offloading schemes, joint optimization of mobility control, data offloading, and resource allocation is proposed using velocity control of multiple robots. According to Lyapunov optimization, the original optimization problem is simplified into minimizing the average completion time of offloading for all robots within UAV and satellite coverage. A multi-agent Q-learning algorithm, including multi-group dual-agent Q-learning, is proposed based on local state observation and global reward calculation. In each dual-agent Q-learning, one agent is responsible for velocity control and communication resource allocation, while the other is responsible for data offloading and computational resource allocation. The convergence of the proposed multi-agent Q-learning algorithm is also theoretically analyzed. Simulation results show that the proposed scheme can effectively reduce the offloading latency by up to 35% in the multi-robot environment over its conventional counterparts.

An N-continuous generalized frequency division multiplexing (GFDM) transceiver architecture is studied with the objective of striking a balanced trade-off between the bit error rate (BER) and the sidelobe suppression performance. More specifically, in the proposed N-continuous GFDM signaling, the basis signals constructed allow one to make the GFDM signal Ncontinuous and attain a compact spectrum as an explicit benefit of sidelobe suppression. We further reveal that compared to conventional N-continuous orthogonal frequency division multiplexing (NC-OFDM), N-continuous GFDM introduces relatively low interference through evaluating the signal-to-interference ratio (SIR). Secondly, a signal recovery algorithm is presented by constructing a recovery matrix to eliminate the interference. Finally, it is demonstrated that the proposed N-continuous GFDM scheme outperforms its N-continuous OFDM counterpart in terms of sidelobe suppression, while achieving moderate BER performance degradation as opposed to original OFDM.