In this paper, we investigate a computing task scheduling problem in space-air-ground integrated network (SAGIN) for delay-oriented Internet of Things (IoT) services. In the considered scenario, an unmanned aerial vehicle (UAV) collects computing tasks from IoT devices and then makes online offloading decisions, in which the tasks can be processed at the UAV or offloaded to the nearby base station or the remote satellite. Our objective is to design a task scheduling policy that minimizes offloading and computing delay of all tasks given the UAV energy capacity constraint. To this end, we first formulate the online scheduling problem as an energy-constrained Markov decision process (MDP). Then, considering the task arrival dynamics, we develop a novel deep risk-sensitive reinforcement learning algorithm. Specifically, the algorithm evaluates the risk, which measures the energy consumption that exceeds the constraint, for each state and searches the optimal parameter weighing the minimization of delay and risk while learning the optimal policy. Extensive simulation results demonstrate that the proposed algorithm can reduce the task processing delay by up to 30% compared to probabilistic configuration methods while satisfying the UAV energy capacity constraint.

With files proactively stored at base stations (BSs), mobile edge caching enables direct content delivery without remote file fetching, which can reduce the end-to-end delay while relieving backhaul pressure. To effectively utilize the limited cache size in practice, cooperative caching can be leveraged to exploit caching diversity, by allowing users served by multiple base stations under the emerging user-centric network architecture. This paper explores delay-optimal cooperative edge caching in large-scale user-centric mobile networks, where the content placement and cluster size are optimized based on the stochastic information of network topology, traffic distribution, channel quality, and file popularity. Specifically, a greedy content placement algorithm is proposed based on the optimal bandwidth allocation, which can achieve (1 - 1/e)-optimality with linear computational complexity. In addition, the optimal user-centric cluster size is studied, and a condition constraining the maximal cluster size is presented in explicit form, which reflects the tradeoff between caching diversity and spectrum efficiency. Extensive simulations are conducted for analysis validation and performance evaluation. Numerical results demonstrate that the proposed greedy content placement algorithm can reduce the average file transmission delay up to 45 percent compared with the non-cooperative and hit-ratio-maximal schemes. Furthermore, the optimal clustering is also discussed considering the influences of different system parameters.

Mobile edge caching aims to enable content delivery within the radio access network, which effectively alleviates the backhaul burden and reduces response time. To fully exploit edge storage resources, the most popular contents should be identified and cached. Observing that user demands on certain contents vary greatly at different locations, this paper devises location-customized caching schemes to maximize the total content hit rate. Specifically, a linear model is used to estimate the future content hit rate. For the case with zero-mean noise, a ridge regression-based online algorithm with positive perturbation is proposed. Regret analysis indicates that the hit rate achieved by the proposed algorithm asymptotically approaches that of the optimal caching strategy in the long run. When the noise structure is unknown, an H _∞ filter-based online algorithm is devised by taking a prescribed threshold as input, which guarantees prediction accuracy even under the worst-case noise process. Both online algorithms require no training phases and, hence, are robust to the time-varying user demands. The estimation errors of both algorithms are numerically analyzed. Moreover, extensive experiments using real-world datasets are conducted to validate the applicability of the proposed algorithms. It is demonstrated that those algorithms can be applied to scenarios with different noise features, and are able to make adaptive caching decisions, achieving a content hit rate that is comparable to that via the hindsight optimal strategy.

Collaboration among industrial Internet of Things (IoT) devices and edge networks is critical to support computation-intensive deep neural network (DNN) inference services which require low delay and high accuracy. Sampling rate adaption which dynamically configures the sampling rates of industrial IoT devices according to network conditions, is the key in minimizing the service delay. In this paper, we investigate the collaborative DNN inference problem in industrial IoT networks. To capture the channel variation and task arrival randomness, we formulate the problem as a constrained Markov decision process (CMDP). Specifically, sampling rate adaption, inference task offloading and edge computing resource allocation are jointly considered to minimize the average service delay while guaranteeing the long-term accuracy requirements of different inference services. Since CMDP cannot be directly solved by traditional reinforcement learning (RL) algorithms due to the intractable long-term constraints, we first transform the CMDP into an MDP by leveraging the Lyapunov optimization technique. Then, a deep RL-based algorithm is proposed to solve the MDP. To expedite the training process, an optimization subroutine is embedded in the proposed algorithm to directly obtain the optimal edge computing resource allocation. Extensive simulation results are provided to demonstrate that the proposed algorithm can significantly reduce the average service delay while preserving long-term inference accuracy with a high probability, as compared to benchmark schemes.

Beam alignment (BA) is to ensure the transmitter and receiver beams are accurately aligned to establish a reliable communication link in millimeter-wave (mmwave) systems. Existing BA methods search the entire beam space to identify the optimal transmit-receive beam pair, which incurs significant BA latency on the order of seconds in the worst case. In this paper, we develop a learning algorithm to reduce BA latency, namely Hierarchical Beam Alignment (HBA) algorithm. We first formulate the BA problem as a stochastic multi-armed bandit problem with the objective to maximize the cumulative received signal strength within a certain period. The proposed algorithm takes advantage of the correlation structure among beams such that the information from nearby beams is extracted to identify the optimal beam, instead of searching the entire beam space. Furthermore, the prior knowledge on the channel fluctuation is incorporated in the proposed algorithm to further accelerate the BA process. Theoretical analysis indicates that the proposed algorithm is asymptotically optimal. Extensive simulation results demonstrate that the proposed algorithm can identify the optimal beam with a high probability and reduce the BA latency from hundreds of milliseconds to a few milliseconds in the multipath channel, as compared to the existing BA method in IEEE 802.11ad.