27 April 2020 in IEEE Transactions on Vehicular Technology
In this paper, we investigate the scheduling design of a mobile edge computing (MEC) system, where active mobile devices with computation tasks randomly appear in a cell. Every task can be computed at either the mobile device or the MEC server. We jointly optimize the task offloading decision, uplink transmission device selection and power allocation by formulating the problem as an infinite-horizon Markov decision process (MDP). Compared with most of the existing literature, this is the first attempt to address the transmission and computation optimization with the random device arrivals in an infinite time horizon to our best knowledge. Due to the uncertainty in the device number and location, the conventional approximate MDP approaches addressing the curse of dimensionality cannot be applied. An alternative and suitable low-complexity solution framework is proposed in this work. We first introduce a baseline scheduling policy, whose value function can be derived analytically with the statistics of random mobile device arrivals. Then, one-step policy iteration is adopted to obtain a sub-optimal scheduling policy whose performance can be bounded analytically. The complexity of deriving the sub-optimal policy is reduced dramatically compared with conventional solutions of MDP by eliminating the complicated value iteration. To address a more general scenario where the statistics of random mobile device arrivals are unknown, a novel and efficient algorithm integrating reinforcement learning and stochastic gradient descent (SGD) is proposed to improve the system performance in an online manner. Simulation results show that the gain of the sub-optimal policy over various benchmarks is significant.
14 February 2024 in IEEE Transactions on Communications
A two-time-scale approximate Markov decision process (MDP) is proposed to optimize the uplink queuing performance of a millimeter wave (mmWave) communication system. Exploiting the wireless sensing techniques, the locations of signal reflectors, blockers, and mobile agents, as well as the mobility pattern of agents, become available knowledge to facilitate predictive scheduling. Notice that the state variation of the wireless channel fading and transmission queue is much faster than that of mobile agents’ locations. The joint optimization of uplink power adaptation, time allocation, and analog beamforming of all the frames is formulated as a two-time-scale MDP with the average uplink energy, queuing length, and buffer overflow rate in the minimization objective. The scheduling in the larger time scale with an infinite horizon, i.e., the analog transceiver beamforming, adapts with the random motion of mobile agents; whereas the scheduling in the smaller time scale with a finite horizon, i.e., uplink power and time allocation, adapts with both the small-scale channel fading, queue dynamics and large-scale random motion. The optimal solution of two-time-scale MDP requests iterative optimization between Bellman’s equations of both time scales, whose computation complexity is prohibitive. A novel low-complexity solution framework is then proposed to obtain the optimal larger-time-scale and sub-optimal smaller-time-scale policies, where the performance of the proposed scheme is bounded analytically. Benefiting from the motion sensing and blockage prediction, the proposed scheduling scheme outperforms existing benchmarks in the numerical simulations.
14 March 2024 in IEEE Transactions on Wireless Communications
Most of the existing research on degrees-of-freedom (DoF) with imperfect channel state information at the transmitter (CSIT) assume the messages are private, which may not reflect reality as the two receivers can request the same content. To overcome this limitation, we therefore consider the hybrid unicast and multicast messages. In particular, we characterize the optimal DoF region for the two-user multiple-input multiple-output (MIMO) broadcast channel (BC) with imperfect CSIT and hybrid messages. For the converse, we establish a three-step procedure to exploit the utmost possible relaxation. For the achievability, since the DoF region is with specific three-dimensional structure regarding antenna configurations and CSIT qualities, we verify the existence or non-existence of corner point candidates via the feature of antenna configurations and CSIT qualities categorization, and provide a hybrid message-aware rate-splitting scheme. Besides, we show that to achieve the strictly positive corner points, it is unnecessary to split the unicast messages into private and common parts. This implies adding a multicast message may mitigate the rate-splitting complexity.
22 November 2024 in ACM Transactions on Sensor Networks
Wireless sensor networks (WSNs) leveraging millimeter wave (mmWave) communication for bandwidth-demanding applications is considered in this article. Despite the large bandwidth, the delivery of delay-sensitive information collected by sensors may still face significant latency due to the vulnerability to intermittent link blockage. Hence, the guarantee of low age of information (AoI) in mmWave WSNs is not straightforward. In this article, the wireless sensing and dynamic programming techniques are jointly exploited to relieve the above issue. The former tracks the human blockers and predicts the chance of link blockage; the latter optimizes the transmission of multiple sensors based on the prediction. Particularly, the long-term optimization of sampling, uplink time and power allocation policies in a sensor network can be formulated as an infinite-horizon Markov decision process (MDP) with discounted cost, where the state transition probabilities can be predicted via wireless sensing. A novel low-complexity solution framework, namely COSMO, with a guaranteed performance in the worst case, is proposed. Simulations show that compared with heuristic benchmarks, benefiting from the prediction of the link blockage, COSMO can significantly suppress the average system cost, which consists of both AoI and energy consumption.
22 May 2026 in IEEE Transactions on Vehicular Technology
Distributed channel access mechanisms such as carrier sense multiple access (CSMA), adopted in Wi-Fi networks, lack centralized coordination and global queue state information. Thus, they are inefficient in supporting delay-sensitive mobile traffic. For this problem, we introduce an adaptive channel contention mechanism aimed at improving queuing performance in a distributed millimeter-wave (mmWave) uplink system equipped with environment and mobility sensing capabilities. Specifically, mobile agents adjust their back-off timer parameters based on locally available information, including uplink queue lengths, current channel quality, and predicted channel statistics, where predictions are enabled by environmental and mobility sensing. We formulate the queuing optimization problem under this adaptive mechanism as a decentralized multi-agent Markov decision process (Dec-MA-MDP). Although contention decisions are made locally by each mobile agent, the corresponding policies for all agents are optimized centrally using system-wide statistics obtained prior to scheduling. In the proposed method, local policies are approximated via analytical models, reducing the policy optimization to a stochastic optimization problem defined along an adaptive Markov chain. To enhance efficiency, we further propose an unbiased gradient estimator, enabling policy refinement through stochastic gradient descent. Simulation results for both indoor and outdoor scenarios show that the proposed method achieves significantly higher optimization efficiency than conventional methods, such as simultaneous perturbation stochastic approximation (SPSA).
16 September 2025 in IEEE Transactions on Communications
In this paper, task offloading from vehicles with random velocities is optimized via a novel dynamic programming framework. Particularly, in a vehicular network with multiple vehicles and base stations (BSs), computing tasks of vehicles are offloaded via BSs to an edge server. Due to the random velocities, the exact locations of vehicles versus time, namely trajectories, cannot be determined in advance. Hence, instead of deterministic optimization, the cell association, uplink time, and throughput allocation of multiple vehicles during a period of task offloading are formulated as a finite-horizon Markov decision process. In order to derive a low-complexity solution algorithm, a two-time-scale framework is proposed. The scheduling period is divided into super slots, each super slot is further divided into a number of time slots. At the beginning of each super slot, we first obtain a reference scheduling scheme of cell association, uplink time and throughput allocation via deterministic optimization, yielding an approximation of the optimal value function. Within the super slot, the actual scheduling action of each time slot is determined by making improvement to the approximate value function according to the system state. Due to the successive improvement framework, a non-trivial average cost upper bound could be derived. In the simulation, the random trajectories of vehicles are generated from a high-fidelity traffic simulator. It is shown that the performance gain of the proposed scheduling framework over the baselines is significant.