In a vehicle-to-grid (V2G) system, electric vehicles (EVs) can be efficiently used as power consumers and suppliers to achieve microgrid (MG) autonomy. Since EVs can act as energy transporters among different regions (i.e., MGs), it is an important issue to decide where and when EVs are charged or discharged to achieve the optimal performance in a V2G system. In this paper, we propose a mobility-aware V2G control algorithm (MACA) that considers the mobility of EVs, states of charge of EVs, and the estimated/actual demands of MGs and then determines charging and discharging schedules for EVs. To optimize the performance of MACA, the Markov decision process problem is formulated and the optimal policy on charging and discharging is obtained by a value iteration algorithm. Since the mobility of EVs and the estimated/actual demand profiles of MGs may not be easily obtained, a reinforcement learning approach is also introduced. Evaluation results demonstrate that MACA with the optimal and learning-based policies can effectively achieve MG autonomy and provide higher satisfaction on the charging.

In federated learning (FL), if the participating mobile devices have low computing power and poor wireless channel conditions and/or they do not have sufficient data for various classes, a long convergence time is required to achieve the desired model accuracy. To address this problem, we first formulate a constrained Markov decision process (CMDP) problem that aims to minimize the average time of rounds while maintaining the numbers of trained data and trained data classes above certain numbers. To obtain the optimal scheduling policy, the formulated CMDP problem is converted into an equivalent linear programming (LP). Additionally, to overcome the problem of the curse of dimensionality in CMDP, we develop a joint client selection and bandwidth allocation algorithm (J-CSBA) that jointly selects appropriate mobile devices and allocates suitable amount of bandwidth to them at each round by considering their data information, computing power, and channel gain. Evaluation results validate that J-CSBA can reduce the convergence time by up to $49\%$ compared to a conventional random scheme.

In battery swapping environments, electric vehicles (EVs) can play roles as battery providers as well as consumers. In this paper, we propose an optimal battery charging algorithm (OBCA) where a battery swapping station (BSS) charges batteries in its storage with the consideration of the profile of the electricity price and the arrival rates of EVs. To maximize the net profit of BSS while maintaining the battery changing probability above a certain level (i.e., maintaining high quality of service (QoS) of BSS), we formulate a constraint Markov decision process (CMDP) problem and the optimal charging schedule for batteries in BSS is obtained by a linear programming (LP). Evaluation results demonstrate that OBCA with the optimal policy can improve the net profit of BSS up to 418% compared to an electric price-aware scheme while maintaining high QoS of BSS.

A novel concept of the edge cloud has recently been introduced to reduce transmission costs in mobile cloud computing services. Heterogeneous networks with diverse radio access networks will be pervasive in the future. In this paper, we propose a spatial and temporal computation offloading decision algorithm (ST-CODA) in edge cloud-enabled heterogeneous networks. In ST-CODA, a mobile device decides where and when to process tasks by means of a Markov decision process (MDP) with consideration of the processing time and energy consumption of different computation nodes and the transmission cost in heterogeneous networks. Extensive evaluation results are given to demonstrate the effectiveness of ST-CODA in terms of the transmission cost, the energy efficiency of the mobile device, and the number of tasks that can be processed before their deadline.

Federated learning (FL) has received significant attention as a practical alternative to traditional cloud-centric machine learning (ML). The performance (e.g., accuracy and convergence time) of FL is hampered by the selection of clients having non-independent and identically distributed (non-IID) data. In addition, a long convergence time is inevitable if clients with poor computation or communication capabilities participate in the FL procedure (i.e., the straggler problem). To minimize convergence time while guaranteeing high learning accuracy, we first formulate an optimization problem on client selection. As a practical solution, we devise a data distribution-aware online client selection (DOCS) algorithm. In DOCS, the FL server finds several clusters having near IID data and then uses a multi-armed bandit (MAB) technique to select the cluster with the lowest convergence time. The evaluation results demonstrate that DOCS can reduce the convergence time by up to $10\% \sim 41\%$ and improve the learning accuracy by up to $4\% \sim 13\%$ compared to the traditional client selection schemes.

Owing to dynamically changing resources and channel conditions of mobile devices (MDs), when a static deadline-based MD selection scheme is used for federated learning, resource utilization of MDs can be degraded. To mitigate this problem, we propose an adaptive deadline determination (ADD) algorithm for MD selection, where a deadline for each round is adaptively determined with the consideration of the performance disparity of MDs. Evaluation results demonstrate that ADD can achieve the fastest average convergence time among the comparison schemes.