To enable the large scale and efficient deployment of Artificial Intelligence (AI), the confluence of AI and Edge Computing has given rise to Edge Intelligence, which leverages on the computation and communication capabilities of end devices and edge servers to process data closer to where it is produced. One of the enabling technologies of Edge Intelligence is the privacy preserving machine learning paradigm known as Federated Learning (FL), which enables data owners to conduct model training without having to transmit their raw data to third-party servers. However, the FL network is envisioned to involve thousands of heterogeneous distributed devices. As a result, communication inefficiency remains a key bottleneck. To reduce node failures and device dropouts, the Hierarchical Federated Learning (HFL) framework has been proposed whereby cluster heads are designated to support the data owners through intermediate model aggregation. This decentralized learning approach reduces the reliance on a central controller, e.g., the model owner. However, the issues of resource allocation and incentive design are not well-studied in the HFL framework. In this article, we consider a two-level resource allocation and incentive mechanism design problem. In the lower level, the cluster heads offer rewards in exchange for the data owners' participation, and the data owners are free to choose which cluster to join. Specifically, we apply the evolutionary game theory to model the dynamics of the cluster selection process. In the upper level, each cluster head can choose to serve a model owner, whereas the model owners have to compete amongst each other for the services of the cluster heads. As such, we propose a deep learning based auction mechanism to derive the valuation of each cluster head's services. The performance evaluation shows the uniqueness and stability of our proposed evolutionary game, as well as the revenue maximizing properties of the deep learning based auction.

Future-generation wireless networks (5G and beyond) must accommodate surging growth in mobile data traffic and support an increasingly high density of mobile users involving a variety of services and applications. Meanwhile, the networks become increasingly dense, heterogeneous, decentralized, and ad hoc in nature, and they encompass numerous and diverse network entities. Consequently, different objectives, such as high throughput and low latency, need to be achieved in terms of service, and resource allocation must be designed and optimized accordingly. However, considering the dynamics and uncertainty that inherently exist in wireless network environments, conventional approaches for service and resource management that require complete and perfect knowledge of the systems are inefficient or even inapplicable. Inspired by the success of machine learning in solving complicated control and decision-making problems, in this article we focus on deep reinforcement- learning (DRL)-based approaches that allow network entities to learn and build knowledge about the networks and thus make optimal decisions locally and independently. We first overview fundamental concepts of DRL and then review related works that use DRL to address various issues in 5G networks. Finally, we present an application of DRL for 5G network slicing optimization. The numerical results demonstrate that the proposed approach achieves superior performance compared with baseline solutions.

Future Internet involves several emerging technologies such as 5G and beyond 5G networks, vehicular networks, unmanned aerial vehicle (UAV) networks, and Internet of Things (IoTs). Moreover, the future Internet becomes heterogeneous and decentralized with a large number of involved network entities. Each entity may need to make its local decision to improve the network performance under dynamic and uncertain network environments. Standard learning algorithms such as single-agent Reinforcement Learning (RL) or Deep Reinforcement Learning (DRL) have been recently used to enable each network entity as an agent to learn an optimal decision-making policy adaptively through interacting with the unknown environments. However, such an algorithm fails to model the cooperations or competitions among network entities, and simply treats other entities as a part of the environment that may result in the non-stationarity issue. Multi-agent Reinforcement Learning (MARL) allows each network entity to learn its optimal policy by observing not only the environments but also other entities’ policies. As a result, MARL can significantly improve the learning efficiency of the network entities, and it has been recently used to solve various issues in the emerging networks. In this paper, we thus review the applications of MARL in emerging networks. In particular, we provide a tutorial of MARL and a comprehensive survey of applications of MARL in next-generation Internet. In particular, we first introduce single-agent RL and MARL. Then, we review a number of applications of MARL to solve emerging issues in the future Internet. The issues consist of network access, transmit power control, computation offloading, content caching, packet routing, trajectory design for UAV-aided networks, and network security issues. Finally, we discuss the challenges, open issues, and future directions related to the applications of MARL in the future Internet.

The maritime Internet of Things (IoT) has recently emerged as a revolutionary communication paradigm where a large number of moving vessels are closely interconnected in intelligent maritime networks. However, the tremendous growth of vessel trajectories, collected from the combined satellite-terrestrial AIS (automatic identification system) base stations, could lead to unsatisfactory maritime safety and efficacy. To promote smart traffic services in maritime IoT, it is necessary to accurately and robustly predict the spatiotemporal vessel trajectories. It is beneficial for collision avoidance, maritime surveillance, and abnormal behavior detection, etc. Motivated by the strong learning capacity of deep neural networks, this work proposes an AIS data-driven trajectory prediction framework, whose main component is a long short term memory (LSTM) network. In particular, the vessel traffic conflict situation modeling, generated using the dynamic AIS data and social force concept, is embedded into the LSTM network to guarantee high-accuracy vessel trajectory prediction. In addition, a mixed loss function is reconstructed to make our prediction results more reliable and robust in different navigation environments. Several quantitative and qualitative experiments have been implemented on realistic AIS-based vessel trajectories. Our results have demonstrated that the proposed method could achieve satisfactory prediction performance in terms of accuracy and robustness.

Federated learning (FL) becomes popular and has shown great potentials in training large-scale machine learning (ML) models without exposing the owners’ raw data. In FL, the data owners can train ML models based on their local data and only send the model updates rather than raw data to the model owner for aggregation. To improve learning performance in terms of model accuracy and training completion time, it is essential to recruit sufficient participants. Meanwhile, the data owners are rational and may be unwilling to participate in the collaborative learning process due to the resource consumption. To address the issues, there have been various works recently proposed to motivate the data owners to contribute their resources. In this paper, we provide a comprehensive review for the economic and game theoretic approaches proposed in the literature to design various schemes for incentivizing data owners to participate in FL training process. In particular, we first present the fundamentals and background of FL, economic theories commonly used in incentive mechanism design. Then, we review applications of game theory and economic approaches applied for incentive mechanisms design of FL. Finally, we highlight some open issues and future research directions concerning incentive mechanism design of FL.

Future generation communication systems, such as 5G and 6G wireless systems, exploit the combined satellite-terrestrial communication infrastructures to extend network coverage and data throughput for data-driven applications. These ground-breaking techniques have promoted the rapid development of Internet of Things (IoT) in maritime industries. In maritime IoT applications, intelligent vessel traffic services can be guaranteed by collecting and analyzing high volume of spatial data flows from automatic identification system (AIS). This AIS system includes a highly integrated automatic equipment including functionalities of core communication, tracking, and sensing. The increased utilization of shipboard AIS devices allows the collection of massive trajectory data. However, the received raw AIS data often suffers from undesirable outliers (i.e., poorly tracked timestamped points for vessel trajectories) during signal acquisition and analog-to-digital conversion. The degraded AIS data will bring negative effects on vessel traffic services (e.g., maritime traffic monitoring, intelligent maritime navigation, and vessel collision avoidance, etc) in maritime IoT scenarios. To improve the quality of vessel trajectory records from AIS networks, we propose to develop a two-phase data-driven machine learning framework for vessel trajectory reconstruction. In particular, a density-based clustering method is introduced in the first phase to automatically recognize the undesirable outliers. The second phase proposes a bidirectional long short-term memory (BLSTM)-based supervised learning technique to restore the timestamped points degraded by random outliers in vessel trajectories. Comprehensive experiments on simulated and realistic datasets have verified the dominance of our two-phase vessel reconstruction framework compared to other competing methods. It thus has the capacity of promoting intelligent vessel traffic services in 6G-enabled maritime IoT systems.