Grant-free non-orthogonal multiple access (GF-NOMA) is a potential multiple access framework for short-packet internet-of-things (IoT) networks to enhance connectivity. However, the resource allocation problem in GF-NOMA is challenging due to the absence of closed-loop power control. We design a prototype of transmit power pool (PP) to provide open-loop power control. IoT users acquire their transmit power in advance from this prototype PP solely according to their communication distances. Firstly, a multi-agent deep Q-network (DQN) aided GF-NOMA algorithm is proposed to determine the optimal transmit power levels for the prototype PP. More specifically, each IoT user acts as an agent and learns a policy by interacting with the wireless environment that guides them to select optimal actions. Secondly, to prevent the Q-learning model overestimation problem, double DQN (DDQN) based GF-NOMA algorithm is proposed. Numerical results confirm that the DDQN based algorithm finds out the optimal transmit power levels that form the PP. Comparing with the conventional online learning approach, the proposed algorithm with the prototype PP converges faster under changing environments due to limiting the action space based on previous learning. The considered GF-NOMA system outperforms the networks with fixed transmission power, namely all the users have the same transmit power and the traditional GF with orthogonal multiple access techniques, in terms of throughput.
A mobile ad hoc network (MANET) is a group of nodes constituting a network of mobile nodes without predefined and pre-established architecture where mobile nodes can communicate without any dedicated access points or base stations. In MANETs, a node may act as a host as well as a router. Nodes in the network can send and receive packets through intermediate nodes. However, the existence of malicious and selfish nodes in MANETs severely degrades network performance. The identification of such nodes in the network and their isolation from the network is a challenging problem. Therefore, in this paper, a simple reputation-based scheme is proposed which uses the consumption and contribution information for selfish node detection and cooperation enforcement. Nodes failing to cooperate are detached from the network to save resources of other nodes with good reputation. The simulation results show that our proposed scheme outperforms the benchmark scheme in terms of NRL (normalized routing load), PDF (packet delivery fraction), and packet drop in the presence of malicious and selfish attacks. Furthermore, our scheme identifies the selfish nodes quickly and accurately as compared to the benchmark scheme.
8 March 2023 in IEEE Transactions on Vehicular Technology
With the use of power domain non-orthogonal multiple access (NOMA) and backscatter communication (BAC), future sixth-generation ultra massive machine-type communications networks are expected to connect large-scale internet of things (IoT) devices. However, due to NOMA co-channel interference, the power allocation to large-scale IoT devices becomes critical. The existing convex optimization-based solutions are highly complex, and therefore it is difficult to find the optimal solution to the resource allocation problem in a highly dynamic environment. To alleviate this problem, this work develops an efficient model-free BAC approach with a NOMA system to assist the base station with complex resource scheduling tasks in a dynamic BAC-NOMA IoT network. The objective is to increase the sum rate of uplink backscatter devices. More specifically, we jointly optimize the transmit power of downlink IoT users and the reflection coefficient of uplink backscatter devices using a reinforcement learning algorithm, namely, the soft-actor critic (SAC) algorithm. With the advantage of entropy regularization, the SAC agent learns to explore and exploit the dynamic BAC-NOMA network efficiently. The proposed algorithm ensures the quality of service (QoS) requirements of downlink users while enhancing the sum rate of uplink backscatter devices. Numerical results reveal the superiority of the proposed algorithm over the conventional optimization (benchmark) approach in terms of the average sum rate of uplink backscatter devices. We show that the network with multiple downlink users obtained a higher reward with respect to a large number of iterations compared to episodes with a lower number of iterations. Moreover, the proposed algorithm outperforms the benchmark scheme and BAC with orthogonal multiple access in terms of the average sum rate with the different number of backscatter devices. Additionally, we show that our proposed algorithm enhances sum rate efficiency with respect to different self-interference coefficients and different noise levels. Finally, we evaluate and show the sum rate efficiency of the proposed algorithm with different QoS requirements and cell radii.
2 February 2024 in IEEE Transactions on Communications
In this paper, we design a resource block (RB) oriented power pool (PP) for semi-grant-free non-orthogonal multiple access (SGF-NOMA) in the presence of residual errors resulting from imperfect successive interference cancellation (SIC). In the proposed method, the BS allocates one orthogonal RB to each grant-based (GB) user, and determines the acceptable received power from grant-free (GF) users and calculates a threshold against this RB for broadcasting. Each GF user as an agent, tries to find the optimal transmit power and RB without affecting the quality-of-service (QoS) and ongoing transmission of the GB user. To this end, we formulate the transmit power and RB allocation problem as a stochastic Markov game to design the desired PPs and maximize the long-term system throughput. The problem is then solved using multi-agent (MA) deep reinforcement learning algorithms, such as double deep Q networks (DDQN) and Dueling DDQN due to their enhanced capabilities in value estimation and policy learning, with the latter performing optimally in environments characterized by extensive states and action spaces. The agents (GF users) undertake actions, specifically adjusting power levels and selecting RBs, in pursuit of maximizing cumulative rewards (throughput). Simulation results indicate computational scalability and minimal signaling overhead of the proposed algorithm with notable gains in system throughput compared to existing SGF-NOMA systems. We examine the effect of SIC error levels on sum rate and user transmit power, revealing a decrease in sum rate and an increase in user transmit power as QoS requirements and error variance escalate. We demonstrate that PPs can benefit new (untrained) users joining the network and outperform conventional SGF-NOMA without PPs in spectral efficiency.
6 December 2022 in IEEE Wireless Communications Letters
We propose a novel semi-centralized framework for Internet-of-Things (IoT) networks with non-orthogonal multiple access to maximize the energy efficiency (EE) of two types of clients, namely grant-based (GB) and grant-free (GF). We use a proximal policy optimization algorithm to maximize the EE of GB clients and a multi-agent deep Q-network to optimize resource allocation for GF clients aided by a gateway node. The proposed algorithm combines the advantages of fully centralized and fully distributed frameworks to compensate for their shortcomings (complexity and long learning time). The numerical results show that the proposed algorithm enhances the EE of GB clients by 6% and 11.5%, respectively, compared with the fixed power allocation and random power allocation strategies. Moreover, the results demonstrate a 47.4% increase in the EE of GF clients over the benchmark scheme. Additionally, we show that the increase in the number of GB clients has a significant impact on the EE of GB and GF clients.
12 August 2024 in IEEE Transactions on Consumer Electronics
The Consumer Internet of Things (CIoT), a key aspect of the IoT, aims to integrate smart technologies into everyday life. In order to improve the spectral efficiency and provide massive connectivity to IoT networks, non-orthogonal multiple access (NOMA) variants like semi-grant-free (SGF) NOMA are employed. This paper aims to maximize secrecy energy efficiency (EE) for SGF-NOMA enabled CIoT in the presence of untrusted users (eavesdroppers) by utilizing a single-agent multi-agent deep reinforcement learning (SAMA-DRL) algorithm to overcome scalability and expensive learning issues. Given the limited long-distance transmission capabilities of CIoT devices, which typically have low transmit power, relay nodes are used to decode and forward data from grant-free (GF) users to the base station. Moreover, to enhance the coverage for GF users, the K-nearest neighbors (KNN) algorithm is utilized to place the relay nodes at an optimal positions. Furthermore, we design a collaborative contribution reward system to discourage user (agent) laziness. Simulation results show that the proposed SAMA-DRL-based SGF-NOMA algorithm for CIoT networks is more effective than baseline algorithms, achieving a 20% increase in secrecy EE compared to DRL-based SGF-NOMA without KNN. Moreover, the proposed scheme outperforms benchmark schemes in terms of EE across different radii. Additionally, we show that the proposed algorithm with quality of service based successive interference cancellation (SIC) is more power efficient as compared to conventional SIC decoding order.