This page has only limited features, please log in for full access.
In scenarios, like critical public safety communication networks, On-Scene Available (OSA) user equipment (UE) may be only partially connected with the network infrastructure, e.g., due to physical damages or on-purpose deactivation by the authorities. In this work, we consider multi-hop Device-to-Device (D2D) communication in a hybrid infrastructure where OSA UEs connect to each other in a seamless manner in order to disseminate critical information to a deployed command center. The challenge that we address is to simultaneously keep the OSA UEs alive as long as possible and send the critical information to a final destination (e.g., a command center) as rapidly as possible, while considering the heterogeneous characteristics of the OSA UEs. We propose a dynamic adaptation approach based on machine learning to improve a joint energy-spectral efficiency (ESE). We apply a Q-learning scheme in a hybrid fashion (partially distributed and centralized) in learner agents (distributed OSA UEs) and scheduler agents (remote radio heads or RRHs), for which the next hop selection and RRH selection algorithms are proposed. Our simulation results show that the proposed dynamic adaptation approach outperforms the baseline system by approximately 67% in terms of joint energy-spectral efficiency, wherein the energy efficiency of the OSA UEs benefit from a gain of approximately 30%. Finally, the results show also that our proposed framework with C-RAN reduces latency by approximately 50% w.r.t. the baseline.
Muhidul Islam Khan; Luca Reggiani; Muhammad Mahtab Alam; Yannick Le Moullec; Navuday Sharma; Elias Yaacoub; Maurizio Magarini. Q-Learning Based Joint Energy-Spectral Efficiency Optimization in Multi-Hop Device-to-Device Communication. Sensors 2020, 20, 6692 .
AMA StyleMuhidul Islam Khan, Luca Reggiani, Muhammad Mahtab Alam, Yannick Le Moullec, Navuday Sharma, Elias Yaacoub, Maurizio Magarini. Q-Learning Based Joint Energy-Spectral Efficiency Optimization in Multi-Hop Device-to-Device Communication. Sensors. 2020; 20 (22):6692.
Chicago/Turabian StyleMuhidul Islam Khan; Luca Reggiani; Muhammad Mahtab Alam; Yannick Le Moullec; Navuday Sharma; Elias Yaacoub; Maurizio Magarini. 2020. "Q-Learning Based Joint Energy-Spectral Efficiency Optimization in Multi-Hop Device-to-Device Communication." Sensors 20, no. 22: 6692.
Blockchain technology has been used recently as a secure method for authenticating digital information in many applications. Inspired by the success of the technology, we envision the potential of the blockchain for secured communication in a decentralised Internet of things (IoT). In this paper, we envisage a framework for a secured IoT and describe the infrastructure and mechanism of the entire system. Also, we provide solutions to overcome some of the limitations of blockchain technology including miner selection and reaching consensus, for a decentralised IoT by incorporating a learning-to-rank method for node selection. We also contemplate using hybrid consensus algorithm in the blockchain to detect faulty node and to improve the node convergence.
Muhidul Islam Khan; Isah A. Lawal. Sec-IoT: A Framework for Secured Decentralised IoT Using Blockchain-Based Technology. Proceedings of the 2nd International Conference on Data Engineering and Communication Technology 2020, 269 -277.
AMA StyleMuhidul Islam Khan, Isah A. Lawal. Sec-IoT: A Framework for Secured Decentralised IoT Using Blockchain-Based Technology. Proceedings of the 2nd International Conference on Data Engineering and Communication Technology. 2020; ():269-277.
Chicago/Turabian StyleMuhidul Islam Khan; Isah A. Lawal. 2020. "Sec-IoT: A Framework for Secured Decentralised IoT Using Blockchain-Based Technology." Proceedings of the 2nd International Conference on Data Engineering and Communication Technology , no. : 269-277.
The vision of Internet of Things (IoT) is to enable systems across the globe to share data using advanced communication technologies. With the recent technological advancements, IoT-based solutions are no longer a challenging vision. IoT will offer numerous, and potentially revolutionary, benefits to today’s digital world. Future personalized and connected heathcare is one of the promising area to see the benefits of IoT. This paper surveys emerging healthcare applications, including detailed technical aspects required for the realization of a complete endto- end solution for each application. The survey explores the key application-specific requirements from the perspective of communication technologies. Furthermore, a detailed exploration from the existing to the emerging technologies and standards that would enable such applications is presented, highlighting the critical consideration of short-range and long-range communications. Finally, the survey highlights important open research challenges and issues specifically related to IoT-based future healthcare systems.
Muhammad Mahtab Alam; Hassan Malik; Muhidul Islam Khan; Tamas Pardy; Alar Kuusik; Yannick Le Moullec. A Survey on the Roles of Communication Technologies in IoT-Based Personalized Healthcare Applications. IEEE Access 2018, 6, 36611 -36631.
AMA StyleMuhammad Mahtab Alam, Hassan Malik, Muhidul Islam Khan, Tamas Pardy, Alar Kuusik, Yannick Le Moullec. A Survey on the Roles of Communication Technologies in IoT-Based Personalized Healthcare Applications. IEEE Access. 2018; 6 (99):36611-36631.
Chicago/Turabian StyleMuhammad Mahtab Alam; Hassan Malik; Muhidul Islam Khan; Tamas Pardy; Alar Kuusik; Yannick Le Moullec. 2018. "A Survey on the Roles of Communication Technologies in IoT-Based Personalized Healthcare Applications." IEEE Access 6, no. 99: 36611-36631.
Device-to-device (D2D) communication is an essential feature for the future cellular networks as it increases spectrum efficiency by reusing resources between cellular and D2D users. However, the performance of the overall system can degrade if there is no proper control over interferences produced by the D2D users. Efficient resource allocation among D2D User equipments (UE) in a cellular network is desirable since it helps to provide a suitable interference management system. In this paper, we propose a cooperative reinforcement learning algorithm for adaptive resource allocation, which contributes to improving system throughput. In order to avoid selfish devices, which try to increase the throughput independently, we consider cooperation between devices as promising approach to significantly improve the overall system throughput. We impose cooperation by sharing the value function/learned policies between devices and incorporating a neighboring factor. We incorporate the set of states with the appropriate number of system-defined variables, which increases the observation space and consequently improves the accuracy of the learning algorithm. Finally, we compare our work with existing distributed reinforcement learning and random allocation of resources. Simulation results show that the proposed resource allocation algorithm outperforms both existing methods while varying the number of D2D users and transmission power in terms of overall system throughput, as well as D2D throughput by proper Resource block (RB)-power level combination with fairness measure and improving the Quality of service (QoS) by efficient controlling of the interference level.
Muhidul Islam Khan; Muhammad Mahtab Alam; Yannick Le Moullec; Elias Yaacoub. Throughput-Aware Cooperative Reinforcement Learning for Adaptive Resource Allocation in Device-to-Device Communication. Future Internet 2017, 9, 72 .
AMA StyleMuhidul Islam Khan, Muhammad Mahtab Alam, Yannick Le Moullec, Elias Yaacoub. Throughput-Aware Cooperative Reinforcement Learning for Adaptive Resource Allocation in Device-to-Device Communication. Future Internet. 2017; 9 (4):72.
Chicago/Turabian StyleMuhidul Islam Khan; Muhammad Mahtab Alam; Yannick Le Moullec; Elias Yaacoub. 2017. "Throughput-Aware Cooperative Reinforcement Learning for Adaptive Resource Allocation in Device-to-Device Communication." Future Internet 9, no. 4: 72.
One of the challenging issues in a distributed computing system is to reach on a decision with the presence of so many faulty nodes. These faulty nodes may update the wrong information, provide misleading results and may be nodes with the depleted battery power. Consensus algorithms help to reach on a decision even with the faulty nodes. Every correct node decides some values by a consensus algorithm. If all correct nodes propose the same value then all the nodes decide on that. Every correct nodes must agree on the same value. Faulty nodes do not reach on the decision that correct nodes agreed on. Binary consensus algorithm and average consensus algorithm are the most widely used consensus algorithm in a distributed system. We apply binary consensus and average consensus algorithm in a distributed sensor network with the presence of some faulty nodes. We evaluate these algorithms for better convergence rate and error rate.
Muhidul Islam Khan; Rajkin Hossain. Efficient and fast convergent consensus algorithms for faulty nodes tracking in distributed wireless sensor networks. 2016 3rd International Conference on Electrical Engineering and Information Communication Technology (ICEEICT) 2017, 1 -6.
AMA StyleMuhidul Islam Khan, Rajkin Hossain. Efficient and fast convergent consensus algorithms for faulty nodes tracking in distributed wireless sensor networks. 2016 3rd International Conference on Electrical Engineering and Information Communication Technology (ICEEICT). 2017; ():1-6.
Chicago/Turabian StyleMuhidul Islam Khan; Rajkin Hossain. 2017. "Efficient and fast convergent consensus algorithms for faulty nodes tracking in distributed wireless sensor networks." 2016 3rd International Conference on Electrical Engineering and Information Communication Technology (ICEEICT) , no. : 1-6.
Wireless sensor networks are creating a new era of pervasive computing applications, such as various monitoring and tracking system. The sensor network consists of so many tiny sensor nodes that have so many critical challenges, since they are battery operated and have limited processing capabilities. Binary sensor networks are modeled in a way that the sensor nodes can communicate with the only 1 b of information. One of the challenges in a binary sensor network is to localize the multiple sources. Very few works have been done considering this challenge. Localization failure may cause the whole system useless. We propose a multiple source localization method. We convert the localization problem into an optimization problem, and we solve that optimization problem using primal dual interior point method. Simulation results show that our proposed method provides better performance in every perspective compared with the existing works.
Muhidul Islam Khan; Kewen Xia. Effective Self Adaptive Multiple Source Localization Technique by Primal Dual Interior Point Method in Binary Sensor Networks. IEEE Communications Letters 2017, 21, 1119 -1122.
AMA StyleMuhidul Islam Khan, Kewen Xia. Effective Self Adaptive Multiple Source Localization Technique by Primal Dual Interior Point Method in Binary Sensor Networks. IEEE Communications Letters. 2017; 21 (5):1119-1122.
Chicago/Turabian StyleMuhidul Islam Khan; Kewen Xia. 2017. "Effective Self Adaptive Multiple Source Localization Technique by Primal Dual Interior Point Method in Binary Sensor Networks." IEEE Communications Letters 21, no. 5: 1119-1122.
Success of any supply chain highly depends on the appropriate use of transportation. It is always a tough decision to select an optimal route in the stages of a supply chain that will balance the reverse condition of cost and time. Q-learning is already getting implied to optimize routes, but it greatly lacks in designing a complete Markov Decision Process (MDP) with enough number of states and actions sets. In this paper, we propose an MDP that can fit the design of a proper multi-stage supply chain network and intend to solve the MDP with SARSA (λ) algorithm. The algorithm was simulated in a java based environment and experimental results show that Q-learning produced higher chunks of rewards but SARSA (λ) was faster in case of convergence speed.
Arafat Habib; Muhidul Islam Khan; Jia Uddin. Optimal route selection in complex multi-stage supply chain networks using SARSA(λ). 2016 19th International Conference on Computer and Information Technology (ICCIT) 2016, 170 -175.
AMA StyleArafat Habib, Muhidul Islam Khan, Jia Uddin. Optimal route selection in complex multi-stage supply chain networks using SARSA(λ). 2016 19th International Conference on Computer and Information Technology (ICCIT). 2016; ():170-175.
Chicago/Turabian StyleArafat Habib; Muhidul Islam Khan; Jia Uddin. 2016. "Optimal route selection in complex multi-stage supply chain networks using SARSA(λ)." 2016 19th International Conference on Computer and Information Technology (ICCIT) , no. : 170-175.
One of the challenging issues in a distributed computing system is to reach on a decision with the presence of so many faulty nodes. These faulty nodes may update the wrong information, provide misleading results and may be nodes with the depleted battery power. Consensus algorithms help to reach on a decision even with the faulty nodes. Every correct node decides some values by a consensus algorithm. If all correct nodes propose the same value, then all the nodes decide on that. Every correct node must agree on the same value. Faulty nodes do not reach on the decision that correct nodes agreed on. Binary consensus algorithm and average consensus algorithm are the most widely used consensus algorithm in a distributed system. We apply binary consensus and average consensus algorithm in a distributed sensor network with the presence of some faulty nodes. We evaluate these algorithms for better convergence rate and error rate. Keywords Wireless sensor networks Consensus algorithm Distributed systems Convergence rate Faulty node tracking Binary consensus Average consensus
Rajkin Hossain; Muhidul Islam Khan. Efficient consensus algorithm for the accurate faulty node tracking with faster convergence rate in a distributed sensor network. EURASIP Journal on Wireless Communications and Networking 2016, 2016, 393 .
AMA StyleRajkin Hossain, Muhidul Islam Khan. Efficient consensus algorithm for the accurate faulty node tracking with faster convergence rate in a distributed sensor network. EURASIP Journal on Wireless Communications and Networking. 2016; 2016 (1):393.
Chicago/Turabian StyleRajkin Hossain; Muhidul Islam Khan. 2016. "Efficient consensus algorithm for the accurate faulty node tracking with faster convergence rate in a distributed sensor network." EURASIP Journal on Wireless Communications and Networking 2016, no. 1: 393.
Cloud computing is a rapidly emerging field, services and applications are more or less 24/7. Resource dimensioning in this field is a great issue. Research is already going on to imply reinforcement learning to automate decision making process in case of addition, reduction, migration and maintenance of the Virtual Machines (VM) to balance the service level performance and VM management cost. Models have been proposed in this case based on Q-learning, a very popular reinforcement learning technique that is used to find optimal action selection policy for any finite Markov Decision Process (MDP). In this paper we propose to work with the challenges like proper initialization of the early stages, designing the states, actions, transitions using Markov Decision Process (MDP) and solving the MDP with two popular reinforcement learning techniques, Q-learning and SARSA(λ).
Arafat Habib; Muhidul Islam Khan. Reinforcement learning based autonomic virtual machine management in clouds. 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV) 2016, 1083 -1088.
AMA StyleArafat Habib, Muhidul Islam Khan. Reinforcement learning based autonomic virtual machine management in clouds. 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV). 2016; ():1083-1088.
Chicago/Turabian StyleArafat Habib; Muhidul Islam Khan. 2016. "Reinforcement learning based autonomic virtual machine management in clouds." 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV) , no. : 1083-1088.
A wireless sensor network (WSN) is composed of a large number of tiny sensor nodes. Sensor nodes are very resource-constrained, since nodes are often battery-operated and energy is a scarce resource. In this paper, a resource-aware task scheduling (RATS) method is proposed with better performance/resource consumption trade-off in a WSN. Particularly, RATS exploits an adversarial bandit solver method called exponential weight for exploration and exploitation (Exp3) for target tracking application of WSN. The proposed RATS method is compared and evaluated with the existing scheduling methods exploiting online learning: distributed independent reinforcement learning (DIRL), reinforcement learning (RL), and cooperative reinforcement learning (CRL), in terms of the tracking quality/energy consumption trade-off in a target tracking application. The communication overhead and computational effort of these methods are also computed. Simulation results show that the proposed RATS outperforms the existing methods DIRL and RL in terms of achieved tracking performance.
Muhidul Islam Khan. Resource-aware task scheduling by an adversarial bandit solver method in wireless sensor networks. EURASIP Journal on Wireless Communications and Networking 2016, 2016, 10 .
AMA StyleMuhidul Islam Khan. Resource-aware task scheduling by an adversarial bandit solver method in wireless sensor networks. EURASIP Journal on Wireless Communications and Networking. 2016; 2016 (1):10.
Chicago/Turabian StyleMuhidul Islam Khan. 2016. "Resource-aware task scheduling by an adversarial bandit solver method in wireless sensor networks." EURASIP Journal on Wireless Communications and Networking 2016, no. 1: 10.
Long term evolution (LTE) network, incompatible with 2G and 3G networks is the most promising technology for wireless communication with higher speed and capacity. Self-organized load balancing is an important research issue for the wireless networks. Game theory provides an efficient way to provide self-organizing properties in a distributed environment like LTE networks. Load balancing means to assign users from highly loaded cells to neighbor lower loaded cells. The amount of load needs to be offloaded or accepted by a particular cell is not really specified and currently totally vendor specified. In our proposed cooperative game theoretic approach, each cell is considered as a player where they trade the load by forming a coalition by satisfying the overall performance of the network. Simulation results show that our proposed method provides better performance in terms of satisfied users and adjusted load values.
Subarno Saha; Rajkin Hossain; Muhidul Islam Khan. Cooperative game theory based load balancing in long term evolution network. 2015 International Conference on Computer and Information Engineering (ICCIE) 2015, 154 -157.
AMA StyleSubarno Saha, Rajkin Hossain, Muhidul Islam Khan. Cooperative game theory based load balancing in long term evolution network. 2015 International Conference on Computer and Information Engineering (ICCIE). 2015; ():154-157.
Chicago/Turabian StyleSubarno Saha; Rajkin Hossain; Muhidul Islam Khan. 2015. "Cooperative game theory based load balancing in long term evolution network." 2015 International Conference on Computer and Information Engineering (ICCIE) , no. : 154-157.
Wireless sensor networks (WSNs) are an attractive platform for monitoring and measuring physical phenomena. WSNs typically consist of hundreds or thousands of battery-operated tiny sensor nodes which are connected via a low data rate wireless network. A WSN application, such as object tracking or environmental monitoring, is composed of individual tasks which must be scheduled on each node. Naturally the order of task execution influences the performance of the WSN application. Scheduling the tasks such that the performance is increased while the energy consumption remains low is a key challenge. In this paper we apply online learning to task scheduling in order to explore the tradeoff between performance and energy consumption. This helps to dynamically identify effective scheduling policies for the sensor nodes. The energy consumption for computation and communication is represented by a parameter for each application task. We compare resource-aware task scheduling based on three online learning methods: independent reinforcement learning (RL), cooperative reinforcement learning (CRL), and exponential weight for exploration and exploitation (Exp3). Our evaluation is based on the performance and energy consumption of a prototypical target tracking application. We further determine the communication overhead and computational effort of these methods.
Muhidul Islam Khan; Bernhard Rinner. Performance Analysis of Resource-Aware Task Scheduling Methods in Wireless Sensor Networks. International Journal of Distributed Sensor Networks 2014, 10, 1 .
AMA StyleMuhidul Islam Khan, Bernhard Rinner. Performance Analysis of Resource-Aware Task Scheduling Methods in Wireless Sensor Networks. International Journal of Distributed Sensor Networks. 2014; 10 (9):1.
Chicago/Turabian StyleMuhidul Islam Khan; Bernhard Rinner. 2014. "Performance Analysis of Resource-Aware Task Scheduling Methods in Wireless Sensor Networks." International Journal of Distributed Sensor Networks 10, no. 9: 1.
Wireless sensor networks (WSN) are an attractive platform for cyber physical systems. A typical WSN application is composed of different tasks which need to be scheduled on each sensor node. However, the severe energy limitations pose a particular challenge for developing WSN applications, and the scheduling of tasks has typically a strong influence on the achievable performance and energy consumption. In this paper we propose a method for scheduling the tasks using cooperative reinforcement learning (RL) where each node determines the next task based on the observed application behavior. In this RL framework we can trade the application performance and the required energy consumption by a weighted reward function and can therefore achieve different energy/performance results of the overall application. By exchanging data among neighboring nodes we can further improve this energy/performance trade-off. We evaluate our approach in an target tracking application. Our simulations show that cooperative approaches are superior to non-cooperative approaches for this kind of applications.
Muhidul Islam Khan; Bernhard Rinner. Energy-aware task scheduling in wireless sensor networks based on cooperative reinforcement learning. 2014 IEEE International Conference on Communications Workshops (ICC) 2014, 871 -877.
AMA StyleMuhidul Islam Khan, Bernhard Rinner. Energy-aware task scheduling in wireless sensor networks based on cooperative reinforcement learning. 2014 IEEE International Conference on Communications Workshops (ICC). 2014; ():871-877.
Chicago/Turabian StyleMuhidul Islam Khan; Bernhard Rinner. 2014. "Energy-aware task scheduling in wireless sensor networks based on cooperative reinforcement learning." 2014 IEEE International Conference on Communications Workshops (ICC) , no. : 871-877.
Wireless Sensor Networks (WSN) consists of small sensor devices with sensing, processing and communication capabilities. Sensor nodes are operated by batteries. As the replacement of these batteries are not practical, this network is very much energy sensitive. Resource coordination is an important issue to make this system energy efficient. Sensor nodes can be applied in various applications. Object tracking, routing, event detection are some common applications in WSN. These application needs to perform some tasks like sensing, transmitting, sleeping, receiving etc. At each time step, the sensor nodes need to perform one task based on its application demand. Scheduling of these tasks is very important aspect for WSN in order to coordinate the resources. In this paper, an effective market based method is proposed for resource coordination in WSN. At first the description of the problem is presented then the combinatorial auction based method is proposed. The simulation results show the efficiency of the proposed method comparing with other existing methods.
Muhidul Islam Khan; Bernhard Rinner; Carlo Regazzoni. Resource coordination in Wireless Sensor Networks by combinatorial auction based method. 2012 IEEE 3rd International Conference on Networked Embedded Systems for Every Application (NESEA) 2012, 1 -6.
AMA StyleMuhidul Islam Khan, Bernhard Rinner, Carlo Regazzoni. Resource coordination in Wireless Sensor Networks by combinatorial auction based method. 2012 IEEE 3rd International Conference on Networked Embedded Systems for Every Application (NESEA). 2012; ():1-6.
Chicago/Turabian StyleMuhidul Islam Khan; Bernhard Rinner; Carlo Regazzoni. 2012. "Resource coordination in Wireless Sensor Networks by combinatorial auction based method." 2012 IEEE 3rd International Conference on Networked Embedded Systems for Every Application (NESEA) , no. : 1-6.
Wireless Sensor Networks (WSN) typically operate in dynamic environments, hence we can not schedule the execution of tasks a priori. This must be done online in a way to minimize the resource consumption. We present a cooperative reinforcement learning approach to schedule the tasks in WSN. A WSN is composed of a large number of tiny sensing nodes capable of interacting with the environment, communicating wirelessly and perform limited processing. In every time step, the sensor nodes need to take decision about some tasks to perform. Our proposed algorithm helps sensor nodes to learn the usefulness of each task based on reinforcement learning. We present an object tracking application with online scheduling of tasks based on our proposed approach. Our simulation studies show a more efficient task scheduling than traditional resource management schemes such as static scheduling, random scheduling and independent reinforcement learning based scheduling of tasks.
Muhidul Islam Khan; Bernhard Rinner. Resource coordination in wireless sensor networks by cooperative reinforcement learning. 2012 IEEE International Conference on Pervasive Computing and Communications Workshops 2012, 895 -900.
AMA StyleMuhidul Islam Khan, Bernhard Rinner. Resource coordination in wireless sensor networks by cooperative reinforcement learning. 2012 IEEE International Conference on Pervasive Computing and Communications Workshops. 2012; ():895-900.
Chicago/Turabian StyleMuhidul Islam Khan; Bernhard Rinner. 2012. "Resource coordination in wireless sensor networks by cooperative reinforcement learning." 2012 IEEE International Conference on Pervasive Computing and Communications Workshops , no. : 895-900.