Zone-level occupancy counting is a critical technology for smart buildings and can be used for applications, such as building energy management, surveillance, and public safety. Existing occupancy counting techniques typically require installation of large number of occupancy monitoring sensors inside a building and an established wireless network. In this paper, in order to achieve occupancy counting, we consider the use of Wi-Fi probe requests that are continuously transmitted from Wi-Fi enabled smart devices for discovering nearby access points. To this end, Wi-Fi Pineapple equipment are used for passively capturing ambient probe requests from Wi-Fi devices, such as smart phones and tablets, where no connectivity to a Wi-Fi network is required. This information is then used to localize users within coarsely defined occupancy zones, and subsequently to obtain occupancy count within each zone at different time scales. An interacting multimodel (IMM) Kalman filter technique is developed to improve occupancy counting accuracy. Our numerical results using Wi-Fi data collected at a university building show that the use of Wi-Fi probe requests in conjunction with IMM-based Kalman filters can be a viable solution for zone-level occupancy monitoring in smart buildings.

In this paper, we develop a novel Physical Layer Security (PLS) technique for a Visible Light Communication (VLC) system composed of light fixtures assisted by mirror array sheets serving as Intelligent Reflecting Surfaces (IRS). Our objective is to optimize the Secrecy Capacity (SC) by finding the optimal beamforming (BF) weights equipped at the VLC fixtures and mirror orientations at the mirror array sheet. Due to many design parameters, including the beamforming weights, the mirror orientations, and the mobility of the users, conventional optimization techniques may not be practical to optimize the SC capacity. Therefore, we proposed a Deep Reinforcement Learning (DRL) solution based on Deep Deterministic Policy Gradient (DDPG) algorithm to solve the highly complex SC problem by adjusting the BF weights and mirror orientations. The DDPG-based algorithm provides an optimized solution that can adapt to the large size of design parameters and act fast to the channel variations due to users’ mobility. Our results show that considering both mirror array sheet and BF vectors provide the highest SC for the system. Moreover, we show the effect of changing the mirror arrangements of the mirror array sheet on SC. We conclude that for a fixed mirror array sheet size, there exists a specific mirror arrangement (i.e., number of mirrors) that optimizes the SC. After this number, the performance of SC saturates. We also show the trade-off between the training complexity and SC performance considering different mirror arrangements in the mirror array sheet.

Passive radio-frequency identification (RFID) systems carry critical importance for Internet of Things (IoT) applications due to their energy harvesting capabilities. RFIDbased position estimation, in particular, is expected to facilitate a wide array of location-based services for IoT applications with low-power requirements. In this paper, considering monostatic and bistatic configurations and 3-D antenna radiation pattern, we investigate the accuracy of received signal strength-based wireless localization using passive ultra high frequency RFID systems. The Cramer-Rao lower bound (CRLB) for the localization accuracy is derived, and is compared with the accuracy of maximum likelihood estimators for various RFID antenna configurations. Numerical results show that due to RFID tag/antenna sensitivity, and the directional antenna pattern, the localization accuracy can degrade at blind locations that remain outside of the RFID reader antennas' main beam patterns. In such cases optimizing elevation angle of antennas are shown to improve localization coverage, while using bistatic configuration improves localization accuracy significantly.

Hybrid radio frequency (RF) and visible light communication (VLC) networks can provide high throughput and energy efficiency with VLC access points (APs) while ensuring ubiquitous coverage with RF APs. Due to dynamic channel conditions and limited resources, the hybrid RF/VLC networks’ resource allocation problem is complex and challenging. Conventional resource allocation techniques fail to overcome these challenges. Heuristic methods can solve high complexity problems; however, they are not robust against changes such as dynamic channel conditions or alternating user requirements. Heuristic methods require centralized control for stability which adds communication overhead between APs. Deep Reinforcement Learning (DRL) based solutions can solve high complexity, dynamic channel conditions, and alternating user requirements while not requiring centralized control. In this paper, we formulate a distributed downlink power allocation problem to optimize the transmit power for users to reach target data rates in hybrid RF/VLC networks. Then, we propose a distributed DRL-based algorithm Deep Deterministic Policy Gradient (DDPG), to solve the formulated computationally-intensive problem. We implement a simulation environment to benchmark the proposed distributed DRL-based method against other methods such as Q-Learning (QL) and Deep Q-Networks (DQN), and centralized heuristic power allocation algorithms. Our simulation results show that the distributed DDPG-based algorithm learns to adapt against changes in the channel or user requirements, while centralized Genetic Algorithm and Particle Swarm Optimization-based algorithms fail to endure against these changes even with coordination between APs. Additionally, we quantify the performance of the DDPG-based algorithm to prevail amid DRL-based algorithms at the expense of higher implementation complexity.

Aiming at protecting device data privacy, Federated Learning (FL) is a framework of distributed machine learning in which devices’ local model parameters are exchanged with a centralized server without revealing the actual data. Hierarchical Federated Learning (HFL) framework was introduced to improve FL communication efficiency where devices are clustered and seek model consensus with the support of edge servers (e.g., base stations). Devices in a cluster submit their local model updates to their assigned local edge server for aggregation at each iteration. The edge servers transmit the aggregated models to a centralized server and establish a global consensus. However, similar to FL, adversaries may threaten the security and privacy of HFL. The client devices within a cluster may deliberately provide unreliable local model updates through poisoning attacks or poor-quality model updates due to inconsistent communication channels, increased device mobility, or inadequate device resources. To address the above challenges, this paper investigates the client selection problem in the HFL framework to eliminate the impact of unreliable clients while maximizing the global model accuracy of HFL. Each FL edge server is equipped with a Deep Reinforcement Learning (DRL)-based reputation model to optimally measure the reliability and trustworthiness of FL workers within its cluster. A Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is utilized to enhance the accuracy and stability of the HFL global model, given the workers’ dynamic behaviors in the HFL environment. The experimental results indicate that our proposed MADDPG improves the accuracy and stability of HFL compared with the conventional reputation model and single-agent DDPG-based reputation model.

Federated learning (FL) has emerged to leverage datasets from multiple devices to improve the performance of a machine learning (ML) model while providing privacy preservation for devices. The training data is collected at the devices, also known as FL workers, which collaboratively train a global learning model and share their local model updates with a central entity or server without sharing their data. However, FL can be susceptible to various adversarial attacks that target its security and privacy. In particular, the workers can upload unreliable local model updates, leading to corruption of the main FL task. Workers may intentionally contribute unreliable local updates by launching poisoning attacks or unintentionally by updating low-quality models caused by high device mobility, limited device resources, or unstable network connection. Consequently, identifying reliable and trustworthy workers becomes critical for FL security. In this article, the concept of reputation is adopted as a metric to evaluate workers’ reliability and trustworthiness. In addition, deep reinforcement learning (DRL)-based reputation mechanism is proposed for optimal selection and evaluation of reliable FL workers. Due to the dynamic nature of worker behavior in the FL environment, the DRL-based algorithm deep deterministic policy gradient (DDPG) is employed to improve the FL model accuracy and stability. We compare the performance of our proposed method with a conventional reputation method and deep $Q$ -networks (DQNs)-based reputation method. Our simulation results demonstrate that our proposed method can improve FL accuracy by more than 30% under various scenarios and achieves better convergence than the other methods.