In this paper, we study reinforcement learning (RL) algorithms to solve real-world decision problems with the objective of maximizing the long-term reward as well as satisfying cumulative constraints. We propose a novel first-order policy optimization method, Interior-point Policy Optimization (IPO), which augments the objective with logarithmic barrier functions, inspired by the interior-point method. Our proposed method is easy to implement with performance guarantees and can handle general types of cumulative multi-constraint settings. We conduct extensive evaluations to compare our approach with state-of-the-art baselines. Our algorithm outperforms the baseline algorithms, in terms of reward maximization and constraint satisfaction.

Reinforcement Learning (RL) algorithms have had tremendous success in simulated domains. These algorithms, however, often cannot be directly applied to physical systems, especially in cases where there are constraints to satisfy (e.g. to ensure safety or limit resource consumption). In standard RL, the agent is incentivized to explore any policy with the sole goal of maximizing reward; in the real world, however, ensuring satisfaction of certain constraints in the process is also necessary and essential. In this article, we overview existing approaches addressing constraints in model-free reinforcement learning. We model the problem of learning with constraints as a Constrained Markov Decision Process and consider two main types of constraints: cumulative and instantaneous. We summarize existing approaches and discuss their pros and cons. To evaluate policy performance under constraints, we introduce a set of standard benchmarks and metrics. We also summarize limitations of current methods and present open questions for future research.

With the proliferation of mobile networks, we face strong diversification of services, demanding the current network to embed more flexibility. To satisfy this daring need, network slicing is embraced as a promising solution for resource utilization, in 5G and future networks. In network slicing, dynamic resource orchestration and network slice management are critical for resource efficiency. However, it is highly complicated such that the traditional approaches can not effectively perform resource orchestration due to the lack of accurate models and hidden problem structures. To address this challenge, we propose a constrained reinforcement learning based approach for network slicing. We formulate the resource allocation problem as a Constrained Markov Decision Process (CMDP) and solve it using constrained reinforcement learning algorithms. Specifically, we use the adaptive interior-point policy optimization and policy safety layer methods to deal with cumulative and instantaneous constraints. Our evaluations show that our method is effective in resource allocation with service demand guarantees and significantly outperforms baselines.

With the emergence of smartphones as an essential part of daily life, the demand for user reauthentication has increased manifolds. The effective and widely practiced biometric schemes are based upon the principle of “who you are” which utilizes inherent and unique characteristics of the user. In this context, the behavioral biometrics such as sliding dynamics and pressure intensity make use of on-screen sliding movements to infer the user’s patterns. In this paper, we present Safeguard, an accurate and efficient smartphone user reauthentication (verification) system based upon on-screen finger movements. The computation and processing is performed at back-end which is transparent to the users. The key feature of the proposed system lies in fine-grained on-screen biometric metrics, i.e., sliding dynamics and pressure intensity, which are unique to each user under diverse scenarios. We first implement our scheme through five machine learning approaches and finally select the support vector machine (SVM)-based approach due to its high accuracy. We further analyze Safeguard to be robust against adversary imitation. We validate the efficacy of our approach through implementation on off-the-shelf smartphone followed by practical evaluation under different scenarios. We process a set of more than 50 000 effective samples derived from a raw dataset of over 10 000 slides collected from each of the 60 volunteers over a period of one month. The experimental results show that Safeguard can verify a user with 0.03% false acceptance rate (FAR) and 0.05% false rejection rate (FRR) within 0.3 s with 15 to 20 slides by the user. The FRR of our system adequately meets the European Standard for Access Control Systems, whereas FAR differs by 0.029%. Our future works aim to integrate multitouch sliding movements in existing scheme.

Over last two decades, Speaker Recognition has primarily been focused on source, system, and prosodic features of the speech. The breath, however, has either been treated as a trivial part of the speech, or considered a noise entity. Our observation reveals that breath is a unique fingerprint of human respiratory system which offers overwhelming results for Speaker Recognition. Moreover, its passive nature, short-duration, fewer occurrences and simple processing results to a light-weight, text-independent and transparent system, which we articulate as BreathID.

As mobile networks proliferate, we are experiencing a strong diversification of services, which requires greater flexibility from the existing network. Network slicing is proposed as a promising solution for resource utilization in 5G and future networks to address this dire need. In network slicing, dynamic resource orchestration and network slice management are crucial for maximizing resource utilization. Unfortunately, this process is too complex for traditional approaches to be effective due to a lack of accurate models and dynamic hidden structures. We formulate the problem as a Constrained Markov Decision Process (CMDP) without knowing models and hidden structures. Additionally, we propose to solve the problem using CLARA, a Constrained reinforcement LeArning based Resource Allocation algorithm. In particular, we analyze cumulative and instantaneous constraints using adaptive interior-point policy optimization and projection layer, respectively. Evaluations show that CLARA clearly outperforms baselines in resource allocation with service demand guarantees.