30 January 2018 in IEEE Transactions on Automatic Control
We consider a restless multi-armed bandit in which each arm can be in one of two states. When an arm is sampled, the state of the arm is not available to the sampler. Instead, a binary signal with a known randomness that depends on the state of the arm is available. No signal is available if the arm is not sampled. An arm-dependent reward is accrued from each sampling. In each time step, each arm changes state according to known transition probabilities which in turn depend on whether the arm is sampled or not sampled. Since the state of the arm is never visible and has to be inferred from the current belief and a possible binary signal, we call this the hidden Markov bandit. Our interest is in a policy to select the arm(s) in each time step to maximize the infinite horizon discounted reward. Specifically, we seek the use of Whittle's index in selecting the arms.
1 May 2007 in Random Structures & Algorithms
7 May 2018 in IEEE Transactions on Control of Network Systems
In this paper, a new model is introduced for traffic on broad roads, where the drivers do not follow lane-discipline. For both longitudinal and lateral motion, the driver reactions are assumed to be influenced by possibly a number of vehicles, obstacles and unmodelled entities in visibility cones to the front and to the sides of each vehicle. The network of influences and the resultant interaction is modelled by “influence graphs”. In congested traffic situations, it is assumed that the influence structure is time invariant and all drivers are forced to behave homogeneously. Then the collection converges to a layered formation with fixed inter-vehicle distances. In sparse and heterogeneous traffic, the velocity and inter vehicle separations in the set of modelled vehicles, though can oscillate continuously, are uniformly bounded. These model based predictions are verified experimentally. Videos of typical traffic on a sample road in Mumbai city, India, are recorded. Detailed motion information of groups of cars is extracted through image processing techniques. The proposed model is initialised with the extracted data and the computed trajectories are compared with the actual ones
17 August 2021 in IEEE Transactions on Network and Service Management
Caching content close to the end users, e.g., at cellular base stations (BSs), WiFi access points (APs), and end user devices is known to improve efficiency and effectiveness of content delivery. This motivates the development of caching-as-a-service where edge networks and devices provide storage capacity to content providers, and enable them to strategically populate these caches to improve user experience in the targeted network. In this paper, we describe Wi-Cache, a prototype for providing caching-as-a-service at the WiFi edge. Wi-Cache is an SDN (Software Defined Networking) based distributed content caching system at the WiFi edge that uses storage at the APs for caching content. Wi-Cache caches content on wireless APs and delivers them to mobile clients when they are requested. It allows content providers to have fine-grained control over the AP-caches and also execute efficient content placement and delivery algorithms at the WiFi edge using a set of APIs that are provided by Wi-Cache. We also show the effectiveness of the Wi-Cache system using an extensive set of experiments.