Reinforcement Learning (RL) is a widely researched area in artificial intelligence that focuses on teaching agents decision-making through interactions with their environment. A key subset includes multi-armed bandit (MAB) and stochastic continuum-armed bandit (SCAB) problems, which model sequential decision-making under uncertainty. This review outlines the foundational models and assumptions of bandit problems, explores non-asymptotic theoretical tools like concentration inequalities and minimax regret bounds, and compares frequentist and Bayesian algorithms for managing exploration–exploitation trade-offs. Additionally, we explore K-armed contextual bandits and SCAB, focusing on their methodologies and regret analyses. We also examine the connections between SCAB problems and functional data analysis. Finally, we highlight recent advances and ongoing challenges in the field.
Wavelength selective switches (WSSs) have been widely deployed in reconfigurable optical add/drop multiplexers (ROADM) networks. The demand for better and more intelligent connectivity drives WSS technology to higher port count, wider spectral coverage and better performance. Due to the complicated fringing field effect within the switching engine, i.e. the liquid crystal on silicon (LCOS) device, of the WSS modules, this requires sophisticated design and calibration of the beam-steering holograms. This process is time-consuming and requires expensive testing equipment, contributing to a significant portion of the manufacturing costs. In this paper, we demonstrated a few-shot transfer learning method that was able to speed up this calibration and optimisation process while meeting the stringent performance requirement. In the proposed process, a digital twin model of the WSS systems was developed to generate the dataset for the training of a convolutional recurrent neural network (RNN). This pre-trained neural network was able to simulate the optimised hologram designs for all the switching scenarios. Subsequently, a very limited experimental characterisation of the actual WSS was performed to update this pre-trained model to reflect the inevitable discrepancy between the digital twin model and the actual system. It was demonstrated that transfer learning reduced the experimental calibration workload by 88% and the training time by 30% while achieving >25 dB worst-case port isolation levels across all the switching scenarios.
7 October 2024 in Journal of Optical Communications and Networking
This paper investigates the practical scalability of wavelength selective switching technology for the emerging multi-band and space-division-multiplexed (SDM) networks. Wavelength selective switching architectures are introduced for multi-band SDM networks. The switching capacity is analyzed for both weakly coupled and strongly coupled SDM networks. Key bottlenecks for scaling up toward multi-band and more spatial modes are identified. Contrary to the conventional view that liquid crystal on silicon (LCOS) was the only technological obstacle, the manufacturability of free-space optics with high numerical apertures and constraints on the optical dimensions also brought significant challenges for the development of highly integrated wavelength selective switches for multi-band SDM networks.
Mediation analysis is a useful tool to study the mechanism of how a treatment exerts effects on the outcome. Classical mediation analysis requires a sequential ignorability assumption to rule out cross-world reliance of the potential outcome of interest on the counterfactual mediator in order to identify the natural direct and indirect effects. In recent years, the separable effects framework has adopted dismissible treatment components to identify the separable direct and indirect effects. In this article, we compare the sequential ignorability and dismissible treatment components for longitudinal outcomes and time-to-event outcomes with time-varying confounding and random censoring. We argue that the dismissible treatment components assumption has advantages in interpretation and identification over sequential ignorability, whereas these two conditions lead to identical estimators for the direct and indirect effects. As an illustration, we study the effect of transplant modalities on overall survival mediated by leukemia relapse in patients undergoing allogeneic stem cell transplantation. We find that Haplo-SCT reduces the risk of overall mortality through reducing the risk of relapse, and Haplo-SCT can serve as an alternative to MSDT in allogeneic stem cell transplantation.
This paper examines the cluster consensus design problem on higher-order interaction networks. Specifically, the higher-order interaction mechanism is captured by matrix-weighted networks that allow the interdependency across the dimensions of the agents' states, and the matrix-valued weight matrices $A_{ij}\in \mathbb{R}^{d\times d}$ associated with specific edges are further assumed to share the same nullspace for design purposes. Under mild assumptions on network connectivity, we first examine the case that the nullspace of positive semi-definite edges is spanned by a nonzero vector $\boldsymbol{\xi}\in \mathbb{R}^{d}$ and show that the predictable cluster consensus can be achieved, which is eventually located in the 1-dimensional linear space determined by span $\{\boldsymbol{\xi}\}$ and the average of agents' initial states. Moreover, the transient state of agents in each cluster can also be explicitly characterized. Namely, the derivative of the average state of agents in each cluster is perpendicular to span $\{\boldsymbol{\xi}\}$ . To generalize the above results, we proceed to examine the case that the nullspace of positive semi-definite edges is spanned by more than one linearly independent $d$ -dimensional vector, in which case, analogous results can be obtained, and the explicit geometric interpretation is also provided.