In the last few years, deep reinforcement learning has been proposed as a method to perform online learning in energy-efficiency scenarios such as HVAC control, electric car energy management, or building energy management, just to mention a few. On the other hand, quantum machine learning was born during the last decade to extend classic machine learning to a quantum level. In this work, we propose to study the benefits and limitations of quantum reinforcement learning to solve energy-efficiency scenarios. As a testbed, we use existing energy-efficiency-based reinforcement learning simulators and compare classic algorithms with the quantum proposal. Results in HVAC control, electric vehicle fuel consumption, and profit optimization of electrical charging stations applications suggest that quantum neural networks are able to solve problems in reinforcement learning scenarios with better accuracy than their classical counterpart, obtaining a better cumulative reward with fewer parameters to be learned.

Quantum neural networks constitute one of the most promising applications of Quantum Machine Learning, as they leverage both the capabilities of classical neural networks and the unique advantages of quantum mechanics. Moreover, quantum mechanics has demonstrated its ability to detect atypical patterns in data that are challenging for classical approaches to recognize. However, despite their potential, there are still open questions such as barren plateau phenomenon and the challenges of scalability and the curse of dimensionality, which become particularly relevant in Reinforcement Learning (RL) when working in environments with high-dimensional state and action spaces. This study delves into the critical realm of representing classical data as quantum states, a topic of keen interest across the scientific community. The aim is to construct streamlined circuits for efficient execution on quantum computers and simulators using minimal qubits and entanglement gates to evade barren plateau phenomena and reducing computational times. Our investigation examines and validates the efficacy of three strategies for data management and dimensionality reduction in real-world, large-scale environments for Quantum Reinforcement Learning, particularly in energy efficiency scenarios. The techniques encompass amplitude encoding, linear layer preprocessing, and data reuploading, supplemented by trainable parameters. This research sheds light on the potential of quantum machine learning in enhancing real-world environments, including energy efficiency scenarios and showcases the capabilities of quantum neural networks in the reinforcement learning landscape.

Combinatorial optimization problems are ubiquitous in industry. In addition to finding a solution with minimum cost, problems of high relevance involve a number of constraints that the solution must satisfy. Variational quantum algorithms (VQAs) have emerged as promising candidates for solving these problems in the noisy intermediate-scale quantum stage. However, the constraints are often complex enough to make their efficient mapping to quantum hardware difficult or even infeasible. An alternative standard approach is to transform the optimization problem to include these constraints as penalty terms, but this method involves additional hyperparameters and does not ensure that the constraints are satisfied due to the existence of local minima. In this paper, we introduce a new method for solving combinatorial optimization problems with challenging constraints using VQAs. We propose the multi-objective variational constrained optimizer (MOVCO) to classically update the variational parameters by a multiobjective optimization performed by a genetic algorithm. This optimization allows the algorithm to progressively sample only states within the in-constraints space, while optimizing the energy of these states. We test our proposal on a real-world problem with great relevance in finance: the cash handling problem. We introduce a novel mathematical formulation for this problem, and compare the performance of MOVCO versus a penalty based optimization. Our empirical results show a significant improvement in terms of the cost of the achieved solutions, but especially in the avoidance of local minima that do not satisfy any of the mandatory constraints.

In recent years, advancements in brain science and neuroscience have significantly influenced the field of computer science, particularly in the domain of reinforcement learning (RL). Drawing insights from neurobiology and neuropsychology, researchers have leveraged these findings to develop novel mechanisms for understanding intelligent decision-making processes in the brain. Concurrently, the emergence of quantum computing has opened new frontiers in artificial intelligence, leading to the development of quantum machine learning (QML). This study introduces a novel model that integrates quantum spiking neural networks (QSNN) and quantum long short-term memory (QLSTM) architectures, inspired by the complex workings of the human brain. Specifically designed for reinforcement learning tasks in energy-efficient environments, our approach progresses through two distinct stages mirroring sensory and memory systems. In the initial stage, analogous to the brain’s hypothalamus, low-level information is extracted to emulate sensory data processing patterns. Subsequently, resembling the hippocampus, this information is processed at a higher level, capturing and memorizing correlated patterns. We conducted a comparative analysis of our model against existing quantum models, including quantum neural networks (QNNs), QLSTM, QSNN and their classical counterparts, elucidating its unique contributions. Through empirical results, we demonstrated the effectiveness of utilizing quantum models inspired by the brain, which outperform the classical approaches and other quantum models in optimizing energy use case. Specifically, in terms of average, best and worst total reward, test reward, robustness, and learning curve.

In recent years, advancements in brain science and neuroscience have significantly influenced the field of computer science, particularly in the domain of reinforcement learning (RL). Drawing insights from neurobiology and neuropsychology, researchers have leveraged these findings to develop novel mechanisms for understanding intelligent decision-making processes in the brain. Concurrently, the emergence of quantum computing has opened new frontiers in artificial intelligence, leading to the development of quantum machine learning (QML). This study introduces a novel model that integrates Quantum Spiking Neural Networks (QSNN) and Quantum Long Short-Term Memory (QLSTM) architectures, inspired by the complex workings of the human brain. Specifically designed for reinforcement learning tasks in energy-efficient environments, our approach progresses through two distinct stages mirroring sensory and memory systems. In the initial stage, analogous to the brain’s hypothalamus, low-level information is extracted to emulate sensory data processing patterns. Subsequently, resembling the hippocampus, this information is processed at a higher level, capturing and memorizing correlated patterns. We conduct a comparative analysis of our model against existing quantum models such as Quantum Neural Networks and their classical counterparts, elucidating its unique contributions. Through empirical results, we aim to underscore the effectiveness of our approach in optimizing energy consumption in practical scenarios.

The Transiting Exoplanet Survey Satellite (TESS) generates photometric time series for hundreds of thousands of stars, far more than can be vetted by hand, motivating machine-learning (ML) triage. We present a low-resource, interpretable pipeline that classifies TESS light curves as transit-bearing or not using a random forest trained on engineered summary features, treating feature-importance analysis as a formal auditing step, and apply it to Sector 68. The positive class comprises TESS-confirmed planet hosts; the negative class comprises Sector 68 targets with no catalogued planet or candidate, assembled from a different source. Auditing the model’s feature-importance ranking exposes a confound: the two dominant features, the flux energy and its logarithm, are for median-normalized light curves essentially proportional to the number of cadences, that is, to observational baseline. A transit-blind ranking by this feature alone already separates the classes with an AUC of 0.734, and because our train/test split cannot control for the classes’ differing provenance, the full model’s held-out ROC-AUC of 0.95 (precision 0.94, recall 0.74, F 1 0.83) is best read as performance under the current sampling procedure; isolating the confound’s precise causal share is left as an open question for a dedicated ablation study. Some evidence of genuine signal survives this caveat: applied to 400 held-out candidates that share the negative class’s baseline and so cannot benefit from the confound, the pipeline still flags 61 (15%) as probable planets, with the highest-probability cases showing genuine periodic transit dips. We give concrete recommendations for detecting and avoiding this failure mode: provenance-matched class construction, baseline-invariant features, controlled ablation, and routine interpretability-based leakage audits. The case study is offered as an instance of auditing scientific machine learning with its own tools, and a cautionary lesson for ML-based exoplanet detection.