A variety of meta-heuristics have shown promising performance for solving multi-objective optimization problems (MOPs). However, existing meta-heuristics may have the best performance on particular MOPs, but may not perform well on the other MOPs. To improve the cross-domain ability, this paper presents a multi-objective hyper-heuristic algorithm based on adaptive epsilon-greedy selection (HH_EG) for solving MOPs. To select and combine low-level heuristics (LLHs) during the evolutionary procedure, this paper also proposes an adaptive epsilon-greedy selection strategy. The proposed hyper-heuristic can solve problems from varied domains by simply changing LLHs without redesigning the high-level strategy. Meanwhile, HH_EG does not need to tune parameters, and is easy to be integrated with various performance indicators. We test HH_EG on the classical DTLZ test suite, the IMOP test suite, the many-objective MaF test suite, and a test suite of a real-world multi-objective problem. Experimental results show the effectiveness of HH_EG in combining the advantages of each LLH and solving cross-domain problems.
The reinforcement learning based hyper-heuristics (RL-HH) is a popular trend in the field of optimization. RL-HH combines the global search ability of hyper-heuristics (HH) with the learning ability of reinforcement learning (RL). This synergy allows the agent to dynamically adjust its own strategy, leading to a gradual optimization of the solution. Existing researches have shown the effectiveness of RL-HH in solving complex real-world problems. However, a comprehensive introduction and summary of the RL-HH field is still blank. This research reviews currently existing RL-HHs and presents a general framework for RL-HHs. This article categorizes the type of algorithms into two categories: value-based reinforcement learning hyper-heuristics and policy-based reinforcement learning hyper-heuristics. Typical algorithms in each category are summarized and described in detail. Finally, the shortcomings in existing researches on RL-HH and future research directions are discussed.
17 August 2023 in IEEE Transactions on Systems, Man, and Cybernetics: Systems
Despite the success of meta-heuristics for many computationally hard optimization problems, the design of meta-heuristics is dependent on the problem domain and needs to be tailored by experts. In contrast, selection hyper-heuristics are emerging cross-domain search methodologies that perform the search over the space of a set of low-level heuristics. In this study, we introduce a novel selection hyper-heuristic to learn, select, and apply low-level heuristics to solve multiobjective optimization problems. The proposed selection hyper-heuristic is an iterative process that includes three main components: 1) the reward balance strategy; 2) the bandit-based learning strategy; and 3) the multistage selection strategy. The performance and behavior of the proposed selection hyper-heuristic are investigated on a range of benchmarks as well as several real-world problems. The empirical results demonstrate the effectiveness and the cross-domain ability of the proposed selection hyper-heuristic by comparing it with several state-of-the-art algorithms.
The spiking neural network (SNN) has attracted the attention of many researchers because of its low energy consumption and strong bionics. However, when the network conversion method is used to solve the difficulty of network training caused by its discrete, too-long inference time, it may hinder the practical application of SNN. This paper proposes a novel model named the SNN with Initialized Membrane Potential and Coding Compensation (IC-SNN) to solve this problem. The model focuses on the effect of residual membrane potential and rate encoding on the target SNN. After analyzing the conversion error and the information loss caused by the encoding method under the low time step, we propose a new initial membrane potential setting method and coding compensation scheme. The model can enable the network to still achieve high accuracy under a low number of time steps by eliminating residual membrane potential and encoding errors in the SNN. Finally, experimental results based on public datasets CIFAR10 and CIFAR100 also demonstrate that the model can still achieve competitive classification accuracy in 32 time steps.
Task scheduling algorithms based on reinforce learning (RL) have been important methods with which to improve the performance of cloud platforms; however, due to the dynamics and complexity of the cloud environment, the action space has a very high dimension. This not only makes agent training difficult but also affects scheduling performance. In order to guide an agent’s behavior and reduce the number of episodes by using historical records, a task scheduling algorithm based on adaptive priority experience replay (APER) is proposed. APER uses performance metrics as scheduling and sampling optimization objectives with which to improve network accuracy. Combined with prioritized experience replay (PER), an agent can decide how to use experiences. Moreover, this algorithm also considers whether a subtask is executed in a workflow to improve scheduling efficiency. Experimental results on Tpc-h, Alibaba cluster data, and scientific workflows show that a model with APER has significant benefits in terms of convergence and performance.
In order to reduce the burden of DBA, the knob tuning method based on reinforcement learning has been proposed and achieved good results in some cases. However, the performance of these solutions is not ideal as the workload features are not considered enough. To address these issues, we propose a database tuning system called ADWTune. In this model, ADWTune employs the idea of multiple sampling to gather workload data at different time points during the observation period. ADWTune uses these continuous data slices to characterize the dynamic changes in the workload. The key of ADWTune is its adaptive workload handling approach, which combines the dynamic features of workloads and the internal metrics of database as the state of the environment. At the same time, ADWTune includes a data repository, which reuses historical data to improve the adaptability of model to workload shifts. We conduct extensive experiments on various workloads. The experimental results demonstrate that ADWTune is better suited for dynamic environments than other methods based on reinforcement learning.