Metaheuristics provide the means to approximately solve complex optimisation problems when exact optimisers cannot be utilised. This led to an explosion in the number of novel metaheuristics, most of them metaphor-based, using nature as a source of inspiration. Thus, keeping track of their capabilities and innovative components is an increasingly difficult task. This can be resolved by an exhaustive classification system. Trying to classify metaheuristics is common in research, but no consensus on a classification system and the necessary criteria has been established so far. Furthermore, a proposed classification system can not be deemed complete if inherently different metaheuristics are assigned to the same class by the system. In this paper we provide the basis for a new comprehensive classification system for metaheuristics. We first summarise and discuss previous classification attempts and the utilised criteria. Then we present a multi-level architecture and suitable criteria for the task of classifying metaheuristics. A classification system of this kind can solve three main problems when applied to metaheuristics: organise the huge set of existing metaheuristics, clarify the innovation in novel metaheuristics and identify metaheuristics suitable to solve specific optimisation tasks.
State-of-the-art Deep Reinforcement Learning Algorithms such as DQN and DDPG use the concept of a replay buffer called Experience Replay. The default usage contains only the experiences that have been gathered over the runtime. We propose a method called Interpolated Experience Replay that uses stored (real) transitions to create synthetic ones to assist the learner. In this first approach to this field, we limit ourselves to discrete and non-deterministic environments and use a simple equally weighted average of the reward in combination with observed follow-up states. We could demonstrate a significantly improved overall mean average in comparison to a DQN network with vanilla Experience Replay on the discrete and non-deterministic FrozenLake8x8-v0 environment.