Algorithm design is crucial for effective problem-solving across various domains. The advent of Large Language Models (LLMs) has notably enhanced the automation and innovation within this field, offering new perspectives and promising solutions. In just a few years, this integration has yielded remarkable progress in areas ranging from combinatorial optimization to scientific discovery. Despite this rapid expansion, a holistic understanding of the field is hindered by the lack of a systematic review, as existing surveys either remain limited to narrow sub-fields or with different objectives. This article seeks to provide a systematic review of algorithm design with LLMs. We introduce a taxonomy that categorizes the roles of LLMs as optimizers, predictors, extractors, and designers, analyzing the progress, advantages, and limitations within each category. We further synthesize literature across the three phases of the algorithm design pipeline and across diverse algorithmic applications that define the current landscape. Finally, we outline key open challenges and opportunities to guide future research.

Vehicle routing problems (VRP) are very important in many real-world applications and has been studied for several decades. Recently, neural combinatorial optimization (NCO) has attracted growing research effort. NCO is to train a neural network model to solve an optimization problem in question. However, existing NCO methods often build a different model for each routing problem, which significantly hinders their application in some areas where there are many different VRP variants to solve. In this work, we make a first attempt to tackle the crucial challenge of cross-problem generalization in NCO. We formulate VRPs as different combinations of a set of shared underlying attributes and solve them simultaneously via a single model through attribute composition. In this way, our proposed model can successfully solve VRPs with unseen attribute combinations in a zero-shot generalization manner. In our experiments, the neural model is trained on five VRP variants and its performance is tested on eleven VRP variants. The experimental results show that the model demonstrates superior performance on these eleven VRP variants, reducing the average gap to around 5% from over 20% and achieving a notable performance boost on both benchmark datasets and real-world logistics scenarios.

Many combinatorial multiobjective optimization problems involve very costly-to-evaluate objectives and constraints. It is very difficult, if not impossible, for traditional heuristics to solve these problems with an acceptable amount of computational time. In this paper, we show that offline machine learning can be very useful to assist multiobjective evolutionary algorithms to tackle this kind of problem. We take a complicated real-life multiobjective routing-packing problem as the test bed. We propose to use offline machine learning methods to replace time-consuming packing heuristics for packing feasibility prediction. Experiments show that the machine learning models can be 1,000 times faster than some commonly used packing heuristics and their accuracy can be as high as 98%. We adopt MOEA/D to decompose the problem into a number of single objective subproblems and solve them in a collaborative manner. We propose an encoding strategy to represent each routing scheme and use genetic operators to generate new routes. Experimental studies have been conducted on 100 instances from HUAWEI’s real-world logistics application and two test suites from the literature. Our proposed method can solve each HUAWEI instance in around one minute. Our solutions on the two test suites are comparable to other existing algorithms, and the overall computational cost of our method is significantly lower than others.