28 June 2014 in Neural Computing and Applications
13 November 2024 in Information & Management
14 November 2019 in IEEE Access
Two dimensional strip packing problem implies a sort of NP-hard combinational optimization, where a set of rectangles are defined all for filling a strip with finite width and infinite height being minimized. Optimization taking no account of rotation and guillotine constraints remains problematic in solving the problem. This paper proposes a new variant of a hybrid heuristic algorithm for optimizing solutions of the strip packing problem. This algorithm involves three key improvements: the building of δ scoring rules for selecting rectangles, the use of the red-black trees that stores rectangle indices for quickly locating the most suitable rectangles, and the embedding of a hierarchical method into a random local search to implement an optimization solution. Comparative studies show that the proposed algorithm variant outperforms state-of-the-art algorithms on almost all benchmark datasets.
15 November 2024 in IEEE Access
To address the issue of the substantial computational resource consumption during the inference phase of large language models due to their vast number of parameters, model sparsification is an effective solution. However, current sparsification methods for large models are costly. We propose a comprehensive two-stage approach called ELO-Mask for the rapid sparsification of large language models using a small calibration dataset. The approach consists of two steps: 1) Mask Reordering Step, this step involves initializing the mask using predefined parameter importance metrics, followed by reordering the model masks in blocks using the Straight-Through Estimator method with a small sample dataset. 2) Mask Fine-Tuning Step, this step involves further fine-tuning the masks obtained from the first step in blocks, using the same small sample dataset. Our experiments demonstrate the effectiveness of this approach. When sparsifying the Llama-7B model, our method shows significant superiority over the standard sparsification plus LoRA fine-tuning approach. It achieves comparable performance in the final sparse model while consuming less computational power, using a smaller dataset, occupying less GPU memory, and not affecting the inference speed of the sparse model.