Transfer learning is a machine learning technique that uses previously acquired knowledge from a source domain to enhance learning in a target domain by reusing learned weights. This technique is ubiquitous because of its great advantages in achieving high performance while saving training time, memory, and effort in network design. In this paper, we investigate how to select the best pre-trained model that meets the target domain requirements for image classification tasks. In our study, we refined the output layers and general network parameters to apply the knowledge of eleven image processing models, pre-trained on ImageNet, to five different target domain datasets. We measured the accuracy, accuracy density, training time, and model size to evaluate the pre-trained models both in training sessions in one episode and with ten episodes.
13 July 2018 in EAI Endorsed Transactions on Smart Cities
INTRODUCTION: Sorting a huge stream of waste accurately within a short period can be done with the support of digitalization, particularly Artificial Intelligence, instead of traditional methods. The overlap of Artificial Intelligence and Circular Economy can flourish many services in the environmental technology domain, in particular smart e-waste recycling, resulting in enabling circular smart cities.
OBJECTIVES: We analyse the growing need for automated e-waste recycling as an essential requirement to cope with the fast-growing e-waste stream and we shed the light on the impact of Artificial Intelligence in supporting the recycling process through smart classification of devices, where the smartphone is our case study.
METHODS: Our study applies transfer learning as a special technique of Artificial Intelligence by fine-tuning the output layers of AlexNet as a pre-trained model and perform the implementation on a small-size dataset that contains 12 classes from 6 smartphone brands.
RESULTS: We evaluate the performance of our model by tuning the learning rate, choosing the best optimizer, and augmenting the original dataset to avoid overfitting. We found that the optimizer of Stochastic Gradient Descent with Momentum and 3 ????????−4 as a learning rate brings almost 98% model accuracy with generalization.
CONCLUSION: Our study supports automated e-waste recycling in decreasing the error-rate of e-waste sorting and investigates the advantages of applying transfer learning as the best scenario to overcome the rising challenges.
Research and applications in artificial intelligence have recently shifted with the rise of large pretrained models, which deliver state-of-the-art results across numerous tasks. However, the substantial increase in parameters introduces a need for parameter-efficient training strategies. Despite significant advancements, limited research has explored parameter-efficient fine-tuning (PEFT) methods in the context of transformer-based models for instance segmentation. Addressing this gap, this study investigates the effectiveness of PEFT methods, specifically adapters and Low-Rank Adaptation (LoRA), applied to two models across four benchmark datasets. Integrating sequentially arranged adapter modules and applying LoRA to deformable attention—explored here for the first time—achieves competitive performance while fine-tuning only about 1–6% of model parameters, a marked improvement over the 40–55% required in traditional fine-tuning. Key findings indicate that using 2–3 adapters per transformer block offers an optimal balance of performance and efficiency. Furthermore, LoRA, exhibits strong parameter efficiency when applied to deformable attention, and in certain cases surpasses adapter configurations. These results show that the impact of PEFT techniques varies based on dataset complexity and model architecture, underscoring the importance of context-specific tuning. Overall, this work demonstrates the potential of PEFT to enable scalable, customizable, and computationally efficient transfer learning for instance segmentation tasks.
Recycling electrical and electronic devices in an automated method can reduce the negative impact on human health and the environment compared with manual dismantling. This approach suggests using multi-sensors by using RGB sensor to classify the devices using a deep learning method and infrared sensor to recognize the internal component of the device. Particularly, the sorting accuracy achieves 98.5% using the state-of-the-art EfficientNet for the tested devices, and infrared images give a significant guide to the main components of the devices. Eventually, this information can be transferred to the next stage of material processing to provide fewer risks and a smarter way to avoid toxic materials.
Selecting pretrained models for image classification often involves computationally intensive finetuning. This study addresses a research gap in the standardized evaluation of transferability scores, which could simplify model selection by ranking pretrained models without exhaustive finetuning. The motivation is to reduce the computational burden of model selection through a consistent approach that guides practitioners in balancing accuracy and efficiency across tasks. This study evaluates 14 transferability scores on 11 benchmark datasets. It includes both Convolutional Neural Network (CNN) and Vision Transformer (ViT) models and ensures consistency in experimental conditions to counter the variability in previous research. Key findings reveal significant variability in score effectiveness based on dataset characteristics (e.g., fine-grained versus coarse-grained classes) and model architectures. ViT models generally show superior transferability, especially for fine-grained datasets. While no single score is best in all cases, some scores excel in specific contexts. In addition to predictive accuracy, the study also evaluates computational efficiency and identifies scores that are suitable for resource-constrained scenarios. This research provides insights for selecting appropriate transferability scores to optimize model selection strategies to facilitate efficient deployment in practice.
About Me
A research assistant in the field of artificial intelligence.
A research assistant in the field of artificial intelligence.