Object detection in kitchen waste faces numerous challenges, including a variety of target categories, significant morphological variations, and complex backgrounds, coupled with the scarcity of targeted datasets, resulting in limited research in this area. In response to these challenges, we integrate insights from real-world industrial scenarios of kitchen waste sorting and create, for the first time, a dataset comprising eight categories specifically designed for kitchen waste object detection. Furthermore, we initiate the integration of Active Learning (AL) methods into the Object Detection Transformer (DETR) network, proposing the AL-DETR model—a high-precision detection model tailored for scenarios with limited samples. To overcome the drawback of DETR, which relies on a large amount of annotated data for training, we employ an active learning strategy, annotating challenging samples with high uncertainty selected from a pool of unlabeled data, and integrating them back into the training set to enhance the model’s learning capability. AL-DETR also models non-local visual correlations between labeled and unlabeled samples, comparing differences in detection categories to identify relevant unlabeled samples. Additionally, we establish a robotic sorting platform with an end-effector integrating gripping and suction functionalities, accompanied by the development of visual detection and robot control software. The experimental results indicate that, in the dataset constructed for this study, which primarily focuses on detecting large objects, our approach achieves a minimum improvement of 9.23% compared to other methods, reaching state-of-the-art performance. Additionally, our method surpasses other active learning approaches using the entire annotated dataset by more than 1.01% in mAP when the original dataset’s annotation data is only 82.5%. In practical testing, the sorting platform successfully accomplishes efficient detection and sorting of kitchen waste.

RGB and Depth (RGB-D) modalities are widely used in planar grasp detection methods. However, most previous grasp detection approaches fail to fully exploit the distinct information contained in these two modalities and simply regard depth information as part of RGB information. In order to better integrate RGB-D information and improve the accuracy of planar grasp detection, a novel generative Multimodal fusion with Channel and Spatial attention Residual Network (MCS-ResNet) is developed, which is specifically designed for two-dimensional planar grasping. The proposed network has two newly designed modules and a distinctive input method which expands Depth information into three channels. First, we propose a novel RGB-D fusion module to fully extract information from both RGB and Depth modalities by disregarding secondary information and enhancing channel and spatial sensitivity. Then, we convert single-channel depth images into three-channel images to enable the network to capture richer Depth features. Finally, a novel residual module is designed to make the model pay more attention to channel-wise information. Besides, to equip the model with classification capabilities, a multi-object solid waste grasp dataset (SWC dataset) with class information is created. In the comparison experiments, MCS-ResNet obtains 95.9% grasping accuracy and 97.8% classification accuracy on SWC dataset, 98.5% grasping accuracy on Cornell dataset, which reflect the superiority of our model. Finally, the real-world robotic experiments conduct in a conveyor belt scenario using an edge computing device, reaching a grasping success rate of 95.3% and the classification success rate of 96.0%.

In skin lesion detection systems, a large number of labeled images are usually required to achieve high segmentation accuracy, which hinders the effectiveness and timeliness of disease diagnosis. To this end, a high-precision skin lesion detection system is proposed in this paper. The hardware part of the system adopts a modular design, fully considering ergonomics, portability, and miniaturization. In the software part, an active learning ensemble with a multi-model fusion method (ALEM) is proposed to achieve efficient and accurate skin lesion region segmentation. The core idea of ALEM is to use multiple uncertainty strategies of active learning to obtain the most uncertain pixels to be marked in the skin lesion image when marking image pixels. The experiment shows that the average Dice coefficient and average Jaccard index of ALEM on ISIC-2016 are 82.81% and 92.4% respectively and that on ISIC-2017 are 87.51% and 79.26% respectively. It is worth noting that ALEM still outperforms in tests with only 80% of the training data and no more than 15% pixel annotation per image on average. Our system achieves an average AUC of 91.01% on the ISIC2017 and is tested for effectiveness on real skin. The skin lesion detection system developed in this paper is expected to bring convenience to doctors and patients and speed up the diagnosis of diseases.

Recognition and early warning of plant diseases is one of the keys to agricultural disaster prevention and mitigation. Deep learning-based image recognition methods give us a new idea for plant disease identification. Due to the harsh conditions in agricultural environment, recent research has focused on exploring ways to lightweight the recognition model for deployment on low-power devices. In this paper, we propose an efficient and feature-guided real-time plant disease recognition model with a multi-classifier architecture, specifically designed for low-power devices. By comparing with other advanced methods, our model reaches the state-of-the-art in the combined metrics of recognition accuracy, the number of parameters and inference speed. First, we propose AMI-NanoNet based on RoofLine theory to significantly reduce the number of parameters and computational complexity. This model can achieve 99.8343% accuracy on PlantVillage by using a Feature-Guided Curriculum Learning (FGCL) with stepwise training strategy. Moreover, we design another training strategy suitable for lightweight ensemble models. Based on this strategy, our model only needs to integrate the classifiers at the end of the network to achieve 99.8708% identification accuracy, and it hardly increases the number of operations and parameters of the network. Extensive evaluations on this dataset demonstrate the effectiveness of our ensemble learning method. Furthermore, we tested our proposed methods on another dataset from other domains to validate its applicability to different scenarios. Overall, our research provides a basis for rapid and intelligent identification of plant diseases.

The key challenge in automated kitchen waste sorting on conveyor belts is how robots execute grasping tasks. Current methods, dependent on object detection, struggle to adopt generative architectures due to data scarcity and algorithmic inefficiency. To solve these issues, this paper proposes solutions in three aspects: dataset construction, annotation data processing, and generative model lightweighting. Firstly, we built the KWG2024 dataset, which includes 3,116 RGB-D images, 18,059 standard annotations, and three new evaluation metrics. Secondly, we proposed two novel annotation processing methods to address the issue of excessive and incorrect grasps generated by generative networks in single-attempt grasp planning of multi-objects. Thirdly, we constructed an efficient network with classification capabilities, AMI-LiteNet, by integrating the advantages of PP-LiteSeg and AMI-NanoNet. On the public Cornell dataset, AMI-LiteNet achieved a grasping accuracy of 98.3% and an efficiency of 118 FPS. In extensive comparative experiments on KWG2024, our model reached a leading computational efficiency compared with recent state-of-the-art methods. It acquired a grasp planning generation accuracy of 87.96% while only requiring a computational effort of 2.79 GFLOPs. Finally, our model was deployed on an edge computing device and achieved a maximum grasping accuracy of 90.91% and a classification accuracy of 97.48% in our actual robotic system. To our knowledge, this is the first time a generative grasp method is applied independently to kitchen waste sorting tasks on a conveyor belt.