Since the Convolutional Neural Network (CNN) has surfaced and fascinated the world, many researchers have exploited CNN for image classification, object detection, semantic segmentation, etc. However, the conventional CNNs have a pyramidal structure and were designed to process images which have the same size. Although some CNNs can accept images of various sizes, performance is degraded for images smaller than the size of images used for training. In this paper, we propose MarsNet, a CNN based end-to-end network for multi-label classification with an ability to accept various size inputs. In order to allow the network to accept such images, dilated residual network (DRN) is modified to get higher resolution feature maps, and horizontal vertical pooling (HVP) is newly designed to efficiently aggregate positional information from the feature maps. Furthermore, multi-label scoring module and threshold estimation module are employed to serve the purpose of multi-label classification. We verify the effectiveness of the proposed network through two distinctive experiments. We first verify our model by inspecting and classifying multiple types of defects occurred in PCB screen printer using solder paste inspection (SPI) datasets. Secondly, we verify our network using VOC 2007 dataset. Our network is pioneering in that no research has attempted to accomplish multi-label classification for defects in addition to being able to take input images of various sizes in SPI field.

Objects can be described in hierarchical semantics, and people also perceive them this way. It leads to the need for hierarchical classification in machine learning. On the other hand, when a new data that belongs to a new class is given, the existing classification methods should be retrained for all data including the new data. To deal with these issues, we propose an adaptive resonance theory-supervised predictive mapping for hierarchical classification (ARTMAP-HC) network that allows incremental class learning for raw data without normalization in advance. Our proposed ARTMAP-HC is composed of hierarchically stacked modules, and each module incorporates two fuzzy ARTMAP networks. Regardless of the level of the class hierarchy and the number of classes for each level, ARTMAP-HC is able to incrementally learn sequentially added input data belonging to new classes. By using a novel online normalization process, ARTMAP-HC can classify the new data without prior knowledge of the maximum value of the dataset. By adopting the prior labels appending process, the class dependency between class hierarchy levels is reflected in ARTMAP-HC. The effectiveness of the proposed ARTMAP-HC is validated through experiments on hierarchical classification datasets. To demonstrate the applicability, ARTMAP-HC is applied to a multimedia recommendation system for digital storytelling.

Artificial intelligence (AI) has demonstrated superior performances in various fields, including computer vision and medical imaging. Further advancement in AI and medical imaging can effectively contribute to increasing the quality of service in both robotic and non-robotic surgical domains by providing enhanced medical perception and predictive assistance. In this regard, we propose a method for successfully recognizing the current surgical actions as well as predicting future surgical actions based on the robotic surgical scenes. We introduce an online robotic tool detection method that can extract visual features that focus on each robotic surgical tool without prior learning. Based on the features, we develop an encoder-decoder framework to recognize the current surgical action and predict the sequence of the next surgical actions to be performed. By performing experiments using the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS) data, it is verified that the proposed method recognizes the current surgical action with high accuracy and effectively predicts a sequence of future surgical actions.

Robotic vision has traditionally relied on high-performance yet resource-intensive computing solutions, which necessitate high-throughput data transmission from vision sensors to remote computing servers, sacrificing energy efficiency and processing speed. A promising solution is data compaction through contour extraction, visualizing only the outlines of objects while eliminating superfluous backgrounds. Here, we introduce an in-sensor multilevel image adjustment method using adjustable synaptic phototransistors, enabling the capture of well-defined images with optimal brightness and contrast suitable for achieving high-clarity contour extraction. This is enabled by emulating dopamine-mediated neuronal excitability regulation mechanisms. Electrostatic gating effect either facilitates or inhibits time-dependent photocurrent accumulation, adjusting photo-responses to varying lighting conditions. Through excitatory and inhibitory modes, the adjustable synaptic phototransistor enhances visibility of dim and bright regions, respectively, facilitating distinct contour extraction and high-accuracy semantic segmentation. Evaluations using road images demonstrate improvement of both object detection accuracy and intersection over union, and compression of data volume.

In human-robot interaction (HRI), classification is one of the most important problems, and it is essential particularly when the robot recognizes the surroundings and chooses a reaction based on a certain situation. Each interaction is different since new people appear or the environment changes, and the robot should be able to adapt to different situations during a brief interaction. Thus, it is imperative that the classification is performed incrementally in real time. In this sense, we propose an online incremental classification resonance network (OICRN) that enables incremental class learning in multi-class classification with high performance online. In OICRN, a scale-preserving projection process is introduced to use the raw input vectors online without a normalization process in advance. The integrated network of the convolutional neural network (CNN) for feature extraction and the OICRN for classification is applied to a robotic system that learns human identities through HRIs. To demonstrate the effectiveness of our network, experiments are carried out on benchmark data sets and on a humanoid robot, Mybot, developed in the Robot Intelligence Technology Laboratory, KAIST.

Reliable recognition and localization of surgical instruments in endoscopic video recordings are foundational for a wide range of applications in computer- and robot-assisted minimally invasive surgery (RAMIS), including surgical training, skill assessment, and autonomous assistance. However, robust performance under real-world conditions remains a significant challenge. Incorporating surgical context - such as the current procedural phase - has emerged as a promising strategy to improve robustness and interpretability. To address these challenges, we organized the Surgical Procedure Phase, Keypoint, and Instrument Recognition (PhaKIR) sub-challenge as part of the Endoscopic Vision (EndoVis) challenge at MICCAI 2024. We introduced a novel, multi-center dataset comprising thirteen full-length laparoscopic cholecystectomy videos collected from three distinct medical institutions, with unified annotations for three interrelated tasks: surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation. Unlike existing datasets, ours enables joint investigation of instrument localization and procedural context within the same data while supporting the integration of temporal information across entire procedures. We report results and findings in accordance with the BIAS guidelines for biomedical image analysis challenges. The PhaKIR sub-challenge advances the field by providing a unique benchmark for developing temporally aware, context-driven methods in RAMIS and offers a high-quality resource to support future research in surgical scene understanding.