Many patients with upper limb defects desire myoelectric prosthetic hands, but they are still not used for some reasons. One of the most significant reasons is its external appearance, which has the discomfort caused by the structural difference between a human hand and a robotic link. The structure must be based on human anatomy to create a more natural-looking prosthesis. This study designed a biomimetic prosthetic hand with bones, ligaments, tendons, and multiple muscles based on the human musculoskeletal system. We verified the proposed prosthetic hand using the viscoelastic angle sensor to determine whether it works like a human hand. We also compared the finger force of the prosthetic hand with that of a human finger. It could be capable of controlling the angle and the stiffness of the joint by multiple extensor and flexor muscles, like humans.
In recent years, developments in deep learning technology have driven significant advancements in research aimed at facilitating communication with individuals who have hearing impairments. The focus has been on enhancing automatic recognition and translation systems for sign language. This study proposes a novel approach using a vision transformer (ViT) for recognizing Japanese Sign Language. Our method employs a pose estimation library, MediaPipe, to extract the positional coordinates of each finger joint within video frames and generate one-dimensional angular feature data from these coordinates. Then, the code arranges these feature data in a temporal sequence to form a two-dimensional input vector for the ViT model. To determine the optimal configuration, this study evaluated recognition accuracy by manipulating the number of encoder layers within the ViT model and compared against traditional convolutional neural network (CNN) models to evaluate its effectiveness. The experimental results showed 99.7% accuracy for the method using the ViT model and 99.3% for the results using the CNN. We demonstrated the efficacy of our approach through real-time recognition experiments using Japanese sign language videos.
Wrist movements play a crucial role in upper-limb motor tasks. As prosthetic and robotic hand technologies have evolved, increasing attention has been focused on replicating the anatomy and functionality of the wrist. Closely imitating the biomechanics and movement mechanisms of human limbs is expected to enhance the overall performance of bionic robotic hands. This study presents the design of a tendon-driven bionic spherical robot wrist, utilizing two pairs of cables that mimic antagonist muscle pairs. The cables are actuated by pulleys driven by servo motors, allowing for two primary wrist motions: flexion–extension and ulnar–radial deviation. The performance Please confirm if the “1583 Iiyama” is necessary. Same as belowof the proposed robot wrist is validated through manipulation experiments using a prototype, demonstrating its capability to achieve a full range of motion for both ulnar and radial deviation. This wrist mechanism is expected to be integrated into robotic systems, enabling greater flexibility and more human-like movement capabilities.
Pathology report generation is challenging as it requires analyzing giga-pixel Whole Slide Images (WSIs), making it a complex and labor-intensive task, yet crucial for clinical decision-making. An AI-driven report generation system can assist in producing accurate and concise reports. To address these needs, we designed a two-stage framework consisting image analysis and text generation, improving diagnostic performance and optimizing it for real-world clinical application. In the first stage, using 752 high resolution WSIs from bladder tumor tissue samples collected at Korea University Medical Center, we leveraged foundation models to extract robust features. Then, we built a multi-label classification based on key diagnostic elements to achieve high performance with an Attention-based Multi-Instance Learning (MIL) model and a Transformer-based MIL with Knowledge Distillation. For report generation stage, we designed a T5-based text-to-text model, simplifying input representations whereas integrating data augmentation to improve stability and generalization. The proposed model achieved a ROUGE score of 0.87, a BLEU-4 score of 0.94, a Jaccard score of 0.89 and a BioLLM score of 0.97. An additional evaluation conducted by the institution maintaining the exclusive K-MEDICON test datasets confirmed a consistent performance yielding an overall score of 0.88, calculated as a weighted sum of aforementioned metrics. Its consistency validated through a Mean Opinion Score (MOS) evaluation by pathologists. By systematically integrating foundation models and modular structure, the benchmark results demonstrate that the task can be effectively solved with limited data and computational resources, indicating promising potential for real-world clinical adaptation.
To improve the accuracy of Japanese finger-spelled character recognition using an RGB camera, we focused on feature design and refinement of the recognition method. By leveraging angular features extracted via MediaPipe, we proposed a method that effectively captures subtle motion differences while minimizing the influence of background and surrounding individuals. We constructed a large-scale dataset that includes not only the basic 50 Japanese syllables but also those with diacritical marks, such as voiced sounds (e.g., “ga”, “za”, “da”) and semi-voiced sounds (e.g., “pa”, “pi”, “pu”), to enhance the model’s ability to recognize a wide variety of characters. In addition, the application of a change-point detection algorithm enabled accurate segmentation of sign language motion boundaries, improving word-level recognition performance. These efforts laid the foundation for a highly practical recognition system. However, several challenges remain, including the limited size and diversity of the dataset and the need for further improvements in segmentation accuracy. Future work will focus on enhancing the model’s generalizability by collecting more diverse data from a broader range of participants and incorporating segmentation methods that consider contextual information. Ultimately, the outcomes of this research should contribute to the development of educational support tools and sign language interpretation systems aimed at real-world applications.
The development of automatic fingerspelling recognition systems is critical for facilitating seamless communication for the deaf and hard-of-hearing community. Historically, recognition technologies relied on wearable sensors, such as data gloves, inertial measurement units (IMUs), and surface electromyography (sEMG), which provided stable kinematic data but suffered from high hardware costs and user discomfort. Non-invasive vision-based sensing has emerged to eliminate wearable overhead but is subject to severe physical bottlenecks: continuous high-speed movements inevitably introduce motion blur and self-occlusion, drastically degrading tracking accuracy. To address these visual sensing constraints, we investigate a visual-language integration framework that combines dynamic visual representations derived from a single RGB video stream with the lexical knowledge of Large Language Models (LLMs), allowing the language model to compensate for uncertainty or errors in the visual representation. Our approach employs a robust visual encoder—comprising a Temporal Convolutional Network (TCN) and a Transformer—to extract and integrate dynamic transition features from 3D skeletal coordinates. Rather than propose a new LLM architecture, this study focuses on how visual information should be represented and transferred to an LLM by comparing discrete-text, probability-level, and latent-feature-level integration strategies under the same experimental conditions. Specifically, we systematically evaluated three integration strategies: Text Input (TI), Projector Probabilities Input (PPI), and Projector Transformer Features Input (PTFI). Experimental results on a 28-word closed-set evaluation show that while the word-level exact-match accuracy (WA) of the visual encoder alone was limited to 11.66% with continuous signing, our LLM-integrated architecture elevated the WA to a maximum of 87.73%. Crucially, the PTFI approach successfully suppressed character-level degradation by grounding linguistic reasoning in latent Transformer-based visual evidence. These results suggest that integrating LLM-based linguistic reasoning with dynamic visual features is a promising approach for closed-set continuous fingerspelling recognition under controlled experimental conditions.