Cervical cancer is the fourth most prevalent disease in women. Accurate and timely cancer detection can save lives. Automatic and reliable cervical cancer detection methods can be devised through the accurate segmentation and classification of Pap smear cell images. This paper presents an approach to whole cervical cell segmentation using a mask regional convolutional neural network (Mask R-CNN) and classifies this using a smaller Visual Geometry Group-like Network (VGG-like Net). ResNet10 is used to make full use of spatial information and prior knowledge as the backbone of the Mask R-CNN. We evaluate our proposed method on the Herlev Pap Smear dataset. In the segmentation phase, when Mask R-CNN is applied on the whole cell, it outperforms the previous segmentation method in precision (0.92±0.06), recall (0.91±0.05) and ZSI (0.91±0.04). In the classification phase, VGG-like Net is applied on the whole segmented cell and yields a sensitivity score of more than 96% with low standard deviation (±2.8%) for the binary classification problem and yields a higher result of more than 95% with low standard deviation (maximum 4.2% in accuracy measurement) for the 7-class problem in terms of sensitivity, specificity, accuracy, h-mean, and F1 score.

Cyber-physical-social systems (CPSS) are expected to support telemedicine and will be the future of independent rehabilitation. However, a connection between the physical and cognitive as one of the social aspects has not been adequately supported. This paper proposes a multiscopic CPSS framework to investigate this issue by developing hand–object interaction (HOI) recognition with visual attention for the block-design test (BDT). We use three vision systems to extract hand features from block interaction. First, a hand-tracking vision is used to collect the hand-skeletal data and finger joint angle feature. We estimate the physical handling postures at the microscopic level. Second, an egocentric vision with an eye tracker is used to get hand and eye movement at the mesoscopic level. We analyze hand-eye coordination through hand gestures and the focus of visual attention when the hand interacts with the block. Third, an evaluation vision is used to classify the color features in each block at the macroscopic level. The system can recognize whether the design matches the task given. The eight-block design test with two scenarios demonstrates that the system can successfully measure human behavior from physical to cognitive during BDT. We investigate the relationship between physical and cognitive evaluation through the experiment in eight healthy persons. The regression and correlation analyses between the dominant and non-dominant hands show that evaluation indices – skewness-kurtosis of handling posture, attention to pattern, and attention to blocks – could indicate improvement during BDT. We believe this study will benefit clinicians and researchers by providing information unavailable in clinical settings. Code and datasets are available online at https://github.com/anom-tmu/bdt-multiscopic.

Most East Asian rehabilitation centers offer chopsticks manipulation tests (CMT). In addition to impaired hand function, approximately two-thirds of stroke survivors have visual impairment related to eye movement. This article investigates the significance of combining finger joint angle estimation and a visual attention measurement in CMT. We present a multiscopic framework that consists of microscopic, mesoscopic, and macroscopic levels. We develop a feature extraction technique to build the finger kinematic model at the microscopic level. At the mesoscopic level, we propose an active perception ability to detect the position and geometry of the finger on the chopsticks. The proposed framework estimates the proximal interphalangeal (PIP) joint angle on the index finger during CMT using fully connected cascade neural networks (FCC-NN). At the macroscopic level, we implement a cognitive ability by measuring visual attention during CMT. We further evaluate the proposed framework with a conventional test that counts the number of peanuts (NP) which are moved from one bowl to another using chopsticks within a particular time frame. We introduce three evaluation indices, namely joint angle estimation movement (JAEM), chopstick attention movement (CAM), and chopstick tip movement (CTM), by detecting the local minima and maxima of the time series data. According to the experiment results, the velocity of these three evaluation indices could indicate improvement in hand and eye function during CMT. We expect this study to benefit therapists and researchers by providing valuable information that is not accessible in the clinic. Code and datasets are available online at https://github.com/anom-tmu/cmt-attention .

Computer vision-based cyber-physical-social systems (CPSS) are predicted to be the future of independent hand rehabilitation. However, there is a link between hand function and cognition in the elderly that this technology has not adequately supported. To investigate this issue, this paper proposes a multiscopic CPSS framework by developing hand–object interaction (HOI) based on visual attention. First, we use egocentric vision to extract features from hand posture at the microscopic level. With 94.87% testing accuracy, we use three layers of graph neural network (GNN) based on hand skeletal features to categorize 16 grasp postures. Second, we use a mesoscopic active perception ability to validate the HOI with eye tracking in the task-specific reach-to-grasp cycle. With 90.75% testing accuracy, the distance between the fingertips and the center of an object is used as input to a multi-layer gated recurrent unit based on recurrent neural network architecture. Third, we incorporate visual attention into the cognitive ability for classifying multiple objects at the macroscopic level. In two scenarios with four activities, we use GNN with three convolutional layers to categorize some objects. The outcome demonstrates that the system can successfully separate objects based on related activities. Further research and development are expected to support the CPSS application in independent rehabilitation.

Background: Pregnancy period is a period for mothers to empower themselves to be safe and comfortable. Pregnant women must acquire pregnancy-related information, such as warning signs of pregnancy, to avoid severe complications and even death during pregnancy and childbirth. Therefore, developing an application for pregnant women would be very helpful. Objective: This study aimed to apply Human-In-The-Loop design with an android application to detect pregnancy risk early and avoid maternal morbidity and mortality. Methods: We collected data from the cohort of 5324 pregnant women at the community health centers in the West Lombok District from 2020 to February 2021. The data included age, parity, height, inter-pregnancy interval, hemoglobin levels, upper arm circumference, previous diseases, and bleeding history. We developed a Human-In-The-Loop mobile application and employed the decision tree for identifying pregnancy danger signs. The midwife (human-in-the-loop) reviewed and clarified the data to generate the final detection and made a recommendation. Results: The ordinal regression model revealed that older patients who have more parity, lower height, the distance of children <2 years, hemoglobin <11 g/dl, upper arm circumference (UPC) <23.5 cm, have positive HBsAg, have HIV disease, have a history of diabetes mellitus (DM), have a history of hypertension, positive protein urine, and have other diseases are more likely to have a high maternal risk. The decision tree outperformed and obtained a high accuracy of 92% ± 0.0351 compared to the nine individual classifiers (Nearest Neighbors, Random Forest, Neural Net, AdaBoost, Gaussian Naïve Bayes, Bagging, Extra Tree, Gradient Boosting, and Stacking). Conclusion: The Human-In-The-Loop mobile app developed in this study can be used by healthcare professionals, especially midwives and nurses, to detect danger indications early in pregnancy, accurately diagnose the high risk of pregnancy, and provide treatment and care recommendations during pregnancy and childbirth.