Many studies have explored emotional and mental services that robots can provide for older adults, such as offering them daily conversation, news, music, or health information. However, the ethical issues raised by using sensors for frail older adults to monitor their daily movements or their medication intake, for instance, are still being discussed. In this study, we develop an older adult-guided, caregiver-monitored robot, Dori, which can detect and recognize movement by sensing human poses in accordance with two factors from the human-centered artificial intelligence (HCAI) framework. To design the care robot’s services based on sensing movement during daily activities, we conducted focus group interviews with two groups—caregivers and medical staff—on the topic of care robot services not for patients but for prefrail and frail elderly individuals living at home. Based on their responses, we derived the focal service areas of cognitive support, emotional support, physical activity support, medication management, and caregiver management. We also found the two groups differed in their ethical judgments in the areas of dignity, autonomy, controllability, and privacy for services utilizing sensing by care robots. Therefore, the pose recognition technology adopted in the present work uses only joint coordinate information extracted from camera images and thus is advantageous for protecting human dignity and personal information.