Personality computing lies on the effort of designing computational models to recognize personality traits from different sources of input such as visual (image), verbal (text) and vocal (audio). To recognize personality traits from facial images, the recognition model needs to extract facial features such as eyes, mouth, nose, etc. Facial features are relevant for personality judgements where they are providing information about human expression and behaviours. For example, by detecting eye’s movement, dimension and size, machines can describe people’s personality. Basically, people with higher score of conscientiousness have greater fluctuations in their pupil size while people who blink faster are more neurotic. The potential of each facial feature to automatically describe human personality has prompted research in personality recognition using CNN-based techniques. On the other hand, convolutional neural networks (CNN) have proven to be greatly success in the field of image processing including face recognition and detection. Since face detection is a key task in personality recognition using facial features, adoption of CNN model is a reliable choice. This study aimed to explore CNN-based models, which have been used and modified in the personality computing field. In this paper, we review several CNN-based models which have been employed for personality traits recognition, with an emphasis on extraction of facial features. Based on the finding, there are four models (VGGNet, ResNet, FaceNet and OpenFace) which are widely used and frequently adopted by previous studies in personality trait recognition. The adoption of CNN- based models in facial features extraction for personality traits recognition helps in achieving a good accuracy result. Thus, there is a huge potential to enhance and modify CNN-based model to generate more efficient model for facial features extraction tasks. Initially, this paper briefly explains personality computing and facial features extraction for personality recognition task. Next, this paper discusses the characteristics of network layer in CNN-based models for facial features extraction and personality recognition. Finally, this paper concludes that each of these models has its own features strength that are potentially useful for facial features extraction in development of personality traits recognition model.
27 May 2025 in International Journal of Innovative Computing
In the field of personality computing research, the analysis on audio-visual input is predominantly used to detect human personality behaviors. With the advancement of computer vision technology, there has been significant enhancement in personality computing. Personality trait recognition is one of the applications under personality computing where the machine can analyze human behaviors and recognize personality traits via video analysis. In video input, there are different audio-visual features characteristics, consisting of visual (images) and audio (sounds) elements. Therefore, it is critical to employ appropriate deep learning techniques to most effectively extract important features from audio-visual input. The maturity of convolutional neural networks (CNNs) has been proven with promising prediction results for feature extraction and selection in image classification, sound detection, and face-emotion recognition. Thus, a variety of CNN-based techniques have been developed with different salient features and CNN layer modifications to learn and extract meaningful patterns and representations from images and videos. Due to the distinct characteristics of audio-visual features, hybrid CNN-based techniques were introduced to optimally analyze these modalities. This study aimed to explore hybrid CNN-based techniques used in video analysis for personality trait recognition systems. This study also provides an overview of current issues in the development of recognition models in personality computing using hybrid CNN-based techniques. The advantages of integrating audio and visual modalities in hybrid techniques are addressed, as well as their performance accuracy. The discussion finally summarizes the findings and potential future research directions.