Facial landmarks represent prominent feature points on the face that can be used as anchor points in many face-related tasks. So far, a lot of research has been done with the aim of achieving efficient extraction of landmarks from facial images. Employing a large number of feature points for landmark detection and tracking usually requires excessive processing time. On the contrary, relying on too few feature points cannot accurately represent diverse landmark properties, such as shape. To extract the 68 most popular facial landmark points efficiently, in our previous study, we proposed a model called EMTCNN that extended the multi-task cascaded convolutional neural network for real-time face landmark detection. To improve the detection accuracy, in this study, we augment the EMTCNN model by using two convolution techniques—dilated convolution and CoordConv. The former makes it possible to increase the filter size without a significant increase in computation time. The latter enables the spatial coordinate information of landmarks to be reflected in the model. We demonstrate that our model can improve the detection accuracy while maintaining the processing speed.

The human face is one of the most viewed visual objects in a person’s life and is used for identifying a person through facial landmarks, which includes the eyes, nose, mouth, and ears that make up a face. It is also possible to communicate nonverbally through the movements of facial landmarks; that is, change of facial expression. Thus, facial landmarks play a crucial role in human-related image analysis. Automatic facial landmark detection is a challenging problem in the field of computer vision, and various studies are underway. The emergence of Deep Neural Networks has played an important role in solving difficult problems in computer vision. Semantic segmentation is a field in which images are classified into pixel units and has also developed rapidly by incorporating deep learning. In this paper, we propose a method for accurately extracting facial landmarks using semantic segmentation. First, we introduce a semantic segmentation architecture for sophisticated landmark detection, and datasets composed of facial images and ground truth pairs. Then, we suggest how improve the performance of pixel classification by adjusting the imbalance of the number of pixels according to the face landmark. Through extensive experiments, we evaluated our approach using the metrics pixel accuracy and intersection over union.