Journal article

Publication year: 2023

Journal: Journal of Imaging

Volume number: 9

Issue number: 5

eISSN: 2313-433X

Open access status: Gold

DOI Link: https://doi.org/10.3390/jimaging9050104

Publisher: MDPI

Abstract: 
The accurate localization of facial landmarks is essential for several tasks, including face recognition, head pose estimation, facial region extraction, and emotion detection. Although the number of required landmarks is task-specific, models are typically trained on all available landmarks in the datasets, limiting efficiency. Furthermore, model performance is strongly influenced by scale-dependent local appearance information around landmarks and the global shape information generated by them. To account for this, we propose a lightweight hybrid model for facial landmark detection designed specifically for pupil region extraction. Our design combines a convolutional neural network (CNN) with a Markov random field (MRF)-like process trained on only 17 carefully selected landmarks. The advantage of our model is the ability to run different image scales on the same convolutional layers, resulting in a significant reduction in model size. In addition, we employ an approximation of the MRF that is run on a subset of landmarks to validate the spatial consistency of the generated shape. This validation process is performed against a learned conditional distribution, expressing the location of one landmark relative to its neighbor. Experimental results on popular facial landmark localization datasets such as 300 w, WFLW, and HELEN demonstrate the accuracy of our proposed model. Furthermore, our model achieves state-of-the-art performance on a well-defined robustness metric. In conclusion, the results demonstrate the ability of our lightweight model to filter out spatially inconsistent predictions, even with significantly fewer training landmarks.

Harvard Citation style: Gdoura, A., Deguenther, M., Lorenz, B. and Effland, A. (2023) Combining CNNs and Markov-like Models for Facial Landmark Detection with Spatial Consistency Estimates, Journal of Imaging, 9(5), Article 104. https://doi.org/10.3390/jimaging9050104

APA Citation style: Gdoura, A., Deguenther, M., Lorenz, B., & Effland, A. (2023). Combining CNNs and Markov-like Models for Facial Landmark Detection with Spatial Consistency Estimates. Journal of Imaging. 9(5), Article 104. https://doi.org/10.3390/jimaging9050104