Robust Facial Expression Recognition using Gabor and LDP Feature Fusion using CCA

##plugins.themes.academic_pro.article.sidebar##

Published Mar 1, 2019
https://doi.org/10.47164/ijngc.v10i1.154
Download
PDF
Statistic

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...
Volume 10, Issue 1, March 2019

##plugins.themes.academic_pro.article.main##

Mahesh M Goyani
Department of Computer Engineering, Government Engineering College, Modasa Gujarat, India.

Abstract

Facial Expression Recognition has become vital considering its numerous applications including Human Computer Interaction, security, gaming, animation, medical field etc. In order to effectively implement these applications, the proposed method aims to increase the overall accuracy and robustness of the recognition system. Fusion of two feature extraction methods, namely Gabor and Local Directional Pattern (LDP) that are complementary in nature is carried out. Gabor Features focuses on the structural details whereas LDP targets the textural and subtler details in a robust manner. Fusion is carried out using Canonical Correlation Analysis (CCA) based fusion, as it effectively utilizes the correlation between the two features for fusion. For the efficient training of the classifiers, optimal feature vector is obtained by projecting the original feature vector on Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) subspace. To prove the robustness of the proposed method, it is tested on benchmark datasets like CK, JAFFE, TFEID and CASIA-VIZ under ideal as well as different conditions such as noisy environment, low resolution, small sample space, different facial components etc. The system is also tested on two spontaneous expression datasets called SFEW (standard) and WESFED (in-house). The proposed method has shown better performance than the state of the art methods.

##plugins.themes.academic_pro.article.details##

How to Cite
Mahesh M Goyani. (2019). Robust Facial Expression Recognition using Gabor and LDP Feature Fusion using CCA. International Journal of Next-Generation Computing, 10(1), 36–55. https://doi.org/10.47164/ijngc.v10i1.154
Crossref
Scopus
Google Scholar
Europe PMC

References

  1. Ahmed, F. and Kabir, M. H. (2012). ˜Directional ternary pattern (dtp) for facial expression recognition. in IEEE International Conference on Consumer Electronics. Las Vegas, pp. 265-266.
  2. Bartlett, M. S. et al. (2003). ˜Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction.. in Computer Vision and Pattern Recognition Workshop, pp. 53-53.
  3. Bashar, F. et al. (2013). ˜Robust facial expression recognition based on median ternary pattern (MTP). in International Conference on Electrical Information and Communication Technology. doi: 10.1109/EICT.2014.6777846.
  4. Butalia, A., Ingle, M. and Kulkarni, P. (2012). ˜Facial Expression Recognition for Security. 2(4), pp. 1449-1453.
  5. C. Shan, Sh. Gong, P. W. M. (2009). ˜Facial expression recognition based on local binary patterns: a comprehensive study. Image and Vision Computing, 27(6), pp. 803-816.
  6. Chen, F. and Kotani, K. (2007). ˜Facial Expression Recognition by SVM-based Two-stage Classifier on Gabor Features. IAPR Conference on Machine Vision Applications, pp. 453-456.
  7. Cid, F. et al. (2013). ˜A Real Time and Robust Facial Expression Recognition and Imitation approach for Affective Human-Robot Interaction Using Gabor filtering. pp. 2188-2193.
  8. Cowie, R. et al. (2001). ˜Emotion recognition in human-computer interaction. Signal Processing Magazine, IEEE, pp. 32-80. doi: 10.1109/79.911197.
  9. Donato, G. et al. (1999). ˜Classifying facial actions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(10), pp. 974-989. doi: 10.1109/34.799905.
  10. Eleftheriadis, S., Rudovic, O. and Pantic, M. (2015). ˜Discriminative shared gaussian processes for multiview and view-invariant facial expression recognition. IEEE Transactions on Image Processing, 24(1), pp. 189-204.
  11. Fasel, B. and Luettin, J. (2003). ˜Automatic facial expression analysis: A survey. Pattern Recognition, 36(1), pp. 259-275.
  12. Gaebel, W. and Wölwer, W. (1992). ˜Facial expression and emotional face recognition in schizophrenia and depression. European Archives of Psychiatry and Clinical Neuroscience, 242(1), pp. 46-52. doi: 10.1007/BF02190342.
  13. Goyani, M. and Patel, N. (2017a). ˜Judgmental Feature Based Facial Expression Recognition Systems and FER Datasets-A Comprehensive Study. International Journal of Next-Generation Computing, 8(1).
  14. Goyani, M. and Patel, N. (2017b). ˜Robust Facial Expression Recognition using Local Mean Binary Pattern. Electronic Letters on Computer Vision and Image Analysis, 16(1), pp. 54-67.
  15. Goyani, M. and Patel, N. (2018). ˜Robust Facial Expression Recognition using Local Haar Mean Binary Pattern. J. Inf. Sci. Eng., 34(5), pp. 1175-1186. doi: 10.6688/JISE.201809.
  16. Guo, Y., Zhao, G. and Pietikainen, M. (2012). ˜Dynamic facial expression ¨ recognition using longitudinal facial expression atlases. ECCV.
  17. Holder, R. P. and Tapamo, J. R. (2017). ˜Improved gradient local ternary patterns for facial expression recognition. EURASIP Journal on Image and Video Processing, 2017(1), pp. 42-57. doi: 10.1186/s13640-017-0190-5.
  18. Huang, D., Shan, C., Ardebilian, M., et al. (2011). ˜Facial Image Analysis Based on Local Binary Patterns : A Survey. IEEE Transactions on Image Processing.
  19. Huang, D., Shan, C., Ardabilian, M., et al. (2011). ˜Local Binary Patterns and its application to facial image analysis : A survey. IEEE Transactions on Systems, Man and Cybernetics, Part C: Applications and Reviews, 41(6), pp. 765-781.
  20. Jabid, T., Kabir, M. H. and Chae, O. (2010). ˜Facial expression recognition using Local Directional Pattern (LDP). 17th IEEE International Conference on Image Processing, pp. 1605-1608. doi: 10.1109/ICPR.2010.373.
  21. Klaser, A. and Marszalek, M. (2008). ˜A spatio-temporal descriptor based on 3d-gradients. BMVC.
  22. Kumari, J., Rajesh, R. and Kumar, A. (2016). ˜Fusion of features for the effective facial expression recognition. International Conference on Communication and Signal Processing, pp. 457-461. doi: 10.1109/ICCSP.2016.7754178.
  23. Liu, M. et al. (2014). ˜Learning expressionlets on spatio-temporal manifold for dynamic facial expression recognition. in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1749-1756. doi: 10.1109/CVPR.2014.226.
  24. Loob, C. et al. (2017). ˜Dominant and Complementary Multi-Emotional Facial Expression Recognition Using C-Support Vector Classification. IEEE 12th International Conference on Automatic Face & Gesture Recogition, pp. 833-838. doi: 10.1109/FG.2017.106.
  25. Luo, Y., Wu, C. M. and Zhang, Y. (2013). ˜Facial expression recognition based on fusion feature of PCA and LBP with SVM. Optik. Elsevier GmbH., 124(17), pp. 2767-2770. doi: 10.1016/j.ijleo.2012.08.040.
  26. Luo, Y., Zhang, T. and Zhang, Y. (2016). ˜A novel fusion method of PCA and LDP for facial expression feature extraction. Optik, 127(2), pp. 718-721. doi: 10.1016/j.ijleo.2015.10.147.
  27. Lyons, M. and Akamatsu, S. (1998). ˜Coding Facial Expressions with Gabor Wavelets. 3rd IEEE Conference on Automatic Face and Gesture Recognition, pp. 200-205. doi: 10.1109/AFGR.1998.670949.
  28. Lyons, M. J. et al. (2000). ˜Classifying Facial Attributes using a 2-D GaborWavelet Representation and Discriminant Analysis. in 4th International Conference on Automatic Face and Gesture Recognition, pp. 202-207.
  29. Martinez, B. et al. (2017). ˜Automatic Analysis of Facial Actions : A Survey. IEEE Transactions on Affective Computing. doi: 10.1109/TAFFC.2017.2731763.
  30. Mehrabian, A. (1968). ˜Communication without words. Psychology Today, 2, pp. 53-55. doi: 10.1016/B978-0-12-384727-0.00014-8.
  31. Meng, Z. et al. (2017). ˜Identity-Aware Convolutional Neural Network for Facial Expression Recognition’. doi: 10.1007/978-3-319-51814-5.
  32. Mollahosseini, A., Chan, D. and Mahoor, M. H. (2015). ˜Going Deeper in Facial Expression Recognition using Deep Neural Networks. in IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1-10. doi: 10.1109/WACV.2016.7477450.
  33. Nasoz, F. et al. (2004). ˜Emotion recognition from physiological signals using wireless sensors for presence technologies. Cognition, Technology & Work, 6(1), pp. 4-14. doi: 10.1007/s10111-003-0143-x.
  34. Ojala, T., Matti, P. and David, H. (1996). ˜A comparative study of texture measures with classification based on featured distributions. Pattern recognition, 29(1), pp. 51-59.
  35. Pantic, M. and Rothkrantz, L. J. M. (2000). ˜Automatic analysis of facial expressions: the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), pp. 1424-1445. doi: 10.1109/34.895976.
  36. Sariyanidi, E., Gunes, H. and Cavallaro, A. (2015). ˜Automatic analysis of facial affect: A survey of registration, representation, and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(6), pp. 1-22. doi: 10.1109/TPAMI.2014.2366127.
  37. Sariyanidi, E., Gunes, H. and Cavallaro, A. (2017). ˜Learning Bases of Activity for Facial Expression Recognition. IEEE Transactions on Image Processing, 26(4), pp. 1965-1978. doi: 10.1109/TIP.2017.2662237.
  38. Sheth, N. A. and Goyani, M. M. (2018). ˜A Comprehensive study of Geometric and Appearance based Facial Expression Recognition Methods. International Journal of Scientific Research in Science, Engineering and Technology, 4(2), pp. 163-175.
  39. Siddiqi, M. H. et al. (2015). ˜Human Facial Expression Recognition Using Stepwise Linear Discriminant Analysis and Hidden Conditional Random Fields. IEEE Transactions on Image Processing, 24(4), pp. 1386-1398. doi: Doi 10.1109/Tip.2015.2405346.
  40. Sun, Y. and Yu, J. (2017). ˜Facial expression recognition by fusing Gabor and Local Binary Pattern features. in International Conference on Multimedia Modeling. Cham, pp. 209-220. doi: 10.1007/978-3-319-51814-5.
  41. Wenfei, G. et al. (2012). ˜Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognition. Elsevier, 45(1), pp. 80-91. doi: 10.1016/j.patcog.2011.05.006.
  42. Xing, Y. and Luo, W. (2016). ˜Facial Expression Recognition Using Local Gabor Features and Adaboost Classifiers. IEEE Internation al Conference on Progress in Informatics and Computing, pp. 1-5.
  43. Yu, T. and Gu, X. (2017). ˜Facial Expression Recognition Using Double-
  44. Stage Sample-Selected SVM. International Conference on Intelligent Computing, Springer, pp. 304-315. doi: 10.1007/978-3-319-63309-1_28.
  45. Zhang, L., Tjondronegoro, D. and Chandran, V. (2014). ˜Random Gabor based templates for facial expression recognition in images with facial occlusion. Neurocomputing. Elsevier, 145, pp. 451-464. doi: 10.1016/j.neucom.2014.05.008.
  46. Zhang, W. et al. (2005). ˜Local Gabor Binary Pattern Histogram Sequence (LGBPHS): A Novel Non-Statical Model for Face Representation and Recognition. in 10th IEEE International Conference on Computer Vision, pp. 786-791.
  47. Zhao, G. et al. (2011). ˜Facial expression recognition from near-infrared videos. Image and Vision Computing, 29(9), pp. 607-619.