POSTER: A Pyramid Cross-Fusion Transformer Network for Facial Expression Recognition

Abstract

Facial expression recognition (FER) is an important task in computer vision, having practical applications in areas such as human-computer interaction, education, health-care, and online monitoring. In this challenging FER task, there are three key issues especially prevalent: inter-class similarity, intra-class discrepancy, and scale sensitivity. While existing works typically address some of these issues, none have fully addressed all three challenges in a unified framework. In this paper, we propose a two-stream Pyramid crOss-fuSion TransformER network (POSTER), that aims to holistically solve all three issues. Specifically, we design a transformer-based cross-fusion method that enables effective collaboration of facial landmark features and image features to maximize proper attention to salient facial regions. Furthermore, POSTER employs a pyramid structure to promote scale invariance. Extensive experimental results demonstrate that our POSTER achieves new state-of-the-art results on RAF-DB (92.05%), FERPlus (91.62%), as well as AffectNet 7 class (67.31%) and 8 class (63.34%). Code is available at https://github.com/zczcwh/POSTER.

Locations

• arXiv (Cornell University) - View - PDF