基于人类注意机制的微表情检测方法

Subjects: Psychology >> Social Psychology submitted time 2023-03-28 Cooperative journals: 《心理科学进展》

李婧婷 东子朝 刘烨 王甦菁 庄东哲

Abstract： Micro-expressions are facial movements that are extremely short and not easily perceived, often generated under high pressure. Micro-expressions can reveal the individual's hidden real emotions and are important non-verbal communication clues, widely used in lies detection and other fields. Due to the difficulty of eliciting, collecting, and labeling micro-expression samples, micro-expression-related research becomes a typical small-sample-size (SSS) problem. In order to enlighten the application of micro-expression analysis technology in complex real-life scenarios such as national security and clinical consultation, this study focuses on the SSS problem and proposes a micro-expression spotting method based on human attention mechanism with multi-branching self-supervised learning through the intersection of computer and psychology. First, this study conducts an exploration related to attentional resources based on the cognitive mechanisms of psychological micro-expressions. A behavioral-experimental paradigm combining eye-movement techniques and a presentation-judgment paradigm with subthreshold emotion priming effects was used to examine the cognitive mechanisms of selective attention allocation in micro-expression recognition and to refine the distinct regions of interest in human recognition of micro-expressions. Thus, the model is effectively and directly enabled to acquire important micro-expression features from the input information. Then the relevant attention modules are further generated from multi-dimensions (time domain, spatial domain, and channel domain) by the deep learning network to improve the performance of the network in extracting micro-expression features with the limited sample size. Second, this study proposes a multi-branching self-supervised learning method based on the human attention mechanism for micro-expression spotting. Training in many unlabeled video samples for the pre-text tasks enables the model to extract features from regions of interest of micro-expressions, including structural and detail features and video dynamic change patterns. Thus, the limitation caused by the SSS problem could be avoided. Finally, the current data released for micro-expressions are video samples and do not include the corresponding depth information. This study will carry out a depth information-based micro-expression spotting method based on the first micro-expression database that includes image depth information being created by our research team. It enables self-supervised learning to learn the corresponding action patterns from the geometric information of the scene. This research will achieve theoretical and technological breakthroughs in the field of automatic micro-expression spotting, improve the accuracy and reliability, and lay the foundation for the application of micro-expression spotting in realistic and complex scenarios. Second, it can achieve the data augmentation of micro-expression samples by mining micro-expression clips in unlabeled videos. Thus, the micro-expression small sample problem could be solved, and the performance improvement of traditional supervised micro-expression spotting methods could be improved.

 Hits 240  Downloads 111  Comment