얼굴 표정 인식 개선을 위한 무표정 감정 분석에 관한 연구

Abstract

As studies have been conducted on how humans think and act to provide human-friendly high value-added services, the methods of interaction between humans and computers began to be researched. As a result, a new field called human-computer interaction (HCI) has been developed to facilitate the interaction between humans and computers. In order to provide human-friendly services and applications, the recognition of human emotions needs be researched. Among various emotion recognition methods, emotion recognition through facial expressions has enabled human-computer interaction and can be used in a variety of applications. However, it is still difficult to recognize expressionless face. Hence, it is necessary to develop a new method to recognize and analyze even fine facial expressions of emotions.

Consequently, in this study, we propose a method of improving emotion recognition through facial expressions by preprocessing facial expression images that cannot be recognized and performing secondary analysis of the emotion of expressionless images recognized as neutral emotion. In the preprocessing step, the facial image is resized and rotated using the Shape Model, which is a face normalization model, and then normalized to 100×100 pixels. The facial features and emotions in the normalized facial images are identified using the Emotion API. If the facial expression is recognized to be a neutral emotion, the emotion is classified again and the representative emotion is extracted and recognized. The recognized emotions were compared with the actual emotions and the accuracy rate of emotion recognition was very high at 87%.

In an experiment conducted with actual subjects using the proposed method, 100 facial images that were determined to have neutral emotion from the secondary classification were compared with the actual emotions. The result showed that the total accuracy rate of emotion recognition was 73.7%. Happiness and sadness had a very high recognition accuracy rate of 94% compared to other emotions. From analyzing the age and sex of the subjects in the images of the secondary classification, we found that the faces of teenagers were most commonly associated with sadness, while the emotions of those over the age of 20 were most commonly associated with happiness. When the subjects were asked to categorize themselves into 8 different facial expression styles, a high proportion of teenagers tended to express their emotions outwardly, and as people became older, fewer people tended to express their emotions overtly. Consequently, to improve facial expression recognition using the method proposed in this study, the emotions of expressionless faces were analyzed again. Then, the results were compared with the actual emotions of the subjects, and we found that the emotion recognition rate can be improved by recognizing fine non-neutral emotions even from even expressionless faces.

The proposed facial expression recognition method with improved emotion recognition rate is expected to be applicable to many industries including education, counseling, telemedicine, and real estate brokerage, and various advanced services such as personalized services, personal recommendation services, and emotional marketing.