보조 분류기를 결합한 멀티 스트림 샴 융합 네트워크 기반 근전도를 이용한 사용자 인식에 관한 연구

Abstract

As information security becomes increasingly important in modern society, methods of user recognition use a fingerprint, face, and iris, which are accessible with less discomfort and have many other convenient advantages. However, they can be collected without consent and is prone to forgery due to lacking liveness. In addition, they are non-cancelable that cannot change the registered information. In order to overcome these problems, biosignals such as the electromyogram(EMG), electrocardiogram(ECG), and electroencephalogram(EEG) expressing an individual's unique physiological characteristics are introduced, which cannot be examined outside the body. Among the biosignals, EMG is a signal that measures the microcurrent generated when a muscle moves and represents complex information reflecting not only physiological information of muscle tissue but also neuromuscular control information. Since the signal changes according to their respective gestures, the non-cancelable issue can be efficiently tackled for user recognition. Existing user recognition techniques using EMG use handcraft features or features generated from neural network-based deep learning structures. Each method has its drawbacks; while the handcraft features lack generality as it uses information extracted in a single domain, deep learning requires model retraining to register new users. In addition, the EMG has non-periodic and non-linear characteristics; thus, data irregularity issues arise, resulting in low performance. This thesis proposes a user recognition method based on a multi-stream siamese fusion network combining an auxiliary classifier to resolve the abovementioned issues regarding user recognition using EMG. In the proposed method, features are extracted by an intrinsic mode function(IMF) using empirical mode decomposition(EMD) in consideration of the time-frequency domain characteristics of the signal. In order to solve the model retraining problem, the proposed method incorporates a model with a multi-stream siamese fusion network. The model consists of two sub networks and one decision network. The two sub networks are designed in a siamese structure to handle the model retraining problem and extract the compatibility function to calculate the similarity of the input data pairs. The decision network computes features from the compatibility function that comes from a convolutional neural network(CNN) organized by a stacking ensemble structure. Finally, the auxiliary classifier utilizing euclidean distance(ED) is employed to solve the data irregularity problem. The structure incorporates a convolutional neural network and an auxiliary classifier to realize the attention mechanism. The proposed multi-stream siamese fusion network combining the auxiliary classifier executed several experiments using CU_sEMG DB, which was built in multi-sessions. First, an experiment was conducted to identify the performance of the feature extraction method utilizing the time-frequency domain characteristics of EMG. As a result of the experiment, when the IMF 1-4 was applied for the multi-stream siamese fusion network without the auxiliary classifier, the average recognition accuracy was 92.01%, higher than the existing time-frequency domain characteristic. Second, in order to solve the data irregularity problem, a user recognition experiment was executed to verify the performance of the decision network combining the auxiliary classifier proposed in this thesis. User recognition using the proposed method classified 100 subjects with 94.35% accuracy, and the performance improved by 2.34% compared to the method without the auxiliary classifier. Next, to check the performance of the siamese network for alleviating the model retraining issue, the experiment proceeded using subjects that were not used for training. The result of the experiment using the EMG of new subjects illustrated that the model retraining problem was much relieved by showing an accuracy of 93.19%. Finally, to verify the proposed method's superiority, using the single-session data of CU_sEMG DB and Ninapro DB2, the proposed method's performance was compared to the existing methods' performance. The existing methods showed a significant decrease in accuracy when the number of subjects increased, indicating that the model was not adequately trained. On the contrary, the proposed method maintained similar performance as the number of subjects increased and outperformed the existing method, proving the superiority of the proposed method. In short, a new user recognition method based on a multi-stream siamese network was proposed in the thesis. The proposed method incorporates an auxiliary classifier with a multi-stream siamese network to deal with the problems of user recognition methods using EMG. The next step will be the research for noise cancellation technologies of EMG and new loss functions to improve user recognition performance. Another research issue will be network optimization to improve computational speed.