가상 심전도 신호 생성을 이용한 앙상블 합성곱 신경망 기반 사용자 인식 시스템에 관한 연구

Abstract

As information and communication technology develops, the smart environment is built to which IoT is applied. While identity authentication used in various fields along with the smart environment is very important, researchers have studied biometrics very secure and easy to use. Biometrics is the technical term for user’s body measurement and calculation based on physiological and behavioral features to send important information to a security system. Typical biometrics is for measuring faces, fingerprints, iris and voice. However, because using the information in the physiological and physical format exposed superficially can be at risk of forgery and modification, problems occur where applied to fin-tech, smart health care, and position-based services requiring high levels of security. To address this issue, user recognition using biometric signals, for example, ECG, has been studied. Researchers have studied the ECG signals as one of biometric signals, which do not involve stimulation and are not prone to be forged or modified to use them as a next-generation user authentication technology. The following problems should be addressed to use the ECG signal-based user recognition in the real environment. While the ECG signals are time-series data acquired as time elapses, it is required to acquire comparative data the same in size as the registered data every time. If the comparative data the same as the registered in size is not acquired, it is impossible to use the data for user recognition because of different data size. Furthermore, the heart rates and waveform of ECG signals change depending on personal physical activities, mental effects, or measured time bands although they are from the same user. If the data acquired while the concerned user state changes is used as a registered data, data regularization occurs to result in overfitting lowering recognition performance for the new data acquired. Therefore, this study suggests a network model of GAN(Generative Adversarial Network) based on an auxiliary classifier able to generate synthetic ECG signals to address the different data size issue. GAN is a method for making the generative model creating data confront the discriminative model discriminating between real data and generated data to improve performance each other. As the number of neural network layers increases, the generative model does not generate data similar to the real ECG signal, and the values of generated ECG signals because of data loss in the training process are distributed on the contrary. In this study, the generative model is designed to have an architecture of 9 convolutional layers, 2 pooling layers and a fully-connected layer. Because the discriminative model simply discriminates the real ECG signal and class information from the synthetic ECG signal and class information, it is designed to be a single convolutional network repeating convolutional operations not deeper than the generative model. Cosine similarity and Cross correlation are used to examine the similarity of synthetic ECG signals. The examination shows that cosine similarity for 89 participants was measured, which lied between 0.974 and 0.998 inclusive, and the average similarity was 0.991. The similarity measured by using the Euclidean distance based on cross correlation lied between 0.136 and 0.364 inclusive, and the average similarity was measured 0.25. This addressed the different data size issue by examining generation of synthetic ECG signals similar to the real ECG signals to create synthetic data although the registered data is not the same as the comparative data in size. Moreover, the multi-layer ensemble network is suggested to address the overfitting issue lowering recognition performance because of user state changes. First, the ECG signals are used as input data into the convolutional neural network in multi-layer architecture while the data showing heart rates and waveform changes are measured from participants before/after physical exercises and while they lie on the bed and stand. Respective convolutional neural networks were set up to have parameters different for each network to detect different features. The resultant ECG signals from each network were classified for each participant, and were integrated into one database to be used as a registered data for re-training. Using the result of low recognition rate resulting from parameters and wrong network design as the registered data contributes rather to lowering recognition performance. The registered data is configured by integrating only the ECG signals in the top-3 network showing good performance, not all result acquired from each network to correct the issue. User recognition is conducted for the reconfigured registered data by retraining the comparative data independent of time for the convolutional neural network. A comparative experiment was conducted by applying the real ECG signals acquired for this study to the single convolutional neural network using 2-dimensional ECG images, the one-dimensional convolutional neural network and the multi-layer ensemble network suggested in this study to address the conventional overfitting problem. In the experiment, the single convolutional neural network using 2-dimensional ECG images and the multi-layer convolutional neural network showed 94.4% and 95.7% of accuracy, respectively. The multi-layer convolutional ensemble neural network suggested in this study showed 98.5% of accuracy, implying better recognition performance than prior studies. Overfitting which occurs when the ECG signal acquired user state changes is used as a comparative data is addressed by the multi-layer ensemble neural network suggested in this study. Because it is hard to acquire a comparative data the same as the registered data in size in the real environment, and heart rates and waveform change as user state changes, recognition performance is lowered. Therefore, the synthetic ECG signals generated to address the different data size issue from the acquired ECG signal in the real environment were made into various combinations for each period to make the comparative data. It was then applied to the multi-layer ensemble network suggested in this study to conduct the user recognition experiment. The experiment showed 98.5% of recognition performance while using 5 periods of the real ECG signals, and 98.7% and 97% of accuracy, respectively, while repeating one synthetic ECG signal period and the last fourth period for the last period of the 4 real ECG signal periods. While using 2 synthetic ECG signal periods for the real 3 ECG signal periods, it showed 97.2% of accuracy, and the accuracy while repeating the last 3 periods for 3 real ECG signal periods was 96% which is 1.2% lower than the performance by using synthetic ECG signals. Therefore, although the registered data is different in size from the comparative data, the different data size and overfitting issues were addressed by applying the synthetic ECG signals generated as user state changes to the multi-layer ensemble network to demonstrate applicability to the real environment.