문서 내 단어 분포차를 고려한 개선된 범주 분류 방법

Abstract

Recently, the population of users of Social Network Service has been increa-sing due to the development of smart devices that is where big-data be are accumulated. Pattern recognition, machine learning, nature language processing can process and analyze the big-data. Nature language processing is one of the methods which deals with human language using computer. This area fo-cuses on a human availability, and many other applications such as keyword extraction, information retrieval, document summarization, document classificat-ion and so on about a human availability.

This paper proposed the method of extracting keyword that special feature words assign high score and without discernment words assign low score in document. First, we extract one of the representative word that same meaning and different shape words such as neologism, abbreviation and synonym word using ‘redirect’ option of Wikipedia. And then, we calculate the deviation to consider of all documents. Principal keyword have high weight value by assigning additional values to words above threshold value.

As a result, the precision rate has been increased up to 1.15% than the TF-IDF method.