Neural network applications for automatic new topic identification on excite web search engine data logs

Abstract

The analysis of contextual information in search engine query logs is an important, yet difficult task. Users submit few queries, and search multiple topics sometimes with closely related context. Identification of topic changes within a search session is an important branch of contextual information analysis. The purpose of this study is to propose a topic identification algorithm using neural networks. A sample from the Excite data log is selected to train the neural network and then the neural network is used to identify topic changes in the data log. As a result, 76% of topic shifts and 92% of topic continuations are identified correctly.