Affiliations: Institute of Computing Technology, Chinese Academy of
Sciences, Beijing 100080, China | Graduate School of Chinese Academy of Sciences,
Beijing 100039, China | Department of Computer Science, Harbin Institute of
Technology, Harbin 150001, China. E-mail: {yhtian, tjhuang,
wgao}@ict.ac.cn
Abstract: The extensive amount of diversified Web-based information
necessitates the development of automated subject-specific Web site
classification techniques. Given that Web sites are in essence heterogeneous,
multi-structured and often accompanied with much noise, it is important to
design Web site classification algorithms that can scale well in the context of
noise and heterogeneity. In this paper, we propose a novel approach for Web
site classification based on the content, structure and context information of
Web sites. In our approach, the site structure is represented as a two-layered
tree, i.e., each page is modeled as a DOM (Document Object Model) tree, and a
page tree is used to hierarchically link all pages within the site. Two context
models are formulated to characterize the topical dependences between nodes in
the two-layered tree. Using the Hidden Markov Tree (HMT) as the statistical
model of page trees and DOM trees, a two-phase Web site classification
algorithm is presented. Moreover, for further improving accuracy while reducing
the classification overheads, a two-stage denoising procedure is adopted to
remove the noise information within sites, and an entropy-based strategy is
introduced to dynamically prune the page trees. The experiments demonstrate
that the proposed approach is able to offer high accuracy and efficient
processing performance.
Keywords: Web site classification, two-layered dependence tree, Hidden Markov Tree model, two-stage denoising, entropy-based pruning
