Affiliations: School of Science and Technology, Communication University of China, Chaoyang, Beijing, P.R. China
Correspondence:
[*]
Corresponding author. Chen Li, School of Science andTechnology, Communication University of China, No.1 Dingfuzhuang East Street, Chaoyang District, Beijing 100024, P.R. China. Tel.: +86 010 65779319; Fax: +86 010 65779134; E-mails: [email protected]; [email protected].
Abstract: This paper proposed a cluster algorithm based on the combination of LDA (Latent Dirichlet allocation) probabilistic topic model and VSM (Vector Space Model), with the three-tier framework adopted containing text, topic and feature word. Although LDA alone has the ability to seek out the hidden topic knowledge, it is hard for the low-dimensional model to remain the integrity of the text information, leading to insufficient capacity for distinguishing texts. The paper is set to launch the cluster analysis in turns of feature words and topic through integrating two model above. With a better mix of LDA and VSM, the clustering effect will be improved, paralleling determining the optimal clustering number K of the K-means algorithms and optimum topic number T of LDA model. In order to design the algorithms more scientifically and effectively, silhouette coefficient and Dunn coefficient have been brought in to make assessments.
Keywords: Text cluster, LDA model, K-means algorithms, VSM model, silhouette
coefficient, Dunn coefficient
