Affiliations: Center of Computational Linguistics, Vytautas Magnus University, Donelaičio 52, 3000 Kaunas, Lithuania, e‐mail: [email protected], [email protected], [email protected] | Faculty of Computer Science, University of Twente, P.O.Box 217 7500 AE Enschede, the Netherlands, e‐mail: [email protected]
Abstract: This paper describes a preliminary experiment in designing a Hidden Markov Model (HMM)‐based part‐of‐speech tagger for the Lithuanian language. Part‐of‐speech tagging is the problem of assigning to each word of a text the proper tag in its context of appearance. It is accomplished in two basic steps: morphological analysis and disambiguation. In this paper, we focus on the problem of disambiguation, i.e., on the problem of choosing the correct tag for each word in the context of a set of possible tags. We constructed a stochastic disambiguation algorithm, based on supervised learning techniques, to learn hidden Markov model's parameters from hand‐annotated corpora. The Viterbi algorithm is used to assign the most probable tag to each word in the text.
Keywords: part of speech tagging, morphological disambiguation, HMM modeling, smoothing, hand‐annotated corpus
Journal: Informatica, vol. 15, no. 2, pp. 231-242, 2004
