Affiliations: School of Knowledge Science, Japan Advanced Institute
of Science and Technology | BIRD, Japan Science and Technology Agency
(JST) | CREST, Japan Science and Technology Agency (JST)
Abstract: Since biomedical texts contain a wide variety of domain specific
terms, building a large dictionary to perform term matching is of great
relevance. However, due to the existence of null boundary between adjacent
terms, this matching is not a trivial problem. Moreover, it is known that
generative words cannot be comprehensively included in a dictionary because
their possible variations are infinite. In this study, we report our approach to dictionary building and
term matching in biomedical texts. Large amount of terms with/without
part-of-speech (POS) and/or category information were gathered, and a
completion program generated ∼1.36 million term variants
to avoid stemming problems when matching terms. The dictionary was stored in a
relational database management system (RDBMS) for quick lookup, and used by a
matching program. Since the matching operation is not restricted to a substring
surrounded by space characters, we can avoid the problem of null boundaries.
This feature is also useful for generative words. Experimental results on GENIA
corpus are promising: nearly half of the possible terms were correctly
recognized as a meaningful segment, and most of the remaining half could be
correctly recognized by some post-processing process, like chunking and further
decomposition. It should be remarked that although we have not used term cost,
connectivity cost, or syntactic information, reasonable segmentation and
dictionary lookup were performed in most cases.
Keywords: text processing, dictionary building, dictionary lookup and matching, sentence segmentation, term boundary
