A Hybrid Generative/Discriminative Model for Rapid Prototyping of Domain-Specific Named Entity Recognition

EasyChair Preprint no. 856

Abstract

We propose PYHSCRF, a novel tagger for domain-specific named entity recognition that only requires a few seed terms, in addition to unannotated corpora, and thus permits the iterative and incremental design of named entity (NE) classes for new domains. The proposed model is a hybrid of a generative model named PYHSMM and a semi-Markov CRF-based discriminative model, which play complementary roles in generalizing seed terms and in distinguishing between NE chunks and non-NE words. It also allows a smooth transition to full-scale annotation because the discriminative model makes effective use of annotated data when available. Experiments involving two languages and three domains demonstrate that the proposed method outperforms baselines.

Keyphrases: Bayesian model, Named Entity Recognition, weakly supervised learning

@Booklet{EasyChair:856,
  author = {Suzushi Tomori and Yugo Murawaki and Shinsuke Mori},
  title = {A Hybrid Generative/Discriminative Model for Rapid Prototyping of Domain-Specific Named Entity Recognition},
  howpublished = {EasyChair Preprint no. 856},
  doi = {10.29007/2zkv},
  year = {EasyChair, 2019}}