Mova0.5 pre-release⌕
☰
home / docs / papers / Modern Hebrew / Neural Modeling for Named Entities and Morphology (NEMO2)

Neural Modeling for Named Entities and Morphology (NEMO2)

What it is about

In Hebrew, named-entity boundaries often fall inside a space-delimited token, because a preposition or the definite article is attached to the name. The authors annotate the Hebrew Treebank (6,143 sentences: 4,937 train, 500 dev, 706 test) twice, with token-level and morpheme-level NER labels. They compare token-single, token-multi and morpheme models, all Bi-LSTM-CRF. Morpheme-based NER is better. With gold morphology, the morpheme model reaches about 80.0 token-level F1 on dev, but with the predicted segmentation of a standard pipeline (YAP) it falls to about 72.8. The proposed hybrid lets the NER tags prune the morphological lattice before disambiguation; this recovers most of the gap (about 78.5 on dev) and beats all token-based models. An error analysis by unseen-entity type shows that names built from a known name plus a known prefix gain most from morpheme models.

Ideas

For Mova

I1 goes into tools/hbo. I2 goes into Prepositions, prefixes, conjunctions and particles, with the gazetteer kept with the Hebrew reader. I3 is the measurement in tools/hbo. The paper is CC BY 4.0; only short phrases with a page may be quoted.

Topics: Named entity recognition

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00404

@article{bareket2021neural,
  title = {{Neural Modeling for Named Entities and Morphology (NEMO2)}},
  author = {Bareket, Dan and Tsarfaty, Reut},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.1162/tacl\_a\_00404},
  url = {https://doi.org/10.1162/tacl_a_00404}
}

References

50 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Paula Czarnowska, Sebastian Ruder, Edouard Grave, Ryan Cotterell, and Ann Copestake. 2019. Don’t forget the long tail! A comprehensive analysis of morphological generalization in bilingual lexicon induction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 974–983, Hong Kong, China. Association for Computational Linguistics. https:// doi.org/10.18653/v1/D19-1090 ↗
  2. Bea Alex, Claire Grover, Rongzhou Shen, and Mijail Kabadjov. 2010. Agile corpus annotation in practice: An overview of manual and automatic annotation of CVs. In Proceedings of the Fourth Linguistic Annotation Workshop, pages 29–37, Uppsala, Sweden. Association for Computational Linguistics. find
  3. Naama Ben-Mordecai. 2005. Hebrew Named Entity Recognition. Master’s thesis, Department of Computer Science, Ben-Gurion University. https://doi.org/10.1007/978-3-540 -70939-8 13 ↗
  4. Kareem Darwish. 2013. Named entity recognition using cross-lingual resources: Arabic as an example. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1558–1567, Sofia, Bulgaria. Association for Computational Linguistics. find
  5. Yassine Benajiba, Paolo Rosso, and José Miguel Benedı́Ruiz. 2007. Anersys: An Arabic named entity recognition system based on maximum entropy. In Computational Linguistics and Intelligent Text Processing, pages 143–153, Berlin, Heidelberg. Springer Berlin Heidelberg. find
  6. Leon Derczynski, Eric Nichols, Marieke van Erp, and Nut Limsopatham. 2017. Results of the WNUT2017 shared task on novel and emerging entity recognition. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 140–147, Copenhagen, Denmark. Association for Computational Linguistics. https://doi.org/10.18653/v1/W17 -4418 ↗
  7. Darina Benikova, Chris Biemann, and Marc Reznicek. 2014. NoSta-D named entity annotation for German: Guidelines and dataset. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014), pages 2524–2531, Reykjavik, Iceland. European Languages Resources Association (ELRA). find
  8. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. https://doi.org/10.1162/tacl a 00051 ↗
  9. Jenny Rose Finkel and Christopher D. Manning. 2009. Nested named entity recognition. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1, EMNLP ’09, pages 141–150, Stroudsburg, PA, USA. Association for Computational Linguistics. https://doi.org/10.3115/1699510 .1699529 ↗
  10. N. Chinchor, E. Brown, L. Ferro, and P. Robinson. 1999. Named entity recognition task definition. The MITRE Corporation and SAIC. find
  11. Karën Fort, Maud Ehrmann, and Adeline Nazarenko. 2009. Towards a methodology for named entities annotation. In Proceedings of the Third Linguistic Annotation Workshop (LAW III), pages 142–145, Suntec, Singapore. Association for Computational Linguistics. https://doi.org/10.3115/1698381 .1698406 random fields for robust character-based part-of-speech tagging. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2736–2743, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  12. Yoav Goldberg. 2014. Hebrew Wikipedia dependency parsed corpus, v.1.0. find
  13. Stav Klein and Reut Tsarfaty. 2020. Getting the ↗
  14. Yoav Goldberg and Reut Tsarfaty. 2008. A single generative model for joint morphological segmentation and syntactic parsing. In Proceedings of ACL-08: HLT, pages 371–379, Columbus, Ohio. Association for Computational Linguistics. find
  15. Spence Green and Christopher D. Manning. 2010. Better Arabic parsing: Baselines, evaluations, and analysis. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 394–402, Beijing, China. Coling 2010 Organizing Committee. find our card
  16. Lingpeng Kong, Chris Dyer, and Noah A Smith. 2015. Segmental recurrent neural networks. arXiv preprint arXiv:1511.06018. ↗
  17. Dilek Küçük and Fazli Can. 2019. A tweet dataset annotated for named entity recognition and stance detection. CoRR, abs/1901.04787. ↗
  18. Onur Güngör, Suzan Üsküdarli, and Tunga Güngör. 2018. Improving named entity recognition by jointly learning to disambiguate morphological tags. CoRR, abs/1807.06683. ↗
  19. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. CoRR, abs/1603.01360. ↗
  20. Nizar Habash and Owen Rambow. 2005. Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 573–580, Ann Arbor, Michigan. Association for Computational Linguistics. https://doi.org/10.3115/1219840 .1219911 ↗ our card
  21. LDC. 2008. ACE (automatic content extraction) english annotation guidelines for entities version 6.6. find
  22. Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3219–3232, Brussels, Belgium. Association for Computational Linguistics. https://doi.org/10.18653/v1/D18 -1360 ↗
  23. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. CoRR, abs/1508.01991. ↗
  24. Katharina Kann, Ryan Cotterell, and Hinrich Schütze. 2016. Neural morphological analysis: Encoding-decoding canonical segments. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 961–967, Austin, Texas. Association for Computational Linguistics. find
  25. Xuezhe Ma and Eduard H. Hovy. 2016. End-to-end sequence labeling via bi-directional LSTM-CNNS-CRF. CoRR, abs/1603.01354. ↗
  26. Behrang Mohit, Nathan Schneider, Rishav Bhowmick, Kemal Oflazer, and Noah A. Smith. 2012. Recall-oriented learning of named enti-Apostolos Kemos, Heike Adel, and Hinrich Schütze. 2019. Neural semi-Markov conditional ties in Arabic Wikipedia. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 162–173, Avignon, France. Association for Computational Linguistics. find
  27. Sampo Pyysalo, Filip Ginter, Juho Heimonen, Jari Björne, Jorma Boberg, Jouni Järvinen, and Tapio Salakoski. 2007. Bioinfer: A corpus for information extraction in the biomedical domain. BMC bioinformatics, 8(1):50. https:// doi.org/10.1186/1471-2105-8-50 ↗
  28. Amir More, Amit Seker, Victoria Basmova, and Reut Tsarfaty. 2019. Joint transition-based models for morpho-syntactic parsing: Parsing strategies for MRLs and a case study from modern Hebrew. Transactions of the Association for Computational Linguistics, 7:33–48. https://doi.org/10.1162/tacl a 00253 ↗ our card
  29. Lev Ratinov and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009), pages 147–155, Boulder, Colorado. Association for Computational Linguistics. https://doi.org/10 .3115/1596374.1596399 ↗
  30. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan McDonald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007. find
  31. The CoNLL 2007 shared task on dependency parsing. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 915–932, Prague, Czech Republic. Association for Computational Linguistics. find
  32. Nils Reimers and Iryna Gurevych. 2017. Optimal hyperparameters for deep LSTM-networks for sequence labeling tasks. CoRR, abs/1707. 06799. find
  33. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 532–1543, Doha, Qatar. Association for Computational Linguistics. https://doi.org/10.3115/v1/D14 -1162 ↗
  34. Barbara Plank, Dirk Hovy, and Anders Søgaard. 2014. Learning part-of-speech taggers with inter-annotator agreement loss. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 742–751, Gothenburg, Sweden. Association for Computational Linguistics. https://doi.org/10.3115/v1/E14-1078 ↗
  35. Djamé Seddah, Reut Tsarfaty, Sandra Kübler, Marie Candito, Jinho D. Choi, Richárd Farkas, Jennifer Foster, Iakes Goenaga, Koldo Gojenola Galletebeitia, Yoav Goldberg, Spence Green, Nizar Habash, Marco Kuhlmann, Wolfgang Maier, Joakim Nivre, Adam Przepiórkowski, Ryan Roth, Wolfgang Seeker, Yannick Versley, Veronika Vincze, Marcin Woliński, Alina Wróblewska, and Eric Villemonte de la Clergerie. 2013. Overview of the SPMRL 2013 shared task: A cross-framework evaluation of parsing morphologically rich languages. In Proceedings of the Fourth Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 146–182, Seattle, Washington, USA. Association for Computational Linguistics. find
  36. Wolfgang Seeker and Özlem Çetinoğlu. 2015. A graph-based lattice dependency parser for joint morphological segmentation and syntactic analysis. Transactions of the Association for Computational Linguistics, 3:359–373. https://doi.org/10.1162/tacl a 00144 ↗ our card
  37. Amit Seker and Reut Tsarfaty. 2020. A pointer network architecture for joint morphological segmentation and tagging. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4368–4378, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020 .findings-emnlp.391 ↗ our card
  38. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL -Shared Task, pages 1–40, Jeju Island, Korea. Association for Computational Linguistics. find our card
  39. Yan Shao, Christian Hardmeier, and Joakim Nivre. 2018. Universal word segmentation: Implementation and interpretation. Transactions of the Association for Computational Linguistics, 6:421–435. https://doi.org/10.1162/tacl a 00033 ral Language Processing (EMNLP-IJCNLP): System Demonstrations, pages 259–264, Hong Kong, China. Association for Computational Linguistics. https://doi.org/10 .18653/v1/D19-3044 ↗ our card
  40. Reut Tsarfaty, Djamé Seddah, Yoav Goldberg, Sandra Kuebler, Yannick Versley, Marie Candito, Jennifer Foster, Ines Rehbein, and Lamia Tounsi. 2010. Statistical parsing of morphologically rich languages (SPMRL) what, how and whither. In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 1–12, Los Angeles, CA, USA. Association for Computational Linguistics. find
  41. Yan Shao, Christian Hardmeier, Jörg Tiedemann, and Joakim Nivre. 2017. Character-based joint segmentation and POS tagging for Chinese using bidirectional RNN-CRF. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 173–183, Taipei, Taiwan. Asian Federation of Natural Language Processing. find
  42. Khalil Sima’an, Alon Itai, Yoad Winter, Alon Altman, and Noa Nativ. 2001. Building a treebank of modern Hebrew text. Traitement Automatique des Langues, 42(2):347–380. find
  43. Hye-Jeong Song, Byeong-Cheol Jo, Chan-Young Park, Jong-Dae Kim, and Yu-Seop Kim. 2018. Comparison of named entity recognition methodologies in biomedical documents. Biomedical Engineering Online, 17(2):158. https:// doi.org/10.1186/s12938-018-0573-6 ↗
  44. Erik F. Tjong Kim Sang. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003 - Volume 4, CONLL ’03, pages 142–147, Stroudsburg, PA, USA. Association for Computational Linguistics. https://doi.org/10.3115/1119176 .1119195 ↗
  45. Gökhan Tür, Dilek Hakkani-tür, and Kemal Oflazer. 2003. A statistical information extraction system for turkish. Natural Language Engineering, 9(2):181–210. https://doi.org/10.1017/S135132490200284X Clara Vania, Andreas Grivas, and Adam Lopez. 2018. What do character-level models learn about morphology? The case of dependency parsing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2573–2583, Brussels, Belgium. Association for Computational Linguistics. https://doi.org/10.18653/v1/D18-1278 ↗
  46. Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Robert Belvin, and Ann Houston. 2013. Ontonotes release 5.0. Linguistic Data Consortium, Philadelphia, PA. find
  47. Reut Tsarfaty, Dan Bareket, Stav Klein, and Amit Seker. 2020. From SPMRL to NMRL: What did we learn (and unlearn) in a decade of parsing morphologically-rich languages (MRLs)? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7396–7408, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020 .acl-main.660 ↗
  48. Jie Yang, Shuailong Liang, and Yue Zhang. 2018. Design challenges and misconceptions in neural sequence labeling. In Proceedings of the 27th International Conference on Computational Linguistics (COLING). https:// doi.org/10.18653/v1/P18-4013 ↗
  49. Jie Yang and Yue Zhang. 2018. NCRF++: An open-source neural sequence labeling toolkit. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. find
  50. Reut Tsarfaty, Shoval Sadde, Stav Klein, and Amit Seker. 2019. What’s wrong with Hebrew NLP? And how to make it right. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu-Seid Muhie Yimam, Iryna Gurevych, Richard Eckart de Castilho, and Chris Biemann. 2013. WebAnno: A flexible, Web-based and visually supported system for distributed annotations. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 1–6, Sofia, Bulgaria. Association for Computational Linguistics. find our card

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/tacl-2021-bareket-neural-modeling-named-entities-morphology/