Mova0.5 pre-release⌕
☰
home / docs / papers / Modern Hebrew / A Pointer Network Architecture for Joint Morphological Segmentation and Tagging

A Pointer Network Architecture for Joint Morphological Segmentation and Tagging

What it is about

Morphological disambiguation (MD) picks one decomposition of each token into morphemes. The authors feed a symbolic lattice of all analyses per token (HebLex for Hebrew, TRMorph2 for Turkish) into a neural pointer network. The lattice is flattened into a sequence of analyses, each analysis is embedded by averaging its morpheme properties, and the decoder points at one analysis per token with copy attention. On the CoNLL18 UD test sets, with infused lattices (gold path forced in), Hebrew joint segmentation and tagging reached aligned-segment F1 94.41 and Turkish 96.6, both at or above the shared-task leaders. With the realistic, uninfused lattices, Hebrew reached 91.3, level with the leader's 91.36. On the SPMRL Hebrew treebank the model beat the earlier non-neural MoreMD on all settings (test uninfused 90.49 against 87.53). Hebrew averages 10 analyses per token against 2.6 in Turkish, and of 850 wrong predictions in Hebrew, 330 had no correct analysis in the lattice.

Ideas

For Mova

I1 is a change to the evaluation in tools/hbo. I2 and I4 are for the segmenter in tools/hbo (tables lex, pre). I3 is for world, and the tool notes are in 13. Biblical Hebrew in Mova: the tools. The neural part is not for Mova; only the lattice-choice framing is.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.findings-emnlp.391

@inproceedings{seker2020pointer,
  title = {{A Pointer Network Architecture for Joint Morphological Segmentation and Tagging}},
  author = {Seker, Amit and Tsarfaty, Reut},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.391},
  url = {https://doi.org/10.18653/v1/2020.findings-emnlp.391}
}

References

42 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Meni Adler and Michael Elhadad. 2006a. An unsupervised morpheme-based HMM for Hebrew morphological disambiguation. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 665–672, Sydney, Australia. Association for Computational Linguistics. find our card
  2. Meni Adler and Michael Elhadad. 2006b. An unsupervised morpheme-based hmm for hebrew morphological disambiguation. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, ACL-44, pages 665– 672, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  3. Roy Bar-haim, Khalil Sima’an, and Yoad Winter. 2008. Part-of-speech tagging of modern hebrew text. Nat. Lang. Eng., 14(2):223–251. find
  4. Dan Bareket and Reut Tsarfaty. 2020. Neural modeling for named entities and morphology (nemo 2 ). find
  5. Bernd Bohnet, Joakim Nivre, Igor Boguslavsky, Richárd Farkas, Filip Ginter, and Jan Hajič. 2013. Joint morphological and syntactic analysis for richly inflected languages. Transactions of the Association for Computational Linguistics, 1:415–428. find
  6. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. ↗
  7. Yoav Goldberg and Michael Elhadad. 2010. An efficient algorithm for easy-first non-directional dependency parsing. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 742–750, Los Angeles, California. Association for Computational Linguistics. find
  8. Nizar Habash and Owen Rambow. 2005. Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 573– 580, Ann Arbor, Michigan. Association for Computational Linguistics. find our card
  9. Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh. 2013. Morphological analysis and disambiguation for dialectal Arabic. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 426–432, Atlanta, Georgia. Association for Computational Linguistics. find
  10. Salam Khalifa, Nasser Zalmout, and Nizar Habash. 2020. Morphological analysis and disambiguation for Gulf Arabic: The interplay between resources and methods. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 3895– 3904, Marseille, France. European Language Resources Association. find our card
  11. Stav Klein and Reut Tsarfaty. 2020. Getting the find
  12. Faisal Ladhak, Ankur Gandhe, Markus Dreyer, Lambert Mathias, Ariya Rastrow, and Björn Hoffmeister. 2016. Latticernn: Recurrent neural networks over lattices. In INTERSPEECH. find
  13. John Lee, Jason Naradowsky, and David A. Smith. 2011. A discriminative model for joint morphological disambiguation and dependency parsing. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 885–894, Portland, Oregon, USA. Association for Computational Linguistics. find
  14. Wang Ling, Chris Dyer, Alan W Black, Isabel Trancoso, Ramón Fermandez, Silvio Amir, Luı́s Marujo, and Tiago Luı́s. 2015. Finding function in form: Compositional character models for open vocabulary word representation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1520–1530, Lisbon, Portugal. Association for Computational Linguistics. find
  15. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412–1421, Lisbon, Portugal. Association for Computational Linguistics. find
  16. Yuval Marton, Nizar Habash, and Owen Rambow. 2013. Dependency parsing of modern standard Arabic with lexical and inflectional features. Computational Linguistics, 39(1):161–194. find our card
  17. Amir More, Özlem Çetinoğlu, Çağrı Çöltekin, Nizar Habash, Benoı̂t Sagot, Djamé Seddah, Dima Taji, and Reut Tsarfaty. 2018. CoNLL-UL: Universal morphological lattices for Universal Dependency parsing. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC’18). find
  18. Amir More, Amit Seker, Victoria Basmova, and Reut Tsarfaty. 2019. Joint transition-based models for morpho-syntactic parsing: Parsing strategies for MRLs and a case study from modern Hebrew. Transactions of the Association for Computational Linguistics, 7:33–48. find our card
  19. Amir More and Reut Tsarfaty. 2016. Data-driven morphological analysis and disambiguation for morphologically rich languages and universal dependencies. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 337–348, Osaka, Japan. find
  20. The COLING 2016 Organizing Committee. find
  21. Thomas Mueller, Helmut Schmid, and Hinrich Schütze. 2013. Efficient higher-order CRFs for morphological tagging. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 322–332, Seattle, Washington, USA. Association for Computational Linguistics. find
  22. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajič, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1659–1666, Portorož, Slovenia. European Language Resources Association (ELRA). find our card
  23. Ryan Roth, Owen Rambow, Nizar Habash, Mona Diab, and Cynthia Rudin. 2008. Arabic morphological tagging, diacritization, and lemmatization using lexeme models and feature ranking. In Proceedings of ACL-08: HLT, Short Papers, pages 117–120, Columbus, Ohio. Association for Computational Linguistics. find our card
  24. Benoı̂t Sagot. 2018. A multilingual collection of CoNLL-u-compatible morphological lexicons. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  25. Haşim Sak, Tunga Güngör, and Murat Saraçlar. 2009. Morphological disambiguation of turkish text with perceptron algorithm. In Proceedings of the 8th International Conference on Computational Linguistics and Intelligent Text Processing, CICLing ’07, page 107–118, Berlin, Heidelberg. Springer-Verlag. find
  26. Wolfgang Seeker and Özlem Çetinoğlu. 2015. A graph-based lattice dependency parser for joint morphological segmentation and syntactic analysis. Transactions of the Association for Computational Linguistics, 3:359–373. find our card
  27. Yan Shao, Christian Hardmeier, and Joakim Nivre. 2018. Universal word segmentation: Implementation and interpretation. Transactions of the Association for Computational Linguistics, 6:421–435. find
  28. Steven Shearing, Christo Kirov, Huda Khayrallah, and David Yarowsky. 2018. Improving low resource machine translation using morphological glosses (non-archival extended abstract). In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Papers), pages 132–139, Boston, MA. Association for Machine Translation in the Americas. find
  29. K. Sima’any, A. Itaiz, Y. Winterz, A. Altmanz, and N. Nativx. 2001. Building a tree-bank of modern hebrew text. volume 42, pages 347–380. Traitement automatique des langues. find
  30. Matthias Sperber, Graham Neubig, Jan Niehues, and Alex Waibel. 2017. Neural lattice-to-sequence models for uncertain inputs. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1380–1389, Copenhagen, Denmark. Association for Computational Linguistics. find
  31. Matthias Sperber, Graham Neubig, Ngoc-Quan Pham, and Alex Waibel. 2019. Self-attentional models for lattice inputs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1185–1197, Florence, Italy. Association for Computational Linguistics. find
  32. Milan Straka and Jana Straková. 2017. Tokenizing, pos tagging, lemmatizing and parsing ud 2.0 with udpipe. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 88–99, Vancouver, Canada. Association for Computational Linguistics. find our card
  33. Jinsong Su, Zhixing Tan, Deyi Xiong, Rongrong Ji, Xiaodong Shi, and Yang Liu. 2017. Lattice-based recurrent neural network encoders for neural machine translation. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17, pages 3302–3308. AAAI Press. find
  34. Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 2692–2700. Curran Associates, Inc. find
  35. Linlin Wang, Zhu Cao, Yu Xia, and Gerard de Melo. 2016. Morphological segmentation with window lstm neural networks. In AAAI. find
  36. Eray Yildiz, Caglar Tirkaz, H. Sahin, Mustafa Eren, and Omer Sonmez. 2016. A morphology-aware network for morphological disambiguation. In AAAI Conference on Artificial Intelligence. find
  37. Nasser Zalmout and Nizar Habash. 2017. Don’t throw those morphological analyzers away just yet: Neural morphological disambiguation for Arabic. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 704– 713, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  38. Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. CoNLL 2018 shared task: Multilingual parsing from raw text to universal dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–21, Brussels, Belgium. Association for Computational Linguistics. find our card
  39. Çağrı Çöltekin. 2014. A set of open source tools for turkish natural language processing. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland. European Language Resources Association (ELRA). find
  40. Dima Taji, Salam Khalifa, Ossama Obeid, Fadhl Eryani, and Nizar Habash. 2018. An Arabic morphological analyzer and generator with copious features. In Proceedings of the Fifteenth Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 140–150, Brussels, Belgium. Association for Computational Linguistics. find our card
  41. Reut Tsarfaty, Dan Bareket, Stav Klein, and Amit Seker. 2020. From SPMRL to NMRL: what did we learn (and unlearn) in a decade of parsing morphologically-rich languages (mrls)? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 7396–7408. find
  42. Reut Tsarfaty, Djamé Seddah, Yoav Goldberg, Sandra Kuebler, Yannick Versley, Marie Candito, Jennifer Foster, Ines Rehbein, and Lamia Tounsi. 2010. Statistical parsing of morphologically rich languages (SPMRL) what, how and whither. In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 1–12, Los Angeles, CA, USA. Association for Computational Linguistics. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/ud-2020-seker-pointer-network-architecture-joint-morphological/