A Truly Joint Neural Architecture for Segmentation and Parsing

What it is about

The paper argues that Hebrew segmentation and dependency parsing should be solved jointly, as pre-neural parsers did. Each token is expanded into a lattice of all morphological analyses from an existing analyzer, and the lattice is linearized into one sequence of segments. A biaffine arc-factored parser scores arcs over that sequence; segments of unused analyses attach to an auxiliary node that is removed at inference, and a constrained maximum spanning tree enforces exactly one analysis per token with no arcs across analyses. On the UD Hebrew HTB split the single joint model gets about 97.7 segmentation F1 and 85.45 dependency F1, above the Stanza and Trankit pipelines in this setup. The error analysis of 50 dev sentences finds 108 parser errors, 70% of them prediction errors, and PP attachment (obl versus nmod) is the largest class.

Ideas

• I1. Constrained lattice in tools/hbo: keep every candidate split from the prefix and suffix tables for a token, let gparse score them, and enforce one split per token and no arcs across splits of the same token. Replace the single segmenter output and measure segmentation F1 and LAS with and without the constraint on a held-out Hebrew set.
• I2. An auxiliary root for unused segments: one extra node that absorbs the segments that the chosen analysis does not use, dropped at inference. The parser then decides which splits are real, and the number of dropped segments per sentence becomes a measure.
• I3. A regression set of the obl/nmod confusions: the paper's Table 8 lists 11 obl and 10 nmod errors, and the same confusions can be collected from treebanks/he/htb into a fixed list for PP attachment in 7. Syntax.

For Mova

I1 and I2 go into tools/hbo (segmenter and gparse), tested on the unpointed and pointed modes. Their lattice needs an analyzer; for Ancient Hebrew Mova has the macula tables, and for Modern Hebrew a lexicon is still missing, so I1 is blocked there until one exists. I3 goes into 7. Syntax as a regression list.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.eacl-long.84

@inproceedings{levi2024truly,
  title = {{A Truly Joint Neural Architecture for Segmentation and Parsing}},
  author = {Levi, Danit Yshaayahu and Tsarfaty, Reut},
  booktitle = {Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.eacl-long.84},
  url = {https://doi.org/10.18653/v1/2024.eacl-long.84}
}

References

30 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Meni Adler and Michael Elhadad. 2006. An unsupervised morpheme-based HMM for Hebrew morphological disambiguation. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 665–672, Sydney, Australia. Association for Computational Linguistics. find our card
2. Idan Brusilovsky and Reut Tsarfaty. 2022. Neural token segmentation for high token-internal complexity. In arXiv:2203.10845v1. ↗
3. Shay B. Cohen and Noah A. Smith. 2007. Joint morphological and syntactic disambiguation. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 208–217, Prague, Czech Republic. Association for Computational Linguistics. find
4. Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency parsing. In In International Conference on Learning Representations (ICLR). find our card
5. Michel Galley and Christopher D. Manning. 2009. Quadratic-time dependency parsing for machine translation. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 773–781, Suntec, Singapore. Association for Computational Linguistics. find
6. Aparna Garimella, Laura Chiticariu, and Yunyao Li. 2021. Domain-aware dependency parsing for questions. pages 4562–4568. find our card
7. Yoav Goldberg and Reut Tsarfaty. 2008. A single generative model for joint morphological segmentation and syntactic parsing. In Proceedings of ACL-08: HLT, pages 371–379, Columbus, Ohio. Association for Computational Linguistics. find
8. Spence Green and Christopher D. Manning. 2010. Better Arabic parsing: Baselines, evaluations, and analysis. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 394–402, Beijing, China. Coling 2010 Organizing Committee. find our card
9. Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear. find
10. Wonseok Hwang, Jinyeong Yim, Seunghyun Park, Sohee Yang, and Minjoon Seo. 2020. Spatial dependency parsing for 2d document understanding. CoRR, abs/2005.00642. ↗
11. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional lstm feature representations. In Transactions of the Association for Computational Linguistics, volume 4, pages 313–327. find our card
12. Dan Kondratyuk and Milan Straka. 2019a. 75 languages, 1 model: Parsing universal dependencies universally. In In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP), pages 2779–2795. find our card
13. Dan Kondratyuk and Milan Straka. 2019b. 75 languages, 1 model: Parsing Universal Dependencies universally. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2779–2795, Hong Kong, China. Association for Computational Linguistics. find our card
14. Amrith Krishna, Ashim Gupta, Deepak Garasangi, Pavankumar Satuluri, and Pawan Goyal. 2020a. Keep it surprisingly simple: A simple first order graph based parsing model for joint morphosyntactic parsing in Sanskrit. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4791–4797, Online. Association for Computational Linguistics. find
15. Amrith Krishna, Bishal Santra, Ashim Gupta, Pavankumar Satuluri, and Pawan Goyal. 2020b. A graph-based framework for structured prediction tasks in Sanskrit. Computational Linguistics, 46(4):785–845. find
16. Jindrich Libovický, Rudolf Rosa, and Alexander Fraser. 2019. How language-neutral is multilingual bert? volume abs/1911.03310. ↗
17. Amir Pouran Ben Veyseh Minh Nguyen, Viet Lai and Thien Huu Nguyen. 2021. Trankit: A light-weight transformer-based toolkit for multilingual natural language processing. In In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, volume 322, pages 80–90. find our card
18. Amir More, Amit Seker, Victoria Basmova, and Reut Tsarfaty. 2019. Joint transition-based models for morpho-syntactic parsing: Parsing strategies for MRLs and a case study from Modern Hebrew. volume 7, pages 33–48, Cambridge, MA. MIT Press. find our card
19. Amir More, Özlem Çetinoğlu, Çagri Çöltekin, Nizar Habash, Benoît Sagot, Djamé Seddah, Dima Taji, and Reut Tsarfaty. 2018. Conll-ul: Universal morphological lattices for universal dependency parsing. In International Conference on Language Resources and Evaluation. find
20. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajič, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1659–1666, Portorož, Slovenia. European Language Resources Association (ELRA). find our card
21. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. In In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 101–108. find our card
22. Jivnesh Sandhan, Amrith Krishna, Ashim Gupta, Laxmidhar Behera, and Pawan Goyal. 2021. A little pretraining goes a long way: A case study on dependency parsing task for low-resource morphologically rich languages. arXiv preprint arXiv:2102.06551. ↗
23. Wolfgang Seeker and Özlem Çetinoğlu. 2015. A graph-based lattice dependency parser for joint morphological segmentation and syntactic analysis. volume 3, pages 359–373, Cambridge, MA. MIT Press. find our card
24. Amit Seker and Reut Tsarfaty. 2020. A pointer network architecture for joint morphological segmentation and tagging. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4368–4378, Online. Association for Computational Linguistics. find our card
25. Amit Seker and Reut Tsarfaty. 2021. Alephbert: A hebrew large pretrained language model to start-off your hebrew nlp application with. In arXiv:2104.04052. ↗ our card
26. Dima Taji, Salam Khalifa, Ossama Obeid, Fadhl Eryani, and Nizar Habash. 2018. An Arabic morphological analyzer and generator with copious features. In Proceedings of the Fifteenth Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 140–150, Brussels, Belgium. Association for Computational Linguistics. find our card
27. Reut Tsarfaty. 2006. Integrated morphological and syntactic disambiguation for Modern Hebrew. In Proceedings of the COLING/ACL 2006 Student Research Workshop, pages 49–54, Sydney, Australia. Association for Computational Linguistics. find our card
28. Reut Tsarfaty, Dan Bareket, Stav Klein, and Amit Seker. 2020. From SPMRL to NMRL: What did we learn (and unlearn) in a decade of parsing morphologically-rich languages (MRLs)? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7396–7408, Online. Association for Computational Linguistics. find
29. Reut Tsarfaty, Djamé Seddah, Yoav Goldberg, Sandra Kuebler, Yannick Versley, Marie Candito, Jennifer Foster, Ines Rehbein, and Lamia Tounsi. 2010. Statistical parsing of morphologically rich languages (SPMRL) what, how and whither. In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 1–12, Los Angeles, CA, USA. Association for Computational Linguistics. find
30. Olcay Taner Yıldız, Begüm Avar, and Gökhan Ercan. 2019. An open, extendible, and fast Turkish morphological analyzer. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 1364–1372, Varna, Bulgaria. INCOMA Ltd. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/eacl-2024-levi-truly-joint-neural-architecture-segmentation/