Temporal Relation Classification in Hebrew

What it is about

The paper asks, for two event mentions in a Hebrew news text, which happened first: temporal relation classification (TRC). No public Hebrew TRC data existed. The authors annotate 78 articles of the Hebrew UD treebank (HTB), giving 7,260 event pairs taken from two-sentence sliding windows. Events are verbs, excluding infinitives and gerunds. Each pair gets BEFORE, AFTER, EQUAL or VAGUE, following the MATRES guidelines with a multi-axis scheme (a main timeline, plus parallel and orthogonal axes; pairs on different axes are VAGUE). Agreement is Cohen's kappa 0.81 over both rounds (0.91 under the relaxed scoring that ignores VAGUE disagreements). Four Hebrew BERT models are fine-tuned with event markers; the best, AlephBERTGimmel with event-start pooling, reaches a relaxed weighted F1 of about 0.78 (macro F1 about 0.65). The dataset and models are released under CC BY-NC-SA 4.0, as the paper states.

Ideas

• I1. Event selection as a rule over UD tags: verbs only, infinitives and gerunds excluded, auxiliaries not part of the event (the paper's Appendix C). Test the rule on the HTB trees before any classifier.
• I2. The multi-axis check as a rule on the event chain: two events may have a non-VAGUE order only if they share a timeline axis, and a quotation or hypothesis opens a parallel axis. A candidate rule for the event order in world, English first.
• I3. A measurement only: use the released pairs as a held-out check of an event-order rule, and report the relaxed score next to the strict one, since VAGUE (16% of pairs) is the hard label. The data are NC, so this is a test, never built into a release.

For Mova

I1 goes into 05. The verb: system, binyanim, tense and use and 11. Modern Hebrew in Universal Dependencies. I2 is a candidate rule for the event chain in world, tried on English first. I3 is a measurement for world and the Hebrew reader; the data are CC BY-NC-SA, so they stay outside the build.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.findings-ijcnlp.24

@inproceedings{yanko2023temporal,
  title = {{Temporal Relation Classification in Hebrew}},
  author = {Yanko, Guy and Pariente, Shahaf and Bar, Kfir},
  booktitle = {Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings)},
  year = {2023},
  doi = {10.18653/v1/2023.findings-ijcnlp.24},
  url = {https://doi.org/10.18653/v1/2023.findings-ijcnlp.24}
}

References

19 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Nathanael Chambers, Taylor Cassidy, Bill McDowell, and Steven Bethard. 2014. Dense event ordering with a multi-pass architecture. Transactions of the Association for Computational Linguistics, 2:273–284. find
2. Avihay Chriqui and Inbal Yahav. 2022. HeBERT & HebEMO: a Hebrew BERT model and a tool for polarity analysis and emotion recognition. INFORMS Journal on Data Science. find our card
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (1), pages 4171–4186. Association for Computational Linguistics. find
4. Eylon Guetta, Avi Shmidman, Shaltiel Shmidman, Cheyn Shmuel Shmidman, Joshua Guedalia, Moshe Koppel, Dan Bareket, Amit Seker, and Reut Tsarfaty. 2022. Large pre-trained models with extra-large vocabularies: A contrastive analysis of Hebrew BERT models and a new one to outperform them all. find our card
5. Rujun Han, I Hsu, Mu Yang, Aram Galstyan, Ralph Weischedel, Nanyun Peng, et al. 2019a. Deep structured neural network for event temporal relation extraction. arXiv preprint arXiv:1909.10094. https://github.com/shahafp/TRC-Hebrew ↗
6. Rujun Han, Qiang Ning, and Nanyun Peng. 2019b. Joint event and temporal relation extraction with shared representations and structured prediction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 434–444, Hong Kong, China. Association for Computational Linguistics. find
7. Rujun Han, Xiang Ren, and Nanyun Peng. 2021. ECONET: Effective continual pretraining of language models for event temporal reasoning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), Punta Cana, Dominican Republic. find
8. Puneet Mathur, Rajiv Jain, Franck Dernoncourt, Vlad Morariu, Quan Hung Tran, and Dinesh Manocha. 2021. TIMERS: Document-level temporal relation extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 524–533, Online. Association for Computational Linguistics. find
9. Ryan T McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith B Hall, Slav Petrov, Hao Zhang, Oscar Täckström, et al. 2013. Universal dependency annotation for multilingual parsing. In Proc. of ACL. find
10. Qiang Ning, Zhili Feng, Hao Wu, and Dan Roth. 2018a. Joint reasoning for temporal and causal relations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2278–2288, Melbourne, Australia. Association for Computational Linguistics. find
11. Qiang Ning, Sanjay Subramanian, and Dan Roth. 2019. An improved neural baseline for temporal relation extraction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6203–6209, Hong Kong, China. Association for Computational Linguistics. find
12. Qiang Ning, Hao Wu, and Dan Roth. 2018b. A multi-axis annotation scheme for event temporal relations. In ACL. find
13. Tim OGorman, Kristin Wright-Bettner, and Martha Palmer. 2016. Richer event description: Integrating event coreference with temporal, causal and bridging annotation. In Proceedings of the 2nd Workshop on Computing News Storylines (CNS 2016), pages 47–56. find
14. Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Greenfeld, and Reut Tsarfaty. 2022. AlephBERT: Language model pre-training and evaluation from sub-word to sentence level. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 46–56, Dublin, Ireland. Association for Computational Linguistics. find our card
15. Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. 2019. Matching the blanks: Distributional similarity for relation learning. CoRR, abs/1906.03158. ↗
16. Reut Tsarfaty. 2013. A unified morpho-syntactic scheme of stanford dependencies. In Proc. of ACL. find
17. Marc Verhagen, Robert Gaizauskas, Frank Schilder, Mark Hepple, Graham Katz, and James Pustejovsky. 2007. Semeval-2007 task 15: Tempeval temporal relation identification. In Proceedings of the fourth international workshop on semantic evaluations (SemEval-2007), pages 75–80. B Table 5 summarizes the non-relaxed F1 scores of our experiments with the different models, as reported in the text. find
18. C Shuaicheng Zhang, Lifu Huang, and Qiang Ning. 2021. Extracting temporal event relation with syntactic-guided temporal graph transformer. arXiv preprint arXiv:2104.09570. ↗
19. Jie Zhou, Shenpo Dong, Hongkui Tu, Xiaodong Wang, and Yong Dou. 2022. RSGT: Relational structure guided temporal relation extraction. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2001–2010, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. 1. Verbal nouns and infinitive verbs are not counted as events. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/narr-2023-yanko-temporal-relation-classification-hebrew/