Mova0.5 pre-release⌕
☰
home / docs / papers / Semitic languages / Arabic Multi-Dialect Segmentation: bi-LSTM-CRF vs. SVM

Arabic Multi-Dialect Segmentation: bi-LSTM-CRF vs. SVM

What it is about

The paper segments four Arabic dialects (Egyptian, Levantine, Gulf, Maghrebi) from raw Twitter text. Only about 350 manually segmented tweets per dialect are used for training, with 5-fold cross-validation and a 70/10/20 split. Two approaches are compared. An SVM ranker generates all segmentations of a word from a closed set of prefixes and suffixes and ranks them, using features such as affix probabilities and whether the stem is in a lexicon. A bi-LSTM with a CRF layer labels each character as a boundary. Three lookup schemes are tried: none, lookup in the dialect training data, and lookup in dialect data then in segmented MSA (the Penn Arabic Treebank). The best results are 94.6 (SVM) and 95.0 (bi-LSTM-CRF) for Egyptian; both reach 91–95% across dialects, against 82–86% for the MSA segmenter Farasa. The authors assume that a word has one segmentation regardless of context. For Egyptian, CODA-normalised text raises the SVM accuracy from 94.6 to 96.8. The main errors are wrong splits of letters that are not affixes, letters that are fused across a boundary, and non-standard spelling.

Ideas

For Mova

I1 and I2 are rules for tools/hbo (segmenter tables lex and pre) and a measure against macula splits. I3 is a held-out test set for tools/hbo and Pronouns, particles and prepositions. The neural model and the pretrained parsers are not part of Mova's build; the cascade and the error classes are.

Topics: Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 1708.05891

@misc{eldesouki2017arabic,
  title = {{Arabic Multi-Dialect Segmentation: bi-LSTM-CRF vs. SVM}},
  author = {Eldesouki, Mohamed and Samih, Younes and Abdelali, Ahmed and Attia, Mohammed and Mubarak, Hamdy and Darwish, Kareem and Laura, Kallmeyer},
  year = {2017},
  eprint = {1708.05891},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1708.05891}
}

References

37 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations. Association for Computational Linguistics, San Diego, California, pages 11–16. find our card
  2. Mohammed Attia, Pavel Pecina, Antonio Toral, Lamia Tounsi, and Josef van Genabith. 2011. An open-source finite state morphological transducer for modern standard arabic. In Proceedings of the 9th International Workshop on Finite State Methods and Natural Language Processing. Association for Computational Linguistics, pages 125–133. find our card
  3. Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks 5(2):157–166. find
  4. Fadi Biadsy, Julia Hirschberg, and Nizar Habash. 2009. Spoken arabic dialect identification using phonotactic modeling. In Proceedings of the EACL 2009 Workshop on Computational Approaches to Semitic Languages. Association for Computational Linguistics, Stroudsburg, PA, USA, Semitic ’09, pages 53– 61. find
  5. Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014. A multidialectal parallel corpus of arabic. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis, editors, Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14). European Language Resources Association (ELRA), Reykjavik, Iceland. find
  6. Tim Buckwalter. 2002. Buckwalter {Arabic} morphological analyzer version 1.0 . find
  7. Rich Caruana, Steve Lawrence, and Lee Giles. 2000. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. In NIPS. pages 402–408. find
  8. Kareem Darwish, Walid Magdy, and Ahmed Mourad. 2012. Language processing for arabic microblog retrieval. In Proceedings of the 21st ACM international conference on Information and knowledge management. ACM, pages 2427–2430. find
  9. Kareem Darwish, Walid Magdy, et al. 2014a. Arabic information retrieval. Foundations and Trends® in Information Retrieval 7(4):239–342. find
  10. Kareem Darwish, Hassan Sajjad, and Hamdy Mubarak. 2014b. Verifiably effective arabic dialect identification. In EMNLP. pages 1465–1468. find
  11. Mohamed Eldesouki, Fahim Dalvi, Hassan Sajjad, and Kareem Darwish. 2016. Qcri@ dsl 2016: Spoken arabic dialect identification using textual features. VarDial 3 page 221. find
  12. Ramy Eskander, Nizar Habash, Owen Rambow, and Nadi Tomeh. 2013. Processing spontaneous orthography. find
  13. Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 . ↗
  14. Alex Graves, Navdeep Jaitly, and Abdel-rahman Mohamed. 2013a. Hybrid speech recognition with deep bidirectional lstm. In Automatic Speech Recognition and Understanding (ASRU), 2013 IEEE Workshop on. IEEE, pages 273–278. find
  15. Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. 2013b. Speech recognition with deep recurrent neural networks. In Acoustics, speech and signal processing (icassp), 2013 ieee international conference on. IEEE, pages 6645–6649. find
  16. Nizar Habash, Mona T Diab, and Owen Rambow. 2012. Conventional orthography for dialectal arabic. In LREC. pages 711–718. find
  17. Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh. 2013. Morphological analysis and disambiguation for dialectal arabic. In Hlt-Naacl. pages 426–432. find
  18. Nizar Habash and Fatiha Sadat. 2006. Arabic preprocessing schemes for statistical machine translation. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers. Association for Computational Linguistics, pages 49–52. find
  19. Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 . ↗
  20. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9(8):1735–1780. find
  21. Zeinab Ibrahim. 2006. Borrowing in modern standard arabic. Innovation and Continuity in Language and Communication of Different Language Cultures 9. Edited by Rudolf Muhr pages 235–260. find
  22. Thorsten Joachims. 2006. Training linear svms in linear time. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pages 217–226. find
  23. Sameer Khurana, Ahmed Ali, and Steve Renals. 2016. Multi-view dimensionality reduction for dialect identification of arabic broadcast speech. arXiv preprint arXiv:1609.05650 . ↗
  24. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. ICML. find
  25. Zachary C Lipton, David C Kale, Charles Elkan, and Randall Wetzell. 2015. A critical review of recurrent neural networks for sequence learning. CoRR abs/1506.00019. ↗
  26. Mohamed Maamouri, Ann Bies, Seth Kulick, Michael Ciul, Nizar Habash, and Ramy Eskander. 2014. Developing an egyptian arabic treebank: Impact of dialectal morphology on annotation and tool development. In LREC. pages 2348–2354. find
  27. Emad Mohamed, Behrang Mohit, and Kemal Oflazer. 2012. Annotating and learning morphological segmentation of egyptian colloquial arabic. In LREC. pages 873–877. find
  28. Will Monroe, Spence Green, and Christopher D Manning. 2014. Word segmentation of informal arabic with domain adaptation. In ACL (2). pages 206–211. find
  29. Hamdy Mubarak and Kareem Darwish. 2014. Using twitter to collect a multi-dialectal corpus of arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP). pages 1–7. find
  30. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan M Roth. 2014. Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. Proc. LREC . find our card
  31. Hassan Sajjad, Kareem Darwish, and Yonatan Belinkov. 2013. Translating dialectal Arabic to English. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Sofia, Bulgaria, ACL ’13, pages 1–6. find
  32. Younes Samih, Mohammed Attia, Mohamed Eldesouki, Hamdy Mubarak, Ahmed Abdelali, Laura Kallmeyer, and Kareem Darwish. 2017. A neural architecture for dialectal arabic segmentation. WANLP 2017 (co-located with EACL 2017) page 46. find our card
  33. Younes Samih, Suraj Maharjan, Mohammed Attia, Laura Kallmeyer, and Thamar Solorio. 2016. Multilingual code-switching identification via lstm recurrent neural networks. In Proceedings of the Second Workshop on Computational Approaches to Code Switching,. Austin, TX, pages 50–59. http://www.aclweb.org/anthology/W/W16/W16-58.pdf#page=62. ↗
  34. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing 45(11):2673–2681. find
  35. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany, pages 1715–1725. http://www.aclweb.org/anthology/P16-1162. ↗
  36. Omar F Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics 40(1):171–202. find
  37. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F. Zaidan, and Chris Callison-Burch. 2012. Machine translation of arabic dialects. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Stroudsburg, PA, USA, NAACL HLT ’12, pages 49–59. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/langstream-2017-eldesouki-arabic-multi-dialect-segmentation-bi/