A Neural Architecture for Dialectal Arabic Segmentation

What it is about

Egyptian Arabic in tweets has no standard spelling and few annotated resources, and the clitics attached to its verbs are missed by MSA tools. The authors segment words character by character with a bidirectional LSTM and a CRF output layer, labelling each character as beginning, middle, end, single or word boundary. The model is trained on only 200 sentences of an annotated set of 350 Egyptian tweets (75 for development, 75 for test), with pre-trained character embeddings and no lexicon. On the 1,449-word test set the BiLSTM-CRF reaches 92.65% accuracy, against 92.47% for MADAMIRA and 88.34% for Farasa; the BiLSTM without the CRF reaches 86.27%. The 0.18-point gap to MADAMIRA is not significant, and 40% of the test words are unseen in training.

Ideas

• I1. Replace the hand-written segmentation tables of tools/hbo with a CRF over boundary labels (B, M, E, S, word boundary), using character features and no LSTM. The CRF enforces legal label sequences, which the tables try to do by hand. Compare its boundary errors with the table segmenter on the same MACULA tokens.
• I2. Score boundary labels per character, not only word accuracy. The paper's test set shows that word accuracy hides where the errors are; the clitic classes (the conjunction w, the negation prefix m, the negative suffix $) are the cases to check.
• I3. A learning curve on the size of the gold set. The paper gets its result with 200 sentences. Measure how much gold the segmenter needs for languages/jpa/ and languages/tmr/ before it beats a table-only baseline.

For Mova

I1 and I2 go into tools/hbo, the segmenter for unpointed text. I3 is a test plan for the Aramaic varieties languages/jpa/ and languages/tmr/. The BiLSTM itself is not reused; the label set and the CRF idea are.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-1306

@inproceedings{samih2017neural,
  title = {{A Neural Architecture for Dialectal Arabic Segmentation}},
  author = {Samih, Younes and Attia, Mohammed and Eldesouki, Mohamed and Abdelali, Ahmed and Mubarak, Hamdy and Kallmeyer, Laura and Darwish, Kareem},
  booktitle = {Proceedings of the Third Arabic Natural Language Processing Workshop},
  year = {2017},
  doi = {10.18653/v1/w17-1306},
  url = {https://doi.org/10.18653/v1/w17-1306}
}

References

35 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16. Association for Computational Linguistics, San Diego, California. find our card
2. Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks, 5(2):157–166. find
3. Fadi Biadsy, Julia Hirschberg, and Nizar Habash. 2009. Spoken arabic dialect identification using phonotactic modeling. In Proceedings of the EACL 2009 Workshop on Computational Approaches to Semitic Languages, Semitic ’09, pages 53–61, Stroudsburg, PA, USA. Association for Computational Linguistics. find
4. Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014. A multidialectal parallel corpus of arabic. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis, editors, Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland, may. European Language Resources Association (ELRA). “is well’, ú æ AÓ mA$iy “OK”. ↗
5. François Chollet. 2015. keras. https://github.com/fchollet/keras. ↗
6. Zeinab Ibrahim. 2006. Borrowing in modern standard arabic. Innovation and Continuity in Language and Communication of Different Language Cultures 9. Edited by Rudolf Muhr, pages 235–260. find
7. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493–2537. find
8. Sameer Khurana, Ahmed Ali, and Steve Renals. 2016. Multi-view dimensionality reduction for dialect identification of arabic broadcast speech. arXiv preprint arXiv:1609.05650. ↗
9. Kareem Darwish, Hassan Sajjad, and Hamdy Mubarak. 2014. Verifiably effective arabic dialect identification. In EMNLP, pages 1465–1468. find
10. Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. 2016. Character-aware neural language models. In Thirtieth AAAI Conference on Artificial Intelligence. find
11. Cıcero dos Santos, Victor Guimaraes, RJ Niterói, and Rio de Janeiro. 2015. Boosting named entity recognition with neural character embeddings. In Proceedings of NEWS 2015 The Fifth Named Entities Workshop, page 25. find
12. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. ICML. find
13. Mohamed Eldesouki, Fahim Dalvi, Hassan Sajjad, and Kareem Darwish. 2016. Qcri@ dsl 2016: Spoken arabic dialect identification using textual. VarDial 3, page 221. find
14. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of NAACL-HLT, pages 260– 270. find
15. Alex Graves and Jürgen Schmidhuber. 2005. Frame-wise phoneme classification with bidirectional lstm and other neural network architectures. Neural Networks, 18(5):602–610. find
16. Zachary C Lipton, David C Kale, Charles Elkan, and Randall Wetzell. 2015. A critical review of recurrent neural networks for sequence learning. CoRR, abs/1506.00019. ↗
17. Alex Graves, Navdeep Jaitly, and Abdel-rahman Mohamed. 2013a. Hybrid speech recognition with deep bidirectional lstm. In Automatic Speech Recognition and Understanding (ASRU), 2013 IEEE Workshop on, pages 273–278. IEEE. find
18. Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional lstm-cnns-crf. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1064–1074, Berlin, Germany, August. Association for Computational Linguistics. find
19. Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. 2013b. Speech recognition with deep recurrent neural networks. In Acoustics, speech and signal processing (icassp), 2013 ieee international conference on, pages 6645–6649. IEEE. find
20. Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850. ↗
21. Mohamed Maamouri, Ann Bies, Seth Kulick, Michael Ciul, Nizar Habash, and Ramy Eskander. 2014. Developing an egyptian arabic treebank: Impact of dialectal morphology on annotation and tool development. In LREC, pages 2348–2354. find
22. Nizar Habash, Mona T Diab, and Owen Rambow. 2012. Conventional orthography for dialectal arabic. In LREC, pages 711–718. find
23. Emad Mohamed, Behrang Mohit, and Kemal Oflazer. 2012. Annotating and learning morphological segmentation of egyptian colloquial arabic. In LREC, pages 873–877. find
24. Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh. 2013. Morphological analysis and disambiguation for dialectal arabic. In Hlt-Naacl, pages 426–432. find
25. Will Monroe, Spence Green, and Christopher D Manning. 2014. Word segmentation of informal arabic with domain adaptation. In ACL (2), pages 206–211. find
26. Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580. ↗
27. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan M Roth. 2014. Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. Proc. LREC. find our card
28. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
29. Hassan Sajjad, Kareem Darwish, and Yonatan Belinkov. 2013. Translating dialectal Arabic to English. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), ACL ’13, pages 1–6, Sofia, Bulgaria. find
30. Younes Samih, Suraj Maharjan, Mohammed Attia, Laura Kallmeyer, and Thamar Solorio. 2016. Multilingual code-switching identification via lstm recurrent neural networks. In Proceedings of the Second Workshop on Computational Approaches to Code Switching,, pages 50–59, Austin, TX. find
31. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673–2681. find
32. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany, August. Association for Computational Linguistics. find
33. Yushi Yao and Zheng Huang. 2016. Bi-directional lstm recurrent neural network for chinese word segmentation. In International Conference on Neural Information Processing, pages 345–353. Springer. find
34. Omar F Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics, 40(1):171–202. find
35. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F. Zaidan, and Chris Callison-Burch. 2012. Machine translation of arabic dialects. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT ’12, pages 49–59, Stroudsburg, PA, USA. Association for Computational Linguistics. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2017-samih-neural-architecture-dialectal-arabic-segmentation/