Mova0.5 pre-release⌕
☰
home / docs / papers / Semitic languages / Probabilistic Modelling of Morphologically Rich Languages

Probabilistic Modelling of Morphologically Rich Languages

What it is about

This thesis asks how the internal structure of words can enter probabilistic language models. Three models are built: a hierarchical Pitman–Yor n-gram model that decomposes compounds, a log-bilinear model that learns morpheme vectors, and an unsupervised model that learns the morphemes themselves. The last part is the one relevant to Semitic work (chapter 5). The author extends adaptor grammars with simple range concatenating grammars, so that a root can be discontiguous and a template can be a second layer. The model is tested on Arabic (Modern Standard and Quranic) and on Hebrew, using segment-border F-score and root identification. For the triliteral templatic grammar the average segment-border F-score rises from 47.8 to 58.0 (about 21% relative) over the concatenative baseline. The thesis also reports that vocalised Hebrew and Arabic are easier than unvocalised text, and that the Quranic data is hard, because of low segmentation quality on it.

Ideas

For Mova

I1 goes into The verb and The lexicon (roots as a table, not a learner). I2 and I3 are tests for tools/hbo (pointed and unpointed modes), with the numbers kept in docs as measurements. The probabilistic model itself does not fit Mova's rule-first design; only its data split, its metric and its error categories are taken.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 1508.04271

@misc{botha2015probabilistic,
  title = {{Probabilistic Modelling of Morphologically Rich Languages}},
  author = {Botha, Jan A.},
  year = {2015},
  eprint = {1508.04271},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1508.04271}
}

References

185 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. H. Adel, N. T. Vu, and T. Schultz. 2013. Combination of Recurrent Neural Networks and Factored Language Models for Code-Switching Language Modeling. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 206–211, Sofia, Bulgaria, August. Association for Computational Linguistics. find
  2. A. Albert, B. MacWhinney, B. Nir, and S. Wintner. 2013. The Hebrew CHILDES corpus: transcription and morphological analysis. Language Resources and Evaluation, 47(4):973–1005. find
  3. A. Alexandrescu and K. Kirchhoff. 2006. Factored Neural Language Models. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, New York City, USA, June. Association for Computational Linguistics. find
  4. A. Allauzen, N. Pechéux, Q. K. Do, M. Dinarelli, T. Lavergne, A. Max, H.-S. Le, and F. Yvon. 2013. LIMSI@ WMT13. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 62–69, Sofia, Bulgaria, August. Association for Computational Linguistics. find
  5. M. Baroni and J. Matiasek. 2002. Predicting the components of German nominal compounds. In Proceedings of the 15th European Conference on Artificial Intelligence (ECAI), pages 470–474, Lyon, France, July. find
  6. M. Baroni and R. Zamparelli. 2010. Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1183–1193, Cambridge, MA. find
  7. M. Baroni, J. Matiasek, and H. Trost. 2002. Unsupervised discovery of morphologically related words based on orthographic and semantic similarity. In Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning, pages 48–57. Association for Computational Linguistics, July. find our card
  8. M. J. Beal. 2003. Variational Algorithms for Approximate Bayesian Inference. Ph.D. thesis, University of London. find
  9. K. R. Beesley and L. Karttunen. 2003. Finite state morphology, volume 18. CSLI Publications Stanford. find
  10. Y. Bengio, R. Ducharme, P. Vincent, and C. Jauvin. 2003. A Neural Probabilistic Language Model. Journal of Machine Learning Research, 3:1137–1155. find
  11. A. Berton, P. Fetter, and P. Regel-Brietzmann. 1996. Compound Words in Large-Vocabulary German Speech Recognition Systems. In Proceedings of Fourth International Conference on Spoken Language Processing. ICSLP ’96, volume 2, pages 1165–1168. IEEE. find
  12. J. A. Bilmes and K. Kirchhoff. 2003. Factored Language Models and Generalized Parallel Backoff. In Proceedings of Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: short papers. Association for Computational Linguistics. find
  13. P. Blunsom and T. Cohn. 2011. A Hierarchical Pitman-Yor Process HMM for Unsupervised Part of Speech Induction. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 865–874, Portland, Oregon, USA, June. Association for Computational Linguistics. find
  14. P. Blunsom, T. Cohn, S. Goldwater, and M. Johnson. 2009. A Note on the Implementation of Hierarchical Dirichlet Processes. In Proceedings of the ACL-IJCNLP 2009 Conference: Short Papers, pages 337–340. Association for Computational Linguistics, August. find
  15. J. A. Botha. 2012. Hierarchical Bayesian Language Modelling for the Linguistically Informed. In Proceedings of the Student Research Workshop at the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 64–73, Avignon, France. Association for Computational Linguistics. find
  16. J. A. Botha and P. Blunsom. 2013. Adaptor Grammars for Learning Non-Concatenative Morphology. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Washington, USA, October. find
  17. J. A. Botha and P. Blunsom. 2014. Compositional Morphology for Word Representations and Language Modelling. In Proceedings of the 31st International Conference on Machine Learning, ICML, pages 1899–1907, Beijing, China, June. find
  18. J. A. Botha, C. Dyer, and P. Blunsom. 2012. Bayesian Language Modelling of German Compounds. In Proceedings of the 24th International Conference on Computational Linguistics (COLING), Mumbai, India. find
  19. A. Boudlal, R. Belahbib, A. Lakhouaja, A. Mazroui, A. Meziane, and M. Bebah. 2009. A Markovian approach for Arabic Root Extraction. The International Arab Journal of Information Technology, 8(1):91–98. find
  20. P. Boullier. 2000. A cubic time extension of context-free grammars. Grammars, 3 (2-3):111–131. find
  21. T. Brants, A. C. Popat, P. Xu, F. J. Och, and J. Dean. 2007. Large language models in machine translation. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), volume 1, pages 858–867. Association for Computational Linguistics, June. find
  22. P. F. Brown, P. V. DeSouza, R. L. Mercer, V. J. Della Pietra, and J. C. Lai. 1992. Class-Based n-gram Models of Natural Language. Computational Linguistics, 18 (4):467–479. find
  23. T. Buckwalter. 2002. Buckwalter Arabic Morphological Analyzer Version 1.0. Technical report, Linguistic Data Consortium. find
  24. J. A. Bullinaria and J. P. Levy. 2007. Extracting Semantic Representations from Word Co-occurrence Statistics: A Computational Study. Behavior research methods, 39(3):510–26, August. find
  25. Ç. Çöltekin. 2010. A Freely Available Morphological Analyzer for Turkish. In Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC2010), pages 820–827. find
  26. V. Chahuneau, E. Schlinger, N. A. Smith, and C. Dyer. 2013a. Translating into Morphologically Rich Languages with Synthetic Phrases. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1677–1687, Seattle, Washington, USA. Association for Computational Linguistics. find
  27. V. Chahuneau, N. A. Smith, and C. Dyer. 2013b. Knowledge-rich morphological priors for Bayesian language models. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1206–1215, Atlanta, Georgia. Association for Computational Linguistics. find
  28. C. Chelba, T. Mikolov, M. Schuster, Q. Ge, T. Brants, P. Koehn, and T. Robinson. 2014. One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling. In Proceedings of the 15th Annual Conference of the International Speech Communication Association (INTERSPEECH), pages 2635–2639, Singapore. find
  29. S. F. Chen and J. Goodman. 1998. An Empirical Study of Smoothing Techniques for Language Modeling. Technical report, Center for Research in Computing Technology, Harvard University, Cambridge, MA. find
  30. S. F. Chen and J. Goodman. 1999. An empirical study of smoothing techniques for language modeling. Computer Speech & Language, 13:359–394, October. find
  31. D. Chiang. 2007. Hierarchical Phrase-Based Translation. Computational Linguistics, 33(2):201–228. find our card
  32. A. Clark. 2001a. Unsupervised induction of stochastic context-free grammars using distributional clustering. In Proceedings of the ACL 2001 Workshop on Computational Natural Language Learning (ConLL), pages 105–112, Toulouse, France. Association for Computational Linguistics. find
  33. A. Clark. 2001b. Partially Supervised Learning of Morphology with Stochastic Transducers. In Proceedings of Natural Language Processing Pacific Rim Symposium, pages 341–348, Tokyo, Japan, November. find
  34. A. Clark. 2002. Memory-Based Learning of Morphology with Stochastic Transducers. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 513–520. find
  35. J. H. Clark, C. Dyer, A. Lavie, and N. A. Smith. 2011. Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 176–181, Portland, Oregon, USA. find
  36. S. B. Cohen, D. M. Blei, and N. A. Smith. 2010. Variational Inference for Adaptor Grammars. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 564–572, Los Angeles, California. Association for Computational Linguistics. find
  37. Y. Cohen-Sygal and S. Wintner. 2006. Finite-state registered automata for non-concatenative morphology. Computational Linguistics, 32(1):49–82. find
  38. T. Cohn, P. Blunsom, and S. Goldwater. 2010. Inducing Tree-Substitution Grammars. Journal of Machine Learning Research, 11:3053–3096. find
  39. R. Collobert and J. Weston. 2008. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning. In Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland. find
  40. M. Creutz and K. Lagus. 2007. Unsupervised Models for Morpheme Segmentation and Morphology Learning. ACM Transactions on Speech and Language Processing, 4(1):1–34. find
  41. K. Darwish. 2002. Building a shallow Arabic morphological analyzer in one day. In Proceedings of the ACL-02 Workshop on Computational Approaches to Semitic Languages, pages 47–54. Association for Computational Linguistics. find
  42. E. Daya, D. Roth, and S. Wintner. 2008. Identifying Semitic Roots: Machine Learning with Linguistic Constraints. Computational Linguistics, 34(3):429–448. find our card
  43. R. G. de Almeida and G. Libben. 2005. Changing morphological structures: The effect of sentence context on the interpretation of structurally ambiguous English trimorphemic words. Language and Cognitive Processes, 20(1-2):373–394. find
  44. C. G. de Marcken. 1996. Unsupervised Language Acquisition. Ph.D. thesis, Massachusetts Institute of Technology. find
  45. A. N. de Roeck and W. Al-Fares. 2000. A morphologically sensitive clustering algorithm for identifying Arabic roots. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, pages 199–206. Association for Computational Linguistics. find our card
  46. J. Duchi, E. Hazan, and Y. Singer. 2011. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. Journal of Machine Learning Research, 12:2121–2159. find
  47. K. Duh, G. Neubig, K. Sudoh, and H. Tsukada. 2013. Adaptation Data Selection using Neural Language Models: Experiments in Machine Translation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 678–683, Sofia, Bulgaria. find
  48. K. Dukes and N. Habash. 2010. Morphological Annotation of Quranic Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC). find our card
  49. C. Dyer. 2009. Using a maximum entropy model to build segmentation lattices for MT. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 406–414, Boulder, Colorado. Association for Computational Linguistics. find
  50. C. Dyer, A. Lopez, J. Ganitkevitch, J. Weese, F. Ture, P. Blunsom, H. Setiawan, V. Eidelman, and P. Resnik. 2010. cdec: A Decoder, Alignment, and Learning Framework for Finite-State and Context-Free Translation Models. In Proceedings of the ACL 2010 System Demonstrations, pages 7–12, Uppsala, Sweden. Association for Computational Linguistics. find
  51. C. Dyer, V. Chahuneau, and N. A. Smith. 2013. A Simple, Fast, and Effective Reparameterization of IBM Model 2. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 644–648, Atlanta, Georgia, USA. Association for Computational Linguistics. find
  52. J. P. Egan. 1975. Signal detection theory and ROC analysis. Series in Cognition and Perception. Academic Press, New York, NY. find
  53. T. Fawcett. 2006. An introduction to ROC analysis. Pattern Recognition Letters, 27 (8):861–874, June. find
  54. T. S. Ferguson. 1973. A Bayesian analysis of some nonparametric problems. Annals of Statistics, (1):209–230. find
  55. R. Finkel and G. Stump. 2002. Generating Hebrew verb morphology by default inheritance hierarchies. In Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages. Association for Computational Linguistics. find our card
  56. L. Finkelstein, E. Gabrilovich, Y. Matias, E. Rivlin, Z. Solan, G. Wolfman, and E. Ruppin. 2002. Placing Search in Context: The Concept Revisited. ACM Transactions on Information Systems, 20(1):116–131. find
  57. M. Forsberg and A. Ranta. 2004. Functional Morphology. In Proceedings of the Ninth ACM SIGPLAN International Conference on Functional Programming, pages 213–223, Utah, September. Association for Computing Machinery. find
  58. S. Frank, F. Keller, and S. Goldwater. 2013. Exploring the utility of joint morphological and syntactic learning from child-directed speech. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Washington, USA. Association for Computational Linguistics. find
  59. M. A. Fullwood and T. J. O’Donnell. 2013. Learning non-concatenative morphology. In Proceedings of the Fourth Annual Workshop on Cognitive Modeling and Computational Linguistics, pages 21–27, Sofia, Bulgaria. Association for Computational Linguistics. find
  60. M. Gasser. 2009. Semitic morphological analysis and generation using finite state transducers with feature structures. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics, pages 309– 317. Association for Computational Linguistics. find our card
  61. P. Geutner. 1995. Using Morphology Towards Better Large-Vocabulary Speech Recognition Systems. In Proceedings of the 1995 International Conference on Acoustics, Speech, and Signal Processing, pages 445–448. IEEE. find
  62. D. Gildea. 2010. Optimal Parsing Strategies for Linear Context-Free Rewriting Systems. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 769–776. Association for Computational Linguistics. find
  63. W. R. Gilks, S. Richardson, and D. J. Spiegelhalter. 1996. Markov Chain Monte Carlo in Practice. Chapman & Hall/CRC Interdisciplinary Statistics. find
  64. J. Goldsmith and T. Reutter. 1998. Automatic Collection and Analysis of German Compounds. In F. Busa F. et al., editor, The Computational Treatment of Nominals, pages 61–69. Universite de Montreal, Canada. find
  65. S. Goldwater. 2007. Nonparametric Bayesian Models of Lexical Acquisition. Ph.D. thesis, Brown University. find
  66. S. Goldwater, T. L. Griffiths, and M. Johnson. 2006. Interpolating Between Types and Tokens by Estimating Power-Law Generators. In Advances in Neural Information Processing Systems, pages 459–466. find
  67. S. Goldwater, T. L. Griffiths, and M. Johnson. 2011. Producing Power-Law Distributions and Damping Word Frequencies with Two-Stage Language Models. Journal of Machine Learning Research, 12:2335–2382. find
  68. I. J. Good. 1953. The Population Frequencies of Species and the Estimation of Population Parameters. Biometrika, 40(3):237–264. find
  69. J. Goodman. 2001. Classes for Fast Maximum Entropy Training. In Proceedings of the 2001 International Conference on Acoustics, Speech, and Signal Processing, volume 1, pages 561–564. IEEE. find
  70. J. T. Goodman. 1998. Parsing Inside-Out. Ph.D. thesis, Harvard University. find
  71. I. Gurevych. 2005. Using the Structure of a Conceptual Network in Computing Semantic Relatedness. In Proceedings of the Second International Joint Conference on Natural Language Processing: Full Papers, pages 767–778. find
  72. M. U. Gutmann and A. Hyvärinen. 2012. Noise-Contrastive Estimation of Unnormalized Statistical Models , with Applications to Natural Image Statistics. The Journal of Machine Learning Research, 13:307–361. find
  73. N. Habash and O. Rambow. 2005. Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 573–580, Ann Arbor, June. Association for Computational Linguistics. find our card
  74. N. Habash and O. Rambow. 2006. MAGEAD: a morphological analyzer and generator for the Arabic dialects. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, volume M, pages 681–688, Sydney, Australia, July. Association for Computational Linguistics. find our card
  75. N. Habash and F. Sadat. 2006. Arabic preprocessing schemes for statistical machine translation. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, pages 49–52. Association for Computational Linguistics. find
  76. N. Habash, O. Rambow, and G. Kiraz. 2005. Morphological Analysis and Generation for Arabic Dialects. In Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, pages 17–24. Association for Computational Linguistics, June. find our card
  77. N. Y. Habash. 2010. Introduction to Arabic natural language processing. Synthesis Lectures on Human Language Technologies, 3(1):1–187. find
  78. H. Hammarström and L. Borin. 2011. Unsupervised Learning of Morphology. Computational Linguistics, 37(2):309–350. find
  79. S. Hassan and R. Mihalcea. 2009. Cross-lingual Semantic Relatedness Using Encyclopedic Knowledge. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1192–1201. Association for Computational Linguistics. find
  80. K. Heafield. 2011. KenLM: Faster and Smaller Language Model Queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 187– 197. Association for Computational Linguistics. find
  81. K. Heafield, I. Pouzyrevsky, J. H. Clark, and P. Koehn. 2013. Scalable Modified Kneser-Ney Language Model Estimation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 690–696, Sofia, Bulgaria, August. Association for Computational Linguistics. find
  82. K. M. Hermann and P. Blunsom. 2013. The Role of Syntax in Vector Space Models of Compositional Semantics. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 894– 904, Sofia, Bulgaria. find
  83. T. Hirsimäki, M. Creutz, V. Siivola, M. Kurimo, S. Virpioja, and J. Pylkkönen. 2006. Unlimited vocabulary speech recognition with morph language models applied to Finnish. Computer Speech and Language, 20(4):515–541, October. find
  84. E. H. Huang, R. Socher, C. D. Manning, and A. Y. Ng. 2012. Improving Word Representations via Global Context and Multiple Word Prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 873–882. Association for Computational Linguistics. find
  85. S. Huang and S. Renals. 2009. A parallel training algorithm for hierarchical Pitman-Yor process language models. In Proceedings of Interspeech, volume 9, pages 2695–2698. find
  86. S. Huang and S. Renals. 2010. Hierarchical Bayesian Language Models for Conversational Speech Recognition. IEEE Transactions on Audio, Speech, and Language Processing, 18(8):1941–1954, November. find
  87. Y. Huang, M. Zhang, and C. L. Tan. 2011. Nonparametric Bayesian Machine Transliteration with Synchronous Adaptor Grammars. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 534–539. Association for Computational Linguistics. find
  88. P. Ircing, P. Krbec, J. Hajic, S. Khudanpur, F. Jelinek, J. Psutka, and W. Byrne. 2001. On large vocabulary continuous speech recognition of highly inflectional language – Czech. In Proceedings of the 7th European Conference on Speech Communication and Technology (EUROSPEECH), volume 1. find
  89. H. Ishwaran and L. F. James. 2001. Gibbs Sampling Methods for Stick-Breaking Priors. Journal of the American Statistical Association, 96(453):161–173. find
  90. H. Ishwaran and L. F. James. 2003. Generalized weighted Chinese restaurant processes for species sampling mixture models. Statistica Sinica, 13:1211–1235. find
  91. F. Jelinek and R. L. Mercer. 1980. Interpolated estimation of Markov source parameters from sparse data. In Proceedings of the Workshop on Pattern Recognition in Practice, pages 381–397, Amsterdam, The Netherlands. find
  92. M. Johnson. 2008. Unsupervised word segmentation for Sesotho using Adaptor Grammars. In Proceedings of the Tenth Meeting of ACL Special Interest Group on Computational Morphology and Phonology, pages 20–27. Association for Computational Linguistics. find
  93. M. Johnson and S. Goldwater. 2009. Improving nonparameteric Bayesian inference: Experiments on unsupervised word segmentation with adaptor grammars. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 317–325. Association for Computational Linguistics. find
  94. M. Johnson, T. L. Griffiths, and S. Goldwater. 2007a. Adaptor Grammars: A Framework for Specifying Compositional Nonparametric Bayesian Models. In Advances in Neural Information Processing Systems, volume 19, page 641. MIT. find
  95. M. Johnson, T. L. Griffiths, and S. Goldwater. 2007b. Bayesian inference for PCFGs via Markov chain Monte Carlo. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics, number April, pages 139–146, Rochester, NY. Association for Computational Linguistics. find
  96. A. K. Joshi. 1985. Tree adjoining grammars: How much context-sensitivity is required to provide reasonable structural descriptions? In D. R. Dowty, L. Karttunen, and A. M. Zwicky, editors, Natural Language Parsing, chapter 6, pages 206–250. Cambridge University Press. find
  97. C. Joubarne and D. Inkpen. 2011. Comparison of Semantic Similarity for Different Languages Using the Google N-gram Corpus and Second-Order Co-occurrence Measures. In Proceedings of the Canadian Conference on Advances in Artificial Intelligence, pages 216–221. Springer-Verlag. find
  98. N. Kalchbrenner and P. Blunsom. 2013. Recurrent Continuous Translation Models. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1700–1709, Seattle, Washington, USA. Association for Computational Linguistics. find
  99. L. Kallmeyer. 2010. Parsing beyond context-free grammars. Springer Verlag. find
  100. L. Kataja and K. Koskenniemi. 1988. Finite-state Description of Semitic Morphology: A Case Study of Ancient Akkadian. In Coling Budapest 1988 Volume 1: International Conference on Computational Linguistics, pages 313–315. find
  101. Y. Kato, H. Seki, and T. Kasami. 2006. Stochastic Multiple Context-Free Grammar for RNA Pseudoknot Modeling. In Proceedings of the International Workshop on Tree Adjoining Grammar and Related Formalisms, pages 57–64, July. find
  102. S. Katz. 1987. Estimation of probabilities from sparse data for the language model component of a speech recognizer. IEEE Transactions on Acoustics, Speech, and Signal Processing, 35(3):400–401, March. find
  103. M. Kay. 1987. Nonconcatenative Finite-State Morphology. In Proceedings of the third conference of the European Chapter of the Association for Computational Linguistics. find
  104. G. A. Kiraz. 2000. Multitiered Nonlinear Morphology Using Multitape Finite Automata: A Case Study on Syriac and Arabic. Computational Linguistics, 26(1): 77–105, March. find our card
  105. K. Kirchhoff, J. Bilmes, and K. Duh. 2008. Factored Language Models Tutorial. Technical Report UWEETR-2008-0004, Department of Electrical Engineering, University ofWashington, Seattle. find
  106. R. Kneser and H. Ney. 1995. Improved Backing-off for m-gram Language Modelling. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 181–184. find
  107. P. Koehn. 2010. Statistical Machine Translation. Cambridge University Press. find
  108. P. Koehn and K. Knight. 2003. Empirical Methods for Compound Splitting. In Proceedings of the 10th Conference of the European Chapter of the Association for Computational Linguistics, pages 187–193. Association for Computational Linguistics. find
  109. P. Koehn, A. Arun, and H. Hoang. 2008. Towards better Machine Translation Quality for the German-English Language Pairs. In Third Workshop on Statistical Machine Translation, pages 139–142. Association for Computational Linguistics, June. find
  110. K. Koskenniemi. 1984. A general computational model for word-form recognition and production. In Proceedings of the 10th International Conference on Computational Linguistics and 22nd Annual Meeting of the Association for Computational Linguistics, pages 178–181. Association for Computational Linguistics. find
  111. M. Kurimo, S. Virpioja, V. T. Turunen, G. W. Blackwood, and W. Byrne. 2010. Overview and Results of Morpho Challenge 2009. In C. Peters, G. M. Nunzio, M. Kurimo, T. Mandl, D. Mostefa, A. Peñas, and G. Roda, editors, Multilingual Information Access Evaluation I. Text Retrieval Experiments, volume 6241 of Lecture Notes in Computer Science, pages 578–597. Springer Berlin / Heidelberg. find
  112. K. Lari and S. J. Young. 1990. The estimation of stochastic context-free grammars using the Inside-Outside algorithm. Computer Speech & Language, 4(1):35–56. find
  113. A. Lavie, A. Itai, U. Ornan, and M. Rimon. 1988. On the Applicabtlity of Two Level Morphology to the Inflection of Hebrew Verbs. Technical Report CS0513, Computer Science Department, Israeli Institute of Technology, Technion. find
  114. A. Lazaridou, M. Marelli, R. Zamparelli, and M. Baroni. 2013. Compositionally Derived Representations of Morphologically Complex Words in Distributional Semantics. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1517–1526, Sofia, Bulgaria. Association for Computational Linguistics. find
  115. H.-S. Le, I. Oparin, A. Allauzen, J.-L. Gauvain, and F. Yvon. 2011. Structured Output Layer Neural Network Language Model. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5524–5527, Prague, Czech Republic. IEEE. find
  116. Y. K. Lee, A. Haghighi, and R. Barzilay. 2011. Modeling syntactic context improves morphological segmentation. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning. Association for Computational Linguistics. find
  117. A. Levenberg, C. Dyer, and P. Blunsom. 2012. A Bayesian Model for Learning SCFGs with Discontiguous Rules. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 223–232, July. find
  118. P. Liang, B. Taskar, and D. Klein. 2006. Alignment by Agreement. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pages 104–111, New York City, USA, June. Association for Computational Linguistics. find
  119. M.-T. Luong, R. Socher, and C. D. Manning. 2013. Better Word Representations with Recursive Neural Networks for Morphology. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning. Association for Computational Linguistics. find
  120. W. Maier. 2010. Direct Parsing of Discontinuous Constituents in German. In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 58–66. Association for Computational Linguistics. find
  121. J. J. McCarthy. 1981. A Prosodic Theory of Nonconcatenative Morphology. Linguistic Inquiry, 12(3):373–418. find
  122. T. Mikolov, M. Karafiát, L. Burget, J. Černocký, and S. Khudanpur. 2010. Recurrent neural network based language model. In Proceedings of Interspeech, pages 1045–1048, September. find
  123. T. Mikolov, S. Kombrink, L. Burget, J. Černocký, and S. Khudanpur. 2011. Extensions of Recurrent Neural Network Language Model. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Prague, Czech Republic. find
  124. T. Mikolov, K. Chen, G. Corrado, and J. Dean. 2013a. Efficient Estimation of Word Representations in Vector Space. In Proceedings of the International Conference on Learning Representations. arXiv:1301.3781. ↗
  125. T. Mikolov, W.-t. Yih, and G. Zweig. 2013b. Linguistic Regularities in Continuous Space Word Representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746–751. Association for Computational Linguistics. find
  126. E. Minkov, K. Toutanova, and H. Suzuki. 2007. Generating Complex Morphology for Machine Translation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 128–135, June. find
  127. J. Mitchell and M. Lapata. 2008. Vector-based Models of Semantic Composition. In Proceedings of ACL-08: HLT, pages 236–244, Columbus, Ohio, June. Association for Computational Linguistics. find
  128. J. Mitchell and M. Lapata. 2010. Composition in distributional models of semantics. Cognitive science, 34(8):1388–429, November. find
  129. A. Mnih and G. Hinton. 2007. Three New Graphical Models for Statistical Language Modelling. In Proceedings of the 24th International Conference on Machine Learning, pages 641–648, Corvallis, Oregon. find
  130. A. Mnih and G. Hinton. 2008. A Scalable Hierarchical Distributed Language Model. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems, pages 1081–1088. find
  131. A. Mnih and Y. W. Teh. 2012. A fast and simple algorithm for training neural probabilistic language models. In Proceedings of the 29th International Conference on Machine Learning, Edinburgh, Scotland. find
  132. A. Mnih, Z. Yuecheng, and G. Hinton. 2009. Improving a statistical language model through non-linear prediction. Neurocomputing, 72(7-9):1414–1418. find
  133. D. Mochihashi, T. Yamada, and N. Ueda. 2009. Bayesian unsupervised word segmentation with nested Pitman-Yor language modeling. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 100–108, Suntec, Singapore. Association for Computational Linguistics. find
  134. F. Morin and Y. Bengio. 2005. Hierarchical probabilistic neural network language model. In R. G. Cowell and Z. Ghahramani, editors, Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, pages 246–252. Society for Artificial Intelligence and Statistics. find
  135. A. E.-D. Mousa, M. A. B. Shaik, R. Schlüter, and H. Ney. 2011. Morpheme Based Factored Language Models for German LVCSR. In Interspeech, pages 1445– 1448. find
  136. V. Nair and G. Hinton. 2010. Rectified Linear Units Improve Restricted Boltzmann Machines. In Proceedings of the 27th International Conference on Machine Learning, pages 807–814, Haifa, Israel. find
  137. R. Neal. 1993. Probabilistic inference using Markov chain Monte Carlo methods. Technical Report CRG-TR-93-1, Department of Computer Science, University of Toronto. find
  138. R. M. Neal. 2003. Slice Sampling. The Annals of Statistics, 31(3):705–741. find
  139. M.-J. Nederhof. 2000. Practical Experiments with Regular Approximation of Context-free Languages. Computational Linguistics, 26(1). find
  140. G. Neubig, M. Mimura, S. Mori, and T. Kawahara. 2010. Learning a Language Model from Continuous Speech. In Interspeech, pages 1053–1056, Chiba, Japan, September. find
  141. H. Ney, U. Essen, and R. Kneser. 1994. On structuring probabilistic dependences in stochastic language modelling. Computer Speech & Language, 8(1):1–38. find
  142. F. J. Och. 2003. Minimum Error Rate Training in Statistical Machine Translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 160–167. find
  143. M. Pachitariu and M. Sahani. 2013. Regularization and nonlinearities for neural language models: when are they needed? arXiv preprint arXiv:1301.5650. ↗
  144. K. Papineni, S. Roukos, T. Ward, W.-j. Zhu, T. J. Watson, and Y. Heights. 2001. Bleu: A Method for Automatic Evaluation of Machine Translation. Technical report, IBM. find our card
  145. R. Pascanu, T. Mikolov, and Y. Bengio. 2012. On the difficulty of training recurrent neural networks. In Proceedings of the 30th International Conference on Machine Learning. find
  146. J. Pitman. 1995. Exchangeable and partially exchangeable random partitions. Probability Theory and Related Fields, 102:145–158. find
  147. J. Pitman and M. Yor. 1997. The Two-Parameter Poisson-Dirichlet Distribution Derived from a Stable Subordinator. The Annals of Probability, 25(2):855–900. find
  148. H. Poon, C. Cherry, and K. Toutanova. 2009. Unsupervised morphological segmentation with log-linear models. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 209–217. Association for Computational Linguistics. find
  149. P. Rodrigues and D. Ćavar. 2007. Learning Arabic Morphology Using Statistical Constraint-Satisfaction Models. In E. Benmamoun, editor, Perspectives on Arabic Linguistics: Proceedings of the 19th Arabic Linguistics Symposium, pages 63–75, Urbana, IL, USA. John Benjamins Publishing Company. find
  150. C. G. Rodríguez and G. Satta. 2009. An Optimal-Time Binarization Algorithm for Linear Context-Free Rewriting Systems with Fan-Out Two. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 985–993. Association for Computational Linguistics. find
  151. R. Rosenfeld, S. F. Chen, and X. Zhu. 2001. Whole-sentence exponential language models: a vehicle for linguistic-statistical integration. Computers Speech & Language, 15(1). find
  152. H. Rubenstein and J. B. Goodenough. 1965. Contextual Correlates of Synonymy. Communications of the Association for Computing Machinery, 8(10):627–633, October. find
  153. H. Schmid and F. Laws. 2008. Estimation of Conditional Probabilities With Decision Trees and an Application to Fine-Grained POS Tagging. In Proceedings of the 22nd International Conference on Computational Linguistics, pages 777–784, Manchester, UK. Association for Computational Linguistics. find
  154. P. Schone and D. Jurafsky. 2000. Knowledge-Free Induction of Morphology Using Latent Semantic Analysis. In Fourth Conference on Computational Natural Language Learning and the Second Learning Language in Logic Workshop, pages 67–72, Lisbon, Portugal. find
  155. H. Schwenk. 2004. Efficient Training of Large Neural Networks for Language Modeling. In Proceedings of the IEEE Joint Conference on Neural Networks, volume 4, pages 3059–3064. IEEE. find
  156. H. Schwenk and P. Koehn. 2008. Large and Diverse Language Models for Statistical Machine Translation. In Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-II. find
  157. H. Schwenk, D. Dechelotte, and J.-L. Gauvain. 2006. Continuous Space Language Models for Statistical Machine Translation. In Proceedings of COLING/ACL, pages 723–730, Sydney, Australia. Association for Computational Linguistics. find
  158. H. Schwenk, A. Rousseau, and M. Attik. 2012. Large, Pruned or Continuous Space Language Models on a GPU for Statistical Machine Translation. In In Proceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT, pages 11–19. Association for Computational Linguistics. find
  159. H. Seki, T. Matsumura, M. Fujii, and T. Kasami. 1991. On multiple context-free grammars. Theoretical Computer Science, 88(2):191–229. find
  160. J. Sethuraman. 1994. A Constructive Definition of Dirichlet Priors. Statistica Sinica, 4:639–650. find
  161. K. Sirts and S. Goldwater. 2013. Minimally-Supervised Morphological Segmentation using Adaptor Grammars. Transactions of the Association of Computational Linguistics, 1:255–266. find
  162. O. Smrž. 2007. Functional Arabic Morphology: Formal System and Implementation. Ph.D. thesis, Charles University, Prague. find
  163. B. Snyder and R. Barzilay. 2008. Unsupervised Multilingual Learning for Morphological Segmentation. In Proceedings of ACL-HLT, pages 737–745. find
  164. A. Stolcke. 2002. SRILM – An extensible language modeling toolkit. In Proceedings of the International Conference on Spoken Language Processing, pages 901–904. find
  165. A. Stolcke and S. M. Omohundro. 1994. Inducing probabilistic grammars by Bayesian model merging. In Grammatical inference and applications, pages 106– 118. Springer. find
  166. G. Stump. 2001. Inflectional Morphology. Cambridge studies in linguistics. Cambridge University Press, Cambridge, UK. find
  167. S. Stymne. 2009. A comparison of merging strategies for translation of German compounds. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 61–69. Association for Computational Linguistics, April. find
  168. Y. W. Teh. 2006a. A hierarchical Bayesian language model based on Pitman-Yor processes. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 985–992. Association for Computational Linguistics. find
  169. Y. W. Teh. 2006b. A Bayesian Interpretation of Interpolated Kneser-Ney. Technical Report TRA2/06, School of Computing, National University of Singapore. find
  170. P. D. Turney and P. Pantel. 2010. From Frequency to Meaning: Vector Space Models of Semantics. Journal of Artificial Intelligence Research, 37:141–188. find
  171. L. van der Maaten and G. Hinton. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research, 9:2579–2605. find
  172. A. Vaswani, Y. Zhao, V. Fossum, and D. Chiang. 2013. Decoding with Large-Scale Neural Language Models Improves Translation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, USA. Association for Computational Linguistics. find
  173. D. Vergyri, K. Kirchhoff, K. Duh, and A. Stolcke. 2004. Morphology-based Language Modeling for Arabic Speech Recognition. In International Conference on Spoken Language Processing, volume 6. find
  174. K. Vijay-Shanker, D. J. Weir, and A. K. Joshi. 1987. Characterizing structural descriptions produced by various grammatical formalisms. In Proceedings of the 25th Annual Meeting of the Association for Computational Linguistics, pages 104–111. find
  175. S. Virpioja, J. J. Väyrynen, M. Creutz, and M. Sadeniemi. 2007. Morphology-Aware Statistical Machine Translation Based on Morphs Induced in an Unsupervised Manner. In Proceedings of the Machine Translation Summit XI, pages 491–498, September. find
  176. R. Wang, M. Utiyama, I. Goto, E. Sumita, H. Zhao, and B.-L. Lu. 2013. Converting Continuous-Space Language Models into N-gram Language Models for Statistical Machine Translation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 845–850. find
  177. L. Wasserman. 2007. All of Nonparametric Statistics. Springer Texts in Statistics. find
  178. E. Williams. 1981. On the Notions “Lexically Related” and “Head of a Word”. Linguistic Inquiry, 12(2):245–274. find
  179. F. Wood and Y. W. Teh. 2009. A Hierarchical Nonparametric Bayesian Approach to Statistical Language Model Domain Adaptation. In Proceedings of the 12th International Conference on Artificial Intelligence and Statistics, pages 607–614, Clearwater Beach, Florida, USA. find
  180. Y. Wu, X. Lu, H. Yamamoto, S. Matsuda, C. Hori, and H. Kashioka. 2012. Factored Language Model based on Recurrent Neural Network. In Proceedings of COLING, pages 2835–2850. find
  181. M. Xiao and Y. Guo. 2013. Domain Adaptation for Sequence Labeling Tasks with a Probabilistic Language Adaptation Model. In Proceedings of the 30th International Conference on Machine Learning, Atlanta, Georgia. find
  182. W. Xu and A. I. Rudnicky. 2000. Can Artificial Neural Networks Learn Language Models? In Proceedings of the International Conference on Statistical Language Processing,. find
  183. T. Zesch and I. Gurevych. 2006. Automatically creating datasets for measures of semantic relatedness. In Proceedings of the Workshop on Linguistic Distances, pages 16–24. Association for Computational Linguistics. find
  184. G. Zipf. 1932. Selective Studies and the Principle of Relative Frequency in Language. Harvard University Press, Cambridge, MA, USA. find
  185. W. Y. Zou, R. Socher, D. Cer, and C. D. Manning. 2013. Bilingual Word Embeddings for Phrase-Based Machine Translation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1393–1398, Seattle, USA. Association for Computational Linguistics. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/langstream-2015-botha-probabilistic-modelling-morphologically-rich-languages/