Towards multi-lingual summarization: A comparative analysis of sentence extraction methods on English and Hebrew corpora

What it is about

The paper compares 31 language-independent methods for scoring sentences for extractive single-document summaries: 15 published ones and 16 graph and vector methods introduced by the same group earlier. The methods use no morphological analysis, so they can be applied to any language. They are tested on English news (DUC 2002, 533 articles) and on a new Hebrew set of 50 Haaretz articles that 70 students summarised by picking sentences. Scores are ROUGE-1 recall. The rankings in the two languages correlate at 0.775 (Pearson), and six of the top ten bilingual methods are the new ones (for example document-coverage with Jaccard or cosine, and graph variants of coverage and keyword scoring). Because Hebrew affixes make word counts unstable, the character-length measure is recommended over the word-length one for multilingual use.

Ideas

• I1. A cross-language check for any Mova summariser or retrieval scorer that is meant to work on Hebrew as well as English: run the same ranking on both languages and report the rank correlation, as the paper does. This is a measurement, not a model, and it fits the English-first RAG work in world and prag.
• I2. Use character length rather than word length as the length feature for Hebrew sentences, because clitics (prefixed particles and pronominal suffixes) change the word count of the same content. A small test on treebanks/he/ sentences would show whether word counts are unstable there.
• I3. The Hebrew experiment design (each article summarised by several students, each student given a fixed sample, a Friedman test and then Bonferroni-Dunn pairwise comparisons) is a template for a small, clean evaluation set when Mova needs its own gold summaries.

For Mova

I1 is a measurement for the summary and retrieval tests of world and prag. I2 goes into 2. Script and spelling as a note on tokenisation and into any length feature in tools/langseg output. I3 is a test-design note for Mova's own held-out sets, not a data source: the corpora here are non-commercial and only for tests.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{litvak2010multi,
  title = {{Towards multi-lingual summarization: A comparative analysis of sentence extraction methods on English and Hebrew corpora}},
  author = {Litvak, Marina and Last, Mark and Kisilevich, Slava and Keim, Daniel and Lipman, Hagay and Gur, Assaf Ben},
  booktitle = {Proceedings of the 4th Workshop on Cross Lingual Information Access},
  year = {2010},
  url = {https://aclanthology.org/W10-4010/}
}

References

26 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Adler, M. (2009). Hebrew morphological disambiguation: An unsupervised stochastic word-based approach. Dissertation. http://www.cs.bgu.ac.il/ adlerm/dat/thesis.pdf. ↗
2. Baxendale, P. (1958). Machine-made index for technical literature-an experiment. IBM Journal of Research and Development, 2, 354–361. find
3. Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems, 30, 107–117. find
4. Dang, H. T. (2006). Overview of DUC 2006. Proceedings of the Document Understanding Conference. find
5. Demsar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1–30. find
6. DUC (2002). Document understanding conference. http://duc.nist.gov. ↗
7. Edmundson, H. P. (1969). New methods in automatic extracting. J. ACM, 16. find
8. Erkan, G., & Radev, D. R. (2004). LexRank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research, 22, 457–479. Our experiments have shown that syntactic filters, which select only lexical units of a certain part of speech, do not significantly improve the performance of the evaluated bilingual scoring methods. find
9. Gong, Y., & Liu, X. (2001). Generic text summarization using relevance measure and latent semantic analysis. Proceedings of the 24th ACM SIGIR conference on Research and development in information retrieval (pp. 19–25). find
10. Kallel, F. J., Jaoua, M., Hadrich, L. B., & Hamadou, A. B. (2004). Summarization at LARIS laboratory. Proceedings of the Document Understanding Conference. find
11. Kupiec, J., Pedersen, J., & Chen, F. (1995). A trainable document summarizer. Proceedings of the 18th annual international ACM SIGIR conference (pp. 68–73). find
12. Lin, C.-Y. (2004). ROUGE: A package for automatic evaluation of summaries. Proceedings of the ACL’04 Workshop: Text Summarization Branches Out (pp. 74–81). find
13. Litvak, M., Last, M., & Friedman, M. (2010). A new approach to improving multilingual summarization using a genetic algorithm. Proceedings of the Association for Computational Linguistics (ACL) 2010. Uppsala, Sweden. find
14. Luhn, H. P. (1958). The automatic creation of literature abstracts. IBM Journal of Research and Development, 2, 159–165. find
15. Mani, I., & Maybury, M. (1999). Advances in automatic text summarization. find
16. Mihalcea, R. (2005). Language independent extractive summarization. AAAI’05: Proceedings of the 20th national conference on Artificial intelligence (pp. 1688–1689). find
17. Nastase, V., & Szpakowicz, S. (2006). A study of two graph algorithms in topic-driven summarization. Proceedings of the Workshop on Graph-based Algorithms for Natural Language. find
18. Neto, J., Santos, A., Kaestner, C., & Freitas, A. (2000). Generating text summaries through the relative importance of topics. Lecture Notes in Computer Science, 300–309. find
19. Radev, D., Blair-Goldensohn, S., & Zhang, Z. (2001). Experiments in single and multidocument summarization using MEAD. First Document Understanding Conference. find
20. Saggion, H., Bontcheva, K., & Cunningham, H. (2003). Robust generic and query-based summarisation. EACL ’03: Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics. find
21. Salton, G., Singhal, A., Mitra, M., & Buckley, C. (1997). Automatic text structuring and summarization. Information Processing and Management, 33, 193–207. find
22. Satoshi, C. N., Satoshi, S., Murata, M., Uchimoto, K., Utiyama, M., & Isahara, H. (2001). Sentence extraction system assembling multiple evidence. Proceedings of 2nd NTCIR Workshop (pp. 319–324). find
23. Schenker, A., Bunke, H., Last, M., & Kandel, A. (2004). Classification of web documents using graph matching. International Journal of Pattern Recognition and Artificial Intelligence, 18, 475–496. find
24. Steinberger, J., & Jezek, K. (2004). Text summarization and singular value decomposition. Lecture Notes in Computer Science, 245–254. find
25. Vanderwende, L., Suzuki, H., Brockett, C., & Nenkova, A. (2007). Beyond SumBasic: Task-focused summarization with sentence simplification and lexical expansion. Information processing and management, 43, 1606–1618. find
26. Wong, K., Wu, M., & Li, W. (2008). Extractive summarization using supervised and semi-supervised learning. Proceedings of the 22nd International Conference on Computational Linguistics (pp. 985–992). find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2010-litvak-multi-lingual-summarization-comparative-analysis/