Multi-document multilingual summarization corpus preparation, Part 2: Czech, Hebrew and Spanish

What it is about

This is the second part of the report on the MultiLing 2013 corpus, which covers ten languages; this part covers Czech, Hebrew and Spanish. A topic is a set of news documents describing an event sequence. Each language needed source documents (translated from English where needed), human summaries of 240–250 words, and human grades of the summaries for overall responsiveness on a 1–5 scale. For Hebrew, five clusters of ten articles were translated and summarised by three people; per article, reading took 2.5 minutes, translation 30 minutes and proofreading 18.5 minutes, and the average grades of the three summarisers were 4.80, 4.40 and 4.13. For Spanish, 15 topics of ten documents were translated by 30 students (104.5 man-hours), and the average human grade was 3.8 out of 5. The authors record that ordering texts by date gave the most fluent summaries, that translation needs expert knowledge, and that machine-translated drafts slowed proofreading.

Ideas

• I1. Hebrew spelling variants: the team left plene and deficient spellings, and foreign words that have both a transliterated and a Hebrew form, to Wikipedia's usage. A rule for languages/he could map plene and deficient forms to one lemma for lookup while keeping both as surface forms.
• I2. Date-ordered event chains: the summarisers found that ordering by publication date gave the most fluent text, but background facts sometimes sat in a later document. In world, order the event chain by date, and let a rule move a background event to the front when the first event refers to it.
• I3. Translate from scratch when building a gold set: the paper found that editing machine output took longer than translating by hand with dictionaries, and that the first text of each cluster took longest. Any Hebrew or Aramaic gold set Mova builds should follow the same rule, and budget the first items of each topic higher.

For Mova

I1 goes into 2. Script and spelling. I2 is an idea for the event chain in world, with the date rule noted in 10. Historical layers: Biblical, Mishnaic, Medieval and Modern Hebrew only as background. I3 goes into 12. Mova and Modern Hebrew: where it stands, what it would need as a gold-set policy.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{elhadad2013multi,
  title = {{Multi-document multilingual summarization corpus preparation, Part 2: Czech, Hebrew and Spanish}},
  author = {Elhadad, Michael and Miranda-Jiménez, Sabino and Steinberger, Josef and Giannakopoulos, George},
  booktitle = {Proceedings of the MultiLing 2013 Workshop on Multilingual Multi-document Summarization},
  year = {2013},
  url = {https://aclanthology.org/W13-3102/}
}

References

12 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Hoa Trang Dang and K. Owczarzak. 2009. Overview of the tac 2009 summarization track, Nov. find
2. Michael Elhadad, Sabino Miranda-Jiménez, Josef Steinberger, and George Giannakopoulos. 2013. Multi-document multilingual summarization corpus preparation, part 2: Czech, hebrew and spanish. In MultiLing 2013 Workshop in ACL 2013, Sofia, Bulgaria, August. find our card
3. Brigitte Endres-Niggemeyer and Elisabeth Wansorra. 2004. Making cognitive summarization agents work in a real-world domain. In Proceedings of NLUCS Workshop, pages 86–96. Citeseer. find
4. Brigitte Endres-Niggemeyer. 2000. Human-style WWW summarization. Technical report. find
5. G. Giannakopoulos, M. El-Haj, B. Favre, M. Litvak, J. Steinberger, and V. Varma. 2011. TAC 2011 MultiLing pilot overview. In TAC 2011 Workshop, Maryland MD, USA, November. find
6. E. Hovy, C. Y. Lin, L. Zhou, and J. Fukumoto. 2005. Basic elements. find
7. Lei Li, Corina Forascu, Mahmoud El-Haj, and George Giannakopoulos. 2013. Multi-document multilingual summarization corpus preparation, part 1: Arabic, english, greek, chinese, romanian. In MultiLing 2013 Workshop in ACL 2013, Sofia, Bulgaria, August. find
8. C. Y. Lin. 2004. Rouge: A package for automatic evaluation of summaries. Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004), pages 25–26. find
9. Annie Louis and Ani Nenkova. 2012. Automatically assessing machine summary content without a gold standard. Computational Linguistics, 39(2):267–300, Aug. find
10. Annie Piolat, Thierry Olive, and Ronald T Kellogg. 2005. Cognitive effort during note taking. Applied Cognitive Psychology, 19(3):291–312. find
11. Horacio Saggion and Sandra Szasz. 2012. The concisus corpus of event summaries. In LREC, pages 2031–2037. find
12. H. Saggion, J. M. Torres-Moreno, I. Cunha, and E. SanJuan. 2010. Multilingual summarization evaluation without human models. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, page 1059– 1067. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2013-elhadad-multi-document-multilingual-summarization-corpus/