Sentence Boundary Detection: A Long Solved Problem?

What it is about

The authors review sentence boundary detection (SBD) for English and find that results are hard to compare. Earlier reports of 99% and more use narrow task definitions. They run nine publicly available SBD systems and toolkits on standard edited corpora (WSJ, Brown, GENIA) and on user-generated Web text, with precision, recall and F1 over boundary points. Out of the box, scores vary with the text type. With a generalized task, which asks for all gold boundaries in running text, performance of 99% is not generally available. Forcing sentence boundaries at paragraph breaks and normalising curly quotes to ASCII or LaTeX quotes lifts the best-case scores on the Web samples, and the authors report that this gain is large for the user-generated text.

Ideas

• I1. en/src/tok.rs: force a sentence boundary at a paragraph break and normalise curly quotes before splitting, as the paper's best-case setting does. Measure F1 over boundary points on EWT test and report the change per genre.
• I2. Report precision, recall and F1 over boundary points in the Mova sentence splitter, not only accuracy, since the paper shows accuracy hides the error on rare boundaries.
• I3. Use the paper's generalized task (all gold boundaries in running text) as the metric for the en parse-eval input stage, and count how many parse errors begin with a missed boundary.

For Mova

The code is en/src/tok.rs, and the command for the metric is en tok-eval. The chapter is 07. Punctuation, capital letters, abbreviations and numbers. The Web and GENIA samples are research-only or NC data, so use them as tests only. Measure first on EWT test with F1 over boundary points, twice, to check the numbers are stable.

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{read2012sentence,
  title = {{Sentence Boundary Detection: A Long Solved Problem?}},
  author = {Read, Jonathon and Dridan, Rebecca and Oepen, Stephan and Solberg, Lars Jørgen},
  booktitle = {Proceedings of COLING 2012: Posters},
  year = {2012},
  url = {https://aclanthology.org/C12-2096/}
}

References

19 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bird, S., Klein, E., and Loper, E. (2009). Natural Language Processing with Python. O’Reilly, Beijing. find
2. Dridan, R. and Oepen, S. (2012). Tokenization. Returning to a long solved problem. A survey, contrastive experiment, recommendations, and toolkit. In Proceedings of the 50th Meeting of the Association for Computational Linguistics, page 378 – 382, Jeju, Republic of Korea. find our card
3. Francis, W. N. and Kucera, H. (1982). Frequency Analysis of English Usage. Houghton Mifflin Co., New York. find
4. Gillick, D. (2009). Sentence boundary detection and the problem with the U.S. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Short Papers, page 241 – 244, Boulder, CO, USA. find our card
5. Grefenstette, G. and Tapanainen, P. (1994). What is a word, what is a sentence? Problems of tokenization. In Proceedings of the 3rd Conference on Computational Lexicography and Text Research, page 79 – 87, Budapest, Hungary. find
6. Kim, J.-D., Ohta, T., Teteisi, Y., and Tsujii, J. (2003). GENIA corpus — a semantically annotated corpus for bio-textmining. Bioinformatics, 19:i180 – i182. find
7. Kiss, T. and Strunk, J. (2002). Viewing sentence boundary detection as collocation identification. In Proceedings of 6. Konferenz zur Varabeitung natürlicher Sprache, page 75 – 82, Saarbrücken, Germany. find
8. Kiss, T. and Strunk, J. (2006). Unsupervised multilingual sentence boundary detection. Computational Linguistics, 32(4):485 – 525. find our card
9. Marcus, M., Santorini, B., and Marcinkiewicz, M. A. (1993). Building a large annotated corpora of English: The Penn Treebank. Computational Linguistics, 19:313 – 330. find
10. Mikheev, A. (2000). Tagging sentence boundaries. In Proceedings of the 1st Meeting of the North American Chapter of the Association for Computational Linguistics, page 264 – 271, Seattle, WA, USA. find
11. Mikheev, A. (2002). Periods, capitalized words, etc. Computational Linguistics, 28(3):289 – 318. find
12. Morante, R. and Blanco, E. (2012). *SEM 2012 shared task: Resolving the scope and focus of negation. In Proceedings of the 1st Joint Conference on Lexical and Computational Semantics, page 265 – 274, Montréal, Canada. find
13. Nunberg, G. (1990). The Linguistics of Punctuation. Number 18 in Lecture Notes. CSLI Publications, Stanford, CA. find
14. Palmer, D. D. and Hearst, M. A. (1997). Adaptive multilingual sentence boundary disambiguation. Computational Linguistics, 23(2):242 – 267. find
15. Read, J., Flickinger, D., Dridan, R., Oepen, S., and Øvrelid, L. (2012). The WeSearch Corpus, Treebank, and Treecache. A comprehensive sample of user-generated content. In Proceedings of the 8th International Conference on Language Resources and Evaluation, Istanbul, Turkey. find
16. Reynar, J. C. and Ratnaparkhi, A. (1997). A maximum entropy approach to identifying sentence boundaries. In Proceedings of the 5th Conference on Applied Natural Language Processing, page 16 – 19, Washington, D.C., USA. find
17. Riley, M. D. (1989). Some applications of tree-based modelling to speech and language. In Proceedings of the DARPA Speech and Natural Language Workshop, page 339 – 352. find
18. Walker, D. J., Clements, D. E., Darwin, M., and Amtrup, J. W. (2001). Sentence boundary detection: A comparison of paradigms for improving MT quality. In Proceedings of the MT Summit VIII, Santiago de Compostela, Spain. find
19. Ytrestøl, G., Oepen, S., and Flickinger, D. (2009). Extracting and annotating Wikipedia subdomains. In Proceedings of the 7th International Workshop on Treebanks and Linguistic Theories, page 185 – 197, Groningen, The Netherlands. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2012-read-sentence-boundary-detection-long-solved/