Experiments on Sentence Boundary Detection

What it is about

The paper looks at sentence boundary detection for the output of automatic speech recognition (ASR). Such text has no reliable punctuation and often no case, so the usual tools for edited text do not transfer. The authors first measure human performance on this task, with several annotators whose agreement is high (a kappa of 0.98 for two annotators who marked the mixed-case text). They then apply a memory-based learner to ASR transcripts and compare the scores on mixed-case and upper-case text. The best reported F-scores are about 0.92 for mixed case and 0.91 for upper case. The paper also reviews an earlier comma-insertion system for ASR output, which reached 75.6% precision and 65.6% recall on 2,317 sentences of the Wall Street Journal.

Ideas

• I1. en/src/tok.rs: a boundary decision on text without case, with the case of each token removed. Measure F1 over boundary points on EWT test in the normal and the lower-cased setting, to see how much the case cue is worth in Mova.
• I2. Report the human-agreement test of the paper for the Mova sentence splitter: two annotators mark the boundaries of a small set of EWT sentences, and the kappa is reported next to the system score.
• I3. A memory-based (nearest-neighbour) variant of the period classifier in en/src/tok.rs, compared with the table-driven one on EWT test, with the same features. The paper's method is the nearest-neighbour type, so this is a direct test of its idea without neural nets.

For Mova

The code is en/src/tok.rs, and the metric is en tok-eval. The chapter is 07. Punctuation, capital letters, abbreviations and numbers. The paper is about speech transcripts and its corpus is not public, so its figures are not Mova figures. Measure first on EWT test with the lower-cased variant, and run it twice to check the same numbers come out.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/974147.974159

@inproceedings{stevenson2000experiments,
  title = {{Experiments on Sentence Boundary Detection}},
  author = {Stevenson, Mark and Gaizauskas, Robert},
  booktitle = {Sixth Applied Natural Language Processing Conference},
  year = {2000},
  doi = {10.3115/974147.974159},
  url = {https://doi.org/10.3115/974147.974159}
}

References

15 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. D. Beeferman, A. Berger, and J. Lafferty. 1998. CY-BERPUNC: A lightweight punctuation annotation system for speech. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 689-692, Seattle, WA. find
2. E. Brill. 1992. A simple rule-based part of speech tagger. In Proceeding of the Third Conference on Applied Natural Language Processing (ANLP-92), pages 152-155, Trento, Italy. find
3. R~ L. Burnard, 1995. Users Reference Guide for the M. Stevenson and Y. Wilks. 1999. Combining weak British National Corpus. Oxford University Com-knowledge sources for sense disambiguation. In puting Services. Proceedings of the Sixteenth International Joint J. Carletta. 1996. Assessing agreement on… find
4. C. van Rijsbergen. 1979. Information Retrieval. find
5. N. Chinchor, P. Robinson, and E. Brown. Butterworths, London. 1998. HUB-4 Named Entity Task Definition (version 4.8). Technical report, SAIC. http ://www. nist. gov/speech/hub4-98. find
6. R. Cole, editor. 1996. Survey of the State of the Art in Human Language Technology. Available at: http://cslu.cse.ogi.edu/HLTsurvey/HLTsurvey.html. Site visited 17/11/99. ↗
7. W. Daelemans, J. Zavrel, K. van der Sloot, and A. van den Bosch. 1999. TiMBL: Tilburg memory based learner version 2.0, reference guide. Technical report, ILK Technical Report 98-03. ILK Reference Report 99-01, Available from http ://ilk. kub. nl/" ilk/papers/ilk9901, ps. gz. find
8. C. Fellbaum, J. Grabowski, S. Landes, and A. Ban-mann. 1998. Matching words to senses in Word-Net: Naive vs. expert differentiation of senses. In C. Fellbaum, editor, WordNet: An electronic lex-ieal database and some applications. MIT Press, Cambridge, MA. find
9. W. Gale and G. Sampson. 1996. Good-Turing frequency estimation without tears. Journal of Quantitave Linguistics, 2(3):217-37. find
10. Y. Gotoh and S. Renals. 2000. Information extraction from broadcast news. Philosophical Transactions of the Royal Society of London, series A: Mathematical, Physical and Engineering Sciences. (to appear). find
11. A. Mikheev. 1998. Feature lattices for maximum entropy modelling. In Proceedings of the 36th Meeting of the Association for Computational Linguistics (COLING-ACL-98), pages 848-854, Montreal, Canada. find
12. R. Moore, J. Dowding, H. Bratt, J. Gawron, Y. Gorfu, and A. Cheyer. 1997. CommandTalk: A Spokcaa-Language Interface to Battlefield Simulations. In Proceedings of the Fifth Conference on Applied Natural Language Processing, pages 1-7, Washington, DC. find
13. D. Palmer and M. Hearst. 1994. Adaptive sentence boundary disambiguation. In Proceedings of the 1994 Conference on Applied Natural Language Processing, pages 78-83, Stutgart, Germany. find our card
14. J. Reynar and A. Ratnaparkhi. 1997. A maximum entropy approach to identifying sentence boundries. In Proceedings of the Fifth Conference on Applied Natural Language Processing, pages 16-19, Washington, D.C. find
15. S. Siegel and N. Castellan. 1988. Nonparametrie Statistics for the Behavioural Sciences. McGraw-Hill, second edition. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2000-stevenson-experiments-sentence-boundary-detection/