Completing and Validating the Re-Aligned Switchboard Dialog Act Corpus

What it is about

The Switchboard Dialog Act corpus (1,155 phone conversations, ~205K utterances, 42 SWBD-DAMSL tags) had speech and text so badly aligned that prosody was useless for tagging. The authors finish a manual re-alignment begun in 2024 (617.5 further conversations re-aligned in Praat, all 1,155 validated) and measure the label agreement between the alignment and validation passes: Cohen's kappa 0.90. Most changed labels are statement-non-opinion → statement-opinion (578 of 723 outgoing changes). An acoustic-only BiLSTM goes from 15.5% accuracy on the old alignment to 57.2%; a RoBERTa text model with a gated acoustic branch reaches 87.8% on 10 classes, against 86.7% for text on the original data.

Ideas

• I1. Note for Mova's act scheme: the border Switchboard's own validators move most is statement vs opinion, the same border where Mova's labeller fails (evaluate→assert 139, express→assert 50 in prag eval). Treat assert/evaluate as a known soft border: report it separately, and do not let teacher-vs-SLM disagreement on it count as an error to repair without a closer rule (an evaluative adjective or verb of judgement as the predicate, see prag/data/cues.tsv eval).
• I2. Otherwise nothing to take: Mova reads written text, and the gain here is from prosody and audio alignment. Mova has no audio channel (chapter 31 §10 lists prosody as absent).

For Mova

31. Speech acts §7.1 (SWBD-DAMSL) and §10 (prosody absent). Code: prag/src/eval.rs (a separate assert/evaluate figure). Measure: the share of act errors on the assert/evaluate border.

Topics: Speech act theory

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.law-main.13

@inproceedings{chen2026completing,
  title = {{Completing and Validating the Re-Aligned Switchboard Dialog Act Corpus}},
  author = {Chen, Run and Tao, Zihao and Prado, John and LaManna, Ignazio and Puterbaugh, Ryan and Datta, Mim and Hirschberg, Julia},
  booktitle = {Proceedings of the 20th Linguistic Annotation Workshop (LAW XX)},
  year = {2026},
  doi = {10.18653/v1/2026.law-main.13},
  url = {https://doi.org/10.18653/v1/2026.law-main.13}
}

References

14 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. James Allen and Mark Core. 1997. Draft of DAMSL: Dialog act markup in several layers. find
2. Paul Boersma and David Weenink. 2026. Praat: Doing phonetics by computer. find
3. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. ↗ our card
4. Barbara Wheatley, George Doddington, Charles Hemphill, John Godfrey, Edward Holliman, Jane Mc-Daniel, and Drew Fisher. 1992. Robust automatic time alignment of orthographic transcriptions with unconstrained speech. In [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 1, pages 533–536. ↗
5. Sasha Calhoun, Jean Carletta, Jason Brenier, Neil Mayo, Dan Jurafsky, Mark Steedman, and David Beaver. 2010. The NXT-format Switchboard Corpus: A rich resource for investigating the syntax, semantics, pragmatics and prosody of dialogue. Language Resources and Evaluation, 44(4):387–419. ↗
6. Jean Carletta, Stefan Evert, Ulrich Heid, and Jonathan Kilgour. 2005. The NITE XML Toolkit: Data model and query language. Language Resources and Evaluation, 39(4):313–334. ↗
7. Run Chen, Eleanor Lin, Shayan Hooshmand, Mariam Mustafa, Ritika Nandi Rose Sloan, Alicia Yang, Andrea Lopez, Ansh Kothary, Isaac Suh, Catherine Lyu, Eric Chen, Sophia Horng, and Julia Hirschberg. 2024. RASwDA: Re-aligned switchboard dialog act corpus for dialog act prediction in conversation. In [Proceedings] International Workshop on Spoken Dialogue Systems Technology. find
8. John Godfrey, Edward Holliman, and Jane McDaniel. 1992. SWITCHBOARD: Telephone speech corpus for research and development. In [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 1, pages 517–520. ↗
9. Dan Jurafsky and Elizabeth Shriberg. 1997. Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual, draft 13. find
10. J. Richard Landis and Gary Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33(1):159–174. ↗
11. Mitchell Marcus, Beatrice Santorini, Mary Ann Marcinkiewicz, and Ann Taylor. 2000. Treebank-3. find
12. Alberto Pettarin. 2017. aeneas. find
13. Elizabeth Shriberg, Andreas Stolcke, Daniel Jurafsky, Noah Coccaro, Marie Meteer, Rebecca Bates, Paul Taylor, Klaus Ries, Rachel Martin, and Carol van Ess-Dykema. 1998. Can prosody aid the automatic classification of dialog acts in conversational speech? Language and Speech, 41(3–4):443–492. PMID: 10746366. ↗
14. SoX. 2015. SoX: Sound eXchange. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/chen-2026-completing-validating-re-aligned-switchboard/