Extracting and evaluating general world knowledge from the Brown Corpus

What it is about

The KNEXT system extracts general "possibilistic" propositions, such as "a person may believe a proposition", from Penn Treebank parses of the Brown corpus. A pattern matcher finds clause and modifier patterns, semantic patterns are built bottom-up, and modifiers are abstracted away. The run yields 137,510 propositions, 117,326 of them distinct, more than two per sentence. Five judges sorted 250 random propositions into six categories. Between 49% and 64% were judged reasonable general claims. Style matters: plain narratives gave about 73% reasonable claims, while abstract philosophical and theological essays gave 41% and under 55%. Agreement is modest (pairwise kappa 0.195 to 0.367 over six categories). The share of propositions that several judges accept falls from 57% for one judge to 35% for three and 28% for five. The authors locate the main limits in the flat SBAR category and in the missing complement–adjunct distinction of the Treebank.

Ideas

• I1. A generic-proposition extractor over Mova's own UD trees rather than Treebank bracketing. It emits "a X may Y" from modal and generic-subject patterns, and stores the sentence id with each proposition, so every fact can say where it came from. Place it next to world/src/knowledge.rs.
• I2. A register gate: classify each source text first (narrative or abstract, using 37. Register and variation) and report the yield per register, as the paper does for its two style groups.
• I3. Use the paper's six labels for Mova's own extracted propositions on a fixed sample, and count survival by the number of judges who agree.
• I4. A pattern-coverage count: the number of distinct UD dependency patterns that the extractor covers, the analogue of the paper's 80 patterns covering thousands of phrase types.

For Mova

Chapters: 13. Verbs (modals), 22. Phrases: noun, adjective, adverb and prepositional phrases, 37. Register and variation. Code: en/src/parse.rs for the trees, world/src/knowledge.rs for storage. Measure: share of propositions judged reasonable on a fixed sample, per register, run twice.

Topics: Treebanks

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{schubert2003extracting,
  title = {{Extracting and evaluating general world knowledge from the Brown Corpus}},
  author = {Schubert, Lenhart and Tong, Matthew},
  booktitle = {Proceedings of the HLT-NAACL 2003 Workshop on Text Meaning},
  year = {2003},
  url = {https://aclanthology.org/W03-0902/}
}

References

17 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eneko Agirre and David Martinez. 2001. Learning class-to-class selectional preferences. In Proc. of the 5th Workshop on Computational Language Learning (CoNLL-2001), Toulouse, France, July 6-7,. find
2. Matthew Berland and Eugene Charniak. 1999. Finding parts in very large corpora. In Proc. of the 37th Ann. Meet. of the Assoc. for Computational Linguistics (ACL-99), Univ. of Maryland, June 22 - 27,. find
3. Jean Carletta. 1996. Assessing agreement on classification tasks: the kappa statistic. Computational Linguistics, 22(2):249–254. find
4. Stephen Clark and David Weir. 1999. An iterative approach to estimating frequencies over a semantic hierarchy. In Proc. of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora. Also available at http://www.cogs.susx.ac.uk/users/davidw/research/papers.html. ↗
5. Ralph Grishman and John Sterling. 1992. Acquisition of selectional patterns. In Proc. of COLING-92, pages 658–664, Nantes, France. find
6. Marti A. Hearst. 1998. Automated discovery of Word-Net relations. In Christiane Fellbaum, editor, Word-Net: An Electronic Lexical Database, pages 131–153? MIT Press. find
7. H. Kucera and W.N. Francis. 1967. Computational Analysis of Present-Day American English. Brown University Press, Providence, RI. find
8. Shari Landes, Claudia Leacock, and Randee I. Tengi. 1998. Building semantic concordances. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database, pages chapter 8, 199–216. MIT Press, Cambridge, MA. find
9. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330, June. 1993. Proc. of the 5th Message Understanding Conference (MUC-5). find our card
10. Morgan Kaufmann, Los Altos, CA. 1995. Proc. of the 6th Message Understanding Conference (MUC-6). find
11. Morgan Kaufmann, Los Altos, CA. 1998. Proc. of the 7th Message Understanding Conference (MUC-7). Morgan Kaufmann, Los Altos, CA, April 29 – May 1, Virginia. find
12. P. Resnik. 1992. A class-based approach to lexical discovery. In Proc. of the 30th Ann. Meet. of the Assoc. for Computational Linguistics (ACL-92), pages 327– 329, Newark, DE. find
13. P. Resnik. 1993. Semantic classes and syntactic ambiguity. In Proc. of ARPA Workshop on Human Language Technology, Plainsboro, NJ. find
14. Ellen Riloff and Rosie Jones. 1999. Learning dictionaries for information extraction by multi-level bootstrapping. In Proc. of the 16th Nat. Conf. on Artificial Intelligence (AAAI-99). find
15. Lenhart K. Schubert. 2002. Can we derive general world knowledge from texts? In Proc. of 2nd Int. Conf. on Human Language Technology Research (HLT 2002), pages 94–97, San Diego, CA, March 24-27. find
16. Uri Zernik and Paul Jacobs. 1990. Tagging for learning: Collecting thematic relations from corpus. In Proc. of the 13th Int. Conf. on Computational Linguistics (COLING-90), pages 34–39, Helsinki. find
17. Uri Zernik. 1992. Closed yesterday and closed minds: Asking the right questions of the corpus to distinguish thematic from sentential relations. In Proc. of COLING-92, pages 1304–1311, Nantes, France, Aug. 23-28,. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/schubert-2003-extracting-evaluating-general-world-knowledge/