Mova0.5 pre-release⌕
☰
home / docs / papers / English / Lexicon and grammar in probabilistic tagging of written English

Lexicon and grammar in probabilistic tagging of written English

Note: the text file is an automatic OCR of the scanned paper, badly damaged in many places (tables and tree figures are unreadable). The summary below rests on the readable prose only.

What it is about

The paper describes the UCREL system at Lancaster, built with IBM support. Its word tagger, CLAWS, assigns part-of-speech tags by a lexicon of about 8,000 word forms, each with tags marked by three levels of probability, and a suffix list for unknown words. Tag selection uses first-order tag-pair probabilities from a tagged corpus. The author reports that CLAWS gives the right tag to over 96% of running words, leaving 3 to 4% for post-editing. The lexicon was extended to about 26,500 entries, from a list of 15,000 words tagged automatically and post-edited. Verbs were subcategorised: about 2,500 verbs coded from concordances, with an initial scheme of eleven verb subcategories. A grammar of about 10,000 context-free productions was built from a parsed databank of about 45,000 words, and a "skeleton parsing" scheme inserts only the uncontroversial brackets, which limits the parses the grammar must consider.

Ideas

For Mova

The chapters are 13. Verbs for subcategorisation and 20. Syntax: the Universal Dependencies frame and 21. The simple clause for the skeleton idea. The code points are en/src/ptag.rs and en/src/tag.rs for the tagger and suffix list, en/src/dict.rs for the marked lexicon, and en/src/parse.rs or gparse for the skeleton parse. The measure: unknown-word tagging accuracy with and without the suffix list, and the number of remaining parses after skeleton pruning, run twice for determinism.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/982023.982049

@inproceedings{beale1988lexicon,
  title = {{Lexicon and grammar in probabilistic tagging of written English}},
  author = {Beale, Andrew David},
  booktitle = {26th Annual Meeting of the Association for Computational Linguistics},
  year = {1988},
  doi = {10.3115/982023.982049},
  url = {https://doi.org/10.3115/982023.982049}
}

References

15 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Erik Akkerman, Pieter Masereeuw and V/ilium Meijs (1985). find
  2. 'Designing a C o m Lexi~n for Linguistic Proposes'. ASCOT Report No. I, CIP-Gegevens KoninHij~e Bib~otheeg. Den Haaf, Netherlm~. Lalit R. Bahl, Frederick Jelinck and Rol~rt L Mercer (1983). find
  3. "A Maximum I.ik~lillood A tO Speech Recognition', IEEE Transactions on Pattern Analysis and Machine In:eUigence, VoL PAMI-5, No. 2, March 1983. find
  4. J. IL Baker (1979). 'Trainable Grammms for Speech Recognition,' Proceedings of the Spring Conference of the Acoustical Society of America. find
  5. Bran Boguraev, Ted Brlscoe, John l l , David and Claire Graver (19873. 'The Derivation of a Grammatically Indexed Lexicon from the Longman Di~onary of Contemporary Engfish', Proceedings of ACL-87, Ste~forrL California. Ted Brise~, Claire Grover, Bran Boguraev, Jolm Carroll (19873. 'A Formalism and Environment for the Develol~nent of a Large Grammar of English', proceedings of IJCAI, Milan. ! Keith Brown (1984)./~nguugi¢$ Today, Fomana, U.K. find
  6. John Carroll, Brml B o , Claire Grover, Ted Briscoe (1988). 'The Grammar Development Environment User M~ual', Cambridge Computer Laboratory Technical Report 127, Cambridge, England. find
  7. Roger Gmside, Geoffrey Leech aad Geoff~y Sampson (19873. The Comp,m~gnal Analysis of English: A Corpus-Based Approach, Longman, London and New York. Claire Graver, Ted Bt~.oe, John Can~ll, Bran Boguraev (1988). 'The Alvey Natural L,mguage Tools Proje:t Grammar:. find
  8. A Wide-Coverage Compalafiooai Grammar of F~Sllxh', Lancaster Papers In 47. of Linguistics. Univorsity of Lma:uler: Mawdt 1988. find
  9. G. Fomey, Jr. (1973). '1"he Viu~oi Algorithm', Proc. IEEE, Vol 61: March 1973, pp. 268-278. find
  10. W. Nelson Franc~ mad Henry (1982). Frequency • Analysis of English Usage: Lexicon and Granmu~, Houghtoo Boston. find
  11. Knut Hofland and Stig Johansson (1982). Word Frequencies in BriOJh and Ismerican EnglisS. Norwegian Computing Cenue for the Humanities. Bergen: Longmmx. Lo~on. find
  12. John E. H o a~! Jeff~'y D. Ullmm (1979). l n w Automata Theory, Languages, and Compum~on, Addlsow Wesley, Reading, MesL Stig J F.~ Atwe~ Roger Gmeide and Geoffrey Leech (1986). Whe Tagged LOB Corpus Users' Mmmal,' Norwegian Computing for the Humanities, Bergen. find
  13. Henry and W. Nelson Francis (19673. Compum:ional Analysis of Present-day Ame~an English, Brown Unive:sity Press, Pmvidmu:e, Rlmde lsla~ Geoffrey L (198"/). 'Parsers' Manual', Depamnmu of !J-m~is~cs, UnivemSy of Lmmu~er. Longman Dicdonary of Conu~pomry E n g / (1978), second edition (19873, Lonmman Group I.imig~ I-Iar~w and l~Jnelmld Randolph Quirk, Sidney G m n : Geoffrey Leech and Jan Svartv~ (19853. A Compre.hens~ Grammar of the English Language, Longm~ Inc., New Yor~ Information Naomi Sager (1981). Namra/ Language Praces~g, Addi-¢on-Wesley, Reading, Mass. find
  14. Geo~ Sampson (1987). "The grammatical database and panm 8 scheme' in Gar~de, Leech and Smnpson, pp 82-96. find
  15. Richard A. Slmmmn (1988). "The Winchesl~r Unification Parsing System', IBM UICSC Report 999: April 1988. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/beale-1988-lexicon-grammar-probabilistic-tagging-written/