A Maximum Entropy Model for Prepositional Phrase Attachment

What it is about

The paper models the noun-or-verb attachment of a prepositional phrase with a maximum-entropy classifier over four head words: the verb, the first noun, the preposition and the second noun. Features are questions on single words, on word classes, and on both; the word classes come from a binary tree of mutual-information clusters. Features are chosen greedily by a ranking of log-likelihood gain, not by exhaustive search. On the Wall Street Journal test events the model scores 81.6% with words and classes (77.7% with words only, 79.1% with classes only); on computer-manual text it scores 84.1%. Humans given the same four head words score 88.2%, and humans who read the whole sentence score 93.2%, so the authors locate the gap in the sentence context.

Ideas

• I1. Cluster features in en: build mutual-information word classes (deterministic, with BTreeMap ordering) from the EWT vocabulary and add class-bit features to the en perceptron in en/src/parse.rs and en/src/ptag.rs; measure EWT test LAS with and without them.
• I2. Feature selection as an explanation: use the paper's greedy log-likelihood ranking to choose feature templates for the en tagger and report the chosen templates with their gain, so that every template is justified in en explain terms.
• I3. Sentence context for attachment: the paper's gap between head words and whole sentence (88.2 to 93.2) motivates testing en/src/ctx.rs document and sentence features on PP attachments in EWT test.

For Mova

The code is en/src/parse.rs, en/src/ptag.rs and en/src/ctx.rs. The chapters are 22. Phrases: noun, adjective, adverb and prepositional phrases and 23-complementation.md. Measure first: the PP-attachment accuracy of the current en parser on EWT test against the paper's 81.6 and its human 93.2 with the sentence, so that the room is clear before any change.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1075812.1075868

@inproceedings{ratnaparkhi1994maximum,
  title = {{A Maximum Entropy Model for Prepositional Phrase Attachment}},
  author = {Ratnaparkhi, Adwait and Reynar, Jeff and Roukos, Salim},
  booktitle = {Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994},
  year = {1994},
  doi = {10.3115/1075812.1075868},
  url = {https://doi.org/10.3115/1075812.1075868}
}

References

10 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jaynes, E. T., "Information Theory and Statistical Mechanics." Phys. Rev. 106, pp. 620-630, 1957. find
2. Kullback, S., Information Theory in Statistics. Wiley, New York, 1959. find
3. Darroch, J.N. and Ratcliff, D., "Generalized Iterative Scaling for Log-Linear Models", The Annals of Mathematical Statistics, Vol. 43, pp 1470-1480, 1972. find
4. Delia Pietra,S., Della Pietra, V., Mercer, R. L., Roukos, S., "Adaptive Language Modeling Using Minimum Discriminant Estimation," Proceedings oflCASSP-92, pp. 1-633-636, San Francisco, March 1992. find
7. Black, E., Garside, R., and Leech, G., 1993. Statistically-driven Computer Grammars of English: The IBM/Lancaster Approach. Rodopi. Atlanta, Georgia. find
8. Black, E., Jelinek, F., Lafferty, J., Magerman, D. M., Mercer, R., and Roukos, S., 1993. Towards History-based Grammars: Using Richer Models for Probabilistic Parsing. In Proceedings of the Association for Computational Linguistics, 1993. Columbus, Ohio. find
9. Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J., 1984. Classification and Regression Trees. Wadsworth and Brooks. Pacific Grove, California. find
10. Brown, P. F., Della Pietra, V. J., deSouza, P. V., Lai, J. C., and Mercer, R. L. Class-based n-gram Models of Natural Language. In Proceedings of the IBM Natural Language 1TL, March, 1990. Paris, France. find
11. Hindle, D. and Rooth, M. 1990. Structural Ambiguity and Lexical Relations. In Proceedings of the June 1990 DARPA Speech and Natural Language Workshop. Hidden Valley, Pennsylvania. find
12. Magerman, D., 1994. Natural Language Parsing as Statistical Pattern Recognition. Ph.D. dissertation, Stanford University, California. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-1994-ratnaparkhi-maximum-entropy-model-prepositional-phrase/