Learning to Solve Arithmetic Word Problems with Verb Categorization

Summary

The paper presents ARIS, an early system for addition and subtraction word problems that does not depend on equation templates. The text is split into fragments, each with a verb, an entity (the noun a number belongs to), a quantity, attributes (modifying adjectives) and one or two containers (an owner or a location). The world state is the amount of each entity in each container, and every sentence is a transition between states. The key learned step classifies each verb into one of seven categories: observation, positive and negative change, positive and negative transfer between two containers, construction and destruction. An SVM predicts, for each (verb, container) pair, whether the quantity goes up or down. The system then updates the states and drops irrelevant sentences by matching entities and attributes against the question. It fills missing containers under a circumscription assumption and builds an equation with one unknown from two adjacent states. The entity-container-transition model is the paper's lasting idea: each state change is an elementary, interpretable step.

How Mova uses it

• math/src/steps.rs: the math solver's main non-lexical features are verb classes (have, get, give, make, lose, move, compare) in the spirit of ARIS's verb categorisation, extended with Roy & Roth 2018. The model sees "how the quantity changes" instead of a bag of words, and question-relevance features follow ARIS and Patel 2021.
• Possession and changes in quantity: the seed on possession and quantity change ("give" decreases the giver and increases the receiver; the question decides which one is asked) is the compiled first-level source of these verb classes. Its verbs block lists the members of each class, and the solver, the story world and pragmatics all share it.
• math/src/qworld.rs: the solver writes the quantitative world of a problem itself, deterministically and ARIS-style. Each number becomes a world command (owner = subject, thing = unit, action from the verb class, rate "X per Y"), each question becomes a query, and an exact rational-number core does the arithmetic. If the world is unknown, it answers "don't know" instead of guessing.

Effectiveness in Mova

Measured only together with other ideas or as whole modules:

• Replacing words and lemmas with non-lexical features (verb classes after ARIS and Roy & Roth 2018, rate features after UnitDep 2017, subject match after Patel 2021) raised SVAMP from 44.7% to 48.0%. The project notes record this combined gain; the share due to ARIS alone is not separated.
• The ARIS-like quantitative world math::qworld v1 answered 31 of 300 SVAMP problems correctly, with 102 "don't know". It was too weak as a third opinion. Trained on verified scripts it reached 34/300, then 97/300 with a learned query template, then 103/300 with question-driven cells for differences.

👨‍🔬💥

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/v1/d14-1058

@inproceedings{hosseini2014learning,
  title = {{Learning to Solve Arithmetic Word Problems with Verb Categorization}},
  author = {Hosseini, Mohammad Javad and Hajishirzi, Hannaneh and Etzioni, Oren and Kushman, Nate},
  booktitle = {Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2014},
  doi = {10.3115/v1/d14-1058},
  url = {https://doi.org/10.3115/v1/d14-1058}
}

References

30 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Antoine Bordes, Nicolas Usunier, and Jason Weston. 2010. Label ranking under ambiguous supervision for learning semantic correspondences. In Proc. International Conference on Machine Learning (ICML). find
2. SRK Branavan, Harr Chen, Luke S. Zettlemoyer, and Regina Barzilay. 2009. Reinforcement learning for mapping instructions to actions. In Proc. of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing of the AFNLP (ACL… find
3. SRK Branavan, Nate Kushman, Tao Lei, and Regina Barzilay. 2012. Learning high-level planning from text. In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL). find
4. Nathanael Chambers and Dan Jurafsky. 2009. Unsupervised learning of narrative schemas and their participants. In Proc. of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing of the AFNLP (ACL-AFNLP). find
5. David Chen, Joohyun Kim, and Raymond Mooney. 2010. Training a multilingual sportscaster: Using perceptual context to learn language. Journal of Artificial Intelligence Research, 37. find
6. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proc. Language Resources and Evaluation Conference (LREC). find our card
7. Jacob Eisenstein, James Clarke, Dan Goldwasser, and Dan Roth. 2009. Reading to learn: Constructing features from semantic abstracts. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP). find
8. Edward A. Feigenbaum and Julian Feldman, editors. 1963. Computers and Thought. McGraw Hill, New York. find
9. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by gibbs sampling. In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL). find
10. Ruifang Ge and Raymond J. Mooney. 2006. Discriminative reranking for semantic parsing. In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL). find
11. Dan Goldwasser and Dan Roth. 2011. Learning from natural instructions. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI). find
12. Dan Goldwasser, Roi Reichart, James Clarke, and Dan Roth. 2011. Confidence driven unsupervised semantic parsing. In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL). find
13. Hannaneh Hajishirzi, Julia Hockenmaier, Erik T. Mueller, and Eyal Amir. 2011. Reasoning about robocup soccer narratives. In Proc. Conference on Uncertainty in Artificial Intelligence (UAI). find
14. Rohit J. Kate and Raymond J. Mooney. 2007. Learning language semantics from ambiguous supervision. In Proc. Conference of the Association for the Advancement of Artificial Intelligence (AAAI). find
15. Nate Kushman and Regina Barzilay. 2013. Using semantic unification to generate regular expressions from natural language. In Proceeding of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics. find
16. Nate Kushman, Yoav Artzi, Luke Zettlemoyer, and Regina Barzilay. 2014. Learning to automatically solve algebra word problems. In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL). find
17. Iddo Lev, Bill MacCartney, Christopher D. Manning, , and Roger Levy. 2004. Solving logic puzzles: From robust processing to precise semantics. In Workshop on Text Meaning and Interpretation at Association for Computational Linguistics (ACL). find
18. Iddo Lev. 2007. Packed Computation of Exact Meaning Representations. Ph.D. thesis, CS, Stanford University. find
19. Percy Liang, Michael I. Jordan, and Dan Klein. 2009. Learning semantic correspondences with less supervision. In Proc. of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing of the AFNLP (ACL-AFNLP). find
20. Dekang Lin. 1998. An information-theoretic definition of similarity. In Proc. International Conference on Machine Learning (ICML). find
21. John McCarthy. 1980. Circumscription—a form of non-monotonic reasoning. Artificial Intelligence, 13. find
22. George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38. find our card
23. Mee Young Park and Trevor Hastie. 2007. L1-regularization path algorithm for generalized linear models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69. find
24. Hoifung Poon and Pedro Domingos. 2009. Unsupervised semantic parsing. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP). find
25. Karthik Raghunathan, Heeyoung Lee, Sudarshan Rangarajan, Nathanael Chambers, Mihai Surdeanu, Dan Jurafsky, and Christopher Manning. 2010. A multi-pass sieve for coreference resolution. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP). find our card
26. Philip Resnik. 1995. Using information content to evaluate semantic similarity in a taxonomy. In International joint conference on Artificial intelligence (IJCAI). find
27. Benjamin Snyder and Regina Barzilay. 2007. Database-text alignment via structured multilabel classification. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI). find
28. Adam Vogel and Daniel Jurafsky. 2010. Learning to follow navigational directions. In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL). find
29. Ian H Witten, Eibe Frank, Leonard E Trigg, Mark A Hall, Geoffrey Holmes, and Sally Jo Cunningham. 1999. Weka: Practical machine learning tools and techniques with java implementations. find
30. Luke S. Zettlemoyer and Michael Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Proc. Conference on Uncertainty in Artificial Intelligence (UAI). find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/hosseini-2014-learning-solve-arithmetic-word-problems/