How a language model learns: guessing the next word, sentence 7
A language model learned by predicting missing or next words in enormous amounts of text.
What happens: learn, predict, amount · To what, where: language model, missing words, enormous amounts, text
This is a sentence of the lesson «How a language model learns: guessing the next word» of the ai course of Mova's primary school. The page gives its syntax tree — a dependency tree in the style of Universal Dependencies — and its pragmatics clause by clause: what happens, when and where, as Mova's frames read it from the tree.
The tree
The words, one row each (CoNLL-U)
| # | word | lemma | UPOS | features | head | relation |
|---|---|---|---|---|---|---|
| 1 | A | a | DET | Definite=Ind PronType=Art | 3 model | det |
| 2 | language | language | NOUN | Number=Sing | 3 model | compound |
| 3 | model | model | NOUN | Number=Sing | 4 learned | nsubj:pass |
| 4 | learned | learn | VERB | Tense=Past VerbForm=Part Voice=Pass | root | root |
| 5 | by | by | SCONJ | _ | 6 predicting | mark |
| 6 | predicting | predict | VERB | Tense=Pres VerbForm=Part | 4 learned | advcl |
| 7 | missing | missing | ADJ | Degree=Pos | 10 words | amod |
| 8 | or | or | CCONJ | _ | 9 next | cc |
| 9 | next | next | ADJ | Degree=Pos | 7 missing | conj |
| 10 | words | word | NOUN | Number=Plur | 6 predicting | obj |
| 11 | in | in | ADP | _ | 13 amounts | case |
| 12 | enormous | enormous | ADJ | Degree=Pos | 13 amounts | amod |
| 13 | amounts | amount | NOUN | Number=Plur | 6 predicting | obl |
| 14 | of | of | ADP | _ | 15 text | case |
| 15 | text | text | NOUN | Number=Sing | 13 amounts | nmod |
| 16 | . | . | PUNCT | _ | 4 learned | punct |
Pragmatics
No stored speech act for this sentence.
| # | verb | tense, mood | who | what, whom | to whom | where | when | other | roles by |
|---|---|---|---|---|---|---|---|---|---|
| 1 | learn | participle · passive | a language model | vn learn-14-2-1 «NP V NP»: subject=Topic | |||||
| 2 | predict | gerund | missing words | enormous amounts of text | vn indicate-78 «NP V NP»: agent=Pivot word=Topic | ||||
| 3 | amount | text | nom amount «possession transitive»: of=patient |
One row per clause the reader finds. «not said here»: a role filled from the clause above. «roles by»: a VerbNet class and frame when one fits the verb, else the reader's rules on the tree.
Licence: CC BY-SA 4.0, as the whole of Mova corpus 0.4, of which this treebank (the agreed school, mova-school-agreed-en.conllu) is part. The lessons were written for Mova's primary school. Sentence school:ai:002-guessing-the-next-word.lesson:7.
The tree: Mova annot: Rust (en) → Haiku 5.5 (medium, Batches API) patches; gates UD 2.18; agreed by treeagree rules (Stanza, Trankit, spaCy trf): R5-strong ×1. The frames are read from the tree by Mova's frames reader (prag) when this page is built, the lesson read in order, so a pronoun may be resolved from an earlier sentence.
See the release 0.4.
This page in Markdown → https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/7/