How a language model learns: guessing the next word, sentence 1

It learned by playing a giant game of "guess the missing word" over and over with more text than you could read in thousands of lifetimes.

Who acts: chatbot, giant game, you · What happens: learn, play, guess, read · To what, where: giant game, missing word, more text, thousands

This is a sentence of the lesson «How a language model learns: guessing the next word» of the ai course of Mova's primary school. The page gives its syntax tree — a dependency tree in the style of Universal Dependencies — and its pragmatics clause by clause: who acts (chatbot, giant game, you), what happens, when and where, as Mova's frames read it from the tree.

The tree

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	It	it	PRON	Case=Nom Gender=Neut Number=Sing Person=3 PronType=Prs	2 learned	nsubj
2	learned	learn	VERB	Mood=Ind Tense=Past VerbForm=Fin	root	root
3	by	by	SCONJ	_	4 playing	mark
4	playing	play	VERB	Tense=Pres VerbForm=Part	2 learned	advcl
5	a	a	DET	Definite=Ind PronType=Art	7 game	det
6	giant	giant	ADJ	Degree=Pos	7 game	amod
7	game	game	NOUN	Number=Sing	4 playing	obj
8	of	of	SCONJ	_	10 guess	mark
9	"	"	PUNCT	_	10 guess	punct
10	guess	guess	VERB	VerbForm=Inf	7 game	acl
11	the	the	DET	Definite=Def PronType=Art	13 word	det
12	missing	missing	ADJ	Degree=Pos	13 word	amod
13	word	word	NOUN	Number=Sing	10 guess	obj
14	"	"	PUNCT	_	10 guess	punct
15	over	over	ADV	_	10 guess	advmod
16	and	and	CCONJ	_	17 over	cc
17	over	over	ADV	_	15 over	conj
18	with	with	ADP	_	20 text	case
19	more	more	ADJ	Degree=Cmp	20 text	amod
20	text	text	NOUN	Number=Sing	10 guess	obl
21	than	than	SCONJ	_	24 read	mark
22	you	you	PRON	Case=Nom Person=2 PronType=Prs	24 read	nsubj
23	could	could	AUX	VerbForm=Fin	24 read	aux
24	read	read	VERB	VerbForm=Inf	20 text	acl
25	in	in	ADP	_	26 thousands	case
26	thousands	thousand	NOUN	Number=Plur	24 read	obl
27	of	of	ADP	_	28 lifetimes	case
28	lifetimes	lifetime	NOUN	Number=Plur	26 thousands	nmod
29	.	.	PUNCT	_	2 learned	punct

Pragmatics

No stored speech act for this sentence.

#	verb	tense, mood	who	what, whom	where	other	roles by
1	learn	past	chatbot				vn learn-14-1 «NP V»: agent=Agent
2	play	gerund	chatbot not said here	a giant game			vn use-105.1 «NP V NP»: agent=Agent game=Theme
3	guess in a quote	infinitive	a giant game not said here	the missing word		with more text; over property	vn conjecture-29.5-1 «NP V NP»: agent=Agent word=Theme
4	read	could	you	more text not said here	thousands of lifetimes		vn transfer_mesg-37.1.1-1 «NP V NP»: agent=Agent text=Topic

One row per clause the reader finds. «not said here»: a role filled from the clause above. «roles by»: a VerbNet class and frame when one fits the verb, else the reader's rules on the tree.

Licence: CC BY-SA 4.0, as the whole of Mova corpus 0.4, of which this treebank (the agreed school, mova-school-agreed-en.conllu) is part. The lessons were written for Mova's primary school. Sentence school:ai:002-guessing-the-next-word.lesson:1.

The tree: Mova annot: Rust (en) → Haiku 5.5 (medium, Batches API) patches; gates UD 2.18; agreed by treeagree rules (Stanza, Trankit, spaCy trf): R5-strong ×1, R6-punct-agree ×1, R7-rel-agree ×1. The frames are read from the tree by Mova's frames reader (prag) when this page is built, the lesson read in order, so a pronoun may be resolved from an earlier sentence.

See the release 0.4.

This page in Markdown → https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/1/