Test 3 · ★★ · Why the bot agrees too easily

"bot said both cats and dogs are fine, it's useless" What is a fair reply?

Mova's choice: option 1 · The right answer: option 2 · wrong ✗ · Who acts: bot, cats · What happens: say, be · To what, where: fair reply · Speech: someone speaks here

This is test 3 · ★★ of lesson 6 «Why the bot agrees too easily» of the AI course of Mova's primary school. The page gives the question's syntax tree — a dependency tree in the style of Universal Dependencies — and its pragmatics clause by clause, then the 4 options as the student saw them, each with its tree and pragmatics. Every text is parsed by Mova itself. How the student answered it, with the score of every option and why: the test at school.

The question

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	"	"	PUNCT	_	3 said	punct
2	bot	bot	NOUN	Number=Sing	3 said	nsubj
3	said	say	VERB	Mood=Ind Number=Sing Person=3 Tense=Past VerbForm=Fin	root	root
4	both	both	CCONJ	_	5 cats	cc:preconj
5	cats	cat	NOUN	Number=Plur	9 fine	nsubj
6	and	and	CCONJ	_	7 dogs	cc
7	dogs	dog	NOUN	Number=Plur	5 cats	conj
8	are	be	AUX	Mood=Ind Number=Plur Person=3 Tense=Pres VerbForm=Fin	9 fine	cop
9	fine	fine	ADJ	Degree=Pos	3 said	ccomp
10	,	,	PUNCT	_	13 useless	punct
11	it	it	PRON	Case=Nom Gender=Neut Number=Sing Person=3 PronType=Prs	13 useless	nsubj
12	's	be	AUX	Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin	13 useless	cop
13	useless	useless	ADJ	Degree=Pos	9 fine	parataxis
14	"	"	PUNCT	_	13 useless	punct
15	What	what	PRON	PronType=Int	19 reply	nsubj
16	is	be	AUX	Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin	19 reply	cop
17	a	a	DET	Definite=Ind PronType=Art	19 reply	det
18	fair	fair	ADJ	Degree=Pos	19 reply	amod
19	reply	reply	NOUN	Number=Sing	9 fine	parataxis
20	?	?	PUNCT	_	3 said	punct

#	verb	tense, mood	who	what, whom	other	roles by
1	say speech: assert in a quote	past	bot			vn say-37.7-1-1 «NP V that S»: agent=Agent
2	is in a quote	present	cats and dogs		fine property	tree rules
3	is in a quote	present	bot		useless property	tree rules
4	be speech: assert	present		a fair reply		tree rules

One row per clause the reader finds. «not said here»: a role filled from the clause above. «roles by»: a VerbNet class and frame when one fits the verb, else the reader's rules on the tree.

The options

Option 1 · ← Mova's choice

It should have said cats, everyone knows that

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	It	it	PRON	Case=Nom Gender=Neut Number=Sing Person=3 PronType=Prs	4 said	nsubj
2	should	should	AUX	VerbForm=Fin	4 said	aux
3	have	have	AUX	VerbForm=Inf	4 said	aux
4	said	say	VERB	Tense=Past VerbForm=Part	root	root
5	cats	cat	NOUN	Number=Plur	4 said	obj
6	,	,	PUNCT	_	8 knows	punct
7	everyone	everyone	PRON	Number=Sing PronType=Tot	8 knows	nsubj
8	knows	know	VERB	Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin	4 said	conj
9	that	that	PRON	Number=Sing PronType=Dem	8 knows	obj

#	verb	tense, mood	who	what, whom	to whom	where	when	other	roles by
1	say speech: assert	should · perfect	It	cats					vn say-37.7-1-1 «NP V NP.topic»: agent=Agent cat=Topic
2	know	present	everyone	that					vn comprehend-87.2-1-1-1 «NP V NP PP.attribute»: agent=Experiencer that=Stimulus

Option 2 · ✓ the right answer

It's a question of taste, so there's no one right answer

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	It	it	PRON	Case=Nom Gender=Neut Number=Sing Person=3 PronType=Prs	4 question	nsubj
2	's	be	AUX	Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin	4 question	cop
3	a	a	DET	Definite=Ind PronType=Art	4 question	det
4	question	question	NOUN	Number=Sing	root	root
5	of	of	ADP	_	6 taste	case
6	taste	taste	NOUN	Number=Sing	4 question	nmod
7	,	,	PUNCT	_	10 's	punct
8	so	so	ADV	_	10 's	advmod
9	there	there	PRON	PronType=Dem	10 's	expl
10	's	be	VERB	Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin	4 question	parataxis
11	no	no	DET	PronType=Neg	14 answer	det
12	one	one	NOUN	Number=Sing	14 answer	compound
13	right	right	ADJ	Degree=Pos	14 answer	amod
14	answer	answer	NOUN	Number=Sing	10 's	nsubj

#	verb	tense, mood	who	what, whom	to whom	where	when	other	roles by
1	is a	present	It	a question				of taste	tree rules
2	be	present	no one right answer			there			vn exist-47.1 «NP V»: agent=Theme

Option 3

It should have said dogs, everyone knows dogs are better

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	It	it	PRON	Case=Nom Gender=Neut Number=Sing Person=3 PronType=Prs	4 said	nsubj
2	should	should	AUX	VerbForm=Fin	4 said	aux
3	have	have	AUX	VerbForm=Inf	4 said	aux
4	said	say	VERB	Tense=Past VerbForm=Part	root	root
5	dogs	dog	NOUN	Number=Plur	4 said	obj
6	,	,	PUNCT	_	8 knows	punct
7	everyone	everyone	PRON	Number=Sing PronType=Tot	8 knows	nsubj
8	knows	know	VERB	Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin	4 said	conj
9	dogs	dog	NOUN	Number=Plur	11 better	nsubj
10	are	be	AUX	Mood=Ind Number=Plur Person=3 Tense=Pres VerbForm=Fin	11 better	cop
11	better	good	ADJ	Degree=Cmp	8 knows	ccomp

#	verb	tense, mood	who	what, whom	other	roles by
1	say speech: assert	should · perfect	It	dogs		vn say-37.7-1-1 «NP V NP.topic»: agent=Agent dog=Topic
2	know	present	everyone			vn comprehend-87.2-1-1-1 «NP V»: agent=Experiencer
3	is	present	dogs		good property	tree rules

Option 4

It means the bot is broken

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	It	it	PRON	Case=Nom Gender=Neut Number=Sing Person=3 PronType=Prs	2 means	nsubj
2	means	mean	VERB	Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin	root	root
3	the	the	DET	Definite=Def PronType=Art	4 bot	det
4	bot	bot	NOUN	Number=Sing	6 broken	nsubj:pass
5	is	be	AUX	Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin	6 broken	aux:pass
6	broken	break	VERB	Tense=Past VerbForm=Part Voice=Pass	2 means	ccomp

#	verb	tense, mood	who	what, whom	to whom	where	when	other	roles by
1	mean	present	It						vn intend-61.2-1 «NP V S»: agent=Agent clause=Topic
2	break	present · passive		the bot					vn hurt-40.8.3-1-1 «NP V NP»: subject=Patient

Text: the tests of Mova's primary school, written for Mova (world/data/school/), as the course runner gave them to the student. Trees: Mova's own parse — the en model on the CPU, when the site is built — not checked by a teacher; they may be wrong. Frames: Mova's frames reader (prag), each text read on its own. Elamur Cosmic License.

This page in Markdown → https://md.mova.elamur.ai/free/en/corpora/questions/ai/6-why-the-bot-agrees-too-easily/t3/