Exam question 1 · Hallucinations: confident made-up answers

Which of the following statements most accurately characterises the disposition designated "sycophantic" in a chatbot?

Mova's choice: option 1 · The right answer: option 2 · wrong ✗ · What happens: follow, characteris, designate · To what, where: disposition, chatbot

This is exam question 1 of lesson 5 «Hallucinations: confident made-up answers» of the AI course of Mova's primary school, asked in stiff, formal language. The page gives the question's syntax tree — a dependency tree in the style of Universal Dependencies — and its pragmatics clause by clause, then the 4 options as the student saw them, each with its tree and pragmatics. Every text is parsed by Mova itself. How the student answered it, with the score of every option and why: the test at school.

The question

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	Which	which	PRON	PronType=Rel	8 characterises	nsubj
2	of	of	ADP	_	5 statements	case
3	the	the	DET	Definite=Def PronType=Art	5 statements	det
4	following	follow	VERB	VerbForm=Ger	5 statements	amod
5	statements	statement	NOUN	Number=Plur	8 characterises	obl
6	most	most	ADV	Degree=Sup	7 accurately	advmod
7	accurately	accurately	ADV	_	8 characterises	advmod
8	characterises	characteris	VERB	Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin	root	root
9	the	the	DET	Definite=Def PronType=Art	10 disposition	det
10	disposition	disposition	NOUN	Number=Sing	8 characterises	obj
11	designated	designate	VERB	Tense=Past VerbForm=Part Voice=Pass	10 disposition	acl
12	"	"	PUNCT	_	13 sycophantic	punct
13	sycophantic	sycophantic	ADJ	Degree=Pos	11 designated	conj
14	"	"	PUNCT	_	13 sycophantic	punct
15	in	in	ADP	_	17 chatbot	case
16	a	a	DET	Definite=Ind PronType=Art	17 chatbot	det
17	chatbot	chatbot	NOUN	Number=Sing	11 designated	obl
18	?	?	PUNCT	_	8 characterises	punct

#	verb	tense, mood	what, whom	where	other	roles by
1	follow	gerund				vn occur-48.3-1 «NP V»: agent=Theme
2	characteris	present	the disposition		of the following statements; accurately property	tree rules
3	designate	participle	the disposition not said here	a chatbot		vn appoint-29.1 «NP V NP»: agent=Agent

One row per clause the reader finds. «not said here»: a role filled from the clause above. «roles by»: a VerbNet class and frame when one fits the verb, else the reader's rules on the tree.

The options

Option 1 · ← Mova's choice

A preference for answering exclusively in verse

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	A	a	DET	Definite=Ind PronType=Art	2 preference	det
2	preference	preference	NOUN	Number=Sing	root	root
3	for	for	SCONJ	_	4 answering	mark
4	answering	answer	VERB	VerbForm=Ger	2 preference	acl
5	exclusively	exclusively	ADV	_	4 answering	advmod
6	in	in	ADP	_	7 verse	case
7	verse	verse	NOUN	Number=Sing	4 answering	obl

#	verb	tense, mood	who	what, whom	to whom	where	when	other	roles by
1	answer speech: assert	gerund	a preference not said here			verse		exclusively property	vn reciprocate-112-1 «NP V»: agent=Agent

Option 2 · ✓ the right answer

An excessive inclination to agree with and to flatter the user

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	An	a	DET	Definite=Ind PronType=Art	3 inclination	det
2	excessive	excessive	ADJ	Degree=Pos	3 inclination	amod
3	inclination	inclination	NOUN	Number=Sing	root	root
4	to	to	PART	_	5 agree	mark
5	agree	agree	VERB	VerbForm=Inf	3 inclination	acl
6	with	with	ADP	_	5 agree	obl
7	and	and	CCONJ	_	9 flatter	cc
8	to	to	PART	_	9 flatter	mark
9	flatter	flatter	VERB	VerbForm=Inf	5 agree	conj
10	the	the	DET	Definite=Def PronType=Art	11 user	det
11	user	user	NOUN	Number=Sing	9 flatter	obj

#	verb	tense, mood	who	what, whom	to whom	where	when	other	roles by
1	agree speech: agree	to-infinitive							vn correspond-36.1-1 «NP V»: agent=Agent
2	flatter	to-infinitive		the user					vn amuse-31.1 «NP V NP»: agent=Stimulus user=Experiencer

Option 3

A propensity to refuse every request presented to it

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	A	a	DET	Definite=Ind PronType=Art	2 propensity	det
2	propensity	propensity	NOUN	Number=Sing	root	root
3	to	to	PART	_	4 refuse	mark
4	refuse	refuse	VERB	VerbForm=Inf	2 propensity	acl
5	every	every	DET	PronType=Tot	6 request	det
6	request	request	NOUN	Number=Sing	4 refuse	obj
7	presented	present	VERB	Tense=Past VerbForm=Part Voice=Pass	6 request	acl
8	to	to	ADP	_	9 it	case
9	it	it	PRON	Case=Acc Gender=Neut Number=Sing Person=3 PronType=Prs	7 presented	obl

#	verb	tense, mood	who	what, whom	to whom	where	when	other	roles by
1	refuse	to-infinitive		every request					vn reject-77.2 «NP V NP»: agent=Agent request=Theme
2	present	participle		every request not said here	request				vn fulfilling-13.4.1 «NP V NP PP.recipient»: agent=Agent to it=Recipient

Option 4

An inability to produce any answer containing praise

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	An	a	DET	Definite=Ind PronType=Art	2 inability	det
2	inability	inability	NOUN	Number=Sing	root	root
3	to	to	PART	_	4 produce	mark
4	produce	produce	VERB	VerbForm=Inf	2 inability	acl
5	any	any	DET	PronType=Ind	6 answer	det
6	answer	answer	NOUN	Number=Sing	4 produce	obj
7	containing	contain	VERB	VerbForm=Ger	6 answer	acl
8	praise	praise	NOUN	Number=Sing	7 containing	obj

#	verb	tense, mood	who	what, whom	to whom	where	when	other	roles by
1	produce	to-infinitive		any answer					vn create-26.4 «NP V NP»: agent=Agent answer=Result
2	contain	gerund	any answer not said here	praise					vn comprise-107.2-2 «NP V NP»: agent=Pivot praise=Theme

Text: the tests of Mova's primary school, written for Mova (world/data/school/), as the course runner gave them to the student. Trees: Mova's own parse — the en model on the CPU, when the site is built — not checked by a teacher; they may be wrong. Frames: Mova's frames reader (prag), each text read on its own. Elamur Cosmic License.

This page in Markdown → https://md.mova.elamur.ai/free/en/corpora/questions/ai/5-hallucinations-confident-made-up-answers/e1/