Exam question 4 · Hallucinations: confident made-up answers

Ascertain the question most likely to elicit a well-founded answer from a chatbot whose correctness is in doubt.

Mova's choice: option 1 · The right answer: option 2 · wrong ✗ · Who acts: you, correctness · What happens: ascertain, elicit, found, be · To what, where: question, founded answer, chatbot, doubt

This is exam question 4 of lesson 5 «Hallucinations: confident made-up answers» of the AI course of Mova's primary school, asked in stiff, formal language. The page gives the question's syntax tree — a dependency tree in the style of Universal Dependencies — and its pragmatics clause by clause, then the 4 options as the student saw them, each with its tree and pragmatics. Every text is parsed by Mova itself. How the student answered it, with the score of every option and why: the test at school.

The question

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	Ascertain	ascertain	VERB	Mood=Imp VerbForm=Fin	root	root
2	the	the	DET	Definite=Def PronType=Art	3 question	det
3	question	question	NOUN	Number=Sing	1 Ascertain	obj
4	most	most	ADV	Degree=Sup	5 likely	advmod
5	likely	likely	ADV	_	1 Ascertain	advmod
6	to	to	PART	_	7 elicit	mark
7	elicit	elicit	VERB	VerbForm=Inf	1 Ascertain	advcl
8	a	a	DET	Definite=Ind PronType=Art	12 answer	det
9	well	well	ADV	Degree=Pos	11 founded	advmod
10	-	-	PUNCT	_	9 well	punct
11	founded	found	VERB	Tense=Past VerbForm=Part Voice=Pass	12 answer	amod
12	answer	answer	NOUN	Number=Sing	7 elicit	obj
13	from	from	ADP	_	15 chatbot	case
14	a	a	DET	Definite=Ind PronType=Art	15 chatbot	det
15	chatbot	chatbot	NOUN	Number=Sing	7 elicit	obl
16	whose	whose	PRON	Poss=Yes PronType=Rel	17 correctness	nmod:poss
17	correctness	correctness	NOUN	Number=Sing	20 doubt	nsubj
18	is	be	AUX	Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin	20 doubt	cop
19	in	in	ADP	_	20 doubt	case
20	doubt	doubt	NOUN	Number=Sing	15 chatbot	acl:relcl
21	.	.	PUNCT	_	1 Ascertain	punct

#	verb	tense, mood	who	what, whom	where	other	roles by
1	ascertain	imperative	you not said here	the question		likely property	vn discover-84 «NP V NP»: agent=Agent question=Topic
2	elicit	to-infinitive	you not said here	a founded answer	a chatbot		vn engender-27.1-1 «NP V NP»: agent=Precondition answer=Patient
3	found	participle				well property	vn establish-55.5-1 «NP V NP»: agent=Agent
4	is	present	doubt's correctness		doubt		tree rules

One row per clause the reader finds. «not said here»: a role filled from the clause above. «roles by»: a VerbNet class and frame when one fits the verb, else the reader's rules on the tree.

The options

Option 1 · ← Mova's choice

A demand that it declare itself certain

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	A	a	DET	Definite=Ind PronType=Art	2 demand	det
2	demand	demand	NOUN	Number=Sing	root	root
3	that	that	SCONJ	_	5 declare	mark
4	it	it	PRON	Case=Nom Gender=Neut Number=Sing Person=3 PronType=Prs	5 declare	nsubj
5	declare	declare	VERB	VerbForm=Inf	2 demand	acl
6	itself	itself	PRON	Case=Acc Gender=Neut Number=Sing Person=3 PronType=Prs Reflex=Yes	5 declare	obj
7	certain	certain	ADJ	Degree=Pos	5 declare	xcomp

#	verb	tense, mood	who	what, whom	to whom	where	when	other	roles by
1	declare speech: other	infinitive	demand	demand				certain property	vn declare-29.4-1-1-1 «NP V NP»: agent=Agent itself=Attribute

Option 2 · ✓ the right answer

A request for the reasons underlying its answer

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	A	a	DET	Definite=Ind PronType=Art	2 request	det
2	request	request	NOUN	Number=Sing	root	root
3	for	for	ADP	_	5 reasons	case
4	the	the	DET	Definite=Def PronType=Art	5 reasons	det
5	reasons	reason	NOUN	Number=Plur	2 request	nmod
6	underlying	underlie	VERB	VerbForm=Ger	5 reasons	acl
7	its	its	PRON	Case=Gen Gender=Neut Number=Sing Person=3 Poss=Yes PronType=Prs	8 answer	nmod:poss
8	answer	answer	NOUN	Number=Sing	6 underlying	obj

#	verb	tense, mood	who	what, whom	to whom	where	when	other	roles by
1	underlie	gerund	the reasons not said here	request's answer					vn contiguous_location-47.8 «NP V NP»: agent=Theme answer=Co-Theme
2	answer		request						nom answer «communication transitive»: poss=agent

Option 3

A repetition of the original question in a louder manner

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	A	a	DET	Definite=Ind PronType=Art	2 repetition	det
2	repetition	repetition	NOUN	Number=Sing	root	root
3	of	of	ADP	_	6 question	case
4	the	the	DET	Definite=Def PronType=Art	6 question	det
5	original	original	ADJ	Degree=Pos	6 question	amod
6	question	question	NOUN	Number=Sing	2 repetition	nmod
7	in	in	ADP	_	10 manner	case
8	a	a	DET	Definite=Ind PronType=Art	10 manner	det
9	louder	loud	ADJ	Degree=Cmp	10 manner	amod
10	manner	manner	NOUN	Number=Sing	6 question	nmod

#	verb	tense, mood	who	what, whom	to whom	where	when	other	roles by
1	repeat		the original question in a louder manner						nom repetition «event intransitive»: of=agent

Option 4

An instruction that it promise to be correct

The words, one row each (CoNLL-U)

#	word	lemma	UPOS	features	head	relation
1	An	a	DET	Definite=Ind PronType=Art	2 instruction	det
2	instruction	instruction	NOUN	Number=Sing	root	root
3	that	that	SCONJ	_	5 promise	mark
4	it	it	PRON	Case=Nom Gender=Neut Number=Sing Person=3 PronType=Prs	5 promise	nsubj
5	promise	promise	VERB	Mood=Ind Number=Plur Person=3 Tense=Pres VerbForm=Fin	2 instruction	acl
6	to	to	PART	_	8 correct	mark
7	be	be	AUX	VerbForm=Inf	8 correct	cop
8	correct	correct	ADJ	Degree=Pos	5 promise	xcomp

#	verb	tense, mood	who	what, whom	to whom	where	when	other	roles by
1	promise speech: promise	present	instruction	a instruction not said here					vn future_having-13.3 «NP V NP.theme (PP)»: agent=Agent instruction=Theme
2	is not said to happen	to-infinitive	instruction not said here					correct property	tree rules

Text: the tests of Mova's primary school, written for Mova (world/data/school/), as the course runner gave them to the student. Trees: Mova's own parse — the en model on the CPU, when the site is built — not checked by a teacher; they may be wrong. Frames: Mova's frames reader (prag), each text read on its own. Elamur Cosmic License.

This page in Markdown → https://md.mova.elamur.ai/free/en/corpora/questions/ai/5-hallucinations-confident-made-up-answers/e4/