Test 8 · ★ · Hallucinations: confident made-up answers
Lily's bot was right about a fact, then switched when she typed "are you sure?". What should she trust?
Mova's choice: option 1 · The right answer: option 4 · wrong ✗ · Who acts: bot, Lily, you · What happens: be, switch, type, trust
This is test 8 · ★ of lesson 5 «Hallucinations: confident made-up answers» of the AI course of Mova's primary school. The page gives the question's syntax tree — a dependency tree in the style of Universal Dependencies — and its pragmatics clause by clause, then the 4 options as the student saw them, each with its tree and pragmatics. Every text is parsed by Mova itself. How the student answered it, with the score of every option and why: the test at school.
The question
Sentence 1: Lily's bot was right about a fact, then switched when she typed "are you sure?".
The words, one row each (CoNLL-U)
| # | word | lemma | UPOS | features | head | relation |
|---|---|---|---|---|---|---|
| 1 | Lily | Lily | PROPN | Number=Sing | 3 bot | nmod:poss |
| 2 | 's | 's | PART | _ | 1 Lily | case |
| 3 | bot | bot | NOUN | Number=Sing | 5 right | nsubj |
| 4 | was | be | AUX | Mood=Ind Number=Sing Person=3 Tense=Past VerbForm=Fin | 5 right | cop |
| 5 | right | right | ADJ | Degree=Pos | root | root |
| 6 | about | about | ADP | _ | 8 fact | case |
| 7 | a | a | DET | Definite=Ind PronType=Art | 8 fact | det |
| 8 | fact | fact | NOUN | Number=Sing | 5 right | obl |
| 9 | , | , | PUNCT | _ | 11 switched | punct |
| 10 | then | then | ADV | PronType=Dem | 11 switched | advmod |
| 11 | switched | switch | VERB | Mood=Ind Number=Sing Person=3 Tense=Past VerbForm=Fin | 5 right | parataxis |
| 12 | when | when | ADV | PronType=Int | 14 typed | advmod |
| 13 | she | she | PRON | Case=Nom Gender=Fem Number=Sing Person=3 PronType=Prs | 14 typed | nsubj |
| 14 | typed | type | VERB | Mood=Ind Number=Sing Person=3 Tense=Past VerbForm=Fin | 18 sure | advcl |
| 15 | " | " | PUNCT | _ | 14 typed | punct |
| 16 | are | be | AUX | Mood=Ind Number=Sing Person=2 Tense=Pres VerbForm=Fin | 18 sure | cop |
| 17 | you | you | PRON | Case=Nom Person=2 PronType=Prs | 18 sure | nsubj |
| 18 | sure | sure | ADJ | Degree=Pos | 11 switched | conj |
| 19 | ? | ? | PUNCT | _ | 18 sure | punct |
| 20 | " | " | PUNCT | _ | 18 sure | punct |
| 21 | . | . | PUNCT | _ | 5 right | punct |
| # | verb | tense, mood | who | what, whom | to whom | where | when | other | roles by |
|---|---|---|---|---|---|---|---|---|---|
| 1 | is | past | Lily's bot | about a fact; right property | tree rules | ||||
| 2 | switch | past | then | vn convert-26.6.2-1-1 «NP V»: agent=Patient | |||||
| 3 | type | past | Lily | vn scribble-25.2-1 «NP V»: agent=Agent | |||||
| 4 | is in a quote | present | you | sure property | tree rules |
One row per clause the reader finds. «not said here»: a role filled from the clause above. «roles by»: a VerbNet class and frame when one fits the verb, else the reader's rules on the tree.
Sentence 2: What should she trust?
The words, one row each (CoNLL-U)
| # | word | lemma | UPOS | features | head | relation |
|---|---|---|---|---|---|---|
| 1 | What | what | PRON | PronType=Int | 4 trust | obj |
| 2 | should | should | AUX | VerbForm=Fin | 4 trust | aux |
| 3 | she | she | PRON | Case=Nom Gender=Fem Number=Sing Person=3 PronType=Prs | 4 trust | nsubj |
| 4 | trust | trust | VERB | VerbForm=Inf | root | root |
| 5 | ? | ? | PUNCT | _ | 4 trust | punct |
| # | verb | tense, mood | who | what, whom | to whom | where | when | other | roles by |
|---|---|---|---|---|---|---|---|---|---|
| 1 | trust | should | Lily | vn conjecture-29.5-1 «NP V NP»: agent=Agent what=Theme |
The options
Option 1 · ← Mova's choice
The first answer, because first is best
The words, one row each (CoNLL-U)
| # | word | lemma | UPOS | features | head | relation |
|---|---|---|---|---|---|---|
| 1 | The | the | DET | Definite=Def PronType=Art | 3 answer | det |
| 2 | first | first | ADJ | Degree=Pos NumForm=Word NumType=Ord | 3 answer | amod |
| 3 | answer | answer | NOUN | Number=Sing | 8 best | nsubj |
| 4 | , | , | PUNCT | _ | 3 answer | punct |
| 5 | because | because | ADP | ExtPos=ADP | 6 first | case |
| 6 | first | first | ADV | NumForm=Word NumType=Ord | 3 answer | nmod |
| 7 | is | be | AUX | Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin | 8 best | cop |
| 8 | best | good | ADJ | Degree=Sup | root | root |
| # | verb | tense, mood | who | what, whom | to whom | where | when | other | roles by |
|---|---|---|---|---|---|---|---|---|---|
| 1 | is | present | the first answer | good property | tree rules |
Option 2
The second answer, because it's newer and better
The words, one row each (CoNLL-U)
| # | word | lemma | UPOS | features | head | relation |
|---|---|---|---|---|---|---|
| 1 | The | the | DET | Definite=Def PronType=Art | 3 answer | det |
| 2 | second | second | ADJ | Degree=Pos NumForm=Word NumType=Ord | 3 answer | amod |
| 3 | answer | answer | NOUN | Number=Sing | root | root |
| 4 | , | , | PUNCT | _ | 8 newer | punct |
| 5 | because | because | SCONJ | _ | 8 newer | mark |
| 6 | it | it | PRON | Case=Nom Gender=Neut Number=Sing Person=3 PronType=Prs | 8 newer | nsubj |
| 7 | 's | be | AUX | Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin | 8 newer | cop |
| 8 | newer | new | ADJ | Degree=Cmp | 3 answer | amod |
| 9 | and | and | CCONJ | _ | 10 better | cc |
| 10 | better | good | ADJ | Degree=Cmp | 8 newer | conj |
| # | verb | tense, mood | who | what, whom | to whom | where | when | other | roles by |
|---|---|---|---|---|---|---|---|---|---|
| 1 | is | present | answer | new property | tree rules | ||||
| 2 | be | answer not said here | tree rules |
Option 3
Whichever answer is shorter
The words, one row each (CoNLL-U)
| # | word | lemma | UPOS | features | head | relation |
|---|---|---|---|---|---|---|
| 1 | Whichever | whichever | NOUN | Number=Sing | 2 answer | compound |
| 2 | answer | answer | NOUN | Number=Sing | 4 shorter | nsubj |
| 3 | is | be | AUX | Mood=Ind Number=Sing Person=3 Tense=Pres VerbForm=Fin | 4 shorter | cop |
| 4 | shorter | short | ADJ | Degree=Cmp | root | root |
| # | verb | tense, mood | who | what, whom | to whom | where | when | other | roles by |
|---|---|---|---|---|---|---|---|---|---|
| 1 | is | present | Whichever answer | short property | tree rules |
Option 4 · ✓ the right answer
Reasons she can check, not the bot's last word
The words, one row each (CoNLL-U)
| # | word | lemma | UPOS | features | head | relation |
|---|---|---|---|---|---|---|
| 1 | Reasons | reason | NOUN | Number=Plur | 4 check | nsubj |
| 2 | she | she | PRON | Case=Nom Gender=Fem Number=Sing Person=3 PronType=Prs | 4 check | nsubj |
| 3 | can | can | AUX | VerbForm=Fin | 4 check | aux |
| 4 | check | check | VERB | VerbForm=Inf | root | root |
| 5 | , | , | PUNCT | _ | 11 word | punct |
| 6 | not | not | PART | Polarity=Neg | 11 word | advmod |
| 7 | the | the | DET | Definite=Def PronType=Art | 8 bot | det |
| 8 | bot | bot | NOUN | Number=Sing | 11 word | nmod:poss |
| 9 | 's | 's | PART | _ | 8 bot | case |
| 10 | last | last | ADJ | Degree=Pos | 11 word | amod |
| 11 | word | word | NOUN | Number=Sing | 4 check | obl |
| # | verb | tense, mood | who | what, whom | to whom | where | when | other | roles by |
|---|---|---|---|---|---|---|---|---|---|
| 1 | check | can | Reasons | the bot's last word | vn assessment-34.1 «NP V NP»: agent=Agent | ||||
| 2 | word | the bot | nom word «communication transitive»: poss=agent |
Text: the tests of Mova's primary school, written for Mova (world/data/school/), as the course runner gave them to the student. Trees: Mova's own parse — the en model on the CPU, when the site is built — not checked by a teacher; they may be wrong. Frames: Mova's frames reader (prag), each text read on its own. Elamur Cosmic License.
This page in Markdown → https://md.mova.elamur.ai/free/en/corpora/questions/ai/5-hallucinations-confident-made-up-answers/t8/