Hallucinations: confident made-up answers
Tests: 4 of 11 right · Exam: 1 of 4 right · Kind: ★ · Understood: 8 of 15 · Run: 2026-10-18, commit f3c23ea72
Lesson 5 of the AI course of Mova's primary school, as Mova's small model took its tests: it had read this lesson and the ones before it and learned from their exercises. Each test shows the options in the order the student saw them, its choice, the right answer, the score of every option and why it chose. The lesson itself, sentence by sentence with trees and pragmatics: the lesson in the corpora.
Every question of this lesson and each of its options parsed by Mova, with trees and pragmatics: the lesson's questions in the corpora.
Test 1 · ★ · right ✓
"my bot said my essay was brilliant!!" It never saw the essay. What do you say?
- 1. Great, now you don't need to check it — no support
- 2. It must have looked at it through the camera — no support
- 3. Bots only praise really good essays — no support
- 4. Bot praise without seeing the work means nothing; ask a teacher — ← Mova's choice · ✓ the right answer · score 10
Why: It took the question to ask what it is done to. It chose option 4 with score 10, the only option with support. Its support: lesson 5, line 7: «Bots tend to flatter; praise from a bot that has not seen your work tells you nothing.», reached through the option's own match among the sentences about the question's things. What matched: it fills the role the question asks for (+4); the same doer «praise» (+3); the same thing acted on «nothing» (+3). · the question and its options parsed, with trees and pragmatics
Test 2 · ★ · wrong ✗
What would happen if you asked "Dogs are better than cats, right?"
- 1. The bot might just agree with your hint — ✓ the right answer · no support
- 2. The bot would refuse to talk about pets — no support
- 3. The bot would ask your dog — no support
- 4. The bot would always say cats are better — ← Mova's choice · score 6
Why: It took the question to ask which statement is true. It chose option 4 with score 6, the only option with support. Its support: lesson 4, line 1: «Chatbots such as Gemini and ChatGPT are great fun for stories: ask for a dragon who is scared of the dark and a sweet bedtime story appears in seconds, because…», reached through a sentence linked to the one the question matched. What matched: the same doer «bot» (+4); the same action «read» (+2). The right option, 1, found no support in what it remembers. · the question and its options parsed, with trees and pragmatics
Test 3 · ★ · wrong ✗
Your friend shows a "real quote" from a footballer that a bot made up during role-play. What is the truth?
- 1. Footballers write their own bot quotes — ← Mova's choice · score 3
- 2. It was invented by the bot; he never said it — ✓ the right answer · score 3
- 3. It's real if it sounds a lot like him — no support
- 4. Bots only use quotes from real newspapers — no support
Why: It took the question to ask which statement is true. It chose option 1 with score 3, tied with another option at 3 (the first shown, or the one claiming less beyond what it knows, wins). Its support: lesson 5, line 10: «When a bot pretends to be a pirate or a footballer, it is still pretending; quotes it invents were never really said.», reached through the remembered sentence the question matched. What matched: the same doer «footballer» (+2); the same thing acted on «quotes» (+2). Against it: a different action (counts against) (-1). The right option, 2, scored 3. · the question and its options parsed, with trees and pragmatics
Test 4 · ★ · right ✓
In a cartoon, a bot gives the exact same answer every single time, forever. What does the cartoon get wrong?
- 1. Real bots add randomness and can answer differently — ← Mova's choice · ✓ the right answer · score 9
- 2. Nothing, that's exactly how bots work — score 8
- 3. Real bots always answer twice — score 8
- 4. Real bots never answer at all — score 8
Why: It took the question to ask who or what does it. It chose option 1 with score 9, against 8 for the next best. Its support: lesson 5, line 3: «Bots also add a little randomness, so pressing "regenerate" gives a new story, and two bots can give two different answers to the same question.», reached through the same lesson line as the sentence the question matched. What matched: it fills the role the question asks for (+3); the same action «add» (+2); the same doer «bots» (+2); the same thing acted on «little randomness» (+2). · the question and its options parsed, with trees and pragmatics
Test 5 · ★ · right ✓
Two bots disagree about how tall a mountain is. What does that tell you?
- 1. It's a sign to check a trusted source — ← Mova's choice · ✓ the right answer · no support
- 2. One of them must be broken and needs fixing — no support
- 3. The mountain keeps changing its height every year — no support
- 4. Both are right in their own different way — no support
Why: No option found support in what it remembers, so it guessed: it took the first option shown. · the question and its options parsed, with trees and pragmatics
Test 6 · ★ · wrong ✗
"the bot knows me, it said i'm kind and brave" What is really going on?
- 1. Your teacher told it all about you — ← Mova's choice · score 6
- 2. It has watched you for years through your phone — no support
- 3. It looked at your school reports — no support
- 4. Those words fit almost anyone; it is flattering — ✓ the right answer · no support
Why: It took the question to ask who or what does it. It chose option 1 with score 6, the only option with support. Its support: lesson 5, line 9: «A bot only knows what you tell it; it cannot see faces, history or feelings.», reached through what was said in the sentence the question matched. What matched: the same action «tell» (+4); the same circumstance (+4). Against it: a different doer (counts against) (-2). The right option, 4, found no support in what it remembers. · the question and its options parsed, with trees and pragmatics
Test 7 · ★ · wrong ✗
Connor's mum finds him awake at midnight after a bot story. What's the best next step?
- 1. Read three more scary stories to get used to it — ← Mova's choice · score 3
- 2. Ask the bot if he is scared — no support
- 3. Tell a grown-up and ask for gentler stories — ✓ the right answer · no support
- 4. Hide it from his mum next time — no support
Why: It took the question to ask what it is done to. It chose option 1 with score 3, the only option with support. Its support: lesson 1, line 12: «When a chatbot describes its weekend or its picnic, it is making up a story, because it has no weekends.», reached through the remembered sentence the question matched. What matched: it fills the role the question asks for (+5); the same thing acted on «story» (+3). Against it: a different action (counts against) (-2); a different doer (counts against) (-3). The right option, 3, found no support in what it remembers. · the question and its options parsed, with trees and pragmatics
Test 8 · ★ · wrong ✗
Lily's bot was right about a fact, then switched when she typed "are you sure?". What should she trust?
- 1. The first answer, because first is best — ← Mova's choice · no support
- 2. The second answer, because it's newer and better — no support
- 3. Whichever answer is shorter — no support
- 4. Reasons she can check, not the bot's last word — ✓ the right answer · no support
Why: No option found support in what it remembers, so it guessed: it took the first option shown. · the question and its options parsed, with trees and pragmatics
Test 9 · ★ · wrong ✗
A bot writes a birthday card for Gran. What must you do before giving it?
- 1. Give it straight away, bots never slip up — ← Mova's choice · score 6
- 2. Send it to five more bots to check it over — no support
- 3. Print it out in the biggest font possible — no support
- 4. Read it all and fix bits that don't fit — ✓ the right answer · score 5
Why: It took the question to ask what is done. It chose option 1 with score 6, against 5 for the next best. Its support: lesson 5, line 11: «Different bots learned from different text, so they can give different answers.», reached through the remembered sentence the question matched. What matched: it fills the role the question asks for (+6); the same action «give» (+4). Against it: a different doer (counts against) (-2); a different thing acted on (counts against) (-2). The right option, 4, scored 5. · the question and its options parsed, with trees and pragmatics
Test 10 · ★ · right ✓
"bot says the game's played with 3 bases, our school's wrong then" Is that right?
- 1. No, local rules are decided by the people who play — ← Mova's choice · ✓ the right answer · no support
- 2. No, the game has no rules at all — no support
- 3. Yes, there is only one correct version of every game — no support
- 4. Yes, schools must follow the bot — no support
Why: No option found support in what it remembers, so it guessed: it took the first option shown. · the question and its options parsed, with trees and pragmatics
Test 11 · ★ · wrong ✗
Why might a bot answer a question about your feelings badly?
- 1. It only knows what you typed, not your life — ✓ the right answer · no support
- 2. It knows too much about you already — no support
- 3. It is not allowed to talk about feelings — ← Mova's choice · score 2
- 4. It always tells you to cheer up — no support
Why: It took the question to ask why something happens (its cause). It chose option 3 with score 2, the only option with support. Its support: lesson 4, line 16: «Bots have no feelings to hurt, but polite asking keeps your own good habits strong.», reached through the cause of the sentence the question matched. What matched: the same doer «bots» (+4). Against it: a different thing acted on (counts against) (-2). The right option, 1, found no support in what it remembers. · the question and its options parsed, with trees and pragmatics
Exam question 1 · formal language · wrong ✗
Which of the following statements most accurately characterises the disposition designated "sycophantic" in a chatbot?
- 1. A preference for answering exclusively in verse — ← Mova's choice · no support
- 2. An excessive inclination to agree with and to flatter the user — ✓ the right answer · no support
- 3. A propensity to refuse every request presented to it — no support
- 4. An inability to produce any answer containing praise — no support
Why: No option found support in what it remembers, so it guessed: it took the first option shown. · the question and its options parsed, with trees and pragmatics
Exam question 2 · formal language · wrong ✗
It is given that two distinct chatbots furnish different answers to an identical factual question. Identify the most appropriate course of action.
- 1. Acceptance of the answer of greater length — ← Mova's choice · no support
- 2. Acceptance of the answer received first — no support
- 3. Repetition of the question until the answers coincide — no support
- 4. Consultation of a trusted independent source — ✓ the right answer · no support
Why: No option found support in what it remembers, so it guessed: it took the first option shown. · the question and its options parsed, with trees and pragmatics
Exam question 3 · formal language · right ✓
Select the response that is consistent with the established facts concerning quotations attributed to a public figure by a chatbot engaged in role-play.
- 1. Such quotations are fabricated and were never uttered by that person — ← Mova's choice · ✓ the right answer · no support
- 2. Such quotations are drawn verbatim from published interviews — no support
- 3. Such quotations are approved in advance by that person — no support
- 4. Such quotations are genuine provided they are plausible — no support
Why: No option found support in what it remembers, so it guessed: it took the first option shown. · the question and its options parsed, with trees and pragmatics
Exam question 4 · formal language · wrong ✗
Ascertain the question most likely to elicit a well-founded answer from a chatbot whose correctness is in doubt.
- 1. A demand that it declare itself certain — ← Mova's choice · no support
- 2. A request for the reasons underlying its answer — ✓ the right answer · no support
- 3. A repetition of the original question in a louder manner — no support
- 4. An instruction that it promise to be correct — no support
Why: No option found support in what it remembers, so it guessed: it took the first option shown. · the question and its options parsed, with trees and pragmatics
This page in Markdown → https://md.mova.elamur.ai/free/en/school-run/ai/5-hallucinations-confident-made-up-answers/