token
part of speech
noun (a token, tokens); adjective (token)
meaning for a pupil
1) A token is a round piece of metal or plastic used instead of a coin, for example in games or machines. tree →
2) A token is a card or voucher that you can exchange for goods: a book token (UK). tree →
3) A token of something is a sign or symbol of a feeling: a token of thanks. tree →
4) As an adjective, token means small and done only for show: a token effort. tree →
5) (computers) A token is a piece of text or a code that a program reads or that proves who you are. tree →
sense in the lessons
In the mystic school: In note 14 the first step of reading cuts a sentence into tokens, as Set cut Osiris; in note 15 "a token knows itself", but not the whole. In the primary lessons: In board games you move your token around the board. In money lessons tokens are like vouchers. In stories a ring or flower can be a token of love. Phrases: a token of thanks, by the same token (for the same reason), a token gesture.
examples
- If that interpretation is the densest region of the learned distribution, ordinary next-token prediction already gives it an advantage. (note 03) tree →
- Characters into tokens, tokens into words, every word separated from its neighbours and laid out in a row. (note 14) tree →
- A token knows itself. (note 15) tree →
- In the note, no explicit rule is needed: if an interpretation is the densest region, ordinary next-token prediction already gives it an advantage, and preference training can reinforce it. (lesson 005) tree →
- The note says every reading is a resurrection. Mova first does what Set did: it cuts characters into tokens, tokens into words, each laid out in a row that says nothing about who did what to whom. Then the parser goes looking like Isis, finding subjects, objects and what belongs to what, until the tree grows back. (lesson 026) tree →
- Put a token in the slot to play the game. tree →
- I got a book token for my birthday. tree →
- She gave him a flower as a token of thanks. tree →
- Move your token three squares forward. tree →
- He made only a token effort to tidy his room. tree →
- By the same token, you must help too. tree →
- The program splits the sentence into tokens. tree →
- “This is a token of my thanks!” tree →
note
Countable noun. «Token» as an adjective (token effort) means not real or serious. «By the same token» = for the same reason. In language models, a token is a small piece of text.
In the corpora
- I have set My bow in the cloud, and it shall be for a token of a covenant between Me and the earth.
- But if this thing be true, that the tokens of virginity were not found in the damsel;
- That frustrateth the tokens of the imposters, and maketh diviners mad; that turneth wise men backward, and maketh their knowledge foolish;
- Have ye not asked them that go by the way; and will ye misdeem their tokens,
Trees and pragmatics
Every meaning and example of «token» parsed by Mova, each sentence with its dependency tree and frames.