roundtrip — where each layer of Mova loses information

roundtrip sends a text down Mova's layers and back up: words → tokens → UD tree → frames → world, then world → frames → tree → tokens → words. It does not try to make the output pretty. It measures, at every layer boundary, what does not come back, puts every loss in a class, and counts what would have to be stored next to the layer's representation for the round trip to be lossless. A class that needs a large residual at a layer is a part of language that layer does not describe; the report ranks those classes with the change to Mova that would remove them.

Two modes:

• exact (--exact): every layer keeps its residual, and whole volumes must come back byte for byte, commas, line wraps and the source's own spellings included. This is the gate: a decoder or a residual that is wrong shows red. The residual's size, in items and bits per sentence, is the measurement.
• normal (--normal): no residual. The text is decoded from each start (tokens, tree, frames, world, and a treebank's gold tree with its frames and world), and its divergence from the original is measured by class, with exact match and chrF. Divergence is allowed here; the deeper the start, the less verbatim the text.

roundtrip mixed is the meta layer test: many languages and styles in one stream, each piece read by its own parser, one world, and back byte for byte (its own section below).

roundtrip blocks is the training gate: every lesson, chapter or run of chapters of every corpus of the training mix, as one block with one discourse and one world, back byte for byte (its section below). With --site-md it takes the site's Markdown pages as blocks too, their trees and frames tables written from the layers, and roundtrip site-learn learns what the train pages teach (the last section).

How it works

The layers, and what goes up and down

The realizer (frames → tree) says what the frames keep, and nothing it guesses. Every frame becomes a clause with the verb group of its tam (tense, aspect, voice, modality: the modal or «will» or «do», «have», «be» of the progressive, «be» of the passive, «to» of an infinitive; a passive says the patient as subject and the agent with its «by», an imperative no «you»); a frame without tam (a nominal event, PRAG_OFF=tam) gets past tense for events and speech, present for general statements, «do not» for a negation, «can» for Fact(can). The agent is the subject unless the frame marks it unsaid (shared with the clause above, an imperative's «you», the noun of a relative clause), and a conjunct does not repeat the noun subject of the clause before; the patient is the object, the recipient and the place take the mention's own preposition (a recipient without one is an indirect object), the obliques theirs; then the role-less noun phrases, the clause's adverbs and discourse words, its markers and conjunction. A copula with an adjective, a noun or a place makes a copular clause. An entity is its pronoun if the frame kept one, the relative word that stands for it, or its head with the mention's determiner or possessor, its adjectives and other one-word modifiers, its prepositional modifiers, appositions and coordinated phrases with their conjunctions. A clause hangs where its link says, with its relation, under the noun of a relative clause; a frame without a link is a conjunct of the first. Punctuation comes from the clause structure (en::lin::punctuate: the full stop; a comma before a coordinated clause with its own subject and in lists of three or more, on a parataxis, a vocative, a discourse word, an apposition). Each step has a switch that gives the old realizer back: PRAG_OFF=rt-shared-subject, rt-unsaid, rt-function (the old «a» then «the», «to», «in», «and», «because», «if»), rt-links (all conjuncts of the first, what was said a ccomp, a cause an advcl), rt-punct (the full stop only), rt-prt-once, rt-when. It reads only the frame's content (frames::F): never Ent::text, the phrase as written.

The referring-expression generator (world → frames) says an entity as its pronoun (its class he, she, it or they from the registry, nominative as the subject and capitalized as the first subject of a sentence, accusative elsewhere) when it was mentioned in this sentence or the one before, it fills no other role of the frame, it has no modifier to say, and prag::frames::Ctx — the reader's own resolver, salience by role, halved at every sentence — would take the pronoun back to it; an unresolved «it» stays a pronoun. ROUNDTRIP_OFF=refgen switches it off (every entity by name).

The world is what prag's discourse and roles::world_docs keep of a text, with every role and link: there is no other world built from text without the large model today. The worlds written by the large model in world's change language (world — worlds, schools and students) are not used here.

Classes

Every residual item and every divergent word has one class: meta (ids, lesson keys: not language), layout (line wraps, blank lines), spacing, punct, case, form (inflection: another form of the same lemma, e.g. were for was), lemma→form (a known word that is not what lemma and tag give: a tag or lemma error), src-spelling (no known word: archaic, dialect, misspelt), typography (’ “ ” for ' "), order, function (determiners, prepositions, conjunctions, markers, particles), pron/name, tense/mod (auxiliaries, modals, the copula, verb tense), negation, quantity (numbers, quantifiers, grammatical number), dropped (content words), lexical (another content word in the same place), attachment (same words, another head or relation), sentence (boundaries), links (next, cause, what was said), quote (direct or reported speech), repetition.

The normal mode aligns the decoded words with the original in steps: the longest common subsequence of forms, then the same form elsewhere (order), typography, case, the same lemma (tense for verbs, form otherwise), a pronoun and a name in the same relation, content words of the same part of speech in the same relation (lexical). What is left is missing or added, each in the class of its word.

Bits

Items are counted exactly; bits are an estimate with fixed rules: a literal string 8 bits per byte; a position among n words log2(n+1); a dictionary lemma 17 bits (110,308 lemmas), a word outside the dictionary its string; a punctuation mark 5 bits, a function word, negation or auxiliary 8; a tag or relation 6; the local order of a head with k dependents log2((k+1)!); a flag 1. A patch item is one field: a word whose lemma and tag both change is two items.

Gates and controls

• Exact gate. Every volume is decoded from the world down with all residuals and compared with the original bytes; the frames, the tree and the tokens are compared on the way. A failure names the layer and the sentence.
• Negative controls (tests): a script without its inserts, a world residual without its patches or boundary toggles, a tree residual with a wrong form — each gives the input back wrong; a decoder that drops negation or punctuation (--drop <class>) shows that class's loss, and no other class goes up. The three generation steps have their own: no tam gives the tense items back (tam_is_read_and_realized), lemma and tag alone give they, was, this (en::morph pronoun_case_and_be_agreement_from_the_tree), the plain world decoder says no pronoun (pronouns_by_salience).
• Determinism. Ordered maps only; volumes run in parallel and merge in their order. Two runs give byte-identical reports.

How to run

cargo build --release -p roundtrip
B=target/release/roundtrip
$B probe "The dog didn't bark. 'Let there be light,' said God."      # every layer of a text, both ways
C=$MOVA_DATA/runs/corpus-0.4
$B run --threads 8 --out $MOVA_DATA/runs/roundtrip-<date>/run \
  lessons=lessons:world/data/school \
  school-gold=conllu:$C/corpus-0.4/treebanks/mova-school-agreed-en.conllu \
  tales03=conllu:$C/corpus-0.3/treebanks/mova-tales-silver-en.conllu \
  torah=tsv:$MOVA_DATA/runs/cycle-torah-2026-10-17/torah.tsv \
  alice=txt:$C/corpus-0.4/books/pg11.txt
$B run --normal --drop negation --out <dir> tales03=conllu:…          # a negative control
$B compare <before dir> <after dir>                                   # two runs side by side: what changed
cargo test --release -p roundtrip                                     # the model tests need $MOVA_DATA/en/models/en.bin

A set is <name>=<kind>:<path>[@<stride>]: txt (a book, paragraphs are blocks), tsv (id<TAB>text lines), lessons (a school directory; title, explain, fact, exercise values are text, every stride-th lesson), conllu (a treebank: its # text, its trees as gold, one volume per document, every stride-th document). The run writes report.md, examples.md (up to four examples of every class at every layer and start), residual.tsv and normal.tsv. The model is en.bin unless --model says otherwise.

Measured (2026-10-18)

Model en.bin of 2026-10-15. Sets:

• lessons: all 23,586 school lessons (91 MB; 515,177 sentences, 12.86 M tokens);
• school-gold: the agreed school treebank (42,759 sentences, 1.14 M tokens; its gold tree used for 42,748);
• tales03: the 0.3 tales treebank (2,087 sentences of 41 books, 51,474 tokens; gold for 2,076);
• torah: the Torah, JPS 1917, torah.tsv of the 2026-10-17 cycle as one volume (6,259 sentences, 184,845 tokens);
• alice: Alice's Adventures in Wonderland (pg11.txt, 1,442 sentences, 35,277 tokens).

Runs, reports and examples: final (on1, on2 with every step on; off with every switch of the frames-complete steps off; compare-full.md is roundtrip compare off on1; ../full.sh). The two runs with every step on gave byte-identical reports and examples. The run with the switches off gives main's report byte for byte on school-gold, tales03, the Torah and Alice; on the lessons only the lesson files of the checkout differ (meta, and 4 bits of dropped the frames-tam run saw in its own copy of the lessons). Earlier measurements: before the frame's tam, the FEATS inflection and the pronoun generator in roundtrip-2026-10-18; with them in final.

Speed. On the shared 20-core server (busy with other jobs, three runs at once), 8 threads: the lessons in 383–387 s (1,300 sentences a second); the Torah, one volume and so one thread, in 447–552 s; 1.1–1.6 GB of memory with the model.

Exact gate. Every volume came back byte for byte: 23,586 lessons, 2,365 school documents, 41 tale books, the Torah and Alice, with the frames, trees and tokens equal on the way down.

Residual per sentence, in bits (ids and lesson keys aside; in brackets with the frames-complete steps off):

Tokens lose almost nothing (Alice: its line wraps). The tree loses 1.1–5.1 bits a token: word order, the book's typography, contractions. The frames lose 6–9 bits a token, two to five times the tree: still the layer where the system stops describing language, but half of what it lost before the frames kept their function words, every content word and the clause links. The world loses 1.3–2.1 bits a token more: which pronoun, the links to frames it drops.

What each step removed (bits per sentence, every set, step off → on; the frames-complete steps from compare-full.md, the others from the frames-tam run):

Each frames-complete step was also measured alone, in order, on the dev subsets (lessons every 20th, school every 5th document, the rest whole; frames-complete-2026-10-18, s0 … s4): the content words took the dropped class from 89–119 to 43–66 bits a sentence, the function words the function class from 65–90 to 28–59, the links the attachment class from 38–47 to 21–33 (and the world's sentence class from 14–34 to 2–6), the punctuation the punct class from 64–104 to 46–85, while costing 2–5 bits of attachment (marks under another head). The world keeps what each mention and clause says, so the world layer did not grow; its pronoun class fell with them (possessors and relative words are no longer names to guess), and its links rose by 0.2–0.8 bits (a link to a frame the world drops). The SLM does not read the new fields: the seven courses gave the same counts with the steps on and off, byte for byte, and the frame bench (dev) stays at F1 0.738. With every switch off the report is byte for byte main's (above).

Normal mode — exact match % / chrF of the text decoded from each start (in brackets with the frames-complete steps off):

• From the tree the main divergence is order (11–19 words per 100 tokens), then case (2–6) and typography (Alice 6.8); inflection is 0.3–1.6.
• From the frames: punctuation 9–17 words per 100 tokens, order 9–12, dropped content 6–8 (14–17 before), function words 4–9 (28–33 before), tense and modality 0.8–2.6.
• From the world, besides: pronoun or name 1.0–2.6 per 100 tokens.
• A gold tree gives less verbatim than our own parse (school 21.9% against 27.4% exact; tales 12.9% against 15.7%; order 13.5 against 11.3 and 17.4 against 14.6 per 100 tokens). lin.rs and the parser learned the same EWT conventions, and the agreed trees follow the Mova dialect. Frames of a gold tree are about as far from the text as frames of our parse.

Negative control (tales03, normal mode, before the tam steps): with --drop negation the tree start's negation goes from 0.0 to 1.1 words per 100 tokens (exact 11.2% → 9.0%); with --drop punct punctuation goes from 0.0 to 15.6 (exact 11.2% → 0.0%, chrF 84.0 → 79.5). With negation dropped no other class moves by more than 0.2; with punctuation dropped the classes that live on punctuation go down with it (typography 2.0 → 0.1, spacing 1.4 → 0.0, order 14.4 → 11.2) and none goes up.

What the residual is made of (examples.md, by kind):

• tree order: the head is the root in 31–41% of the fixes (the main clause: inversion, fronted obliques, «said Alice»), then conj 11–18%, parataxis 8–13%, ccomp (Alice 14%); names come out reversed («by Carroll Lewis»);
• tree form: contractions ('s, n't, ca, 'm, 're, 'll — 40–55% of what is left), dashes, «an» before a consonant letter; were and these where agreement in the tree is wrong;
• tree spelling: the Torah's shalt, hath, hast, -eth (2,382 escapes);
• tree spacing: quotes next to punctuation (.” ,” “I — the largest kinds in Alice and the tales, about a third of Alice's; 24% of the Torah's), hyphenated compounds (-offering, burnt-: 24% of the Torah's);
• frames, punctuation: the text's marks the generator does not make (79–90%: commas after fronted clauses and obliques, around non-restrictive relative clauses, quotes, colons and semicolons), its own marks the text does not have (10–21%);
• frames, dropped content: conjuncts of adjectives and verbs the frames do not make (missing conj 7–16%), amod and compounds of nouns that are no entity (a noun in no role, a predicate noun's own modifiers), advmod of adjectives and adverbs, parataxis, subjects that are no entity; a pronoun subject the frames kept where the text shares it (realized nsubj 4–9%);
• frames, function words: the prepositions, determiners and conjunctions of words that are in no frame (missing case 29–37%, det 20–26%, cc 11–17%), «'s» and «of» pieces of fixed expressions;
• frames, attachment: punctuation under another head (12–17%), a determiner aligned to another noun of the same lemma, amod for compound (the frames keep a noun compound among the adjectives), obl for nmod;
• frames, tense (what tam leaves): the realizer's agreement (VBP for VBZ), get-passives, the copula and auxiliaries of clauses the frames do not make;
• frames, quantity: nummod, all / some / every in the words the frames do not keep;
• world, pronoun or name: the generator's pronoun where the text repeats the name (cheap: one bit), possessives (his, its, their), a pronoun two sentences or more after the name.

The residual ranked, and the changes it asks for

The mean over the five sets of bits per sentence (report.md has every class with every set):

The top changes left, with what each would remove (bits per sentence are the mean over the sets):

1. Punctuation placed after the order (en): 61 bits a sentence (20%); the generator works before lin.rs, so it cannot put a comma after a fronted clause or around a parenthetical, and the marks it makes land under another head (12–17% of the attachment class). A step after the order, on the ordered words, with the rules of the clause structure, would reach the marks that depend on position.
2. The rest of the content words (prag): 51 bits (17%) and most of the 34 bits of function words that hang on them: conjuncts of adjectives and verbs, the modifiers of a noun in no role.
3. Word order in lin.rs: 22.6 bits (7%) at the tree, the main divergence from the tree start (11–19 words per 100 tokens); from the frames start order is now 9–12 words per 100 tokens.
4. Longer reference in the world's decoder: 17 bits (6%); possessives and pronouns past the sentence before.
5. Quantities in the entity (prag): 12 bits (4%).

Done on 2026-10-18 and measured above: tense, aspect and modality in the frame (30 → 3 bits), inflection from the tree's FEATS (20 → 10 bits), the pronoun generator (27 → 23 bits), every content word in the frames (103 → 51), the function words (94 → 34), the clause links (52 → 27; the world's sentence class 21 → 3), punctuation from clause structure (79 → 61).

Smaller and cheap: typography as a document bit (one bit instead of 48 bits a sentence in Alice), contractions as a bit, spacing rules for quotes and hyphens, an EModE generation layer for the Torah's spellings.

Limits

• The frame realizer and the world decoder say only what the frames and the world keep, with fixed rules: much of what is left at the frames layer (punctuation that depends on word order, the words of phrases in no frame) is what these decoders do not try to say. The ranking says which parts of language the frames do not carry, not how good a realizer can be.
• The frames layer is compared on the tree: the residual is a tree edit script, so a word the frames keep in another form counts as a lemma or tag patch, and attachment patches count the realizer's clause structure against the parser's.
• Bits are estimates by fixed rules, not an entropy code; per-document regularities (one apostrophe style for a whole book, one line width) are counted per word, which the proposals point out.
• The en.bin model was trained on the school's parse database, so the school sets are in its domain; the tales and the Torah are not.
• The tales treebank's odd half is the test set of the tales parsing numbers; this diagnostic trains nothing on it.
• prag frames are read from en's tree without world's matrix repairs (world::frames::read adds them).

The meta layer test: roundtrip mixed

Texts mix languages and styles: a commentary quotes Hebrew and Aramaic inside English, a lesson puts slang next to an exam question. roundtrip mixed feeds such mixtures to the skeleton and asks it to see that the pieces are different languages, read each with its own parser, keep one world for all of them, and give everything back byte for byte — comma for comma, error for error, each piece in its own language.

How it works

The sample (seed 20261018, at most --n sentences a variety, 3–40 words):

The streams. mono-<variety>: one variety, a sentence a line. lines: every sentence of every variety shuffled, a sentence a line (the same items as the mono streams together). docs: 300 documents of an English carrier sentence with one or two quotes of Hebrew or Aramaic — cut by the quote (A «Q» B), followed by it (A «Q»), or by two (A «Q1» (Q2)) — and the verse pairs of Ruth and Jonah (the JPS sentences of a verse, then « the Hebrew verse »). A stream is cut into volumes of 25 lines; a volume is one world. The streams, their gold spans and the items are written to <out>/streams/.

The meta layer, text ↔ pieces. langseg segments each line. A segment is cut where a document's quote marks or brackets (« » “ ” ( ) [ ]) stand between its words: langseg reads a Hebrew-script run whole, so two quotes side by side come out as one segment otherwise. Neighbouring segments of one route make one piece (English always; Hebrew and Aramaic unless quote marks part them). The gap between two pieces is cut at its last run of whitespace: what comes before it ends the piece before, the whitespace is the joint, opening marks start the next piece. Down, the pieces are joined with one space; another joint is residual.

The routes.

• English (en, translit): en tokens and tree, prag frames, exactly as the single-language round trip above. A sentence's residual goes to the item that holds most of its bytes.
• Hebrew (hbo, hbo-x-mishnaic, he): hbo's segmenter and gparse (the pointed or unpointed model by the text; one gparse annotate call per model for all the pieces of all the runs, a failure waited out ten minutes, three times at most). Text ↔ tokens: the piece without its quote marks, in NFC (a word that is not NFC is kept as written), cut by hbo::text::tokenize; the decoder writes a space between tokens, none after a maqaf or before a mark. Tokens ↔ tree: each word's form is the commonest form of its lemma, tag and FEATS in the parser's training trees (a lookup generator), its accents residual when the lookup gives the form without them; a prefix (ו ה ב כ ל מ ש) joins the next word and a non-subject pronoun the word before in one written word; the swallowed article (a bare ה after a preposition) is not written. Tree ↔ frames: one frame per verb (lemma, subject, object, indirect object, obliques with their preposition, negation, adverbs; a verb of saying quotes its ccomp), a nominal clause is a fact, and macula prag's rules read the sentence (form, act, speech, carried through the volume) into the first frame. Down: the simplest honest realizer and an edit script to the parsed tree, as for English.
• Text (the Aramaic varieties, other scripts): there is no Aramaic parser yet. The piece has the Hebrew route's token layer, and its tokens are the whole residual (class unparsed); nothing above them holds it.

One world. The frames of all English and Hebrew sentences of a volume, in order, make one world (world::encode_by, each residual item counted for its sentence's item). An entity named in Hebrew and one named in English are one entity only if linked: by the verse — a Hebrew piece found in an index of the WLC's verses and an English piece of the same volume found in an index of JPS 1917's sentences, joined when the English id is among the English words (and their lemmas) of the Hebrew lemma in tanakh-en.tsv (MACULA's commonest glosses and the OpenScriptures definition) — or by the name, a Hebrew and an English name anywhere in the volume, by the same dictionary. A Hebrew lemma joins at most one English id and the reverse. Down, the decoder gives a Hebrew sentence its Hebrew names back. Links are measured against MACULA's gloss of the word in that verse (the gold trees' Gloss=, read in context).

Bits. As in the single-language round trip; a character of the Hebrew script (a letter, a point, an accent) is 8 bits, as an English letter, not the 16 of its two UTF-8 bytes; a Hebrew lemma of the parser's training trees is log2 of their number.

Runs. Every mono stream with langseg and with the gold routing (oracle); lines and docs with langseg, the oracle, and two negative controls: everything to English (all-en), and langseg's languages shuffled over the segments of a volume (shuffle); the linking control link-any (the gloss alone, any Hebrew piece with any English piece, no verse); the ablation of the cut at quote marks. --damage is the gate's negative control: the exact decode drops the Hebrew tree residual and the joints.

Run

cargo build --release -p roundtrip
roundtrip mixed --out $MOVA_DATA/runs/roundtrip-mixed-<date>/run --threads 8 \
  --school world/data/school --tales <mova-tales-silver-en.conllu> --judaism $MOVA_DATA/corpora/judaism \
  --shakespeare $MOVA_DATA/corpora/emode/text/shakespeare.tsv --ud <ud-2.18> --hbo $MOVA_DATA/models/hbo \
  --langseg $MOVA_DATA/langseg/model.tsv --gparse <gparse binary>      # [--n 120] [--docs 300] [--seed N] [--volume 25] [--damage]

It writes report.md (every table below), residual.tsv (run, mode, variety, layer, class, bits), items.tsv (every item's bits by layer in every run), links.tsv (every link of docs), the parses (hbo-pointed.conllu, hbo-unpointed.conllu) and streams/. The outputs hold Modern Hebrew under CC BY-SA and CC BY-NC: they stay in $MOVA_DATA. hbo and macula are libraries as well as commands for this (hbo::{segment_line, annotate}, macula::prag::read_sentence).

Measured (2026-10-18)

1,693 items: 120 a variety, 44–45 for each of the three pointings of Ruth and Jonah; docs 410 documents. Model en.bin of 2026-10-15, langseg model of 2026-10-18, the hbo pointed and unpointed models of 2026-10-18. Runs in roundtrip-mixed-2026-10-18 (run6, run7, damage; run.sh). Two runs gave byte-identical report.md and items.tsv. A whole run (43 runs of streams, two gparse calls for 1,129 Hebrew pieces) takes 16 s and 2.0 GB on the shared server.

Exact gate. Every volume came back byte for byte in every run: mono 71/71 (langseg and oracle), lines 68/68 and docs 17/17 in all five modes, with the frames, trees and tokens equal on the way down. With --damage the gate shows red: lines 0/68 and docs 0/17 in every mode that routes Hebrew (all-en, which routes none, stays 68/68), mono 45/71.

Routing (share of langseg's words given the variety's language / sent to its route):

Confusion (report.md, tokens): Modern Hebrew gets another code for 3.4% (wiki) and 5.2% (HTB) of its words, mostly hbo and hbo-x-mishnaic (still the Hebrew route), 0.9% and 1.4% an Aramaic code or English; Onkelos goes to hbo and hbo-x-mishnaic for 1.8% (the Hebrew route); 22 words of hbo-pointed to hbo-x-mishnaic; English to a Hebrew-script code only where an English item holds Hebrew (an exam question quoting a verse). The two Yerushalmi varieties and much of the Bible are in langseg's training data and their labels are langseg's own, so their routing is not a held-out measure.

Residual per sentence (bits), lines with langseg; the last column mono:

No joint between pieces was other than one space (0.00 bits a line in lines and docs). The Hebrew tree layer is mostly forms the lookup does not give back: for hbo-accented form 337, src-spelling 274 (no lemma, or a written word that is not its words' forms) and accent 103 bits a sentence; for hbo-x-mishnaic src-spelling 394 and form 290; for he-wiki src-spelling 423. Its text ↔ tokens bits for the Yerushalmi are words not in NFC (76 and 58).

Normal mode (chrF of the text decoded from the tree / the frames / the world, lines): English as alone (tree 80.6–91.8, frames 34.0–48.3, world 33.1–47.4); hbo-accented 39.6 / 18.4 / 18.4, hbo-pointed 57.7 / 21.5 / 21.5, hbo-unpointed 69.0 / 22.9 / 22.9, hbo-x-mishnaic 40.2 / 13.4 / 12.7, he-wiki 27.7 / 8.8 / 8.5, he-htb 28.2 / 9.7 / 8.8. The Hebrew generator writes no accents, so the accented text loses most from the tree. From the tree every chrF is the same in mono and lines; from the frames English moves by 0.1–0.5 (en-jps 1.4), Hebrew not at all.

One world. In docs (langseg), 99 verse pairs gave 129 verse links and 4 name links: 111 right and 22 wrong against MACULA's contextual gloss (83.5%), 58 gold pairs missed. By hand, 17 of the 22 join the same word under another English word than MACULA's (LORD / God, storm / tempest, favor / favour, shade / shadow, deep / depth, region / field, food / bread, Salmah / Salmon, you / thou), and 5 join different things (שָׂדֶה = wife, שָׁנָה = one, אֵת = thou, זֶרַע = you, measure): 128 of 133 by hand. The oracle routing gives the same links. The control link-any (no verse) joins 241 in docs: 109 right, 76 wrong (59% of the verified), 56 not verifiable; in lines, where a verse pair is rare (4 pairs), 15 links against 227 for the control.

The cost of mixing (all bits per sentence; in docs each item against itself alone):

Δ context = lines − mono with the oracle; Δ routing = what langseg adds in lines over the oracle, less what it adds alone. Read: shuffling the languages line by line costs nothing that routing explains (|Δ routing| ≤ 0.1 bits); the shared discourse moves English by −3.9…+1.5 bits and Hebrew by −4.6…−1.2 (a world with more entities, the realizer's first mentions); in documents every quote costs 26.5–39.7 bits more and an English carrier −1.4…+11.6.

Negative controls (lines): everything to English costs 883.9 bits a line against 745.5 (docs: 2147.5 against 1757.8), langseg's languages shuffled 964.2 (2347.3). From the frames, chrF for every Hebrew and Aramaic variety falls to 0.0–2.3 under all-en (from 4.8–22.9). In bits per sentence all-en loses on Biblical Hebrew (hbo-accented 2361.8 against 1282.2, pointed 1126.1 / 912.4, unpointed 849.1 / 624.4) and Aramaic (jpa 1216.4 / 1034.7, Onkelos 1098.0 / 860.5, arc 2826.1 / 1295.4) — but not on Mishnaic and Modern Hebrew (hbo-x-mishnaic 1234.6 against 1440.1, he-wiki 977.8 / 1179.3, he-htb 792.4 / 931.9): there the Biblical Hebrew parser's tree costs more to rebuild than the words kept opaque. English loses nothing under all-en (it is English's own route) and 140–360 bits under shuffle.

Ablation of the cut at quote marks (docs, langseg): without it, langseg's segment ran over two neighbouring quotes in a number of documents, making one piece of two items; all bits per document rise from 1757.8 to 1766.0, and the bits of the shorter quote are counted for the longer (Δ docs of the Hebrew and Aramaic quotes +34…+100 instead of +27…+40).

Where the meta layer leaks

From the item-against-itself comparison (report.md, «Where the meta layer leaks»):

1. The quote marks of an embedded quote: +26.6…+31.4 bits per quote at text ↔ tokens (punct) for every Hebrew and Aramaic variety — « and » stored as literal strings on the quote's token layer. This is nearly all of each quote's Δ docs.
2. The English carrier cut by its quote: −1.4…+11.6 bits per carrier, mostly tree ↔ frames dropped and function words (en-formal +9.1 and +4.9, en-plain +7.5, en-tale +5.9): the English parser reads two fragments, «A» and «B», not one sentence with a quoted object.
3. Discourse shared across languages: ±4 bits a sentence (Δ context); its sign varies by variety; the largest cell is he-wiki's frames ↔ world sentence boundaries (−4.2).
4. Routing inside a document: none measurable once segments are cut at quote marks (Δ routing ≤ 0.1; Δ docs langseg and oracle agree within 1.6 bits for 13 of 16 varieties; en-slang +3.2, he-wiki −2.1, he-htb −4.5).

The system changes, ranked by the bits they would remove

1. A Hebrew generator (tokens ↔ tree): the lookup of form by lemma, tag and FEATS misses 337 (form) + 274 (no lemma, written word) + 103 (accents) bits a sentence of accented Biblical Hebrew, 684 of Mishnaic, 495 of Modern. A rule generator for the prefixes' vowels (וַ before a wayyiqtol, the article's dagesh), construct and suffixed forms, and the accents as a layer of their own (they follow the verse's syntax), would remove most of it.
2. A parser for Mishnaic and Modern Hebrew: on them the Biblical Hebrew route is worse than opaque words in bits (1440 against 1235, 1179 against 978, 932 against 792), though it alone gives frames (chrF 13.4 / 8.8 / 9.7 against ≤ 2.3). Modern Hebrew's open treebanks are CC BY-SA or NC (measure only), so it needs our own trees; Mishnaic Hebrew needs trees from the Mishnah.
3. An Aramaic parser: 860–1295 bits a sentence of Aramaic are kept as text, with no frames and nothing in the world. MACULA's Biblical Aramaic trees (268 sentences) and the Targum's alignment to the Hebrew verses are the start.
4. A quote in the meta layer, not in the text: a piece's quote marks as one record of the document (which marks, which piece) instead of literals: 27–31 bits a quote down to a few.
5. The quote as a constituent of its carrier: parse the English carrier with the quote as one node (the object or apposition of «says», «renders»), and keep that node in the frame, pointing to the quote's own frames; removes the 3–12 bits a carrier and gives the world the link «this English sentence quotes that Hebrew one».
6. Cross-language entity links by the verse's word alignment, not by dictionary words: 58 gold pairs missed and 5 of 133 links wrong with the dictionary; MACULA's contextual glosses (or a learned alignment of the verse) give the word, and an entity may need more than one English name (LORD and God).
7. langseg cuts at a document's quote marks itself, as the meta layer now does for it (the ablation above), and does not let a Hebrew-script run decide two quotes together.

Limits

• The Hebrew and Aramaic sentences are few in places: 44–45 a pointing of Ruth and Jonah. Ruth and Jonah are the Hebrew parser's dev books (its choices were made on them), and langseg was trained on most of the Bible, the Targum and the two Yerushalmi tractates; the Yerushalmi's labels are langseg's own. None of this is a held-out measure of the parsers or of langseg.
• The Hebrew frames reader and realizer are the simplest honest ones, and the Hebrew generator is a lookup; their residual says what they do not try to say, not how good a Hebrew realizer can be. The English and the Hebrew estimators share the bit rules but not the trees, so bits compare within a variety (mono against mixed), less across languages.
• A piece's residual goes to the item that holds most of its bytes (an English sentence to its own item); when langseg merges two items into one piece the attribution is coarse — the ablation shows how coarse.
• Link correctness is judged against MACULA's gloss in context; synonyms count as wrong there (17 of 22 «wrong» by hand). Names and pronouns are joined only within a volume of 25 lines.
• The documents are built by template; real commentaries also transliterate (translit), abbreviate and cite by verse number, which this test does not sample.
• Modern Hebrew (UD IAHLT, CC BY-SA; HTB, CC BY-NC) is only read here; nothing built from it is kept.

The block gate: roundtrip blocks

In training every lesson of the corpus mix must unfold back into itself bit for bit, not sentence by sentence but as whole blocks. A block is a whole lesson, a book chapter, or a run of whole chapters up to 2,500 words (about 4–5 A4 pages at 13 pt), cut only at chapter or lesson boundaries; a single chapter longer than that stays whole. This is a hard training target, not a claim that the layers keep every word and meaning: the residual says what they lose.

How it works

Block makers (roundtrip::blocks, a library for tools/translate too). Each corpus is read into chapters in its own order; blocks(corpus, &chapters, max_words) groups them greedily into runs that stay inside a chapter's group.

A book is cut at lines starting with CHAPTER, PART, BOOK, LECTURE, STORY or SECTION after a blank line, and in the tales and Alice also at a title line in capitals alone between blank lines; the text before the first heading is a chapter too. A "chapter" longer than 10,000 words means the maker sees no chapter structure (a dictionary, a handbook numbered by sections only, OCR without headings): it is cut at blank lines into runs of at most 2,500 words and the block says so (para_cut). The chapters of a file together are the file byte for byte (tests).

Whole-block encoding. A block is one volume: the reader's discourse (prag::frames::Ctx), the realizer and the world are one for the whole block; headings, speakers and ids are metadata pieces, paragraphs and blank lines layout pieces, and sentence boundaries a class of the world's residual — all of it must come back. Exact mode: every block is decoded from its world down with every residual and compared byte for byte. Normal mode: the whole block decoded from each start (tokens, tree, frames, world) against the block's sentences on one line, exact % and chrF, and sentence by sentence as before. Hebrew goes through the Hebrew route of roundtrip mixed (one volume a block).

Units (pipeline::Unit): block (the gate); sentence (the discourse, the realizer and the world reset at every sentence: the frames themselves change, a pronoun the reader cannot resolve stays a pronoun); world (the frames read with the block's discourse, the realizer and the world reset at every sentence: the same frames as block, so the bits compare). Hebrew has block and the verse alone.

Controls, on every 10th block of each corpus: damage-<layer> loses the first residual item of a layer in the exact decode (Hebrew: the whole tree residual) and the block must be red; flat (the block's sentences joined by a space) against shuffle (the same sentences shuffled inside the block). A block that fails to go up or come down (an error or a panic) is a red block with its reason, not the end of the run.

Run

cargo build --release -p roundtrip -p jobs
jobs run --name rt-blocks --mem-gb 8 -- roundtrip blocks --out $MOVA_DATA/runs/roundtrip-blocks-<date>/run --threads 8 \
  --model <en.bin> --school world/data/school --hbo $MOVA_DATA/models/hbo --langseg <langseg model> --gparse <gparse>
  # defaults: --judaism, --shakespeare, --corpora under $MOVA_DATA/corpora; tales and books under $MOVA_DATA/runs/corpus-0.4
  # [--only school,tales,alice,tanakh,talmud,shakespeare,grammars,hbo] [--stride N (lessons)] [--controls N] [--max-words 2500] [--hbo-books GEN,EXO,…]

It writes report.md (every table below), the training signal residual-blocks.tsv (block unit: corpus, block, layer, class, items, bits — what the trainers and the agents read), blocks.tsv (a row per block and unit: chapters, words, sentences, tokens, exact, bits per layer, metadata bits, total), total.tsv (per run and corpus: blocks, exact, total bits per block, sentence and token), and per unit block/, sentence/, world/ with residual.tsv, normal.tsv, examples.md in the format of roundtrip run (roundtrip compare reads them). The outputs stay in $MOVA_DATA.

API for other tools: blocks::{school, book, books_dir, jps, wlc, talmud_en, shakespeare, grammars} → Vec<Chapter> (id, group, vol, para_cut); blocks::blocks(corpus, &chapters, max) → Vec<Block> (corpus, id, chapters, words, para_cut, vol; text() is the block byte for byte); blocks::like(corpus, &chapters, &blocks) cuts another text of the same chapters as blocks (the WLC in the JPS blocks: the two sides of a translation are the same block); blocks::run::flat(a, &volume, shuffle); pipeline::Opts { unit, damage, .. }.

Measured (2026-10-18)

Model en.bin of 2026-10-15; the hbo models and langseg of 2026-10-18. Run run1 (run.sh): 1,897 s wall, 12,656 s CPU, peak 1.97 GB on the shared server, 8 threads, all units and controls. Determinism: run2 (every 20th lesson and Alice, the Tanakh, the Talmud, Shakespeare and the Hebrew Torah whole, the Hebrew parsed again on the GPU) gave every one of its 6,025 block rows and 42,462 residual rows byte for byte as run1, and a third Hebrew run (run1-hbo) the same again: a block's numbers do not depend on the other blocks or on the run.

Blocks:

The gate. Every block of every corpus came back byte for byte in every unit: 27,197 blocks (school 23,586, tales 1,248, Alice 13, Tanakh 312, Talmud 47, Shakespeare 403, grammars 1,508, Hebrew 80), with frames, trees and tokens equal on the way down. Damage turns it red: every block that had an item of the layer to lose failed — the tree and the frames layers 2,715 of 2,715 English blocks of the control set and the Hebrew 80 of 80, the world 2,713 of 2,713 (one tale and one grammar block have no world item), the token layer 278 of 278 (the school, the Tanakh, the Talmud and Shakespeare have no token residual to lose).

Residual of the block unit, bits per sentence (ids, lesson keys, headings and speakers aside):

The corpus total, the number to drive down: 404.7 M bits (50.6 MB) over 27,197 blocks. The grammars' text ↔ tokens is the OCR's double spaces (157.7 bits a sentence of spacing).

Normal mode, the whole block (chrF; no block comes back verbatim from the tree or deeper): from the tree 78.1–87.5, from the frames 52.4–74.0, from the world 50.6–70.2 (school 66.3, tales 70.2, Alice 64.4, Tanakh 67.9, Talmud 68.9, Shakespeare 62.3, grammars 50.6). Decoded from per-sentence worlds the block reads 0.8–4.9 chrF higher (school 71.2, Talmud 72.9): the one world says things where the text does not (below). Hebrew is measured sentence by sentence only (chrF 46.8 / 19.1 / 18.1 from tree / frames / world).

From sentence to block (bits per sentence: sentence unit / world per sentence / block unit; unresolved pronouns: mentions whose entity is a pronoun the reader did not resolve, per 100 sentences):

What one world for the block changes against a world per sentence, the same frames (mean over the seven English corpora, bits per sentence):

The tree ↔ frames layer moves by less than 1 bit a sentence between the units.

Read: the block makes the world hold what the sentence unit could not — who «he» is (unresolved pronouns 8–88 per 100 sentences alone, 0.6–2.8 in a block), and the sentence-unit bits are lower because nothing pays for that identity. With the identity kept (the world unit), one world still costs 8–13 bits a sentence more at frames ↔ world than a world per sentence: of the classes expected to fall at block level only pronoun against name falls; sentence planning and entity reintroduction rise, because the world decoder plans the block from one registry (every modifier at the first mention, sentences cut at unlinked events, links across boundaries). These are the decoders the block gate asks the training to learn.

Shuffle (bits per sentence at frames ↔ world, in order → shuffled): tales 41.4 → 42.1, Talmud 35.5 → 36.8, Shakespeare 28.5 → 28.9, Alice 38.4 → 38.4, grammars 18.2 → 18.2, Hebrew 89.9 → 89.9, school 32.5 → 32.2, Tanakh 45.0 → 44.9; pronoun or name rises in the narrative corpora (tales +0.6, Talmud +1.2, Shakespeare +0.4) and by at most 0.1 elsewhere. The world residual is almost blind to sentence order: the reader re-resolves the shuffled pronouns by the same salience the decoder uses, and the world records what was read, not whether the story coheres. As a control of coherence this one is not red; a world that pays for order (who could be «he» here, given the story so far) is a change the block gate asks for, not something it measures today.

What loses most at block level (every class of the block unit by its share of the 404.7 M bits):

report.md has the same ranked by bits per sentence (each corpus weighing the same) with the full list of changes.

Limits

• Chapter headings are found by fixed rules: 413 tale blocks and 1,299 grammar blocks are cut at paragraphs because no heading was seen within 10,000 words (Kruisinga's handbooks are numbered by sections, Wright's Bible Word-Book is a dictionary, Sweet's primer has none), and 541 tale and 923 grammar blocks are single chapters over 2,500 words.
• The JPS, Talmud and Shakespeare blocks are rebuilt from sentence and segment records: their headings (GEN 1) are ours, their paragraphs the source's records, not a printed page; the school's lesson keys are metadata (266 M bits, counted aside).
• The sentence unit changes the frames themselves, so its bits measure a different, poorer representation; compare bits with the world unit.
• The shuffle control is not red at the world layer (above); the damage controls are the gate's.
• Hebrew is the Torah in the JPS blocks; its normal mode is per sentence only; the Hebrew reader keeps no pronoun entities.

The site's Markdown as blocks: blocks::site_md

The Markdown twin of mova.elamur.ai (free/en/**.md: the docs, the corpora pages with their ```conllu trees and frames tables, the dictionaries, the school runs) is a corpus of the block gate. A page is a chapter; a block is a page or a run of sibling pages (one folder, one Bible book) up to 2,500 words of prose, cut only at page boundaries.

How it works

• Prose up, markup aside (site_md::page_volume). Paragraphs, list items, table cells with letters, headings and link texts are text blocks and go up the layers as English. The rest is layout of class markup: front matter, <!-- … --> markers, heading and list marks, table pipes and number cells, code fences; inside a paragraph, link targets, emphasis, backticks, escapes and inline HTML are marks at byte offsets of the block (vol::Block::marks), so the prose reads without them and the page comes back byte for byte. A paragraph a marker gives another language (lang=hbo, dir=rtl: the Hebrew verse, the Targum) is kept as written (unparsed).
• Markup copied (vol::Copier). Each markup literal is a copy of an earlier one of the block (its common prefix and suffix kept, the middle stored) or stored whole: a flag, the earlier literal (one bit when it is the one after the literal the last copy took — a page copied line after line from the page before — else its number), the edit (one bit when the same). With a dictionary learned on the train pages (--site-dict), each block starts with its section's frequent markup.
• Repeated prose copied (site_md::repeat, class repeat). A text block that is a copy of the prose before it in the block for at most a quarter of its bytes (a template's sentence with other names: «This is Deuteronomy 12:5 · JPS 1917 in the English of …») is coded as that copy and does not go up the layers. ROUNDTRIP_OFF=site-repeat keeps every block.
• Trees and tables rebuilt from the layers (site_code, classes conllu at the tree layer and frames-table at the frames layer). An English CoNLL-U is en's annotation of the sentence its # text points to (a prose block or a given text), with a patch per field that differs (lemma 17 bits or its string, UPOS 5, tag 6, a FEATS pair 7, head log2(n+1), relation 6, MISC its string, log2(n+1) to name the word); its comment lines are copied. A Hebrew CoNLL-U (MACULA's, right to left) is written from its verse and a lexicon of the train pages (site_heb, --site-heb): the verse cut into MACULA's written tokens, each token into its words with its commonest analysis (form, lemma, UPOS, FEATS, relation, head inside the token, Gloss, Strong, Type), heads outside the token by the commonest offset of their UPOS and relation, Ref and MId from the verse and the token's number; a token cut into another number of words than the page is given whole, every other difference is a field patch (FEATS and MISC per pair). The English frames table is prag::frames_md::table of the frames prag reads from the rebuilt tree, one discourse state through the block's trees (a bit per table: start afresh or not; a bit: the note left out); the Hebrew verbs' table is prag::frames_md::hebrew_table of the rebuilt Hebrew tree (MACULA's Frame=). What a table still differs in is a line script. The page builder (tera/tools/papers) calls the same two functions, so a rebuilt table is the page's byte for byte.
• The gate decodes every derived piece from its code, the block's prose and the model alone, and compares it with the page; the damage controls lose a field patch of a tree (damage-tree) or a line of a table script (damage-frames) and must show red.
• Sampling and split. Pages are listed per section (corpora/torah, corpora/tales, docs, …) in the order of their paths, numbers as numbers; the runs are cut per section; --site-target N takes every k-th run so that about N are taken per section; a block goes to train, dev or test by the hash of its first page (test 1 in 10, dev 1 in 10: site_md::split_of, fixed).

What the train pages teach: roundtrip site-learn

On the train split (the same sample, the Tanakh pages of tools/translate's dev and test books left out whatever their split): the markup dictionary (markup.tsv: per section the literals repeated at least 3 times, at most 4,096, the most bytes saved first), the Hebrew trees' lexicon (hebrew.tsv), en's order generator adapted (en-lin.bin: lin.rs trained again on its own trees and on en's parses of the train pages' prose sentences, Model::adapt; the tagger and the parser are the same, so every parse is too), and every fourth of those sentences for tools/translate's standard encoder (english.tsv). Negative controls: --lin-shuffle (each sentence's words shuffled before it is learned from) and --shuffle-tables (each section gets the next section's dictionary, each Hebrew token another token's analysis).

Run

cargo build --release -p roundtrip -p jobs
S=<a frozen copy of the md twin>/free/en          # the live tree is rebuilt in place: measure on a copy
roundtrip site-learn --site-md $S --site-target 1000 --out $L --model <en.bin> --threads 8 --lin-every 2
jobs run --name site --mem-gb 8 -- roundtrip blocks --only site --site-md $S --site-target 1000 --site-split dev \
  --out <dir> --threads 8 --model $L/en-lin.bin --site-dict $L/markup.tsv --site-heb $L/hebrew.tsv
  # [--site-sections corpora/torah,docs] [--units block,sentence,world,controls] [--site-split train|dev|test|all]

The report adds the site's tables: bits per block and per 1,000 words of prose with markup, CoNLL-U, frames tables and the prose layers apart; for each kind of derived piece how many there are, how many need nothing stored, and the bits of storing them whole against their code; what the codes store by kind.

Measured (2026-10-19)

A copy of the md twin taken at 12:10 (387,595 pages; md-snapshot), --site-target 1000, the dev split: 1,040 blocks, 6,040 pages, 2.18 M words of prose, 140,319 sentences in 18 sections. Model en.bin of 2026-10-15. Runs in ae-md-2026-10-19 (site.sh; site-before, site-after1, site-after2, learn5, learn5-control): 97 s wall, 620 s CPU, 0.98 GB with every unit and control; the listing of all pages takes 5 s.

The gate. Every dev block came back byte for byte in the block, sentence and world units, before and after the training; every damage control that had an item to lose was red. Determinism: two runs before and two after gave byte-identical report.md, blocks.tsv, residual-blocks.tsv, total.tsv and examples.

Bits (dev; before: en.bin and no table of the train pages; after: the tables and the adapted order of site-learn):

The prose layers include the repeat copies (1.48 M bits after). Stored whole, the dev blocks' CoNLL-U blocks and frames tables would cost 110.6 M bits, more than everything else together; written from the layers they cost 4.93 M. Their codes save 98.4–99.5% for the English trees, 91.3% for the school's agreed trees, 78.3–84.1% for the Hebrew trees, 90.5–99.9% for the English tables (the tales' 90.5%: the page builder's discourse state starts at the chapter, the block's at its first page) and 100% for the Hebrew tables (667 of 667 rebuilt exactly). The markup, 53.1 M bits stored whole, costs 16.1 M copied.

What each learned part removed (dev, block unit, bits; the order class is tokens ↔ tree order):

The seven courses (torah, biology, religion-ethics, history, world, mysticism, everyday) gave byte-identical output with en-lin.bin and en.bin, and with this branch's prag and main's.

The test split, once (1,050 blocks, 6,115 pages, 2.22 M words; site-test, site-test-before): every block byte for byte; 77.48 M bits before, 67.45 M after (−12.9%); after, 64,240 bits a block and 30,426 per 1,000 words.

What loses most after (share of the dev bits): markup 25.7%, frames dropped 22.0%, frames punct 10.9%, frames function 6.5%, conllu 6.1% (Hebrew tokens given whole 1.44 M, Hebrew heads 0.41 M, English comment lines 0.49 M, English MISC 0.25 M), frames quantity 4.8%, unparsed 4.2% (the Targum's Aramaic, the Hebrew lines no tree points to), frames attachment 3.0%, layout 2.5%, repeat 2.3%, tree order 2.0%. The tables' largest kind is the «who» column (0.64 M bits): the frames reader's state from pages before the block.

What the residual asks for next

1. Markup from the page's id and template (25.7%): crumbs, the HTML link and the front matter follow from the page id and the section's template; copying them from the page before still pays for every new id.
2. The frames' content words and punctuation (prag, lin.rs: 39% together), as for every other corpus.
3. The Hebrew lexicon's back-off (1.44 M bits of tokens given whole): a token never seen cut by its prefixes and suffix (ו ה ב כ ל מ ש, the pronoun suffixes) and its stem looked up; the heads by the token's place in the clause.
4. The discourse state from the chapter's start: the frames tables' «who» column and the block cut at chapter boundaries for the corpora pages.

Limits

• The live md tree is rebuilt in place (it was rebuilt during this work: 378 k → 388 k pages); runs measure a frozen copy.
• repeat takes templated prose out of the layers: its bits are those of the copy, and the layers' classes measure the rest; ROUNDTRIP_OFF=site-repeat gives them all.
• The copier's correctness is checked on every literal (a code that does not decode is stored whole); the exact gate copies markup literals, it does not decode them.
• A block cut inside a chapter starts the frames tables' discourse state afresh; the English tables of the tales' and Tanakh pages pay for it.

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/tools/roundtrip/