📘 wordgraph-tamp — the Elephant's tamping step over papers

🐘 The Elephant keeps the word graph (wordgraph, the animals). Tamping is its pass over what papers say: statements that many papers say alike — how they train with Adam, what CIFAR-10 holds, that results are averaged over seeds — are collapsed into shared approach nodes, and each paper keeps what is its own. A node is a reverse distillation of what large models absorb from repetition, built natively: Mova reads, Rust counts, no language model writes or judges anything here.

It is a separate crate next to the graph because it reads papers through the Whales' reader (tools/whale, reader.rs), whose dependencies stay out of the root workspace. It uses the graph (lemma ids, Graph::freq, typed neighbour profiles) through the wordgraph library.

CARGO_TARGET_DIR=… cargo build --release --offline --manifest-path wordgraph/tamp/Cargo.toml
elephant-tamp read STORE ROOT [--jobs N] [--limit N]           # Mova reads every ROOT/*/eprint.md once
elephant-tamp build STORE ROOT OUT [--graph WG] [--tau X] [--min-papers N] [--min-cos X]
                    [--min-typicality X] [--min-lift X] [--max-anchor-share X] [--min-score X] [--jobs N]
elephant-tamp controls STORE ROOT [--graph WG] [same options]  # the negative controls (JSON)
elephant-tamp probe STORE ROOT LEMMA                           # statements using a lemma (TSV)

Runs need WORLD_EN_MODEL (the en model) for read; --graph takes a word-graph file. Heavy runs go through tools/jobs. Outputs hold sentences of the papers: internal, under the papers' licences, never published.

Model

Statements. read keeps, per paper, every admitted sentence of a read part (reader::read: references, acknowledgement headings and LaTeX math are not read) with its restated clauses and the parse's lemma and UPOS per token (reader::Sentence::lemmas). The record is cached under the text's SHA-256 and the reader version (compose::WRITER + record layout); a changed text or reader is read again.

Normalization (norm.rs). A statement's content sequence: lemma ids of the tokens its clauses restate (all tokens when none is restated), UPOS NOUN, PROPN, VERB, ADJ, ADV or NUM, case-folded, every number (and a lemma starting with a digit, 1e-4) one id, light verbs and connectives left out. Dictionary lemmas keep the en dictionary number (the word graph's ids); others OOV_TAG | fnv. Shingles: ordered pairs at distance one or two. Statements with fewer than 5 content lemmas are scored but not clustered.

Synonyms (synonym.rs). WordNet proposes, the graph confirms: v in w's first-sense synonyms (same part of speech, both used by ≥ 2 papers) and the cosine of their typed neighbour profiles in the word graph (relation, direction, neighbour; PMI weights) ≥ 0.15; British/American spelling variants without the graph. Union-find, the smallest id represents a class.

Template nodes (cluster.rs): near-identical statements. MinHash LSH, 64 bands × 3 rows (candidate probability 0.40 at Jaccard 0.2, 0.83 at 0.3, 0.985 at 0.4), exact Jaccard verification, an edge at J ≥ τ = 0.4 between different papers. Star clustering, no chaining: sets in order of statement document frequency, an unassigned set is a centre and takes its unassigned neighbours; ≥ 3 papers make a node.

Anchor nodes (anchor.rs): what papers typically say around a term. For a noun/proper-noun lemma t used by 5 papers up to 40 % of them, with T(t) its statements: the profile Π(t) holds the shingles of T(t) that occur in at least max(3, 5 % of its papers) papers and are said in T(t) at least 10 times more often than in all statements (lift). Typicality τ_t(s) = |sh(s) ∩ Π(t)| / |sh(s)|. A member has τ ≥ 0.5 with ≥ 3 shared pairs; a statement typical for several anchors goes to the highest τ × salience, salience = ln(corpus share / Wikipedia share in the word graph). Anchors whose nodes take the same name merge. Name: the adjacent pair with the anchor that half the members use («Adam optimizer», «beam search»).

The order test (both kinds). A match must hold in the statement's own order: τ (or J) must beat the best of 8 seeded shuffles of the same words by 0.2 (norm::order_holds). Without it the shuffle control was red (below).

Canonical explanation: Mova's restated clauses of the canonical member — for an anchor a definitional member («t is/consists/computes …», «t (…»), else the most typical; for a template the centre — plus core, the member words at least half of the members share, in its order, and every variant with its fit and paper.

Importance (importance.rs): score = 0.35 rarity + 0.25 specificity + 0.20 own + 0.15 result + 0.05 part, × 0.25 for boilerplate. Rarity = 1 − ln(df)/ln(N) with df the papers holding a statement of J ≥ τ (0.5 when too short to cluster); specificity = mean of the three highest lemma idfs; own = first person or «this paper/work» with a contribution verb; result = a number with %, a metric or a comparison verb; part +1 abstract/contributions/conclusion, +0.5 introduction, −1 related work/background. A node member is boilerplate unless it carries a payload: a lemma outside the node's shared words used by at most 1 % of the papers (then it stays, and the node is still referenced). Important: score ≥ 0.5 and not boilerplate. The weights are a stated operating point, not fitted.

Compression per paper: each boilerplate member is replaced by one reference [approach ID: NAME], written once per node. ratio_refs = words / words with references; ratio_important = words / (important statements + refs).

Invariants. Deterministic: sorted maps, seeded hashes, ties by text and index; output bytes do not depend on the thread count. A node member is within τ of its centre (template) or passes the anchor's tests (anchor); every reference names an existing node; a paper record is bound to its text hash and reader.

Cost. Reading ≈ 0.7 CPU-s per paper (Mova parse plus the grammar gate per clause); build O(U·(K + B log U)) for U statements, K = 192 hashes, B = 64 bands, plus per-anchor profile counting O(Σ|T(t)|·|sh|).

Prospective predictions (written 2026-11-06 before the first build) and measurements

The predictions were wrong by an order of magnitude: in these papers repeated explanations are rarely near-identical statements. Statements mentioning «Transformer» (582 papers) have typicality p90 0.19, max 0.57 — each paper words it its own way — so no Transformer node forms; Adam, beam search, BLEU, CIFAR-10, COCO do.

Final build (defaults above, word graph wiki-50k.wg): 557 nodes (194 anchor, 363 template), 3,122 statements, 43,171 of 9,568,093 words boilerplate; 144,169 important statements; ratio_important 2.83. Coherence: template member pairs with J < 0.2: 0.6 %; anchor member pairs: 57 % (anchor members share the profile, not each other's words). Single linkage at the same τ would join 195 sets of 162 papers into one component; the star keeps nodes apart. Without the graph: 534 nodes, 3,039 statements (the graph adds 551 synonym pairs and the salience).

Hand check (2026-11-06, a small finite sample): every 10th of the first 200 nodes of the τ 0.4 / typicality 0.5 build before the order test, its canonical member and its two worst-fitting members from other papers (60 statements): 20/20 nodes on one topic; 1 node («FID») groups result statements rather than an explanation; 1 («leftmargin») was LaTeX residue, now excluded. Synonyms: 13 of 16 sampled pairs are synonyms in context.

Controls (controls, final code): shuffled-only corpus 11 nodes (4 anchor, 7 template) against 557 real; real plus a shuffled copy of every statement as another paper: 113 copies join a node (50 of them shuffles that left the order nearly unchanged, J ≥ 0.5 to the original; 385 shuffles are identities); foreign statements (from papers that never use the anchor) asked against each anchor's profile: 0 of 1,348 pass. Two identical runs and a run with 3 threads give byte-identical outputs.

Writer hook

mova_whale::compose::Approaches::from_record(record, paper_text) reads a paper record (refused for another text or reader); compose_with_approaches(reading, known, guard, max, &approaches) writes the description with each node's repeated explanation replaced by one sentence naming the node, and over the word budget keeps the most important sentences first. API only: the Owls' configuration does not use it.

Sources

Library ids are folders of the paper library; works marked «outside» have no card yet (docs/usage.tsv).

• Shingles and Jaccard resemblance: broder-1997 (resemblance and containment; syntactic clustering of the web).
• MinHash banding as locality-sensitive hashing: Indyk & Motwani 1998 (outside); NLP use ravichandran-2005; near-duplicates at web scale henzinger-2006, manku-2007; random-projection relative charikar-2002.
• Star clustering: Aslam, Pelekhov & Rus 2004 (outside).
• Term specificity and tf-idf: Sparck Jones 1972 (outside), salton-1975, robertson-2009 (BM25 as the probabilistic alternative); synonym profiles after baroni-2010.

Limits

• Repeated explanations of an approach across papers are mostly paraphrases, not near-identical: at these thresholds the nodes are dominated by setup and reporting boilerplate (hyper-parameters, dataset sizes, «comparison with the state of the art», funding, checklists). The compression ratio is small (≈ 1.003); paraphrase-level merging needs frames or meaning, not shingles.
• Duplicate library entries (two versions of one paper) count as two papers.
• Anchors are single lemmas; «GPT» covers GPT-2/3/4 (the tokenizer splits «GPT-3»).
• Template + payload is decided by word rarity only.
• The shuffle control still lets short statements through (5–7 content lemmas keep runs under a random shuffle).
• No word senses: «Adam» the optimizer and Adam of Genesis share an anchor; the profile keeps the majority sense.

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/wordgraph/tamp/