Gali, a small large language model
Gali is Elamur's small large language model (sLLM), built around Mova, Elamur's small language model (SLM) for parsing. Gali itself is not released; this site holds the draft papers on its architecture, versioned with Mova.
Mova analyses text into dependency trees, pragmatics and frames, deterministically (its pipeline). Gali generates text: a frozen multilingual autoencoder reads and writes tokens with Mova's markup, and Gali reasons between its halves, thinking in English. Every training target is known without any AI — Mova's parses, human translations and tasks built by code.
Papers
Gali and Mova: a closed sLLM that learns together with an open parser
Start here. Gali is Elamur's small large language model: closed, Elamur's property, its weights and code unpublished. It is built with Mova, Elamur's open-source parser, which reads text deterministically while Gali writes. Mova's markup teaches Gali's autoencoder, and the autoencoder's confident disagreements come back to Mova as fixed rules, so the two get better in turns. The paper says what each part does, how they share their data, and how the rest of this set is organised.
draft, preview 0.6Gali: a small large language model between the halves of a frozen multilingual autoencoder
The main paper of the preview describes the whole design. A multilingual encoder–decoder, trained on 30 languages with Mova's markup and then frozen, reads and writes for Gali. Gali is a 20-block trunk with long skips and two branches: one writes the answer, the other writes its work, a thought in English and memory cells. It answers in passes that read the input with the previous answer and work, and its streams carry seeds of language and role. The paper gives the training tasks, the measurements with their controls and the limitations.
draft, preview 0.6The multilingual autoencoder that reads and writes for Gali
Gali never reads or writes characters itself: one encoder–decoder, trained before Gali and then frozen, does it for about 140 languages. It rebuilds a text from its bare words together with the markup Mova would write, and two synthetic tasks align the languages' tokens. Because the markup weighs only a tenth of its loss, its confident disagreements with Mova point at parser errors. Its figure shows the three tasks and how the disagreements come back to Mova as rules. The paper gives the model, its tasks, the layer probe, the skip test and the limits of a small model for many languages.
draft, preview 0.6A vocabulary with a shared half for 79 languages
The second paper lays out the 8192 token ids that every model of Gali uses. The first 4096 mean the same in every language: markup, language tags, signs, WordNet concepts and seed tokens. The other 4096 belong to the language in use and are fitted on its own Wikipedia prose, for each of the 79 languages of the Elamur sites. Each shared block has single tokens for frequent items and codes for any number of rare ones. The paper also explains why digits are deliberately not shared, and how each language's bank turns text into tokens and back without losing a character.
draft, preview 0.6Marking languages, continua and dialects in Gali's tokens
Every stream Gali reads or writes carries a tag that says its language, and in Gali 0.6 that tag is a path in a language graph: the root language, then a code for each node down to the variety, and a marker for each further parent. The graph also decides which bank of tokens a text is written with. Where one language ends and another begins is often settled by states, scripts and markets. What can be supported is settled by the corpora that exist. The paper gives the decisions and what each costs.
draft, preview 0.6Versions of these papers
The papers are versioned with Mova, not with Gali, which is not released. Gali 0.6 preview describes the design built on Mova 0.6 and Mova Logic 0.6. Until Mova 0.6 is released, the drafts are edited freely, so a page may change from one day to the next. At Mova's release the set is fixed and stays readable at its own address, so a citation keeps pointing at the text it meant, with its figures and numbers. The next set starts with the next version of Mova and describes the design built on it.
0.6 previewStudies
Short notes on single measurements, each with its control and its limits; not full papers.
Where a long skip keeps the token
A skip from block 4 into block 8 of the autoencoder's encoder was compared with the same run without it, on the same data in the same order. A probe per layer reads the token at each position from the layer's states. It shows that the skip does not keep more of the token at the top: the blocks before the skip let the token fade, and the skip brings it back. Whole lines come out better with the skip, the markup slightly worse. The note gives the table, our reading and the limits of one pair of runs.
study, 19 Nov 2026Continuing a multilingual autoencoder as it grows
The autoencoder keeps gaining tasks and languages, and we continue it from its checkpoint instead of starting over. Each change is followed by a dip: when seeds and roles were added, chains of numbers fell from 95 % to 47 % and came back to 78 % within 10 000 steps. New languages cost the old skills less than new token roles do. Markup follows the data it is measured on, so it is read as a trend. The note gives the dips and recoveries we measured, why continuing still costs less than starting over, and why the same holds for Gali.
study, 20 Nov 2026When the model copies the answer from its context
A negative result. Gali was trained in social scenes where the noisy answers of other copies of Gali sit beside the source sentence. It then translated the same with the right source and with another source, while chains of numbers, which need the source, were mostly right. Making half of the translation items plain did not cure it. The lesson is a control for every context that holds a version of the answer: the right source against another, with everything else in the input kept as it was.
study, 20 Nov 2026In practice
Production examples: what the design does for Elamur's own tools.
An autoencoder that finds errors in a deterministic parser
Mova parses text by hand-written rules, so its errors repeat on every sentence of a kind. The autoencoder gives Mova's markup only a light weight in its loss, so its confident disagreements with Mova point at errors on one side or the other. They are read by hand, and the real errors become general rules in Mova. Four of six leads were real; after the fixes 67 more treebank tokens got the right labelled head. A planted-error control shows that about two errors in three are caught.
practice, 20 Nov 2026Lexical banks for 140 languages from open text
Half of Gali's token ids belong to the language in use, so every language needs a bank of its own. Each bank is fitted on open Wikipedia prose with bot-made stub articles removed, which in some Wikipedias are most of the articles. Dialects read through their standard's bank, a continuum shares one, and a language in another script gets its own. Not one paragraph changes in the round trip, in any of about 140 languages. The note gives the pipeline, the densities and what sharing a bank costs.
practice, 20 Nov 2026