What question did this study set out to answer?

May 25, 20260 citationsOpen Access

Sub-lexical Regularity in the Voynich Manuscript: BPE Morpheme Signature Discriminates Competing Hypotheses and Reveals a Concentrated Suffix System

Key Points

To classify the Voynich Manuscript using byte-pair encoding mean vocabulary morpheme length (BPE VMML) and assess its suffix system.
Applied BPE VMML to 21 corpora, including gibberish and various languages, to identify writing system characteristics.
Analyzed the suffix inventory in the Voynich Manuscript, comparing it against Latin and German corpora.
The Voynich VMML of 5.918 exceeds the maximum of the alphabetic cluster (4.51), demonstrating distinct sub-lexical features.
37 distinct 3-character endings constitute 80% of the Voynich corpus, a stark contrast to Latin (172) and German (109).
Stable suffix patterns are observed across different scribal sections, indicating consistency in the manuscript's language structure.

Abstract

The Voynich Manuscript (Beinecke MS 408, c. 1404-1438 CE) resists classification because standard statistical metrics are reproduced by both natural language and human-produced gibberish (Gaskell & Bowern 2022). We introduce Byte-Pair Encoding mean vocabulary morpheme length (BPE VMML) as a writing system classifier and apply it to 21 corpora spanning gibberish, 15 alphabetic languages, two cipher controls, and two logographic references. Three clusters emerge: gibberish (range 2.0-4.1), alphabetic natural and constructed language (4.06-5.25), and high sub-word regularity (5.62-5.92, containing Chinese Pinyin and Voynich). The Voynich VMML of 5.918 (95% CI 5.77-6.05) lies entirely above the alphabetic cluster maximum (4.51), with no overlap. Five mechanistic hypotheses are tested. The central finding is a systematic suffix inventory analysis: 37 distinct 3-character endings account for 80% of the entire Voynich corpus (vs. 172 for Latin, 109 for German). BPE applied to the last four characters of Voynich tokens alone yields VMML=5.868, matching the full-corpus value and exceeding Latin and German full-corpus values. The suffix inventory is stable across scribal sections (Currier A and B share a core of 9 dominant endings). The rank ordering Gibberish = Pinyin is preserved across all nine BPE parameter configurations tested. All code and data are publicly available.

问 AI

Bookmark

View Full Paper