Introduction

Note

Versions of the documentation will be published as revisions of UTN #57, Encoding and Shaping of the Mongolian Script (the Mongolian UTN).

Background

The ISO/IEC 10646 and the Unicode Standard have supported the Mongolian script for approximately two decades, providing code charts (see UTR #54) that delineate the fundamental information concerning characters and their variants. The absence of systematic standards concerning the encoding of the Mongolian script has led to the standardization of related text processing progressing through several stages. The existing framework faces insurmountable practical obstacles.

Early international and national standards lacked standardized shaping rules. Following the encoding of the Mongolian script, the ISO/IEC 10646 and the Unicode Standard have merely catalogued all Mongolian characters and their variants. It has come to light that these lists are incomplete and contain errors. However, there is no specific description of how a sequence of characters is shaped into a sequence of correct variants. The Chinese national standard was also lacking in this regard, as it did not provide such a detailed description.

This situation has two consequences. First, when relevant information technology products undergo acceptance testing related to character sets, they must reference an unpublished, internally maintained document that is constantly being updated. Second, different vendors have developed mutually incompatible shaping rules based on their own interpretations and arbitrary specifications. This has resulted in extremely poor cross-platform compatibility for the Mongolian script.

The new standards establish uniform regulations, but the text is not readable or practical. The issue of fragmentation within the industry has been addressed. Multiple Chinese organizations have joined forces to launch the Project for the Development and Sharing of Digital Resources on the Mongolian Language and Script, with the standardization of shaping rules as its core focus. After extensive coordination, multiple rounds of validation, and reviews, the new Chinese national standard GB/T 25914—2023 (Chinese version, English version) has been released. Revisions to the national standards for Todo, Sibe, and Manchu are currently underway, in line with this standard.

The current national standard text uses the Extended Backus-Naur Form to describe the conditions under which different variants of each character are selected. It lists all expressions in a flat manner, without structuring the text-shaping process as a hierarchical, step-by-step procedure. While this approach is convenient for testers, it creates a very high barrier to entry for developers. Developers find it difficult to determine the order in which the rules apply, making it easy for them to overlook or misinterpret requirements. This directly increases the development and testing costs associated with compliance implementation. It is imperative that we provide supporting interpretive materials to make implementation easier.

The Unicode Standard requires more detailed explanations in many places, but this level of detail goes beyond the scope of the various existing documentation structures. As mentioned above, the Mongolian script combines multiple writing systems. Some characters are shared, while others are not. A given character shares some variants with others, but each uses different ones. This means that descriptions of shaping rules are specific to each writing system. Consequently, neither the Core Specification, the Unicode Character Database, nor Code Charts can handle the complexity of such data structures, which is why they refer to UTN #57 instead. Therefore, UTN #57 needs to address the questions that arise from these data and texts.

Introduction to and explanation of the relevant datasets and toolchains. This UTN provides standardized datasets and an automated toolchain to support the automatic construction and testing of fonts, as well as long-term version maintenance. It meets the needs of font development and maintenance and standards compliance testing. These data are complex and difficult to understand quickly. This UTN is also responsible for explaining the datasets and toolchain it maintains.

Content

The UTN will cover the following four sections, as outlined in the background information above.

A glyph set that is fully aligned with Chinese national standards. This UTN fully encompasses the characters, variants, and ligatures defined in the various Chinese national standards. It also lists the corresponding FVS assignments for each variant to ensure accurate correspondence with the respective Chinese national standards. To this end, this UTN performs phoneme analysis on all characters and grapheme analysis on all variants. It uses the resulting transcriptions to represent all characters and variants. It then creates datasets of the characters and variants included in various Chinese national standards.

A layered, phased process that ensures the final result is consistent with Chinese national standards. UTN is different from national standards. National standards rely on a flattened, expanded BNF notation without process sequencing. UTN reorganizes all variant selection constraints into sequential, hierarchical shaping steps. These steps reflect the runtime execution flow of text shaping. By breaking down a single BNF rule into an actionable, step-by-step workflow, this document reduces the cognitive load on developers and eliminates the rule sequencing issues that previously led to inconsistent rendering outputs across different vendors. This UTN uses test sets developed for the validation of various Chinese national standards. These test sets ensure that the shaping steps defined in this UTN produce final results consistent with the expectations of those standards.

A retrospective analysis of the historical evolution of Mongolian encoding. This UTN provides a historical overview of the encoding of the Mongolian script and examines the evolution of national and international standards regarding its encoding. This clearly explains how the Mongolian script provisions and descriptions in the Unicode Standard and ISO/IEC 10646 are historically constrained. This addresses the expectation that these standards would reference this UTN. Developers should refer to this UTN when supporting Mongolian script or any of its writing systems.

An introduction to the standardized, structured datasets and automated toolchains maintained alongside the UTN version. This UTN maintains datasets, shaping rules, font templates, test modules, and web pages related to the encoding of the Mongolian script. It also provides clear instructions on how to use these resources for the work.

Structure

The specifications provided in this document are oriented towards the first three layers of a five-layer system of text processing techniques, that is:

Character Layer. The code charts of the Unicode Standard and ISO/IEC 10646 specify the code points, names, with annotation about characters and character sequences. The characters specified in this document correspond to the characters in the code charts (see UTR #54).

Glyph Layer. The Unicode Standard and its various supplementary standards further provide the characters with additional behavioral specifications, including character properties (general category, cursive joining type, etc.) and algorithms (normalization, collation, line breaking, text segmentation, bidirectional, vertical text layout, etc.), thus specifying the association of characters with glyphs and showing how this association is affected by factors such as character properties, contextual environment and localization.

Glyph Image Layer. The text shaping engine and the font project file implement the rules for associating characters and glyphs as glyph images in the design space, substitution and positioning relations between glyph images, and text shaping operations performed by the text shaping engine. Based on specific design decisions, there is freedom in constructing glyph image sets and writing shaping rules, so this document cannot directly specify the framework of glyph image sets and shaping rules that each font project file should contain. However, the step-by-step shaping rules provided in this document can directly correspond to the shaping rules in the font project file, and therefore can provide a reference for the construction and conversion of the font project files.

Typesetting Layer. The typesetting application invokes the text shaping engine with the font files to render the glyph images in the typeset pages.

Physical Layer. Devices such as monitors and printers convert typeset pages into physical objects.

Relation

Relationship to the Unicode Standard. This UTN provides a detailed description of the characters and their behavior in the Mongolian script as defined by the Unicode Standard. It also lists the variants contained in each Mongolian character and explains the process by which a sequence of Mongolian characters should be converted into a sequence of glyphs. Note that UTR #54 only provides a historical version of the Code Charts for the Mongolian block in the Unicode Standard. The variant set has changed significantly from this version. Do not use the variant descriptions and FVS assignments provided in this UTR as a reference during implementation.

Relationship to Chinese national standards. This UTN definitively interprets several Chinese national standards. It definitively defines the implementation of the encoding and related data, offering clear guidance on supporting these Chinese national standards. This UTN will be updated promptly as the Chinese national standards are revised.

Content boundaries. The current version of UTN does not specify more detailed requirements for U+1878 MONGOLIAN LETTER CHA WITH TWO DOTS and U+1898 MONGOLIAN LETTER TODO ALI GALI TA. The former concerns the specifications for Buryat Mongolian; as WG2 N4781 indicates, this goes beyond the scope of the current version of the UTN. The latter does not appear in any Mongolian writing system; it is a character established by early standards developers, and the current Chinese national standard supports it solely for compatibility purposes. Therefore, this UTN does not impose any requirements on these two characters.