Up to Home Programming

Site Map | Text version

Internationalization Notes

[What is Internationalization?] [Unicode] [Links] [Further notes] [References] [Chronology]

This page was written because I kept forgetting issues involving (& mislaying good web pages about) Internationalization. -- Fred Curtis.

What is Internationalization?

Internationalization - sometimes abbreviated 'i18n' (i followed by 18 letters followed by n) -

Localization - sometimes abbreviated 'l10n' (l followed by 10 letters followed by n) -

Unicode

Unicode (see www.unicode.org) is a standard that attempts to encompass characters from all currently used languages, several historical languages and technical / mathematical symbols. It is synonymous with the Universal Character Set (UCS) of ISO 10646 - the standards are periodically synchronised.

See the WikiPredia entry for Unicode for discussion on the advantages and disadvantages of unicode.

Text Layout

Most European languages are written horizontally left-to-right (LTR) with rows running top-to-bottom (TTB). Hebrew, Arabic & Farsi are written right-to-left (RTL) with rows read TTB. Traditional Japanese and Chinese are written vertically TTB with columns running RTL; both are now commonly written horizontally LTR/TTB. Older Mainland Chinese and Mongolian scripts are written vertically but the columns are read LTR. [More on writing directions; references].

Unicode caters for horizontal LTR and RTL scripts, and regards the vertical rendering of scripts as a formatting style. Sometimes the text directions are mixed, e.g. in an English (LTR) document quoting an Arabic (RTL) phrase or vice-versa, or in Arabic when digits in a number are written LTR. Unicode copes with these mixtures by listing characters in logical order (the order in which they should be read), using special control characters to indicate LTR / RTL rendering on a page -- see "Bidirectional Behaviour" in section 3 of the Unicode standard.

Etemad^[3] has an excellent discussion on vertical/horizontal text layout.

Glyphs vs. Characters

In Unicode, a character may be represented visually in a variety of context-dependent glyphs, e.g.:

• In arabic, the appearance of a letter can depend on its position (initial, medial, final, entire) in a word. Unicode arabic characters correspond to the logical letters and not the different glyphs; it is up to a drawing application to use the appropraite glyph for each letter depending on its position within a word.
• In some variants of Greek, the letter sigma appears differently depending on position (final vs. initial/medial).
• In Japanese, punctuation marks appear differently depending on the direction (vertical, horizontal) of writing.
• Korean has the Hangul, a large number of glyphs which are the written representation of Jamo, the phonetic components of Korean syllables. Hangul can be represented in Unicode 3.0 as one of the 11,172 Unicode Hangul glyphs, or as combinations of far fewer Jamo.
• Over the centuries since Japan and Korea adopted Chinese (Han) characters, some of the characters have diverged slightly in appearance. Unicode has merged each of these orthographic variants into a single code; this makes text searches less language-dependent but means additional language information is needed to display Han characters where appearance is important - for example, in documents containing a mixture of Chinese and Japanese text. This means that there can never be a universal Unicode font unless there is some mechanism within the font to allow for language information. See this Wikipedia article on Han unification for more detail.
• In English, some dipthongs may be represented either as ligatures or separate letters, e.g. 'æ' vs 'a e'.
• An open-parenthesis charater, rendered "(" in a left-to-right context, will be rendered ")" in a right-to-left context.

Several unicode characters may combine to yield a single glyph. The character LATIN A followed by the character COMBINING DIAERESIS is rendered "Ä". The Unicode encodings of Devanagari and Arabic are quite complex to render.

ANSI/POSIX locales

X/Open language independent messages

Links

General:

• Open Directory listings for globalization, internationlization, localization
• WC3Internationalization / Localization
• Markus Kuhn's article on ISO 8601 standard for representing dates & times.

HTML:

• RFC 2070 - HTML Internationalization advocates, among other things, extension of the HTML &#<decimal>; entities to include 16-bit Unicode characters.
• Test pages:
  • Announcement of Tenth International Unicode Conference - this web page contains examples of many differenct scripts
  • Japanese Test Page by Hayato Yamana. In WinNT Netscape 4.75 I can only view this page under the 'Japanese (Auto Detect)' character set

Win32:

• Microsoft Layer for Unicode (MSLU) provides Unicode support on Windows platforms that don't provide their own support, including Windows 98, Windows 95, and Windows Me
• MS Global Software Development - which indicates what a pain it is to develop international applications under older versions of windows.
• Support for ANSI/POSIX locales?
• MSJ article on unicode apps that run under win2000 & win98 - illustrating the scary incompatibilities between windows versions.
• Making Font Selection Multilingual

Fonts:

• Alan Wood's Unicode fonts for Windows computers lists several fonts, including Ariel Unicode MS [Free download (with licence restrictions) from Arial Unicode MS Font for Publisher 2000]

Further notes

Writing directions

Ancient Numidian scripts and traditional (Mangyan) Philippine scripts like Hanunoo were written vertically bottom-to-top (BTT) with columns running left-to-right (LTR). Several ancient languages flirted with with alternating horizontal RTL / LTR rows (called boustrophedon, Greek for "turning like [plowing] oxen"), but seem invariably to have settled into either RTL or LTR. Mayan was written in paired vertical columns reading from left to right and top to bottom in zigzag pattern. Ogham was carved border-like around the edges of rocks and doorways. Foucault^[1] refers to "Mexicans" writing in spirals. Gaur^[2] lists many writing directions, but provides no corresponding examples of scripts for each direction.

Books written in Asian RTL vertical scripts and in Hebrew / Arabic RTL horizontal scripts have pages ordered in reverse to the common European style, so, e.g., the front cover of such a book is where an English reader would expect to find the back cover; another layout issue affected by writing direction is the order in which panels in a cartoon book are read.

There's an excellent overview of writing systems at www.omniglot.com and world languages at www.ethnologue.com.

References

Chronology

• 10 Feb 2002 - Page started prior to internationalizing a tiny Win32 application
• 13 Feb 2002 - Notes on writing directions ; unicode characters vs glyphs
• 8 Jul 2005 - Updated link to Robust Vertical Layout by Elika J. Etemad, now a Unicode technical report. Thanks to Elika for the email!

This page last changed: 5 Aug 2005 [Validate HTML]

Donate free food & land

Anzwers Google Websters Dict. Wiktionary Roget's Thes. Open Directory WikiPedia Google News FTP -- English FTP -- Russian Tellaseek Tucows FileDudes Debian Linux MetaCrawler Alta Vista Yahoo RPM Find Perl Modules Search4Science f2.org - This site for

Feedback by email or Web form