9 #=======================================================================
10 # File name: DEVANAGA.TXT
12 # Contents: Map (external version) from Mac OS Devanagari
13 # encoding to Unicode 2.1 and later.
15 # Copyright: (c) 1995-2002, 2005 by Apple Computer, Inc., all rights
18 # Contact: charsets@apple.com
22 # c02 2005-Apr-05 Update header comments; add section on
23 # roundtrip considerations. Matches internal
24 # xml <c1.1> and Text Encoding Converter 2.0.
25 # b3,c1 2002-Dec-19 Update URLs. Matches internal utom<b1>.
26 # b02 1999-Sep-22 Update contact e-mail address. Matches
27 # internal utom<b1>, ufrm<b1>, and Text
28 # Encoding Converter version 1.5.
29 # n04 1998-Feb-05 First version; matches internal utom<n9>,
35 # Apple, the Apple logo, and Macintosh are trademarks of Apple
36 # Computer, Inc., registered in the United States and other countries.
37 # Unicode is a trademark of Unicode Inc. For the sake of brevity,
38 # throughout this document, "Macintosh" can be used to refer to
39 # Macintosh computers and "Unicode" can be used to refer to the
42 # Apple Computer, Inc. ("Apple") makes no warranty or representation,
43 # either express or implied, with respect to this document and the
44 # included data, its quality, accuracy, or fitness for a particular
45 # purpose. In no event will Apple be liable for direct, indirect,
46 # special, incidental, or consequential damages resulting from any
47 # defect or inaccuracy in this document or the included data.
49 # These mapping tables and character lists are subject to change.
50 # The latest tables should be available from the following:
52 # <http://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/>
54 # For general information about Mac OS encodings and these mapping
55 # tables, see the file "README.TXT".
60 # Three tab-separated columns;
61 # '#' begins a comment which continues to the end of the line.
62 # Column #1 is the Mac OS Devanagari code or code sequence
63 # (in hex as 0xNN or 0xNN+0xNN)
64 # Column #2 is the corresponding Unicode or Unicode sequence
65 # (in hex as 0xNNNN or 0xNNNN+0xNNNN).
66 # Column #3 is a comment containing the Unicode name or sequence
67 # of names. In some cases an additional comment follows the
70 # The entries are in two sections. The first section is for pairs of
71 # Mac OS Devanagari code points that must be mapped in a special way.
72 # The second section maps individual code points.
74 # Within each section, the entries are in Mac OS Devanagari code order.
76 # Control character mappings are not shown in this table, following
77 # the conventions of the standard UTC mapping tables. However, the
78 # Mac OS Devanagari character set uses the standard control characters
79 # at 0x00-0x1F and 0x7F.
81 # Notes on Mac OS Devanagari:
82 # ---------------------------
84 # This is a legacy Mac OS encoding; in the Mac OS X Carbon and Cocoa
85 # environments, it is only supported via transcoding to and from
88 # Mac OS Devanagari is based on IS 13194:1991 (ISCII-91), with the
89 # addition of several punctuation and symbol characters. However,
90 # Mac OS Devanagari does not support the ATR (attribute) mechanism of
93 # 1. ISCII-91 features in Mac OS Devanagari include:
95 # a) Overloading of nukta
97 # In addition to using the nukta (0xE9) like a combining dot below,
98 # nukta is overloaded to function as a general character modifier.
99 # In this role, certain code points followed by 0xE9 are treated as
100 # a two-byte code point representing a character which may be
101 # rather different than the characters represented by either of
102 # the code points alone. For example, the character DEVANAGARI OM
103 # (U+0950) is represented in ISCII-91 as candrabindu + nukta.
105 # b) Explicit halant and soft halant
107 # A double halant (0xE8 + 0xE8) constitutes an "explicit halant",
108 # which will always appear as a halant instead of causing formation
109 # of a ligature or half-form consonant.
111 # Halant followed by nukta (0xE8 + 0xE9) constitutes a "soft
112 # halant", which prevents formation of a ligature and instead
113 # retains the half-form of the first consonant.
115 # c) Invisible consonant
117 # The byte 0xD9 (called INV in ISCII-91) is an invisible consonant:
118 # It behaves like a consonant but has no visible appearance. It is
119 # intended to be used (often in combination with halant) to display
120 # dependent forms in isolation, such as the RA forms or consonant
123 # d) Extensions for Vedic, etc.
125 # The byte 0xF0 (called EXT in ISCII-91) followed by any byte in
126 # the range 0xA1-0xEE constitutes a two-byte code point which can
127 # be used to represent additional characters for Vedic (or other
128 # extensions); 0xF0 followed by any other byte value constitutes
129 # malformed text. Mac OS Devanagari supports this mechanism, but
130 # does not currently map any of these two-byte code points to
133 # 2. Mac OS Devanagari additions
135 # Mac OS Devanagari adds characters using the code points
136 # 0x80-0x8A and 0x90-0x91 (the latter are some Devanagari additions
139 # 3. Unused code points
141 # The following code points are currently unused, and are not shown
142 # here: 0x8B-0x8F, 0x92-0xA0, 0xEB-0xEF, 0xFB-0xFF. In addition,
143 # 0xF0 is not shown here, but it has a special function as described
146 # Unicode mapping issues and notes:
147 # ---------------------------------
149 # 1. Mapping the byte pairs
151 # If one of the following byte values is encountered when mapping
152 # Mac OS Devanagari text - 0xA1, 0xA6, 0xA7, 0xAA, 0xDB, 0xDC, 0xDF,
153 # 0xE8, or 0xEA - then the next byte (if there is one) should be
154 # examined. If the next byte is 0xE9 - or also 0xE8, if the first
155 # byte was 0xE8 - then the byte pair should be mapped using the
156 # first section of the mapping table below. Otherwise, each byte
157 # should be mapped using the second section of the mapping table
160 # - The Unicode Standard, Version 2.0, specifies how explicit
161 # halant and soft halant should be represented in Unicode;
162 # these mappings are used below.
164 # If the byte value 0xF0 is encountered when mapping Mac OS
165 # Devanagari text, then the next byte should be examined. If there
166 # is no next byte (e.g. 0xF0 at end of buffer), the mapping
167 # process should indicate incomplete character. If there is a next
168 # byte but it is not in the range 0xA1-0xEE, the mapping process
169 # should indicate malformed text. Otherwise, the mapping process
170 # should treat the byte pair as a valid two-byte code point with no
171 # mapping (e.g. map it to QUESTION MARK, REPLACEMENT CHARACTER,
174 # 2. Mapping the invisible consonant
176 # It has been suggested that INV in ISCII-91 should map to ZERO
177 # WIDTH NON-JOINER in Unicode. However, this causes problems with
178 # roundtrip fidelity: The ISCII-91 sequences 0xE8+0xE8 and 0xE8+0xD9
179 # would map to the same sequence of Unicode characters. We have
180 # instead mapped INV to LEFT-TO-RIGHT MARK, which avoids these
183 # 3. Additional loose mappings from Unicode
185 # These are not preserved in roundtrip mappings.
187 # U+0958 0xB3+0xE9 # DEVANAGARI LETTER QA
188 # U+0959 0xB4+0xE9 # DEVANAGARI LETTER KHHA
189 # U+095A 0xB5+0xE9 # DEVANAGARI LETTER GHHA
190 # U+095B 0xBA+0xE9 # DEVANAGARI LETTER ZA
191 # U+095C 0xBF+0xE9 # DEVANAGARI LETTER DDDHA
192 # U+095D 0xC0+0xE9 # DEVANAGARI LETTER RHA
193 # U+095E 0xC9+0xE9 # DEVANAGARI LETTER FA
195 # 4. Roundtrip considerations when mapping to decomposed Unicode
197 # Both ISCII-91 (hence Mac OS Devanagari) and Unicode provide multiple
198 # ways of representing certain Devanagari consonants. For example,
199 # DEVANAGARI LETTER NNNA can be represented in Unicode as the single
200 # character 0x0929 or as the sequence 0x0928 0x093C; similarly, this
201 # consonant can be represented in Mac OS Devanagari as 0xC7 or as the
202 # sequence 0xC6 0xE9. This leads to some roundtrip problems. First
203 # note that we have the following mappings without such problems:
205 # ISCII/ standard decomposition of reverse mapping
206 # Mac OS Unicode mapping standard mapping of decomposition
207 # ------ ----------------------- ---------------- ----------------
208 # 0xC6 0x0928 ... LETTER NA 0x0928 (same) 0xC6
209 # 0xCD 0x092F ... LETTER YA 0x092F (same) 0xCD
210 # 0xCF 0x0930 ... LETTER RA 0x0930 (same) 0xCF
211 # 0xD2 0x0933 ... LETTER LLA 0x0933 (same) 0xD2
212 # 0xE9 0x093C ... SIGN NUKTA 0x093C (same) 0xE9
214 # However, those mappings above cause roundtrip problems for the
215 # the following mappings if they are decomposed:
217 # ISCII/ standard decomposition of reverse mapping
218 # Mac OS Unicode mapping standard mapping of decomposition
219 # ------ ----------------------- ---------------- ----------------
220 # 0xC7 0x0929 ... LETTER NNNA 0x0928 0x093C 0xC6 0xE9
221 # 0xCE 0x095F ... LETTER YYA 0x092F 0x093C 0xCD 0xE9
222 # 0xD0 0x0931 ... LETTER RRA 0x0930 0x093C 0xCF 0xE9
223 # 0xD3 0x0934 ... LETTER LLLA 0x0933 0x093C 0xD2 0xE9
225 # One solution is to use a grouping transcoding hint with the four
226 # decompositions above to mark the decomposed sequence for special
227 # treatment in transcoding. This yields the following mappings to
228 # decomposed Unicode:
231 # Mac OS Unicode mapping
232 # ------ ----------------
233 # 0xC7 0xF860 0x0928 0x093C
234 # 0xCE 0xF860 0x092F 0x093C
235 # 0xD0 0xF860 0x0930 0x093C
236 # 0xD3 0xF860 0x0933 0x093C
238 # Details of mapping changes in each version:
239 # -------------------------------------------
242 # Section 1: Map the following byte pairs as indicated:
243 # (ZWNJ means ZERO WIDTH NON-JOINER, ZWJ means ZERO WIDTH JOINER)
244 # (Also see note about 0xF0 in comments above)
245 # Section 2: Map the remaining bytes as follows:
251 0x00 - 0x7F = 0x0000 -
344 #0xE8+0xE8 = 0x094D+0x200C
345 #0xE8+0xE9 = 0x094D+0x200D
projects / FreeBSD / releng / 9.2.git / blob