4 .\" install as magic.4 on USG, magic.5 on V7 or Berkeley systems.
7 .Dt MAGIC 5 "Public Domain"
11 .Nd file command's magic number file
13 This manual page documents the format of the magic file as
16 command, version 4.21.
19 command identifies the type of a file using,
21 a test for whether the file begins with a certain
24 .Pa /usr/share/misc/magic
25 specifies what magic numbers are to be tested for,
26 what message to print if a particular magic number is found,
27 and additional information to extract from the file.
29 Each line of the file specifies a test to be performed.
30 A test compares the data starting at a particular offset
31 in the file with a 1-byte, 2-byte, or 4-byte numeric value or
33 If the test succeeds, a message is printed.
34 The line consists of the following fields:
35 .Bl -tag -width indent
37 A number specifying the offset, in bytes, into the file of the data
38 which is to be tested.
40 The type of the data to be tested.
41 The possible values are:
42 .Bl -tag -width indent
46 A two-byte value (on most systems) in this machine's native byte order.
48 A four-byte value (on most systems) in this machine's native byte order.
51 The string type specification can be optionally followed
55 flag compacts whitespace in the target, which must contain
56 at least one whitespace character.
59 consecutive blanks, the target needs at least
61 consecutive blanks to match.
64 flag treats every blank in the target as an optional blank.
67 flag, specifies case insensitive matching: lowercase characters
68 in the magic match both lower and upper case characters in the
69 targer, whereas upper case characters in the magic, only much
70 uppercase characters in the target.
72 A pascal style string where the first byte is interpreted as the an
78 A four-byte value interpreted as a
82 A four-byte value interpreted as a
84 date, but interpreted as
85 local time rather than UTC.
87 A two-byte value (on most systems) in big-endian byte order.
89 A four-byte value (on most systems) in big-endian byte order.
91 A four-byte value (on most systems) in big-endian byte order,
96 A four-byte value (on most systems) in big-endian byte order,
99 date, but interpreted as local time rather
102 A two-byte unicode (UCS16) string in big-endian byte order.
104 A two-byte value (on most systems) in little-endian byte order.
106 A four-byte value (on most systems) in little-endian byte order.
108 A four-byte value (on most systems) in little-endian byte order,
113 A four-byte value (on most systems) in little-endian byte order,
116 date, but interpreted as local time rather
119 A two-byte unicode (UCS16) string in little-endian byte order.
121 A four-byte value (on most systems) in middle-endian (PDP-11) byte order.
123 A four-byte value (on most systems) in middle-endian (PDP-11) byte order,
128 A four-byte value (on most systems) in middle-endian (PDP-11) byte order,
131 date, but interpreted as local time rather
134 A regular expression match in extended
136 regular expression syntax
138 The type specification can be optionally followed by
140 for case-insensitive matches.
141 The regular expression is always
142 tested against the first
146 is the given offset, thus it
147 is only useful for (single-byte encoded) text.
151 will match the beginning and end of individual lines, respectively,
152 not beginning and end of file.
154 A literal string search starting at the given offset.
155 It must be followed by
156 .Li / Ns Aq Ar number
157 which specifies how many matches shall be attempted (the range).
158 This is suitable for searching larger binary expressions with variable
161 escapes for special characters.
165 The numeric types may optionally be followed by
168 to specify that the value is to be AND'ed with the
169 numeric value before any comparisons are done.
172 to the type indicates that ordered comparisons should be unsigned.
173 .Bl -tag -width indent
175 The value to be compared with the value from the file.
178 is specified in C form; if it is a string, it is specified as a C string
179 with the usual escapes permitted (e.g.\& \en for new-line).
182 may be preceded by a character indicating the operation to be performed.
185 to specify that the value from the file must equal the specified value,
187 to specify that the value from the file must be less than the specified
190 to specify that the value from the file must be greater than the specified
193 to specify that the value from the file must have set all of the bits
194 that are set in the specified value,
196 to specify that the value from the file must have clear any of the bits
197 that are set in the specified value, or
199 the value specified after is negated before tested, or
201 to specify that any value will match.
202 If the character is omitted,
211 specifies that the line matches if the test does
215 Numeric values are specified in C form; e.g.\&
223 For string values, the byte string from the
224 file must match the specified byte string.
232 can be applied to strings.
233 The length used for matching is that of the string argument
235 This means that a line can match any string, and
236 then presumably print that string, by doing
238 (because all strings are greater than the null string).
240 The message to be printed if the comparison succeeds.
244 format specification, the value from the file (with any specified masking
245 performed) is printed using the message as the format string.
248 Some file formats contain additional information which is to be printed
249 along with the file type or need additional tests to determine the true
251 These additional tests are introduced by one or more
253 characters preceding the offset.
256 on the line indicates the level of the test; a line with no
258 at the beginning is considered to be at level 0.
259 Tests are arranged in a tree-like hierarchy:
260 If a the test on a line at level
262 succeeds, all following tests at level
264 are performed, and the messages printed if the tests succeed, until a line
268 For more complex files, one can use empty messages to get just the
269 "if/then" effect, in the following way:
270 .Bd -literal -offset indent
272 >0x18 leshort <0x40 MS-DOS executable
273 >0x18 leshort >0x3f extended PC executable (e.g., MS Windows)
276 Offsets do not need to be constant, but can also be read from the file
278 If the first character following the last
282 then the string after the parenthesis is interpreted as an indirect offset.
283 That means that the number after the parenthesis is used as an offset in
285 The value at that offset is read, and is used again as an offset
287 Indirect offsets are of the form:
288 .Em (x[.[bslBSL]][+\-][y]) .
291 is used as an offset in the file.
292 A byte, short or long is read at that offset
296 The capitalized types interpret the number as a big endian value, whereas
297 a small letter versions interpret the number as a little endian value;
300 type interprets the number as a middle endian (PDP-11) value.
301 To that number the value of
303 is added and the result is used as an offset in the file.
305 if one is not specified is long.
307 That way variable length structures can be examined:
308 .Bd -literal -offset indent
309 # MS Windows executables are also valid MS-DOS executables
311 >0x18 leshort <0x40 MZ executable (MS-DOS)
312 # skip the whole block below if it is not an extended executable
314 >>(0x3c.l) string PE\e0\e0 PE executable (MS-Windows)
315 >>(0x3c.l) string LX\e0\e0 LX executable (OS/2)
318 This strategy of examining has one drawback: You must make sure that
319 you eventually print something, or users may get empty output (like, when
320 there is neither PE\e0\e0 nor LE\e0\e0 in the above example).
322 If this indirect offset cannot be used as-is, there are simple calculations
324 .Em [+-*/%&|^]<number>
325 inside parentheses allows one to modify
326 the value read from the file before it is used as an offset:
327 .Bd -literal -offset indent
328 # MS Windows executables are also valid MS-DOS executables
330 # sometimes, the value at 0x18 is less that 0x40 but there's still an
331 # extended executable, simply appended to the file
333 >>(4.s*512) leshort 0x014c COFF executable (MS-DOS, DJGPP)
334 >>(4.s*512) leshort !0x014c MZ executable (MS-DOS)
337 Sometimes you do not know the exact offset as this depends on the length or
338 position (when indirection was used before) of preceding fields.
340 specify an offset relative to the end of the last uplevel field using
342 as a prefix to the offset:
343 .Bd -literal -offset indent
346 >>(0x3c.l) string PE\e0\e0 PE executable (MS-Windows)
347 # immediately following the PE signature is the CPU type
348 >>>&0 leshort 0x14c for Intel 80386
349 >>>&0 leshort 0x184 for DEC Alpha
352 Indirect and relative offsets can be combined:
353 .Bd -literal -offset indent
356 >>(4.s*512) leshort !0x014c MZ executable (MS-DOS)
357 # if it's not COFF, go back 512 bytes and add the offset taken
358 # from byte 2/3, which is yet another way of finding the start
359 # of the extended executable
360 >>>&(2.s-514) string LE LE executable (MS Windows VxD driver)
363 Or the other way around:
364 .Bd -literal -offset indent
367 >>(0x3c.l) string LE\e0\e0 LE executable (MS-Windows)
368 # at offset 0x80 (-4, since relative offsets start at the end
369 # of the uplevel match) inside the LE header, we find the absolute
370 # offset to the code area, where we look for a specific signature
371 >>>(&0x7c.l+0x26) string UPX \eb, UPX compressed
375 .Bd -literal -offset indent
378 >>(0x3c.l) string LE\e0\e0 LE executable (MS-Windows)
379 # at offset 0x58 inside the LE header, we find the relative offset
380 # to a data area where we look for a specific signature
381 >>>&(&0x54.l-3) string UNACE \eb, ACE self-extracting archive
384 Finally, if you have to deal with offset/length pairs in your file, even the
385 second value in a parenthesed expression can be taken from the file itself,
386 using another set of parentheses.
387 Note that this additional indirect offset
388 is always relative to the start of the main indirect offset.
389 .Bd -literal -offset indent
392 >>(0x3c.l) string PE\e0\e0 PE executable (MS-Windows)
393 # search for the PE section called ".idata"...
394 >>>&0xf4 search/0x140 .idata
395 # ...and go to the end of it, calculated from start+length;
396 # these are located 14 and 10 bytes after the section name
397 >>>>(&0xe.l+(-4)) string PK\e3\e4 \eb, ZIP self-extracting archive
416 are system-dependent; perhaps they should be specified as a number
417 of bytes (2B, 4B, etc),
418 since the files being recognized typically come from
419 a system on which the lengths are invariant.
422 .Pa /usr/share/misc/magic
424 .Pa /usr/share/misc/magic.mgc
427 .Dq Li "cd /usr/share/misc && file -C -m magic"
432 .\" From: guy@sun.uucp (Guy Harris)
433 .\" Newsgroups: net.bugs.usg
434 .\" Subject: /etc/magic's format isn't well documented
435 .\" Message-ID: <2752@sun.uucp>
436 .\" Date: 3 Sep 85 08:19:07 GMT
437 .\" Organization: Sun Microsystems, Inc.
440 .\" Here's a manual page for the format accepted by the "file" made by adding
441 .\" the changes I posted to the S5R2 version.
443 .\" Modified for Ian Darwin's version of the file command.
444 .\" @(#)$Id: magic.man,v 1.30 2006/02/19 18:16:03 christos Exp $