1 .\" $OpenBSD: sort.1,v 1.31 2007/08/21 21:22:37 millert Exp $
4 .\" Copyright (c) 1991, 1993
5 .\" The Regents of the University of California. All rights reserved.
7 .\" This code is derived from software contributed to Berkeley by
8 .\" the Institute of Electrical and Electronics Engineers, Inc.
10 .\" Redistribution and use in source and binary forms, with or without
11 .\" modification, are permitted provided that the following conditions
13 .\" 1. Redistributions of source code must retain the above copyright
14 .\" notice, this list of conditions and the following disclaimer.
15 .\" 2. Redistributions in binary form must reproduce the above copyright
16 .\" notice, this list of conditions and the following disclaimer in the
17 .\" documentation and/or other materials provided with the distribution.
18 .\" 3. Neither the name of the University nor the names of its contributors
19 .\" may be used to endorse or promote products derived from this software
20 .\" without specific prior written permission.
22 .\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
23 .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
24 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
25 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
26 .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
27 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
28 .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
29 .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
30 .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
31 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
34 .\" @(#)sort.1 8.1 (Berkeley) 6/6/93
41 .Nd sort or merge records (lines) of text and binary files
45 .Op Fl bcCdfghiRMmnrsuVz
47 .Op Fl k\ \& Ar field1 Op , Ar field2
62 utility sorts text and binary files by lines.
63 A line is a record separated from the subsequent record by a
64 newline (default) or NUL \'\\0\' character (-z option).
65 A record can contain any printable or unprintable characters.
66 Comparisons are based on one or more sort keys extracted from
67 each line of input, and are performed lexicographically,
68 according to the current locale's collating rules and the
69 specified command-line options that can tune the actual
71 By default, if keys are not given,
73 uses entire lines for comparison.
75 The command line options are as follows:
77 .It Fl c, Fl Fl check, Fl C, Fl Fl check=silent|quiet
78 Check that the single input file is sorted.
79 If the file is not sorted,
81 produces the appropriate error messages and exits with code 1,
90 This is a "silent" version of
92 .It Fl m , Fl Fl merge
94 The input files are assumed to be pre-sorted.
95 If they are not sorted the output order is undefined.
96 .It Fl o Ar output , Fl Fl output Ns = Ns Ar output
97 Print the output to the
99 file instead of the standard output.
100 .It Fl S Ar size, Fl Fl buffer-size Ns = Ns Ar size
103 for the maximum size of the memory buffer.
104 Size modifiers %,b,K,M,G,T,P,E,Z,Y can be used.
105 If a memory limit is not explicitly specified,
107 takes up to about 90% of available memory.
108 If the file size is too big to fit into the memory buffer,
109 the temporary disk files are used to perform the sorting.
110 .It Fl T Ar dir , Fl Fl temporary-directory Ns = Ns Ar dir
111 Store temporary files in the directory
113 The default path is the value of the environment variable
120 .It Fl u , Fl Fl unique
122 Suppress all lines that have a key that is equal to an already
124 This option, similarly to
126 implies a stable sort.
132 also checks that there are no lines with duplicate keys.
135 This option maintains the original record order of records that have
137 This is a non-standard feature, but it is widely accepted and used.
139 Print the version and silently exits.
141 Print the help text and silently exits.
144 The following options override the default ordering rules.
145 When ordering options appear independently of key field
146 specifications, they apply globally to all sort keys.
147 When attached to a specific key (see
149 the ordering options override all global ordering options for
150 the key they are attahced to.
151 .Bl -tag -width indent
152 .It Fl b, Fl Fl ignore-leading-blanks
153 Ignore leading blank characters when comparing lines.
154 .It Fl d , Fl Fl dictionary-order
155 Consider only blank spaces and alphanumeric characters in comparisons.
156 .It Fl f , Fl Fl ignore-case
157 Convert all lowercase characters to their uppercase equivalent
158 before comparison, that is, perform case-independent sorting.
159 .It Fl g, Fl Fl general-numeric-sort, Fl Fl sort=general-numeric
160 Sort by general numerical value.
163 this option handles general floating points, which have a much
164 permissive format than those allowed by
166 but it has a significant performance drawback.
167 .It Fl h, Fl Fl human-numeric-sort, Fl Fl sort=human-numeric
168 Sort by numerical value, but take into account the SI suffix,
170 Sort first by numeric sign (negative, zero, or
171 positive); then by SI suffix (either empty, or `k' or `K', or one
172 of `MGTPEZY', in that order); and finally by numeric value.
173 The SI suffix must immediately follow the number.
174 For example, '12345K' sorts before '1M', because M is "larger" than K.
175 This sort option is useful for sorting the output of a single invocation
180 options (human-readable).
181 .It Fl i , Fl Fl ignore-nonprinting
182 Ignore all non-printable characters.
183 .It Fl M, Fl Fl month-sort, Fl Fl sort=month
184 Sort by month abbreviations.
185 Unknown strings are considered smaller than the month names.
186 .It Fl n , Fl Fl numeric-sort, Fl Fl sort=numeric
187 Sort fields numerically by arithmetic value.
188 Fields are supposed to have optional blanks in the beginning, an
189 optional minus sign, zero or more digits (including decimal point and
190 possible thousand separators).
191 .It Fl R, Fl Fl random-sort, Fl Fl sort=random
192 Sort by a random order.
193 This is a random permutation of the inputs except that
194 the equal keys sort together.
195 It is implemented by hashing the input keys and sorting
197 The hash function is choosen randomly.
198 The hash function is randomized by
200 content, or by file content if it is specified by
201 .Fl Fl random-source .
202 Even if multiple sort fields are specified,
203 the same random hash function is used for all of them.
204 .It Fl r , Fl Fl reverse
205 Sort in reverse order.
206 .It Fl V, Fl Fl version-sort
207 Sort version numbers.
208 The input lines are treated as file names in form
209 PREFIX VERSION SUFFIX, where SUFFIX matches the regular expression
210 "(\.([A-Za-z~][A-Za-z0-9~]*)?)*".
211 The files are compared by their prefixes and versions (leading
212 zeros are ignored in version numbers, see example below).
213 If an input string does not match the pattern, then it is compared
214 using the byte compare function.
215 All string comparisions are performed in C locale, the locale
216 environment setting is ignored.
217 .Bl -tag -width indent
219 .It $ ls sort* | sort -V
225 .It sort-1.024.003.tgz
226 .It sort-1.024.07.tgz
227 .It sort-1.024.009.tgz
231 The treatment of field separators can be altered using these options:
232 .Bl -tag -width indent
233 .It Fl b , Fl Fl ignore-leading-blanks
234 Ignore leading blank space when determining the start
235 and end of a restricted sort key (see
240 is specified before the first
242 option, it applies globally to all key specifications.
245 can be attached independently to each
247 argument of the key specifications.
251 .Fl k\ \& Ar field1 Op , Ar field2 , Fl Fl key Ns = Ns Ar field1 Op , Ar field2
254 Define a restricted sort key that has the starting position
256 and optional ending position
261 option may be specified multiple times,
262 in which case subsequent keys are compared when earlier keys compare equal.
265 option replaces the obsolete options
269 but the old notation is also supported.
270 .It Fl t Ar char , Fl Fl field-separator Ns = Ns Ar char
273 as a field separator character.
276 is not considered to be part of a field when determining key offsets.
279 is significant (for example,
281 delimits an empty field).
284 is not specified, the default field separator is a sequence of
285 blank space characters, and consecutive blank spaces do
287 delimit an empty field, however, the initial blank space
289 considered part of a field when determining key offsets.
290 To use NUL as field separator, use
293 .It Fl z , Fl Fl zero-terminated
294 Use NUL as record separator.
295 By default, records in the files are supposed to be separated by
296 the newline characters.
297 With this option, NUL (\'\\0\') is used as a record separator character.
301 .Bl -tag -width indent
302 .It Fl Fl batch-size Ns = Ns Ar num
303 Specify maximum number of files that can be opened by
306 This option affects behavior when having many input files or using
308 The default value is 16.
309 .It Fl Fl compress-program Ns = Ns Ar PROGRAM
310 Use PROGRAM to compress temporary files.
311 PROGRAM must compress standard input to standard output, when called
313 When called with argument
315 it must decompress standard input to standard output.
318 must exit with error.
319 An example of PROGRAM that can be used here is bzip2.
320 .It Fl Fl random-source Ns = Ns Ar filename
321 In random sort, the file content is used as the source of the 'seed' data
322 for the hash function choice.
323 Two invocations of random sort with the same seed data will use
324 the same hash function and will produce the same result if the input is
330 Print some extra information about the sorting process to the
332 %%THREADS%%.It Fl Fl parallel
333 %%THREADS%%Set the maximum number of execution threads.
334 %%THREADS%%Default number equals to the number of CPUs.
335 .It Fl Fl files0-from Ns = Ns Ar filename
336 Take the input file list from the file
338 The file names must be separated by NUL
339 (like the output produced by the command "find ... -print0").
341 Try to use radix sort, if the sort specifications allow.
342 The radix sort can only be used for trivial locales (C and POSIX),
343 and it cannot be used for numeric or month sort.
344 Radix sort is very fast and stable.
347 This is a universal algorithm that can always be used,
348 but it is not always the fastest.
350 Try to use quick sort, if the sort specifications allow.
351 This sort algorithm cannot be used with
356 Try to use heap sort, if the sort specifications allow.
357 This sort algorithm cannot be used with
362 Try to use file memory mapping system call.
363 It may increase speed in some cases.
366 The following operands are available:
367 .Bl -tag -width indent
369 The pathname of a file to be sorted, merged, or checked.
372 operands are specified, or if a
376 the standard input is used.
379 A field is defined as a maximal sequence of characters other than the
380 field separator and record separator (newline by default).
381 Initial blank spaces are included in the field unless
384 the first blank space of a sequence of blank spaces acts as the field
385 separator and is included in the field (unless
388 For example, all blank spaces at the beginning of a line are
389 considered to be part of the first field.
391 Fields are specified by the
393 .Fl k\ \& Ar field1 Op , Ar field2
398 is missing, the end of the key defaults to the end of the line.
407 and can be followed by one or more of the modifiers
412 which correspond to the options discussed above.
415 is specified it applies only to
419 where it is specified while the rest of the modifiers
420 apply to the whole key field regardless if they are
428 position specified by
430 is interpreted as the
432 character from the beginning of the
441 indicating the first character of the
447 is counted from the first non-blank character in the
451 refers to the first non-blank character in the
457 character from the beginning of the line;
460 is greater than the length of the line, the field is taken to be empty.
463 positions are always counted from the field beginning, even if the field
464 is shorter than the number of specified positions.
465 Thus, the key can really start from a position in a subsequent field.
469 position specified by
471 is interpreted as the
473 character (including separators) from the beginning of the
478 indicates the last character of the
483 designates the end of a line.
486 is synonymous with the obsolete option
487 .Cm \(pl Ns Ar v-\&1.x-\&1
494 .Cm \(pl Ns Ar v-\&1.x-\&1
499 option is still supported, except for
507 Locale settings to be used to determine the collation for
510 Locale settings to be used to case conversion and classification
511 of characters, that is, which characters are considered
514 Locale settings that determine the language of output messages
519 Locale settings that determine the number format used in numeric sort.
521 Locale settings that determine the month format used in month sort.
523 Locale settings that override all of the above locale settings.
524 This environment variable can be used to set all these settings
525 to the same value at once.
527 Used as a last resort to determine different kinds of locale-specific
528 behavior if neither the respective environment variable, nor
531 %%NLS%%.It Ev NLSPATH
532 %%NLS%%Path to NLS catalogs.
534 Path to the directory in which temporary files will be stored.
537 may be overridden by the
540 .It Ev GNUSORT_NUMERIC_COMPATIBILITY
543 will not override the locale numeric symbols, that is, thousand
544 separators and decimal separators.
545 By default, if we specify
547 with the same symbol as the thousand separator or decimal point,
548 the symbol will be treated as the field separator.
549 Older behavior was less definite; the symbol was treated as both field
550 separator and numeric separator, simultaneously.
551 This environment variable enables the old behavior.
554 .Bl -tag -width Pa -compact
555 .It Pa /var/tmp/.bsdsort.PID.*
558 Default seed file for the random sort.
563 utility shall exit with one of the following values:
565 .Bl -tag -width flag -compact
567 Successfully sorted the input files or if used with
571 the input file already met the sorting criteria.
573 On disorder (or non-uniqueness) with the
588 utility is compliant with the
594 are extensions to the POSIX specification.
596 All long options are extensions to the specification, some of them are
597 provided for compatibility with GNU versions and some of them are
600 The old key notations
604 come from older versions of
606 and are still supported but their use is highly discouraged.
610 command first appeared in
613 Gabor Kovesdan <gabor@FreeBSD.org>,
615 Oleg Moskalenko <mom040267@gmail.com>
617 This implementation of
619 has no limits on input line length (other than imposed by available
620 memory) or any restrictions on bytes allowed within lines.
622 The performance depends highly on locale settings,
623 efficient choice of sort keys and key complexity.
624 The fastest sort is with locale C, on whole lines,
627 In general, locale C is the fastest, then single-byte
628 locales follow and multi-byte locales as the slowest but
629 the correct collation order is always respected.
630 As for the key specification, the simpler to process the
631 lines the faster the search will be.
633 When sorting by arithmetic value, using
635 results in much better performance than
637 so its use is encouraged