2 .\" This file and its contents are supplied under the terms of the
3 .\" Common Development and Distribution License ("CDDL"), version 1.0.
4 .\" You may only use this file in accordance with the terms of version
7 .\" A full copy of the text of the CDDL should have accompanied this
8 .\" source. A copy of the CDDL is also available via the Internet at
9 .\" http://www.illumos.org/license/CDDL.
11 .\" Copyright (c) 2012 by Delphix. All rights reserved.
19 .Nd find, execute, and log the results of tests
26 .Op Fl uxX Ar username
35 .Op Fl uxX Ar username
47 command has three basic modes of operation.
53 processes the arguments provided on
54 the command line, adding them to the list for this run.
57 is an executable file, it is added as a test.
60 is a directory, the behavior depends upon the presence of
64 is specified, the directory is treated as a test group.
71 simply descends into the directory looking for executable files.
72 The tests are then executed, and the results are logged.
77 finds tests in the manner described above.
78 Rather than executing the tests and logging the results, the test configuration
81 which can be used in future invocations, or edited
82 to modify which tests are executed and which options are applied.
83 Options included on the command line with
85 become defaults in the
93 which can specify a series of tests and test groups to be executed.
94 The tests are then executed, and the results are logged.
97 A test group is comprised of a set of executable files, all of which exist in
99 The options specified on the command line or in a
101 apply to individual tests in the group.
102 The exception is options pertaining to pre and post scripts, which act on all
104 Rather than running before and after each test,
105 these scripts are run only once each at the start and end of the test group.
107 The specified tests run serially, and are typically assigned results according
109 Tests that exit zero and non-zero are marked
114 When a pre script fails for a test group, only the post script is executed,
115 and the remaining tests are marked
117 Any test that exceeds
120 is terminated, and marked
123 By default, tests are executed with the credentials of the
126 Executing tests with other credentials is done via
129 be configured to allow execution without prompting for a password.
130 Environment variables from the calling shell are available to individual tests.
131 During test execution, the working directory is changed to
137 will print one line on standard output at the conclusion
138 of each test indicating the test name, result and elapsed time.
139 Additionally, for each invocation of
141 a directory is created using the ISO 8601 date format.
142 Within this directory is a file named
145 test output with timestamps, and a directory for each test.
146 Within the test directories, there is one file each for standard output,
147 standard error and merged output.
148 The default location for the
151 .Pa /var/tmp/test_results .
155 is an INI-style configuration file that describes a test run.
156 The file has one section named
158 which contains configuration option
159 names and their values in
160 .Sy name No = Ar value
162 The values in this section apply to all the subsequent sections,
163 unless they are also specified there, in which case the default is overridden.
164 The remaining section names are the absolute pathnames of files and directories,
165 describing tests and test groups respectively.
166 The legal option names are:
167 .Bl -tag -width "tests = ['filename', …]"
168 .It Sy outputdir No = Ar pathname
169 The name of the directory that holds test logs.
170 .It Sy pre No = Ar script
173 prior to the test or test group.
174 .It Sy pre_user No = Ar username
175 Execute the pre script as
177 .It Sy post No = Ar script
180 after the test or test group.
181 .It Sy post_user No = Ar username
182 Execute the post script as
184 .It Sy quiet No = Sy True Ns | Ns Sy False
187 only the results summary is printed to standard out.
188 .It Sy tests No = [ Ns Ar 'filename' , No … ]
192 Only the basename of the absolute path is required.
193 This option is only valid for test groups, and each
195 must be single quoted.
196 .It Sy timeout No = Ar n
200 .It Sy user No = Ar username
201 Execute the test or test group as
206 .Bl -tag -width "-o outputdir"
210 to be consumed by the run command.
213 Execute no tests, but print a description of each test that would have been run.
215 Enable kmemleak reporting (Linux only)
217 Create test groups from any directories found while searching for tests.
218 .It Fl o Ar outputdir
219 Specify the directory in which to write test results.
223 prior to any test or test group.
227 after any test or test group.
229 Print only the results summary to the standard output.
233 as a failsafe after any test is killed.
235 Execute the failsafe script as
238 Specify a timeout value of
242 Execute tests or test groups as
245 Specify the name of the
249 Execute the pre script as
252 Execute the post script as
258 .It Sy Example 1 : No Running ad-hoc tests .
259 This example demonstrates the simplest invocation of
262 .No % Nm run Ar my-tests
263 Test: /home/jkennedy/my-tests/test-01 [00:02] [PASS]
264 Test: /home/jkennedy/my-tests/test-02 [00:04] [PASS]
265 Test: /home/jkennedy/my-tests/test-03 [00:01] [PASS]
270 Running Time: 00:00:07
271 Percent passed: 100.0%
272 Log directory: /var/tmp/test_results/20120923T180654
274 .It Sy Example 2 : No Creating a Ar runfile No for future use .
275 This example demonstrates creating a
277 with non-default options.
279 .No % Nm run Fl p Ar setup Fl x Ar root Fl g Fl w Ar new-tests.run Ar new-tests
280 .No % Nm cat Pa new-tests.run
289 outputdir = /var/tmp/test_results
291 [/home/jkennedy/new-tests]
292 tests = ['test-01', 'test-02', 'test-03']
projects / FreeBSD / FreeBSD.git / blob