$FreeBSD$ A brief how-to -------------- Each nsswitch regression test does 2 kinds of actions: 1. It runs a series of queries and tests the correctness of results. There are 2 basic criterias which are used for that: - numbers must be in the correct range - certain pointers should not be NULL 2. It makes a snapshot of the results of all queries that were made. The idea of snapshots is to test that nsswitch-related function calls behave equally (i.e. return same results for the same queries) between system upgrades. When the test is executed and the snapshot is already created, the test will compare the result of each query with the appropriate result from the snapshot and will signal if they differ. In order for nsswitch tests to be as useful as possible you should use them in the following way: Step 1 (before upgrading your system). Build the tests with "make" command and execute them with "prove -v" command. If there are errors during the execution, then appropriate nsswitch functions should be checked. Note, that errors on this state can happen only if the particular function return incorrect data. After the stage 1 a number of "snapshot_[test name]" files will appear in your test folder. Step 2 (after upgrading you system). Rebuild the tests with "make clean; make" command and execute them again with "prove -v" (check that "snapshot_[test name]" files are in the same folder with tests). On this stage regression tests will catch not only the correctness errors, but will also determine the changes in nsswitch functions behaviour. In case of the test failure you will get the following message: To get more details about the error you should do the following: Step 1. Run the test alone with debug output enabled. Step 2. Mail the snapshot file and the debug test output to the freebsd-current@ mailing list. Example testing session for getpwXXX() family of functions ---------------------------------------------------------- 1. make 2. prove -v ./test-getpw.t test-getpw....1..8 ok 1 - getpwnam() ok 2 - getpwuid() ok 3 - getpwent() ok 4 - getpwent() 2-pass ok 5 - building snapshot, if needed ok 6 - getpwnam() snapshot ok 7 - getpwuid() snapshot ok 8 - getpwent() snapshot ok All tests successful. Files=1, Tests=8, 1 wallclock secs ( 0.00 cusr + 0.20 csys = 0.20 CPU) 3. Upgrading the system. 4. make clean; make 5. prove -v ./test-getpw.t (suppose that something has gone wrong) test-getpw....1..8 ok 1 - getpwnam() ok 2 - getpwuid() ok 3 - getpwent() ok 4 - getpwent() 2-pass ok 5 - building snapshot, if needed not ok 6 - getpwnam() snapshot ok 7 - getpwuid() snapshot not ok 8 - getpwent() snapshot FAILED tests 6, 8 Failed 2/8 tests, 75.00% okay Failed 1/1 test scripts, 0.00% okay. 2/8 subtests failed, 75.00% okay. 6. We see that test number 6 failed. According to get-getpw.t, this test is executed like this: do_test 6 'getpwnam() snapshot' '-n -s snapshot_pwd' To determine why the test has failed, we need to run it in debug mode - it means adding "-d" to the options list. 7. ./test-getpw -dn -s snapshot_pwd ... testing getpwnam() with the following data: toor:*:0:0:0::ne-again Superuser:/root::0:4831 testing correctness with the following data: toor:*:0:0:0::Bourne-again Superuser:/root::0:4831 correct not ok 8. Here we can see that the data from snapshot (first "toor" line) and the data received from the getpwnam() call (second "toor" line) are different. It is the reason why the test has failed. If you can't (or don't want) to investigate the problem by yourself, mail the test debug output and the snapshot file to the developers list. Notes on using standalone nsswitch tests ---------------------------------------- All nsswitch tests have [-d] optional command line argument which enables debug output. The debug output can be extremely helpful to determine the cause of test failure. In all nsswitch tests -s command line argument specifies the snapshot file. If this file doesn't exist, it would be built during test execution. If it already exists then it will be used to check the results of particular function calls. This argument is mostly optional, but some tests (test-getaddr and test-getusershell) force it to be specified. test-gethostby and test-getaddr require the list of hostnames, that should be queried during the test. This list must be specified via -f command line argument. Each hostname should occupy exactly one line in the file. Detailed tests description -------------------------- ./test-getaddr - tests the getaddrinfo() function. Usage: test-getaddr [-d] [-46] [-s ] -f -d - enable debug output -4 - force IPv4 usage -6 - force IPv6 usage -s - build/use specified snapshot file -f - use specified hostnames list for testing ./test-getgr Usage: test-getgr -nge2 [-d] [-s ] -d - enable debug output -n - test getgrnam(3) -g - test getgrgid(3) -e - test getgrent(3) -2 - test getgrent(3) in 2-pass mode -s - build/use specified snapshot file ./test-gethostby Usage: test-gethostby -na2i [-o] [-d] [-m46] [-s ] -f -n - test gethostbyname2(3) -a - test gethostbyaddr(3) -2 - test gethostbyname2(3) results to be equal with getaddrinfo(3) results for the similar query -i - test gethostbyaddr(3) results to be equal with getnameinfo(3) results for the similar query -o - use getipnodebyname(3)/getipnodebyaddr(3) for testing instead of gethostbyname2(3)/gethostbyaddr(3) -d - enable debug output -m - force IPv4-to-IPv6 mapping -4 - force IPv4 usage -6 - force IPv6 usage -s - build/use specified snapshot file -f - use specified hostnames list for testing ./test-getproto Usage: test-getproto -nve2 [-d] [-s ] -d - enable debug output -n - test getprotobyname(3) -v - test getprotobynumber(3) -e - test getprotoent(3) -2 - test getprotoent(3) in 2-pass mode -s - build/use specified snapshot file ./test-getpw Usage: test-getpw -nue2 [-d] [-s ] -d - enable debug output -n - test getpwnam(3) -u - test getpwuid(3) -e - test getpwent(3) -2 - test getpwent(3) in 2-pass mode -s - build/use snapshot file ./test-getrpc Usage: test-getrpc -nve2 [-d] [-s ] -d - enable debug output -n - test getrpcbyname(3) -v - test getrpcbynumber(3) -e - test getrpcent(3) -2 - test getrpcent(3) in 2-pass mode -s - build/use specified snapshot file ./test-getserv Usage: test-getserv -npe2 [-d] [-s ] -d - enable debug output -n - test getservbyname(3) -p - test getservbyport(3) -e - test getservent(3) -2 - test getservent(3) in 2-pass mode -s - build/use specified snapshot file ./test-getusershell Usage: test-getusershell [-d] -s -d - enable debug output -s - build/use specified snapshot file