From: Ian Jackson Date: Thu, 22 Feb 2007 17:44:04 +0000 (+0000) Subject: documentation of debian/tests X-Git-Tag: converted-from-bzr~32^3~29 X-Git-Url: http://www.chiark.greenend.org.uk/ucgi/~ianmdlvl/git?a=commitdiff_plain;h=6ea6528a5a7ce3baffec24f0f5ff9ff99a7fe313;p=autopkgtest.git documentation of debian/tests --- diff --git a/doc/README.package-tests b/doc/README.package-tests new file mode 100644 index 0000000..97b2b84 --- /dev/null +++ b/doc/README.package-tests @@ -0,0 +1,120 @@ + AUTOPKGTEST - DEFINING TESTS FOR PACKAGES + ========================================= + +This document describes how the autopkgtest tester core (the program +`adt-run') interprets and executes tests found in source packages. + + + +The source package provides a test metadata file +debian/tests/control. This is a file containing zero or more +RFC822-style stanzas, along these lines: + + Tests: fred bill bongo + Restrictions: needs-root breaks-computer + +This example defines three tests, called `fred', `bill' and `bongo'. +The tests will be performed by executing debian/tests/fred, +debian/tests/bill, etc. Each test program should, on success, exit +with status 0 and print nothing to stderr; if a test exits nonzero, or +prints to stderr, it is considered to have failed. + +The cwd of each test is guaranteed to be the root of the source +package which will have been built. HOWEVER note that the tests must +test the INSTALLED version of the program. Tests may not modify the +source tree (and may not have write access to it). + +If the file to be executed has no execute bits set, chmod a+x is +applied to it (this means that tests can be added in patches without +the need for additional chmod; contrast this with debian/rules). + +During execution of the test, the environment variable TMPDIR will +point to a directory for the execution of this particular test, which +starts empty and will be deleted afterwards (so there is no need for +the test to clean up files left there). + +Tests must declare all applicable Restrictions - see below. + + +The fields which may appear in the RFC822-style stanza are: + + Tests: [ ...] + + This field is mandatory. It names the tests which are defined by + this stanza. All of the other fields in the same stanza apply to + all of the named tests. + + Test names are separated by whitespace and should contain only + characters which are legal in package names, plus `/'. + + Restrictions: [ ...] + + Declares some restrictions or problems with the tests defined in + this stanza. Depending on the test environment capabilities, user + requests, and so on, restrictions can cause tests to be skipped or + can cause the test to be run in a different manner. Tests which + declare unknown restrictions will be skipped. + + Features: [ ...] + + Declares some additional capabilities or good properties of the + tests defined in this stanza. Any unknown features declared will + be completely ignored. Currently there are no defined features + but this field is recognised and ignored for future expansion. + + Depends: + + Declares that the specified packages must be installed for the + test to go ahead. `@' stands for the package(s) generated by the + source package containing the tests; each dependency (strictly, + or-clause, which may contain `|'s but not commas) containing `@' + is replicated once for each such binary package, with the binary + package name substituted for each `@' (but normally `@' should + occur only once and without a version restriction). + + If no Depends field is present, `Depends: @' is assumed. Note + that the source tree's Build-Dependencies are _not_ necessarily + installed, and if you specify any Depends, no binary packages from + the source are installed unless explicitly requested. + + Tests-Directory: + + Replaces the path segment `debian/tests' in the filenames of the + test programs with . Ie, the tests are run by executing + /path/to/built/source/tree//. must be + a relative path and is interpreted starting from the root of the + built source tree. + + This allows tests to live outside the debian/ metadata area, so + that they can more palatably be shared with non-Debian + distributions. + +Any unknown fields will cause the whole stanza to be skipped. + + +The defined Restrictions are: + + rw-build-tree + + The test(s) needs write access to the built source tree (so it may + need to be copied first). Even with this restriction, the test is + not allowed to make any change to the built source tree which (i) + isn't cleaned up by debian/rules clean, (ii) affects the future + results of any test, or (iii) affects binary packages produced by + the build tree in the future. + + breaks-testbed + + The test, when run, is liable to break the testbed system. This + includes causing data loss, causing services that the machine is + running to malfunction, or permanently disabling services; it does + not include causing services on the machine to temporarily fail. + + When this restriction is present the test will usually be skipped + unless the testbed's virtualisation arrangements are sufficiently + powerful, or alternatively if the user explicitly requests. + + needs-root + + The test script must be run as root. +