| 1 | Core GIT Tests |
| 2 | ============== |
| 3 | |
| 4 | This directory holds many test scripts for core GIT tools. The |
| 5 | first part of this short document describes how to run the tests |
| 6 | and read their output. |
| 7 | |
| 8 | When fixing the tools or adding enhancements, you are strongly |
| 9 | encouraged to add tests in this directory to cover what you are |
| 10 | trying to fix or enhance. The later part of this short document |
| 11 | describes how your test scripts should be organized. |
| 12 | |
| 13 | The mechanism that powers this testsuite is directly imported from the |
| 14 | Core GIT Tests, in directory t/ of the git repository. Files are base |
| 15 | on Core GIT version 1.3.0.rc4.g5069. |
| 16 | |
| 17 | |
| 18 | Running Tests |
| 19 | ------------- |
| 20 | |
| 21 | The easiest way to run tests is to say "make -C t". This runs all |
| 22 | the tests. |
| 23 | |
| 24 | *** t0000-basic.sh *** |
| 25 | * ok 1: .git/objects should be empty after git-init-db in an empty repo. |
| 26 | * ok 2: .git/objects should have 256 subdirectories. |
| 27 | * ok 3: git-update-index without --add should fail adding. |
| 28 | ... |
| 29 | * ok 23: no diff after checkout and git-update-index --refresh. |
| 30 | * passed all 23 test(s) |
| 31 | *** t0100-environment-names.sh *** |
| 32 | * ok 1: using old names should issue warnings. |
| 33 | * ok 2: using old names but having new names should not issue warnings. |
| 34 | ... |
| 35 | |
| 36 | Or you can run each test individually from command line, like |
| 37 | this: |
| 38 | |
| 39 | $ sh ./t3001-ls-files-killed.sh |
| 40 | * ok 1: git-update-index --add to add various paths. |
| 41 | * ok 2: git-ls-files -k to show killed files. |
| 42 | * ok 3: validate git-ls-files -k output. |
| 43 | * passed all 3 test(s) |
| 44 | |
| 45 | You can pass --verbose (or -v), --debug (or -d), and --immediate |
| 46 | (or -i) command line argument to the test. |
| 47 | |
| 48 | --verbose:: |
| 49 | This makes the test more verbose. Specifically, the |
| 50 | command being run and their output if any are also |
| 51 | output. |
| 52 | |
| 53 | --debug:: |
| 54 | This may help the person who is developing a new test. |
| 55 | It causes the command defined with test_debug to run. |
| 56 | |
| 57 | --immediate:: |
| 58 | This causes the test to immediately exit upon the first |
| 59 | failed test. |
| 60 | |
| 61 | |
| 62 | Naming Tests |
| 63 | ------------ |
| 64 | |
| 65 | The test files are named as: |
| 66 | |
| 67 | tNNNN-commandname-details.sh |
| 68 | |
| 69 | where N is a decimal digit. |
| 70 | |
| 71 | Here is a proposal for numbering, loosely based on the Core GIT |
| 72 | numbering conventions. |
| 73 | |
| 74 | First two digit tells the particular command we are testing: |
| 75 | |
| 76 | 00 - stgit itself |
| 77 | 10 - branch |
| 78 | 11 - clone |
| 79 | 12 - push |
| 80 | |
| 81 | Third and fourth digit (optionally) tells the particular switch or |
| 82 | group of switches we are testing. |
| 83 | |
| 84 | If you create files under t/ directory (i.e. here) that is not |
| 85 | the top-level test script, never name the file to match the above |
| 86 | pattern. The Makefile here considers all such files as the |
| 87 | top-level test script and tries to run all of them. A care is |
| 88 | especially needed if you are creating a common test library |
| 89 | file, similar to test-lib.sh, because such a library file may |
| 90 | not be suitable for standalone execution. |
| 91 | |
| 92 | |
| 93 | Writing Tests |
| 94 | ------------- |
| 95 | |
| 96 | The test script is written as a shell script. It should start |
| 97 | with the standard "#!/bin/sh" with copyright notices, and an |
| 98 | assignment to variable 'test_description', like this: |
| 99 | |
| 100 | #!/bin/sh |
| 101 | # |
| 102 | # Copyright (c) 2005 Junio C Hamano |
| 103 | # |
| 104 | |
| 105 | test_description='xxx test (option --frotz) |
| 106 | |
| 107 | This test registers the following structure in the cache |
| 108 | and tries to run git-ls-files with option --frotz.' |
| 109 | |
| 110 | |
| 111 | Source 'test-lib.sh' |
| 112 | -------------------- |
| 113 | |
| 114 | After assigning test_description, the test script should source |
| 115 | test-lib.sh like this: |
| 116 | |
| 117 | . ./test-lib.sh |
| 118 | |
| 119 | This test harness library does the following things: |
| 120 | |
| 121 | - If the script is invoked with command line argument --help |
| 122 | (or -h), it shows the test_description and exits. |
| 123 | |
| 124 | - Creates an empty test directory with an empty .git/objects |
| 125 | database and chdir(2) into it. This directory is 't/trash' |
| 126 | if you must know, but I do not think you care. |
| 127 | |
| 128 | - Defines standard test helper functions for your scripts to |
| 129 | use. These functions are designed to make all scripts behave |
| 130 | consistently when command line arguments --verbose (or -v), |
| 131 | --debug (or -d), and --immediate (or -i) is given. |
| 132 | |
| 133 | |
| 134 | End with test_done |
| 135 | ------------------ |
| 136 | |
| 137 | Your script will be a sequence of tests, using helper functions |
| 138 | from the test harness library. At the end of the script, call |
| 139 | 'test_done'. |
| 140 | |
| 141 | |
| 142 | Test harness library |
| 143 | -------------------- |
| 144 | |
| 145 | There are a handful helper functions defined in the test harness |
| 146 | library for your script to use. |
| 147 | |
| 148 | - test_expect_success <message> <script> |
| 149 | |
| 150 | This takes two strings as parameter, and evaluates the |
| 151 | <script>. If it yields success, test is considered |
| 152 | successful. <message> should state what it is testing. |
| 153 | |
| 154 | Example: |
| 155 | |
| 156 | test_expect_success \ |
| 157 | 'git-write-tree should be able to write an empty tree.' \ |
| 158 | 'tree=$(git-write-tree)' |
| 159 | |
| 160 | - test_expect_failure <message> <script> |
| 161 | |
| 162 | This is the opposite of test_expect_success. If <script> |
| 163 | yields success, test is considered a failure. |
| 164 | |
| 165 | Example: |
| 166 | |
| 167 | test_expect_failure \ |
| 168 | 'git-update-index without --add should fail adding.' \ |
| 169 | 'git-update-index should-be-empty' |
| 170 | |
| 171 | - test_debug <script> |
| 172 | |
| 173 | This takes a single argument, <script>, and evaluates it only |
| 174 | when the test script is started with --debug command line |
| 175 | argument. This is primarily meant for use during the |
| 176 | development of a new test script. |
| 177 | |
| 178 | - test_done |
| 179 | |
| 180 | Your test script must have test_done at the end. Its purpose |
| 181 | is to summarize successes and failures in the test script and |
| 182 | exit with an appropriate error code. |
| 183 | |
| 184 | |
| 185 | Tips for Writing Tests |
| 186 | ---------------------- |
| 187 | |
| 188 | As with any programming projects, existing programs are the best |
| 189 | source of the information. However, do _not_ emulate |
| 190 | t0000-basic.sh when writing your tests. The test is special in |
| 191 | that it tries to validate the very core of GIT. For example, it |
| 192 | knows that there will be 256 subdirectories under .git/objects/, |
| 193 | and it knows that the object ID of an empty tree is a certain |
| 194 | 40-byte string. This is deliberately done so in t0000-basic.sh |
| 195 | because the things the very basic core test tries to achieve is |
| 196 | to serve as a basis for people who are changing the GIT internal |
| 197 | drastically. For these people, after making certain changes, |
| 198 | not seeing failures from the basic test _is_ a failure. And |
| 199 | such drastic changes to the core GIT that even changes these |
| 200 | otherwise supposedly stable object IDs should be accompanied by |
| 201 | an update to t0000-basic.sh. |
| 202 | |
| 203 | However, other tests that simply rely on basic parts of the core |
| 204 | GIT working properly should not have that level of intimate |
| 205 | knowledge of the core GIT internals. If all the test scripts |
| 206 | hardcoded the object IDs like t0000-basic.sh does, that defeats |
| 207 | the purpose of t0000-basic.sh, which is to isolate that level of |
| 208 | validation in one place. Your test also ends up needing |
| 209 | updating when such a change to the internal happens, so do _not_ |
| 210 | do it and leave the low level of validation to t0000-basic.sh. |