Coverage Testing
Coverage testing is a technique used to see how much of the code is exercised by the unit tests. It is important to remember that a high level of coverage is a necessary but not sufficient condition for having effective tests. Coverage testing can be useful for identifying whole functions or classes which are not tested, or for finding certain conditions which are never tested.
The [coverage.py] utility can be used for coverage testing. To install it, simply download the file from the website and place it somewhere on your path, ensuring it has execute rights. To run the coverage testing there are three steps.
1. Delete the results from any previous runs.
coverage.py -e
2. Execute the unit tests. The syntax for doing this exactly the same as for running any of the unit tests as described [here] except with coverage.py -x in front of it.
coverage.py -x ./test --slow
3. Examine the coverage results.
PYTHON_PATH=lib FILTER='tests\|neuroimaging/utils\|sandbox' coverage.py -r -m $(find $PYTHON_PATH/neuroimaging -name "*py" | grep -v $FILTER)
This will product a report listing all the python files, the number of lines of code, number of lines executed, percentage of lines executed and a list of lines which were not covered by the tests. An example of the output is shown below
[jarrod@bic127 nipy-trunk]$ python coverage.py -r -m $(find $PYTHON_PATH/neuroimaging -name "*py" | grep -v $FILTER) Name Stmts Exec Cover Missing ------------------------------------------------------------------------------------- lib/neuroimaging/__init__ 42 32 76% 80, 92-95, 98, 101-104 lib/neuroimaging/algorithms/__init__ 0 0 100% lib/neuroimaging/algorithms/fwhm 246 25 10% 23-32, 35-48, 51, 54, 57-71, 77-93, 97, 108-126, 129-136, 140-170, 173-177, 180-188, 192-278, 281-284, 293-295, 299-351 lib/neuroimaging/algorithms/interpolation 48 40 83% 18, 25, 33, 53, 79-82 lib/neuroimaging/algorithms/kernel_smooth 92 14 15% 21-28, 31-35, 44-63, 67-113, 116-118, 121-135 lib/neuroimaging/algorithms/onesample 74 26 35% 24-57, 62-77, 81, 92-93, 99-101, 104, 109, 112, 117, 120, 125, 128, 133, 136 lib/neuroimaging/algorithms/regression 78 20 25% 15-21, 29-35, 38, 41-57, 60, 63-67, 74-77, 80, 83-97, 100, 107-111, 114 lib/neuroimaging/algorithms/statistics/__init__ 1 1 100% lib/neuroimaging/algorithms/statistics/classification 4 0 0% 1-5 lib/neuroimaging/algorithms/statistics/nlsmodel 50 39 78% 39, 43-45, 48-50, 64, 70-73 lib/neuroimaging/algorithms/statistics/onesample 106 17 16% 12, 22, 25, 30-37, 40-43, 46-56, 60-96, 100-141, 148-149, 156, 164-181 lib/neuroimaging/algorithms/statistics/regression 45 14 31% 14-15, 23, 30-41, 53-60, 63-64, 67, 70, 73, 76-82 lib/neuroimaging/algorithms/statistics/rft 314 245 78% 18, 52, 130, 136, 142, 146-147, 167, 216-219, 247, 250, 255, 266-268, 280, 283-288, 309-324, 329, 332-335, 351, 356, 438, 441, 519, 527-533, 546, 553-558, 625, 678-687, 690-691, 695-699 lib/neuroimaging/algorithms/utils 101 6 5% 9, 15, 20-81, 85-153 lib/neuroimaging/core/__init__ 1 1 100% lib/neuroimaging/core/image/__init__ 0 0 100% lib/neuroimaging/core/image/base_image 22 21 95% 30 lib/neuroimaging/core/image/image 89 89 100% lib/neuroimaging/core/image/intent 229 0 0% 1-396 lib/neuroimaging/core/image/roi 116 0 0% 8-217 lib/neuroimaging/core/reference/__init__ 2 2 100% lib/neuroimaging/core/reference/axis 64 63 98% 250 lib/neuroimaging/core/reference/coordinate_system 60 60 100% lib/neuroimaging/core/reference/grid 139 111 79% 62, 82, 114-133, 170, 176, 183, 191, 199-203 lib/neuroimaging/core/reference/iterators 154 154 100% lib/neuroimaging/core/reference/mapping 169 169 100% lib/neuroimaging/core/reference/mni 9 7 77% 11, 22 lib/neuroimaging/core/reference/slices 50 50 100% lib/neuroimaging/data_io/__init__ 130 92 70% 40-47, 65-68, 82-83, 85, 96-97, 103, 109-117, 121-122, 126, 134-135, 159-160, 167-168, 189, 218-220 lib/neuroimaging/data_io/formats/__init__ 63 47 74% 71, 80, 85, 96, 111, 119, 126, 133, 137, 171-172, 182-186 lib/neuroimaging/data_io/formats/analyze 157 145 92% 142, 211, 272, 286, 311-321, 326 lib/neuroimaging/data_io/formats/binary 106 68 64% 60, 70-73, 102, 106, 114-115, 121-123, 128-139, 143-145, 150-158, 162-168, 175 lib/neuroimaging/data_io/formats/ecat7 216 79 36% 242-307, 313-317, 325, 332, 343-351, 357-358, 363, 376-401, 404-427, 434, 445-493, 500-501, 507, 514-520, 526-529, 536, 543, 551, 557, 563-566, 571-572, 576 lib/neuroimaging/data_io/formats/nifti1 198 183 92% 239, 249, 267-271, 341, 386, 404-408, 414, 449 lib/neuroimaging/data_io/formats/nifti1_ext 30 12 40% 24-624, 628-630 lib/neuroimaging/data_io/formats/utils 57 49 85% 58-62, 66-69, 72 lib/neuroimaging/defines 22 13 59% 14-17, 49-52, 56 lib/neuroimaging/modalities/__init__ 0 0 100% lib/neuroimaging/modalities/fmri/__init__ 59 36 61% 24, 41-60, 83-85, 100, 113-115 lib/neuroimaging/modalities/fmri/filters 115 70 60% 21-32, 48-55, 62-69, 93-95, 113, 116, 142-145, 148-149, 174-175, 178, 199-200, 203-204, 207 lib/neuroimaging/modalities/fmri/fmristat/__init__ 1 1 100% lib/neuroimaging/modalities/fmri/fmristat/delay 189 48 25% 47-85, 102-106, 115-173, 177-193, 197-242, 245-247, 250-254, 259-265, 282, 327-346 lib/neuroimaging/modalities/fmri/fmristat/invert 40 35 87% 31, 50, 53, 56, 59 lib/neuroimaging/modalities/fmri/fmristat/utils 199 28 14% 31-45, 48-51, 65-100, 103-109, 113-138, 142-163, 167-196, 209-236, 254-331, 336-343 lib/neuroimaging/modalities/fmri/functions 185 149 80% 59-60, 65, 76, 82-84, 87, 165-175, 178-193, 209-216 lib/neuroimaging/modalities/fmri/hrf 73 62 84% 59-63, 105, 137-140, 151 lib/neuroimaging/modalities/fmri/pca 150 31 20% 41-50, 53, 56, 59-68, 82-128, 136-184, 206-209, 216-227, 241-289 lib/neuroimaging/modalities/fmri/protocol 212 173 81% 53, 56, 133, 136-142, 159-160, 175-194, 245, 260, 270-277, 284, 299, 318-320, 347 lib/neuroimaging/modalities/fmri/regression 108 31 28% 27, 35-43, 46, 54-62, 65-77, 80-84, 88, 94-99, 102-115, 118, 123-125, 131-133, 137-149, 152-163 lib/neuroimaging/modalities/fmri/utils 54 51 94% 95-96, 99 lib/neuroimaging/ui/__init__ 0 0 100% lib/neuroimaging/ui/tools/__init__ 20 0 0% 3-36 lib/neuroimaging/ui/tools/analyzeinfo 27 0 0% 1-40 lib/neuroimaging/ui/visualization/__init__ 0 0 100% lib/neuroimaging/ui/visualization/cmap/__init__ 20 19 95% 25 lib/neuroimaging/ui/visualization/montage 76 24 31% 34-53, 57-94, 100-101, 107-122 lib/neuroimaging/ui/visualization/multiplot 54 21 38% 51-88 lib/neuroimaging/ui/visualization/qview 11 0 0% 1-14 lib/neuroimaging/ui/visualization/slices 140 86 61% 23, 37-41, 76-80, 86-88, 131-137, 141-150, 160, 164, 176-183, 186, 212, 217-225, 237-246 lib/neuroimaging/ui/visualization/viewer 191 35 18% 40-44, 47-51, 69-114, 117-131, 135-193, 196-206, 209, 213, 217, 221-237, 240-255, 259-290, 294-298 ------------------------------------------------------------------------------------- TOTAL 5208 2794 53%
