Changes between Version 11 and Version 12 of TestingGuidelines

Timestamp:

01/21/07 06:03:22 (6 years ago)

Author:

jarrod.millman

Comment:

--

TestingGuidelines

@@ -28 +28 @@
 Ideally, every Python code, extension module, or subpackage in the !SciPy package directory should have a corresponding {{{test_<name>.py}}} file.  This file should define classes derived from the {{{NumpyTestCase}}} (or the {{{unittest.TestCase}}}) class and have names starting with {{{test_}}}. The methods of these classes whose names start with {{{bench_}}}, {{{check_}}}, or {{{test_}}}, are passed on to the {{{unittest}}} machinery. In addition, the value of the first optional argument of these methods determines the level of the corresponding test. (Default level is 1.)
 
-=== test_yyy.py ===
+=== {{{test_yyy.py}}} ===
 Suppose you have a !SciPy module {{{scipy/xxx/yyy.py}}} containing a function {{{zzz()}}}. To test this you would start by creating a test module called {{{test_yyy.py}}}. This test file should include a class that tests {{{zzz()}}}. The test class has test methods that test various aspects of {{{zzz()}}}. Within these test methods, {{{assert()}}} is used to test whether some case is true. If the assert fails, the test fails. The {{{NumpyTest().run()}}} function actually runs the test suite. A minimal example of a {{{test_yyy.py}}} file that implements tests for a Scipy package module {{{scipy.xxx.yyy}}}, is shown below:
 {{{
@@ -54 +54 @@
 {{{NumpyTestCase}}} is derived from {{{unittest.TestCase}}} and it basically only implements an additional method {{{measure(self, code_str, times=1)}}}. Note that all classes that are inherited from {{{TestCase}}} class, are picked up by the test runner when using {{{testall}}}. For more detailed information on defining test classes see the official documentation for the [http://docs.python.org/lib/module-unittest.html Python Unit testing framework].
 
-=== The ``tests/`` directory ===
+=== {{{tests/}}} ===
 Rather than keeping the code and the tests in the same directory, we put all the tests for a given subpackage in a {{{tests/}}} subdirectory. For our example, if it doesn't all ready exist you will need to create a {{{tests/}}} directory in {{{scipy/xxx/}}}. So the path for {{{test_yyy.py}}} is {{{scipy/xxx/tests/test_yyy.py}}}.
 
@@ -67 +67 @@
 }}}
 
-=== The {{{__init__.py}}} file ===
+=== {{{__init__.py}}} ===
 Usually however, adding the {{{tests/}}} directory to the python path isn't desirable. Instead it would better to invoke the test straight from the module {{{xxx}}}. To this end, simply place the following two lines at the end of your package's {{{__init__.py}}} file:
 {{{
@@ -85 +85 @@
 # your tests are included and run automatically!
 }}}
-
-== {{{numpy.testing}}} ==
-
-This package holds a few helper routines. Several were discussed in the previous section. A couple of others are something of a verbose {{{assert()}}} function that give feedback as to what error occured. One of them, {{{assert_array_equal()}}}, is useful for comparing values in two arrays. The library of assert functions should grow to test for other general cases.
-
-=== Testing Wishes ===
-
- * '''Speed testing:''' It'd be nice to have some speed testing facilities so that we could keep track of whether new code hurts or helps our speed. It would provide some simple benchmarking facilities also. To do this correctly, we'd need something like {{{start_timer()}}} and {{{stop_timer()}}} calls that could be placed in the {{{check_xxx()}}} methods. This would allow the set up and error checking code at the beginning and end of check methods to be ignored for timing purposes. If these functions weren't used in the check method, then the entire check method would be timed.
- * '''Regression testing and database:''' If each test could be stored in a database, including timing information, that would allow us to see how speed is changing over time (using a reference machine).
- * '''Web output of test results:''' Haven't looked much at the {{{TestRunner}}} stuff, but I imagine it is possible to grab info from the test suite and write it out to an HTML file. We should do this on several architectures every evening on the [http://www.scipy.org SciPy] site.