One of the goal of numpy.scons branch is to enable easier customization of the build process by users. For this purpose:

• The way optimized libraries are detected has been totally revamped, so that it is more robust to version changes, and so on.
• The build process internally uses separate logic for compilation options which only affect generated code, and options which are necessary for the binary to work (e.g. you can modify the compilation flags for optimization without risk to disable option necessary to build python modules on your platform). This was not easy to do with distutils because using CFLAGS/LDFLAGS *overwrites* options instead of merging them.

Handling optimized libraries

Here are some basic info to build numpy.scons branch with MKL, Sunperf, etc... Generally, if you don't use default path, you have to customize a site.cfg file so that scons can find the libraries. For example, when I test numpy.scons with the MKL, which is installed somewhere in my home directory, I do the following:

site.cfg file:

[mkl]
include_dirs = /home/david/opt/intel/mkl/9.1.023/include
library_dirs = /home/david/opt/intel/mkl/9.1.023/lib/32

This will tell the configuration system to use this MKL whenever possible (BLAS, LAPACK, etc..). To see if MKL is correctly picked up, look for "Checking MKL ... " lines while building numpy. You can also check once numpy is built, using the show_config function:

import numpy
print numpy.show_config()

Which will returns something like

package numpy.linalg configuration:
    lapack
        Using MKL
        Customized items site.cfg:
          rpath : ['/home/david/opt/intel/mkl/9.1.023/lib/32']
          libpath : ['/home/david/opt/intel/mkl/9.1.023/lib/32']
          libs : ['lapack', 'mkl', 'guide', 'm']
          cpppath : ['/home/david/opt/intel/mkl/9.1.023/include']
          Version is : Not checked
package numpy.core configuration:
    cblas
        Using MKL
        Customized items site.cfg:
          rpath : ['/home/david/opt/intel/mkl/9.1.023/lib/32']
          libpath : ['/home/david/opt/intel/mkl/9.1.023/lib/32']
          libs : ['mkl', 'guide', 'm']
          cpppath : ['/home/david/opt/intel/mkl/9.1.023/include']
          Version is : Not checked

Note that the checks are pretty 'strong', in the sense that the mere libraries presence is not enough: numpy.scons will actually check whether it can run code compiled with the library, and if it is not possible, it will not use it. This is to avoid runtime library path problems, wrong ABI and so on.

If a checker fails, numpy.scons should give you a message why it failed. For the exact reason (with the failed test code), you should take a look at config.log (this file is per module, in build/scons). I hope soon to be able to have a separate configuration step with a configuration summary before building, as the current situation is a bit ackward.

Disabling a library

You can disable a library by doing VAR=None python setup.py build, where VAR is ATLAS, MKL, Sunperf, etc... This can be useful to testing purposes.

Supported libraries

For now, the following 'perf' checkers are supported:

• ATLAS
• MKL
• Accelerate and vecLib frameworks (on mac os X)
• Sunperf (on solaris only for now).

Generally, numpy use those through a standard API such as CBLAS, LAPACK, etc... those are called 'meta checkers'. For now, only CBLAS and LAPACK are available. Soon, FFT-related code will be provided.

Customizing optimization flags

Compilation flags are handled through compiler configuration objects: http://projects.scipy.org/scipy/numpy/browser/branches/numpy.scons/numpy/distutils/scons/core/default.py. For each supported compiler, warning flags, optimization flags, debugging flags are separated (this is meant so that you can keep optimization flags while compiling swig or pyrex extensions, which generate may spurious warnings).

As an example, on solaris, with a PIV CPU, I got a 300 % speed increase on common linear algebra by using -xtarget=pentium4 in optim and link_optim arguments:

cfg = CompilerConfig(optim = ['-fast', '-xtarget=pentium4'],
                     debug_symbol = ['-g'],
                     link_optim = ['-xtarget=pentium4'])

There is no way yet to customize those in a friendly way, but this is on the TODO list.