Milestone 1.1.0
This is the first minor release since the 1.0 release since October 2006. There are a few major changes, which introduce some minor API breakage. In addition this release includes tremendous improvements in terms of bug-fixing, testing, and documentation.
Major Changes
New implementation of masked arrays
MaskedArray is now a subclass of ndarray and the _data section can now be any subclass of ndarray. Apart from a couple of issues, the behavior of the new MaskedArray class reproduces the old one. API changes in the new MaskedArray implementation can be found here: http://scipy.org/scipy/numpy/wiki/MaskedArrayApiChanges
The old ma.py interface has been moved to the oldnumeric compatibility layer (i.e., numpy/oldnumeric/ma.py).
Complete overhaul of the IO system started
The IO code in both NumPy and SciPy is undergoing a major reworking. NumPy will provide basic code for reading and writing NumPy arrays, while SciPy will house file readers and writers for various data formats (data, audio, video, images, matlab, excel, etc.). As a first step in this direction:
- NumPy has a new standard binary file format (.npy/.npz) for arrays/groups_of_arrays. This is the new default method of storing arrays; pickling arrays is discouraged.
Support for numscons
Numscons extends SCons (http://www.scons.org/) to provide an modern and flexible build system for NumPy. While still under active development, Numscons all ready replaces most of the functionalities of numpy.distutils and provides much additional functionality. SCons is a software construction tool designed to replace traditional tools like make. One of the most attractive features of SCons is that its configuration files are actually Python scripts, and has a sane API for configuration test. Some of the advantages of numscons are: fine-grained control of compilation flags (up to per compiled object file), possibility of building ctypes-based extensions, automatic dependency handling, autoconf-like tests.
Note that the current build system still uses numpy.distutils by default and NumPy does not include numscons. This release does, however, include hook in numpy.distutils to call SCons, the SCons scripts, as well as some refactoring of existing code to make it easier to use numscons. You can find out more information about numscons (and where to get it) on NumScons page.
Semantic change for histogram
The histogram function is undergoing some changes to improve its consistency with the other histogram functions (histogram2d and histogramdd) and resolve long standing confusion regarding the handling of outliers. Specifically, the past behavior for histogram was to store upper outliers in the rightmost bin and ignore lower outliers. This behavior was a side effect of bins being specified by their left edges only, implicitly placing the rightmost bin edge at infinity. This made the interpretation of the density potentially confusing and the overall behavior irritating for new users.
In version 1.3, histogram will accept and return all the bin edges, including the rightmost edge, so that
>>> hist, edges = histogram(a, n)
will return arrays of length n (hist) and n+1 (edges) respectively. Also, values lying outside of the user specified range (lower and upper outliers) will not be tallied.
In order to minimize code breakage, in the current release the histogram function retains its old behavior, however
- A warning is raised at the first invocation of histogram, notifying users of the upcoming changes.
- Use of the range keyword will raise a warning about the coming changes to outliers handling.
- Use of explicit bin edges will raise a warning notifying users of the coming change regarding the edges definition.
- Use of explicit bin edges and normed=True will raise an error.
The new behavior for histogram, along with added support for weights, is available using the new=True argument. In version 1.1, new is set to False by default. In version 1.2, new will default to True, and in 1.3, it should disappear altogether. Setting new=True will allow users to call histogram without being annoyed by warnings.
New Features
New binary installers for Microsoft Windows
We provide new win32 installer aimed at solving the recurring problem of non-working atlas on different sets of CPU. This installer simply checks which cpu you have, and installs the appropriate NumPy accordingly (without atlas if your cpu is not supported).
Technical details:
- ATLAS is 3.8.0, and built with cygwin. Built on Core 2 duo for SSE3, pentium 4 xeon for SSE2.
- BLAS/LAPACK are built with mingw.
- only SSE3 and SSE2 are supported. If you don't have at least sse2, it will not use ATLAS, but netlib blas/lapack instead.
- the numpy installers are prepared with build_wininst for each different supported architecture
- the numpy installers are then bundled in another installer, based on nsis.
- Nsis is an open source installer, based on a small scripting language, and is extensible by plug-ins. I wrote a plug-in to detect the cpu type, and the installer will simply execute a different numpy installer depending on the cpu.
Official binary installers for MacOS X
For the first time, we provide an official Universal Mac binary.
Technical details:
- Built on OSX 10.5.2, Intel Core 2 Duo
- Using XCode 3.0 with gcc 4.0.1 and the Accelerate.framework 1.4.2, gfortran 4.2.1 and Python 2.5.2 from python.org (MacPython?)
- Used bdist_mpkg 0.4.3 to make the distribution
Since it uses the MacPython?, it will install NumPy under here:
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
Median
This release introduces 'axis' support for numpy.median(). This is not just useful on its own, but it is also a first step in making median() consistent with mean(), min(), max(), var(), std(), sum(), prod(), etc.
While our goal is to harmonize the median() call signature and functionality with mean() and the rest, we also need to minimize API breakage for this release. So for 1.1, the new call signature is:
def median(a, axis=0, out=None, overwrite_input=False)
This returns the median of the array elements where the median is taken over the first axis of the array by default, otherwise over the specified axis. Then in the 1.2 release, we will allow API breakage and will, thus, change it to:
def median(a, axis=None, out=None, overwrite_input=False)
Var and Std
ddof keyword
Currently var and std are computed using 1/N. Traditionalists often like to use 1/(N-1) in these calculations. Some analysis methods suggest using 1/(N-alpha) for a wide-variety of alpha's depending on what you are trying to accomplish. In 1.0.5, var, and std methods and functions now take a ddof keyword. The divisor used in these calculations will be (N-ddof). The default is still 0, leaving the calculation unchanged.
complex data-types
For complex-valued data-types, the var and std calculations have been changed so that var = real(sum(x*conj(x)))/N and std = sqrt(var).
Bug-fixes
- #69
- Why numpy headers are installed using add_data_dir and not add_headers?
- #419
- Make web page links to NumPy bug report (ticket) system
- #420
- Add numpy.floatc for C float type
- #430
- numpy fails with PyWin build 210 (PyWin32-210.win32-py2.5.exe)
- #460
- better integration of matrices using properties
- #502
- Unbiased estimates of variance/ std deviation
- #551
- numpy.ndarray messed up after unpickling
- #555
- setting random seed does not work with a numpy.int64
- #558
- 'axis' support for numpy.median()
- #562
- Reference leaks
- #570
- Vectorize leaks
- #571
- Support universal gfortran compiler on OS X
- #573
- numpy does not build when LDFLAGS are set
- #581
- random.set_state does not reset state of random.standard_normal
- #583
- fromiter: memory error on invalid dtype
- #587
- f2py fails on INTEGER(8) return statement in f90 code
- #589
- problem installing with latest version of Intel MKL
- #593
- Avoiding polluting public namespace in numpy headers
- #594
- Major slowdown when mixing array with Python's list
- #596
- setuptools' develop mode broken
- #597
- assigning negative # to a uint64 array element gives wrong answer on 32-bit machine
- #601
- function for computing powers of a matrix
- #603
- numpy 1.0.3.1, python crash when indexing an array with a int32 ndarray (see example)
- #607
- segfault from particular indexing of empty array
- #608
- <ufunc 'absolute'> should clear the sign bit.
- #609
- numpy/linalg/lapack_litemodule.c:lapack_lite_zgeqrf int vs. long
- #610
- Let numpy.interp accept scalar values as first argument
- #611
- [Vista - NPY_OWNDATA] data created in C code deleted on return to python
- #612
- NumPy 1.0.4 fails to compile on Cygwin
- #613
- resize on an empty array fails in some cases, very weird
- #614
- SegFault/double free with simple array mask operation
- #615
- numpy.array([])[arange(10000)>=0] = 123 segfaults
- #616
- arange: incorrect output byteorder
- #617
- numpy.random.randint error with negative numbers
- #618
- numpy freebsd fix
- #620
- Error in the typemaps in numpy.i
- #621
- wrapping f95 fails
- #622
- function for computing the condition number
- #623
- loadtxt fails with record arrays
- #624
- Statistics-related array method are not self-consistent.
- #625
- ImportError: No module named test_ufunc
- #626
- percentile() and clamp()
- #627
- linalg.svd takes 100% CPU and never ends
- #628
- polyfit uses 100% CPU and does not stop (Windows XP, Python 2.5.1 NumPy 1.0.4)
- #630
- If float('123.45') works, so should numpy.float32('123.45')
- #631
- incorrect description of numpy.linalg.pinv in linalg/info.py
- #632
- numpy.histogram fails with bin=<list>
- #633
- segfault in assignment to pyobject array
- #634
- masked array with dtype bool, sum to bool instead of int.
- #635
- numpy-1.0.4 crashes when doing linalg.inv on P3 Windows machine. Invalid op code.
- #636
- Faster array version of ndindex
- #637
- typo in scalarmathmodule.c.src for ctype_negative
- #638
- var should take absolute value for complex numbers.
- #639
- test_format.py:283: ImportError: No module named nose.tools
- #641
- numpy.sqrt(numpy.array([-1.0], dtype=numpy.complex256)) segfaults
- #643
- Tracebacks in Turkish locales
- #644
- distutils/cpuinfo fails to detect some nocona based hardware
- #645
- maskedarray extrema operations break on non-array types
- #646
- incorrect calculation on int64 array
- #647
- Bus Error with object arrays on big endian Solaris system
- #648
- Bus Error with object arrays on big endian system
- #649
- numpy.linalg.lstsq segfaults on i686 32-bit and ia64 linux
- #650
- fromfile and fromstring differ in parsing ascii integer strings
- #651
- typos in _compiled_base.c function arr_bincount
- #652
- linalg.eig takes 100% cpu and never ends
- #653
- v1.0.4 incorrectly identifies Xeon as 64-bit
- #654
- Numpy tests failure on solaris
- #656
- numpy.histogram fails if bins is a sequence
- #657
- x.flat[:] interprets non-native byteorder arrays incorrectly
- #658
- Segfault on reshape
- #660
- Crash in numpy.unique1D (release only)
- #661
- maximum handles nan improperly
- #662
- eigh hangup
- #663
- Support for multi formatting elements in savetxt
- #664
- more accurate representation of polynomials
- #665
- 100% cpu error consolidation, tickets 627, 628, 652
- #667
- _dotblas.so only built if ATLAS is installed
- #668
- Explain why numpy headers are installed using add_data_dir
- #670
- cmp_arg_types bug
- #671
- linalg ImportError on Solaris 10 systems
- #672
- Fatal Python error: PyEval_RestoreThread: NULL tstate
- #674
- Impossible to rename object fields
- #676
- fortran-order .flatten() fails for multidimensional arrays
- #677
- format string / substitutable parameter mismatch in numpy/core/src/arrayobject.c
- #678
- fromfile() : reading past end of file causes unhandled exception
- #679
- ERROR: check_MyPyLong_AsUnsignedLongLong (test_regression.TestRegression)
- #680
- Bug in numpy/core/defmatrix.py
- #682
- fromfile/tofile broken under Windows
- #683
- MyPyLong_AsUnsignedLongLong fails on 64bit *nix platforms
- #684
- Intp takes 2 arguments on all platforms except Windows x86 64 MSVC
- #685
- Intp takes 2 arguments on all platforms except Windows x86 64 MSVC
- #686
- Intp takes 2 arguments on all platforms except Windows x86 64 MSVC
- #687
- site.cfg: ConfigParser in Python 2.6 does not allow section 'DEFAULT'
- #688
- Numpy and scipy should give informative error messages when run from source dir
- #689
- SyntaxError while installing v1.0.5 on MacTel running OSX v10.4.11
- #690
- No test for r4806
- #691
- No test for r4819
- #692
- No test for r4822
- #693
- No test for r4826
- #694
- No test for r4827 (possibly)
- #696
- No test for r4853
- #697
- ValueError: shape mismatch: objects cannot be broadcast to a single shape
- #699
- corrcoef incorrect behaviour
- #700
- Test for mean fails, incorrect processing of weights
- #702
- Memory leak in generic 1d array
- #703
- ma.masked_where can make a defective array
- #704
- Building SciPy crashes if no FORTRAN compiler available
- #705
- [ numpy-Bugs-601052 ] SVD did not converge
- #706
- [ numpy-Bugs-601052 ] SVD did not converge
- #708
- Setuptools call to numpy trunk causes numpy.scons error
- #710
- Reference count error, in array_reshape?
- #711
- object array refcount bug
- #712
- scalar math test failure on solaris
- #713
- numpy.any segfaults for object arrays with >=10000 elements
- #715
- The interactive mode of setup.py incorrect lower-cases prefix settings
- #716
- Inconsistent behavior, i686 vs x86_64
- #717
- numpy.loadtxt fails when missing values are present
- #719
- No test for 'fromregex' in r4960
- #720
- Changeset 4945 requires docstrings and tests
- #723
- gfortran installed in Program Files is not found
- #724
- numpy.ma.std and var should match numpy.std and var behaviour
- #726
- r4980 doesn't test assert_ functions with nan arrays
- #728
- numpy.r_ incorrectly casts with mixed types
- #731
- numpy.linalg.eigvals can't take nested lists while numpy.linalg.eig can
- #732
- error string on IndexError is wrong
- #733
- check_object_casting failing on Python 2.6
- #734
- [PATCH] interactive docstring search (lookfor)
- #735
- FAIL: test_record (numpy.lib.tests.test_io.Testloadtxt) -- On Solaris 10
- #736
- Inconsistent integer conversion from strings
- #737
- String to integer conversion inconsistent.
- #738
- Test trying to write file in wrong location
- #739
- Please don't use yellow in the output of setup.py
- #742
- Don't require bz2 or gzip
- #743
- Array content zeroed by |= even though exception raised
- #744
- Loss of dimension bug in numpy.ma.__getitem__ for arrays of shape (n, 1) or (1, n) or (n, 1, 1, ...)
- #746
- Segfault calling logical_not on own object
- #747
- Reference counting bug in call to ufunc object loops.
- #748
- Ifft pads incorrectly
- #749
- incorrect behaviour when using mask to index masked array
- #751
- distutils does not seem to handle the ALL section in site.cfg
- #753
- numpy.ma.morestats requires scipy
- #759
- spurious underflow warning in numpy.lib.machar
- #760
- scalar indexing of matrices -> deprecation warning
- #761
- operator += fails silently when adding arrays with diffenent base types
- #767
- numpy.i typo in error messages
- #770
- numpy.core.tests.test_multiarray.TestView failures on big-endian machines
- #776
- array resize shifts values in array
- #777
- array resize shifts values in array
- #778
- Markup errors in trunk/numpy/ma/API_CHANGES.txt
- #779
- Markup errors in trunk/numpy/ma/API_CHANGES.txt
- #780
- Markup errors in trunk/numpy/ma/API_CHANGES.txt
- #781
- Markup errors in trunk/numpy/ma/API_CHANGES.txt
- #782
- Markup errors in trunk/numpy/ma/API_CHANGES.txt
- #783
- Markup errors in trunk/numpy/ma/API_CHANGES.txt
- #788
- astype sometimes fails to return a copy
- #789
- Segfault if ndarray.compress can't safely cast into the "out" array
- #790
- import error with latest numpy
- #791
- Possible bug: std/var not properly calling array_finalize
- #792
- set -D_FORTIFY_SOURCE=1 instead of 2
- #793
- recarray.list() raises RunTimeError
