Ticket #893 (closed defect: fixed)

Opened 2 years ago

Last modified 2 years ago

Numpy incompatible with python -OO

Reported by: cscott Owned by: pierregm
Priority: normal Milestone: 1.2.0
Component: numpy.ma Version: none
Keywords: Cc:

Description

In line 3256 of numpy/ma/core.py, the following code appears:

inner.doc += doc_note("Masked values are replaced by 0.")

This line fails when python is invoked with the -OO option, which removes doc strings to save memory, since inner.doc will be None and a TypeError? will be raised trying to add a string to None. Adding a guard like: 'if inner.doc:' would fix this bug.

Attachments

numpy-OO.patch (3.6 KB) - added by cscott 2 years ago.
Patch to allow numpy to work when -OO option is passed to python.

Change History

  Changed 2 years ago by cscott

There are a number of other instances:

  • numpy/lib/financial.py, lines 64, 90, 118, 160, 196 (needs guard with if)
  • numpy/ma/core.py, lines 3256, 3273 (needs guard with if)
  • numpy/ma/extras.py, line 896 (guard with 'if _g[nfunc].func_doc')

Patch attached.

Changed 2 years ago by cscott

Patch to allow numpy to work when -OO option is passed to python.

  Changed 2 years ago by charris

I fixed core.py but extras.py has changed and needs to be looked at a bit more.

  Changed 2 years ago by pierregm

All, Let me take care of that. I wish I could do it now, but I can't compile the latest revision (cf mailing list), so I can't test anything.

  Changed 2 years ago by charris

financial.py has changed and I don't see any problems with the current code.

follow-up: ↓ 6   Changed 2 years ago by cscott

$ python2.5 -OO -c 'import numpy'

is a good way to validate that all of the doc issues have been fixed.

in reply to: ↑ 5   Changed 2 years ago by pierregm

Replying to cscott:

{{{ $ python2.5 -OO -c 'import numpy' }}} is a good way to validate that all of the doc issues have been fixed.

That's what I did of course, but I run into other problems:

Traceback (most recent call last):
  File "<string>", line 1, in ?
  File "/usr/lib64/python2.4/site-packages/numpy/__init__.py", line 125, in ?
    import add_newdocs
  File "/usr/lib64/python2.4/site-packages/numpy/add_newdocs.py", line 9, in ?
    from lib import add_newdoc
  File "/usr/lib64/python2.4/site-packages/numpy/lib/__init__.py", line 4, in ?
    from type_check import *
  File "/usr/lib64/python2.4/site-packages/numpy/lib/type_check.py", line 8, in ?
    import numpy.core.numeric as _nx
  File "/usr/lib64/python2.4/site-packages/numpy/core/__init__.py", line 34, in ?
    from numpy.testing import Tester
  File "/usr/lib64/python2.4/site-packages/numpy/testing/__init__.py", line 12, in ?
    from utils import *
  File "/usr/lib64/python2.4/site-packages/numpy/testing/utils.py", line 9, in ?
    from nosetester import import_nose
  File "/usr/lib64/python2.4/site-packages/numpy/testing/nosetester.py", line 88, in ?
    class NoseTester(object):
  File "/usr/lib64/python2.4/site-packages/numpy/testing/nosetester.py", line 292, in NoseTester
    _docmethod(_test_argv, '(testtype)')
  File "/usr/lib64/python2.4/site-packages/numpy/testing/nosetester.py", line 85, in _docmethod
    meth.__doc__ = meth.__doc__ % {'test_header':test_header}
TypeError: unsupported operand type(s) for %: 'NoneType' and 'dict'

Until nosetester is fixed, I'm blind.

I'm using r5708 slightly modified (commenting out the trunc function in umathmodule.c.src that prevents me to compile numpy).

  Changed 2 years ago by charris

Looks like doc tests are going to fail, natch. I think you should open a ticket on that. Did you do anything that required the tester? If not, I don't know why it is being imported. I think Robert made some fixes along those lines for lib as part of speeding up numpy import. Maybe you should open two tickets.

  Changed 2 years ago by alan.mcintyre

I would expect the doctests option to run just fine, since doctest isn't going to find any tests. I was going to test this, but when I run

python2.5 -OO -c "import numpy;numpy.test(doctests=True,verbose=2)"

I get a glibc message "python2.5: free(): invalid next size (fast): 0x..." when running test_convolve_empty. Does this happen for anyone else? (it happens without the doctests argument as well)

  Changed 2 years ago by charris

I see it also.

  Changed 2 years ago by ChrisBarker

Just to add another note: this issue prevents numpy from being used with py2exe's default setting (which I believe include -OO). py2exe can be set to not optimize as a work-around, but it would be nice if numpy "just worked" with py2exe.

  Changed 2 years ago by charris

Financials has real documentation now and no longer has a problem.

Hmmm,

python -OO -c"import numpy; numpy.test(verbose=2)"
...
Ticket #789, changeset 5217. ... ok
Convolve should raise an error for empty input array. ... *** glibc detected *** python: free(): invalid next size (fast): 0x09361fe8 ***

How is the test finding the doc strings, I thought they not supposed to be imported?

  Changed 2 years ago by charris

OK the problem is with assert:

$[charris@f9 ~]$ python -OO
Python 2.5.1 (r251:54863, Jun 15 2008, 18:24:51) 
[GCC 4.3.0 20080428 (Red Hat 4.3.0-8)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> assert 1 == 0, "bummer"
>>> 
$[charris@f9 ~]$ python
Python 2.5.1 (r251:54863, Jun 15 2008, 18:24:51) 
[GCC 4.3.0 20080428 (Red Hat 4.3.0-8)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> assert 1 == 0, "bummer"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError: bummer
>>>

Which probably creates a problem for failUnlessRaises. That function looks to be in TestCase?.

  Changed 2 years ago by charris

But really, the problem is in convolve, which misuses assert to report a runtime error. Assert is for debugging, which is not what -OO is about.

>>> raise AssertionError, "bummer"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError: bummer
>>> assert 1 == 0, "bummer"
>>>

  Changed 2 years ago by charris

Ok, I fixed the segfault by changing convolve to explicitly raise a ValueError?. However, there are a number of new errors that now show up running with -OO:

====================================================================== FAIL: Test two arrays with different shapes are found not equal.


Traceback (most recent call last):

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 46, in test_array_diffshape

self._test_not_equal(a, b)

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 18, in _test_not_equal

raise AssertionError?("a and b are found equal but are not")

AssertionError?: a and b are found equal but are not

====================================================================== FAIL: Test two different array of rank 1 are found not equal.


Traceback (most recent call last):

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 32, in test_array_rank1_noteq

self._test_not_equal(a, b)

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 18, in _test_not_equal

raise AssertionError?("a and b are found equal but are not")

AssertionError?: a and b are found equal but are not

====================================================================== FAIL: Test two arrays with different shapes are found not equal.


Traceback (most recent call last):

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 46, in test_array_diffshape

self._test_not_equal(a, b)

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 18, in _test_not_equal

raise AssertionError?("a and b are found equal but are not")

AssertionError?: a and b are found equal but are not

====================================================================== FAIL: Test two different array of rank 1 are found not equal.


Traceback (most recent call last):

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 32, in test_array_rank1_noteq

self._test_not_equal(a, b)

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 18, in _test_not_equal

raise AssertionError?("a and b are found equal but are not")

AssertionError?: a and b are found equal but are not

====================================================================== FAIL: Test rank 1 array for all dtypes.


Traceback (most recent call last):

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 65, in test_generic_rank1

foo(t)

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 61, in foo

self._test_not_equal(c, b)

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 18, in _test_not_equal

raise AssertionError?("a and b are found equal but are not")

AssertionError?: a and b are found equal but are not

====================================================================== FAIL: Test rank 3 array for all dtypes.


Traceback (most recent call last):

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 84, in test_generic_rank3

foo(t)

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 80, in foo

self._test_not_equal(c, b)

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 18, in _test_not_equal

raise AssertionError?("a and b are found equal but are not")

AssertionError?: a and b are found equal but are not

====================================================================== FAIL: Test arrays with nan values in them.


Traceback (most recent call last):

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 98, in test_nan_array

self._test_not_equal(c, b)

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 18, in _test_not_equal

raise AssertionError?("a and b are found equal but are not")

AssertionError?: a and b are found equal but are not

====================================================================== FAIL: Test record arrays.


Traceback (most recent call last):

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 124, in test_recarrays

self._test_not_equal(c, b)

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 18, in _test_not_equal

raise AssertionError?("a and b are found equal but are not")

AssertionError?: a and b are found equal but are not

====================================================================== FAIL: Test two arrays with different shapes are found not equal.


Traceback (most recent call last):

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 109, in test_string_arrays

self._test_not_equal(c, b)

File "/usr/lib/python2.5/site-packages/numpy/testing/tests/test_utils.py", line 18, in _test_not_equal

raise AssertionError?("a and b are found equal but are not")

AssertionError?: a and b are found equal but are not

====================================================================== SKIP: test_umath.TestComplexFunctions?.test_branch_cuts_failing


Traceback (most recent call last):

File "/usr/lib/python2.5/site-packages/nose/case.py", line 203, in runTest

self.test(*self.arg)

File "/usr/lib/python2.5/site-packages/numpy/testing/decorators.py", line 93, in skipper

raise nose.SkipTest?, 'This test is known to fail'

SkipTest?: This test is known to fail


Ran 1719 tests in 6.241s

FAILED (failures=9)

  Changed 2 years ago by charris

  • status changed from new to closed
  • resolution set to fixed

All tests pass as of r5720, so I'm closing the ticket. Please open a new ticket if you turn up any more problems.

Note: See TracTickets for help on using tickets.