Ticket #156 (reopened defect)

Opened 1 year ago

Last modified 8 months ago

ipython crashes when input specific characters

Reported by: hongqn Assigned to: fperez
Priority: low Milestone:
Component: ipython Version:
Severity: normal Keywords:
Cc:

Description

When input the Chinese character '阿' (UTF-8: '\xe9\x98\xbf', a quite frequently used character in Chinese language) in ipython, it crashed. Other characters have no such effect. 

Python 2.5 (r25:51908, Apr 14 2007, 16:13:48) 
Type "copyright", "credits" or "license" for more information.

IPython 0.8.1 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import sys

In [2]: sys.stdin.encoding
Out[2]: 'UTF-8'

In [3]: '你好'
Out[3]: '\xe4\xbd\xa0\xe5\xa5\xbd'

In [4]: '    ��'
WARNING: 
********
You or a %run:ed script called sys.stdin.close() or sys.stdout.close()!
Exiting IPython!

However the same input is ok in a normal python shell: 

Python 2.5 (r25:51908, Apr 14 2007, 16:13:48) 
[GCC 4.1.2 (Gentoo 4.1.2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.stdin.encoding
'UTF-8'
>>> '你好'
'\xe4\xbd\xa0\xe5\xa5\xbd'
>>> '阿'
'\xe9\x98\xbf'

Produced on a amd64 gentoo linux laptop running Xfce 4.4.1. locale is set to en_US.UTF-8. using SCIM to input Chinese characters.

Change History

10/05/07 02:01:15 changed by Truck

I had a similar problem when inputting ä - very frequent in Finnish:

ikanim:~ truck$ ipython Python 2.4.4 (#1, Sep 25 2007, 16:06:09) Type "copyright", "credits" or "license" for more information.

IPython 0.8.1 -- An enhanced Interactive Python. ? -> Introduction to IPython's features. %magic -> Information about IPython's 'magic' % functions. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: print "ä" WARNING: ******** You or a %run:ed script called sys.stdin.close() or sys.stdout.close()! Exiting IPython!

- This worked, as above, in python. HOWEVER.

My Locale for LC_CTYPE was set to C, after changing it to UTF8, the issue goes away for the ä ...

ikanim:~ truck$ export LC_CTYPE="en_IE.UTF-8" ikanim:~ truck$ ipython Python 2.4.4 (#1, Sep 25 2007, 16:06:09) Type "copyright", "credits" or "license" for more information.

IPython 0.8.1 -- An enhanced Interactive Python. ? -> Introduction to IPython's features. %magic -> Information about IPython's 'magic' % functions. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: print "ä" ä

In [2]:

However it does NOT go away for the Chinese character:

ikanim:~ truck$ export LC_CTYPE="zh_HK.UTF-8" ikanim:~ truck$ ipython Python 2.4.4 (#1, Sep 25 2007, 16:06:09) Type "copyright", "credits" or "license" for more information.

IPython 0.8.1 -- An enhanced Interactive Python. ? -> Introduction to IPython's features. %magic -> Information about IPython's 'magic' % functions. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: '你好' Out[1]: '\xe4\xbd\xa0\xe5\xa5\xbd'

In [2]: ' ##' WARNING: ******** You or a %run:ed script called sys.stdin.close() or sys.stdout.close()! Exiting IPython!

It is my hope that this helps you to track down the problem; or at least propose a workaround for some folks experiencing this issue.

10/05/07 02:03:47 changed by Truck

darn it, I forgot that trac's wiki should be formatted differently. For ease of reading (Chinese UTF8 first, C second, English(Irish) UTF8 third: 

ikanim:~ truck$ export LC_CTYPE="zh_HK.UTF-8"
ikanim:~ truck$ ipython
Python 2.4.4 (#1, Sep 25 2007, 16:06:09) 
Type "copyright", "credits" or "license" for more information.

IPython 0.8.1 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: '你好'
Out[1]: '\xe4\xbd\xa0\xe5\xa5\xbd'

In [2]: '    ##'
WARNING: 
********
You or a %run:ed script called sys.stdin.close() or sys.stdout.close()!
Exiting IPython!
ikanim:~ truck$ export LC_CTYPE="C"
ikanim:~ truck$ ipython
Python 2.4.4 (#1, Sep 25 2007, 16:06:09) 
Type "copyright", "credits" or "license" for more information.

IPython 0.8.1 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: print "ä"
WARNING: 
********
You or a %run:ed script called sys.stdin.close() or sys.stdout.close()!
Exiting IPython!
ikanim:~ truck$ export LC_CTYPE="C"
ikanim:~ truck$ export LC_CTYPE="en_IE.UTF-8"
ikanim:~ truck$ ipython
Python 2.4.4 (#1, Sep 25 2007, 16:06:09) 
Type "copyright", "credits" or "license" for more information.

IPython 0.8.1 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: print "ä"
ä

In [2]:

01/30/08 05:18:10 changed by stefanv

Can anyone else reproduce this behaviour with latest SVN? It won't break on my machine (32-bit Ubuntu Linux).

01/30/08 10:31:41 changed by fperez

I can reproduce the original crash with the chinese character on my ubuntu gutsy 32-bit box: 

In [1]: '你好'
Out[1]: '\xe4\xbd\xa0\xe5\xa5\xbd'

In [2]: ��'
WARNING:
********
You or a %run:ed script called sys.stdin.close() or sys.stdout.close()!
Exiting IPython!

The Finnish one is OK though: 

In [3]: print "ä"
ä

This means we still have unicode issues somewhere deep...

02/20/08 16:27:53 changed by vivainio

  • status changed from new to closed.
  • priority changed from normal to low.
  • resolution set to fixed.
  • severity changed from major to normal.

This is probably somewhere deeper than IPython - if the stdin closes, we are screwed.

I'm setting this as lower priority, since this will probably never be fixed. If you use unicode characters, you can expect some problems, but that's the way it'll be before Python 3.0.

02/20/08 16:28:59 changed by vivainio

  • status changed from closed to reopened.
  • resolution deleted.