Numpy API Analysis

This is the list of all of the numpy api functions classified based on how we are going to deal with them during the refactor. Functions are classified as one of:

* core: This function will be moved into the core library. There will be a simple wrapper in the interface layer that calls the core function.

* interface: This function will be part of the language interface.

* split: This function will be split. Some part of the functionality will be in the core library, but the language interface layer will also contain significant functionality.

arrayobject.c

Function	Type	Notes
PyArray_Size	core
PyArray_CopyObject	interface
PyArray_TypeNumFromName	core	This referenced userdescr and should be changed at the same time as the other functions that use this. The function should probably also be moved.
PyArray_SetStringFunction	interface
PyArray_SetDateTimeFunction	interface
PyArray_CompareUCS4	core
PyArray_CompareString	core
PyArray_ElementStrides	core
PyArray_CheckStrides	core

calculation.c:

Function	Type	Notes
PyArray_ArgMax	core	Calls PyArray_FromAny and PyArray_ContiguousFromAny, but passes in a PyArrayObject, so this will need to be fixed.
PyArray_ArgMin	interface	Types define max func, not min & this depends on PyInt_FromLog, PyNumber_Subtract so stays interface for now.
PyArray_Max	interface	Depends on PyArray_GenericReduceFunction & n_ops
PyArray_Min	interface	Depends on PyArray_GenericReduceFunction & n_ops
PyArray_Ptp	interface	Depends on PyArray_Min/Max, n_ops
PyArray_Std	core
PyArray_Sum	core
PyArray_Prod	core
PyArray_CumSum	core
PyArray_CumProd	core
PyArray_Round	core	Calls PyArray_SetAttrString when dealing with complex, but these will turn into calls to array_real_set and array_imag_set, so this can be fixed.
PyArray_Std	interface	This function is written using the Python C API and the numpy C API. It is almost at the Python level. It looks like we could rewrite it and make it core. (Should we?)
PyArray_Mean	interface	Should be core, but would need to refactored. Uses PyNumber functions.
PyArray_Any	core
PyArray_All	core
PyArray_Clip	split	The fastclip functionality should probably be in the core.
PyArray_Conjugate	core
PyArray_Trace	core

NOTE: Actually, almost all of the above functions are technically split since they

return a numpy scalar object instead of an array at times. The plan is the core functions will return a 0-d array and the wrapper function will convert to a scalar if needed by calling PyArray_Return.

conversion_utils.c:

Function	Type	Notes
PyArray_Converter	interface	since it is for use with PyArgs_ParseTuple
PyArray_OutputConverter	interface	since it is for use with PyArgs_ParseTuple
PyArray_IntpConverter	interface	since it uses sequence API.
PyArray_BufferConverter	interface
PyArray_AxisConverter	interface
PyArray_BoolConverter	interface
PyArray_ByteorderConverter	interface
PyArray_SortkindConverter	interace
PyArray_SearchsideConverter	interface
PyArray_PyIntAsInt	interface
PyArray_PyIntAsIntp	interface
PyArray_IntpFromSequence	interface
PyArray_TypestrConvert	core
PyArray_IntTupleFromIntp	interface

convert.c:

Function	Type	Notes
PyArray_ToList	interface
PyArray_ToFile	split	the binary file writing should be part of the core. The text file writing using the Python C API, but could perhaps be refactored so it does not.
PyArray_ToString	interface	maybe split
PyArray_FillWithScalar	split	the interface layer needs to handle converting the obj.
PyArray_NewCopy	core
PyArray_View	core	Calls PyObject_SetAttrString to set dtype, but we should be able to change this.

convert_datatype.c:

Function	Type	Notes
PyArray_CastToType	core
PyArray_GetCastFunc	core	we need to deal with castdict.
PyArray_CastTo	core
PyArray_CastAnyTo	core
PyArray_CanCastSafely	core
PyArray_CanCastTo	core
PyArray_CanCastScalar	core
PyArray_ValidType	core
PyArray_Zero	interface	but probably should not be.
PyArray_One	interface	but probably should not be.
PyArray_ObjectType	interface
PyArray_ConvertToCommonType	interface

ctors.c:

Function	Type	Notes
PyArray_MoveInto	core
PyArray_NewFromDescr	core	Calls __array_finalize__ if there is one so that part has to call back to the interface. This needs to be core since it is used a lot.
PyArray_New	core
PyArray_FromAny	interface
PyArray_CheckFromAny	interface	Added NpyArray_CheckFromArray in core, needed by NpyArray_CheckAxix
PyArray_FromArray	core
PyArray_FromStructureInterface	interace
PyArray_FromInterface	interface
PyArray_FromArrayAttr	interface
PyArray_DescrFromObject	interface
PyArray_FromDimsAndDataAndDescr	core	Not converted yet: comment says that it's 'old, should just call PyArray_NewFromDescr', waiting to see if it's needed.
PyArray_FromDims	core
PyArray_EnsureArray	interface	May need to be split, called by core functions such as PyArray_ArgMax
PyArray_EnsureAnyArray	interface	Ditto, may need to be split with part in core
PyArray_CopyAnyInto	core
PyArray_CopyInto	core
PyArray_CheckAxis	core	calls PyArray_CheckFromAny for some of the flags support but the arg is always an array, so it should be able to do the same with only core functions.
PyArray_Zeros	split	For object array fills with PyInts.
PyArray_Empty	split	For object arrays fills with Py_None
PyArray_Arange	split?	Uses setitem and PyFloats. Can this be refactored? JM: Do we need it in the core? I don't see it used in any core functions.
PyArray_ArangeObj	interface.
PyArray_FromFile	split	Keep at interface? Text file uses scanfunc on the given type, could by a Python method I think
PyArray_FromBuffer	interface
PyArray_FromString	split
PyArray_FromIter	interface

NOTE: It seems like we need to make PyArray_NewFromDescr a core function. This means that we will have to put something into the language interface to the with calling __array_finalize_ Generally array subtypes are something I'm not sure how we are going to deal with.

datetime.c:

Function	Type	Notes
PyArray_DatetimeStructToDatetime	core
PyArray_TimedeltaStructToTimedelta	core
PyArray_DatetimeToDatetimeStruct	core
PyArray_TimedeltaToTimedeltaStruct	core

descriptor.c:

Function	Type	Notes
PyArray_DescrNewFromType	core
PyArray_DescrConverter2	interface
PyArray_DescrConverter	interface
PyArray_DescrNew	core
PyArray_DescrAlignConverter	interface
PyArray_DescrAlignConverter2	interface
PyArray_DescrNewByteorder	core

NOTE: There is an issue with desriptors containing python objects, so all of the "core" cases above are probably really split, or only core after a refactor.

flagsobject.c:

Function	Type	Notes
PyArray_NewFlagsObject	interface
PyArray_UpdateFlags	core

itemselection.c:

Function	Type	Notes
PyArray_TakeFrom	split	indices is a PyObject, but gets converted to an array.
PyArray_PutTo	split	indices and values get converted to arrays
PyArray_PutMask	split
PyArray_Repeat	split
PyArray_Choose	split	op is converted to arrays
PyArray_Sort	core
PyArray_ArgSort	core
PyArray_LexSort	split	Takes a sequence of arrays as an argument.
PyArray_SearchSorted	split	op2 is converted to an array
PyArray_Diagonal	interface	uses a python list. Also uses PyArray_EnsureAnyArray, but I don't understand why.
PyArray_Compress	interface	uses PyArray_Nonzero which returns a tuple
PyArray_Nonzero	interface	Returns a tuple of index arrays? Why not an array?

NOTE: PyArray_Sort uses a global for the qsort compare function. This is an issue for multi-threaded environments. Also PyArray_ArgSort

iterators.c

Function	Type	Notes
PyArray_IterNew	split	Needed in core, interface to create PyObject wrapper
PyArray_BroadcastToShape	split
PyArray_IterAllButAxis	core
PyArray_RemoveSmallest	core
PyArray_Broadcast	core
PyArray_MultiIterFromObjects	interface
PyArray_MultiIterNew	split	Needed in core, interface to create PyObject wrapper
PyArray_NeighborhoodIterNoew	split	Needed in core, interface to create PyObject wrapper

NOTE: Are iterators and multi-iterators ever stored in the core? Do we need to reference count them in the core or only in the interface layer?

methods.c:

Function	Type	Notes
PyArray_GetField	core
PyArray_SetField	split	Uses PyArray_CopyObject.
PyArray_Byteswap	core
PyArray_Dump	interface	uses cpickle.
PyArray_Dumps	interface	uses cpickle.

multiarraymodule.c:

Function	Type	Notes
PyArray_GetPriority	interface	Looks for __array_priority__ attr.
PyArray_MultiplyIntList	core
PyArray_MultiplyList	core
PyArray_OverflowMultiplyList	core
PyArray_GetPtr	core
PyArray_CompareLists	core
PyArray_AsCArray	core
PyArray_As1D	deprecated core
PyArray_As2D	deprecated core
PyArray_Free	core
PyArray_Concatenate	split	Revisit to see if there is much code worth refactoring into core. Doesn't appear that there is a good API function, maybe a utility func.
PyArray_ScalarKind	core
PyArray_CanCoerceScalar	core
PyArray_InnerProduct	split
PyArray_MatrixProduct	split
PyArray_CopyAndTranspose	split
PyArray_Correlate2	split
PyArray_Correlate	split
PyArray_OrderConverter	interface
PyArray_ClipmodeConverter	interface
PyArray_EquivTypes	core
PyArray_EquivTypenums	core
PyArray_GetNDArrayCVersion	core
PyArray_GetNDArrayCFeatureVersion	core
PyArray_GetEndianness	core
PyArray_Where	split
PyArray_SigintHandler	?
PyArray_GetSigintBuf	?

number.c:

Function	Type	Notes
PyArray_SetNumericOps	interface with ufuncs
PyArray_GetNumericOps	interface with ufuncs
PyArray_GenericReduceFunction	core	Required by PyArray_Max, others but uses Py API, may need to be split
PyArray_GenericAccumulateFunction	core	Uses Py API may need to be split
PyArray_GenericBinaryFunction	core	Part of ufuncs library?
PyArray_GenericUnaryFunction	core	Part of ufuncs library?

NOTE: PyArray_GenericReduceFunction calls several Py API functions such as GetAttrString? and PyObject_Call. It would be a whole lot better if we can process native types without going back to the interface. It would also be particularly nice if we don't always have to return a generic object, though I'm not sure how to avoid it without replacing all invocations of it with big switch statements.

refcount.c:

Function	Type	Notes
PyArray_Item_INCREF	interface
PyArray_Item_XDECREF	interface
PyArray_INCREF	interface
PyArray_XDECREF	interface
PyArray_FillObjectArray	interface

NOTE: We need think about how to deal with object arrays. I think that the core should treat object arrays a void* arrays, but have come callback functions that are called when the array is copied to deal with refcount or GC handles. This definitely requires some more thought.

scalarapi.c:

Function	Type	Notes
PyArray_ScalarAsCtype	interface
PyArray_CastScalarToCtype	interface
PyArray_CastScalarDirect	interface
PyArray_FromScalar	interface
PyArray_ScalarFromObject	interface
PyArray_DescrFromTypeObject	interface
PyArray_FieldNames	interface
PyArray_DescrFromScalar	interface
PyArray_TypeObjectFromType	interface
PyArray_Scalar	interface
PyArray_Return	interface

NOTE: All of these are interface functions since we are assuming that the core does not have scalar objects.

shape.c:

Function	Type	Notes
PyArray_Resize	split	looks at weakreflist. Weak references are going to have to be part of the interface layer, so we'll need to either call back for this or check before we go into the core.
PyArray_Newshape	core
PyArray_Reshape	interface
PyArray_Squeeze	core
PyArray_SwapAxes	core
PyArray_Transpose	core
PyArray_Ravel	core
PyArray_Flatten	core

usertypes.c:

Function	Type	Notes
PyArray_InitArrFuncs	core
PyArray_RegisterDataType	core
PyArray_RegisterCastFunc	core
PyArray_RegisterCanCast	core

NOTE: I'm really not sure about these since I'm not sure I understand the relationship

between user types and scalars.