[SciPy-dev] new sparsetools + umfpack

[Robert Cimrman]

Nathan Bell wrote:
> On 1/10/07, Robert Cimrman <cimrman3 at ntc.zcu.cz> wrote:
> I remain unconvinced that this is a real performance problem.  Given
> the speed of ensure_sorted_indices(), (it's even faster than copy()) I
> can't see how repeated calls would increase the total run time by more
> than a few %.  Do you have evidence that this is a bottleneck?

I have got my code running, so:
with
<29339x29339 sparse matrix of type '<type 'numpy.float64'>'
        with 927955 stored elements (space for 927955)
        in Compressed Sparse Row format>
note that my matrix has sorted indices apriori, so the time spent in
ensure_sorted_indices() is minimal possible

sfe: Stokes...
nls: iter: 0, out-of-balance: 1.812362e-02 (rel: 1.000000e+00)
->>>>>>>>>> 0.24667596817
  rezidual:    0.05 [s] ... rezidual assembling
     solve:    3.29 [s] ... umfpack solution
    matrix:    0.31 [s] ... matrix assembling
nls: iter: 1, out-of-balance: 1.036336e-16 (rel: 5.718151e-15)
sfe: Navier-Stokes...
nls: iter: 0, out-of-balance: 1.818563e-04 (rel: 1.000000e+00)
->>>>>>>>>> 0.249101161957
  rezidual:    0.08 [s]
     solve:    3.27 [s]
    matrix:    0.76 [s]
nls: iter: 1, out-of-balance: 4.214136e-07 (rel: 2.317289e-03)
->>>>>>>>>> 0.244355916977
  rezidual:    0.10 [s]
     solve:    3.29 [s]
    matrix:    0.78 [s]
nls: iter: 2, out-of-balance: 2.091538e-09 (rel: 1.150105e-05)

ensure_sorted_indices() call is denoted by '->>>>>>>>>>' so it is
definitely more than 10%. Note that I have a modified version of
umfpack.py, which calls ensure_sorted_indices() in symbolic() only (call
in numeric() is not needed).

> Personally, I think it's best to hide these details from the user as
> much as possible.  If the performance penalty is more than 10%, then
> perhaps it should be changed.  Otherwise I think simplicity wins.
> 
> Keep in mind that many SciPy users have little interest in learning
> the underlying implementation of things like sparse matrix formats.
> Informing them to use lil_matrix for construction but csr_matrix for
> arithmetic is fine (they can just take your word for it), but
> requiring that they ensure_sorted_indices() before calling umfpack is
> a bit too much IMO.

I agree generally, but on the other hand installing umfpack itself is
not trivial either - such people usually know what they are doing.
The default solver code path would not be touched, of course...

r.