 pkg/TODO 2005/08/19 17:01:11 868
+++ pkg/TODO 2008/04/23 11:23:50 2175
@@ 1,47 +1,8 @@
+ Check for DimNames propagation in coercion and other operations.
+
 Report the problem in the Linux ldexp manual page. The second and
third calls in the Synopsis should be to ldexpf and ldexpl.
 [,] indexing: for sparse "works", but not yet for negative indices!

 group generics: "Arith" (partly done),
 but also "Compare", "Math" etc;
 see ?Math and the examples in ?SetGeneric

 methods for rbind and cbind where they make sense
 (Kurt is particularly interested in dgCMatrix ones, for pkg 'arules')
 ****> UNFEASIBLE as long cbind is function(..., *) !!!! <*******
 > follow John's proposition cbind(x, ...) via Generic cbind2(x,y)?

 consider moving alloc3Darray from ./src/Mutils.c to
 $(RSRC)/src/base/array.c

 data/ : names 'mm' and even more 'y' are ``too short''.
 If we really want to keep them, don't use "LazyData"
 (such that one needs data(*) explicitly);
 But MM would rather want something like ex.mm and ex.y

 slot "factors" maybe should move up to "Matrix"



We have a (at least one) basic problem :
 Currently the show() method fail sometime after coercion:
 e.g. 'sy' show()s wrongly, even though it "str()" fine :

 (po < crossprod(Matrix(0:3, 2))) # ok
 (ge < as(po, "dgeMatrix")) # ok
 (sy < as(po, "dsyMatrix")) # BAD
 str(sy) # looks fine

 or
 example(expand) # > ex$L and ex$U look bad, however
 as(ex$L, "dgeMatrix") # `works'

 {Of course, we don't need a workaround but must understand
 and solve the problem}



 provide methods for "dspMatrix" and "dppMatrix"!
 implement (more) methods for supporting "packed" (symmetric / triangular)
@@ 50,35 +11,174 @@
(have some dtr* <> dtp*)
 implement diagonal Matrix class "ddiMatrix" etc
 using constructor function Diagonal() or Diag().

 FIXME: dtpMatrix(... diag = "U") (i.e., unitdiagonal packed triangular)
  *does* need 'x' entries for the diagonal but these are never looked at.
 > change doc {or code ?  depends on what Lapack expects !}

BUG: rcond() of a singular dpoMatrix gives a LaPack error instead of just 0:
 MM < crossprod(M < Matrix(c(1:4,9:6), 2,4)) ; rcond(MM)
 ##> Error in rcond(MM) : Lapack routine dpotrf returned error code 4




 combine the C functions for multiplication by special forms and
solution wrt special forms by using a 'right' argument and a
'classed' argument.
[done with dgeMatrix_matrix_mm(); not yet for other classes;
and for _crossprod()]
 add more comprehensive examples / tests for Schur decomposition
+
 arithmetic for sparse matrices:
 o {  }
 should return a sparse matrix for at least "+" and "*" , also %%,
 and "/" and "%/%" at least when the RHS is nonzero a scalar.



 in lmer.c check all instances of the use of ZtX and XtX and change
 them so that having a negative last element of nc means use the
 response only (but look for it in the right place).
+ "Math2" , "Math", "Arith":
+ keep triangular and symmetric Matrices when appropriate:
+ particularly desirable for "Math2": round(), signif()
+
+ For triangular matrices, more specifically make sure the four rules of
+ "triangular matrix algebra" (Golub+Van Loan 1996, 3.1.8, p.93) are
+ fulfilled; now(20080306) ok for Csparse; not yet for %*%
+
+ "d" <> "l" coercion for all "[TCR]" sparse matrices is really trivial:
+ "d" > "l" : drops the 'x' slot
+ "l" > "d" : construct an 'x' slot of all '1'
+ We currently have many of these conversions explicitly, e.g.
+ setAs("dsTMatrix", "lsTMatrix",
+ function(from) new("lsTMatrix", i = from@i, j = from@j, uplo = from@uplo,
+ Dim = from@Dim, Dimnames = from@Dimnames))
+ but I would rather want to automatically construct all these coercion
+ methods at once by a ``method constructor'', i.e.,
+ for all "dsparse*" > "lsparse*" and vice versa.
+ How can one do this {in a documented way} ?
+
+ Think of constructing setAs(...) calls automatically in order to
+ basically enable all ``sensible'' as(fromMatrix, toMatrix) calls,
+ possibly using canCoerce(.)
+
+ setAs(, "[dln]Matrix") for in {Matrix or denseMatrix + sparseMatrix}
+
+ When we have a packed matrix, it's a waste to go through "full" to "sparse":
+ ==> implement
+ setAs("dspMatrix", "sparseMatrix")
+ setAs("dppMatrix", "sparseMatrix")
+ setAs("dtpMatrix", "sparseMatrix")
+ and the same for "lsp" , "ltp" and "nsp" , "ntp" !
+
+ tcrossprod(x, y) : do provide methods for y != NULL
+ calling Lapack's DGEMM for "dense"
+ [200512xx: done for dgeMatrix at least]
+
+ BUGlet: Shouldn't lose factorization here:
+ h6 < Hilbert(6); chol(h6) ; str(h6) # has factor
+ str(H6 < as(h6, "dspMatrix")) # has lost factor
+ ## and the same in a similar situation involving "dpo", "dpp"
+
+ Factorizations: LU done; also Schur() for *sparse* Matrices.
+
+ is.na() method for all our matrices [ ==> which(*, arr.ind=TRUE) might work ]
+
+ use .Call(Csparse_drop, M, tol) in more places,
+ both with 'tol = 0.' to drop "values that happen to be 0" and for
+ zapsmall() methods for Csparse*
+
+ implement .Call(Csparse_scale, ....) interfacing to cholmod_scale()
+ in src/CHOLMOD/Include/cholmod_matrixops.h : for another function
+ specifically for multiplying a cholmod_sparse object by a diagonal matrix.
+ Use it in %*% and [t]crossprod methods.
+
+ chol() and determinant() should ``work'': proper result or "good" error
+ message.
+
+ make sure *all* group methods have (maybe "bailout") setMethod for "Matrix".
+ e.g. zapsmall() fails "badly"
+
+ sum(): implement methods which work for *all* our matrices.
+
+ Implement expand(.) for the Cholesky() results
+ "dCHMsimpl" and "dCHMsuper"  currently have no *decent* way to get at
+ the matrix factors of the corresponding matrix factorization !!
+
+ rbind2(, ) does not work (e.g. , )
+
+ %*% {also in crossprod/tcrossprod} currently always
+ returns , since > Csparse_dense_prod > cholmod_sdmult
+ and that does only return dense.
+ When the sparse matrix is very sparse, i.e. has many rows with only zero
+ entries, it would make much sense to return sparse.
+
+ sparsesymmetric + diagonal should stay sparsesymmetric
+ (only stays sparse): Matrix(0, 4, 4) + Diagonal(4, 1:4)
+ > R/diagMatrix.R ('FIXME')
+ but also R/Ops.R to ensure spsym. + spsym. > spsym. etc
+
+ Diagonal(n) %*% A  too slow!! > ~/R/MM/Pkgex/Matrix/diagTamasex.R
+
+ ! loses symmetry, both for dense and sparse matrices.
+ !M where M is "sparseMatrix", currently always gives dense. This only
+ makes sense when M is ``really sparse''.
+
+ msy < as(matrix(c(2:1,1:2),2), "dsyMatrix"); str(msy)
+
+ shows that the Cholesky factorization is computed ``too quickly''.
+ Can be a big pain for largish matrices, when it is unneeded.
+
+ example(Cholesky, echo=FALSE) ; cm < chol(mtm); str(cm); str(mtm)
+
+ shows that chol() does not seems to make use of an already
+ present factorization and rather uses one with more '0' in x slot.
+
+ diag(m) < val currently automatically works via m[cbind(i,i)] < val
+ This (`[<` method) is now "smart" for diagonalMatrix, but needs also to
+ be for triangularMatrix, and probably also "dense*general*Matrix" since the
+ above currently goes via "matrix" and back instead of using the 'x' slot
+ directly; in particular, the triangular* "class property" is lost!
+
+ examples for solve( Cholesky(.), b, system = c("A", "LDLt"....))
+ probably rather in man/CHMfactorclass.Rd than man/Cholesky.Rd
+
+ LDL() looks relatively easy; via "tCsparse_diag()"
+ {diagonal entries of *triangular* Csparse}
+ > see comment in determinant() in R/dsCMatrix.R, will give
+ faster determinant
+
+ tr(A %*% B) {and even tr(A %*% B %*% C) ...} are also needed
+ frequently in some computations {conditional normal distr. ...}.
+ Since this can be done faster than by
+ sum(diag(A %*% B)) even for traditional matrices, e.g.
+ sum(A * t(B)) or {even faster for "full" mat}
+ crossprod(as.vector(A), as.vector(B))
+ and even more so for, e.g. %*%
+ {used in Soeren's 'gR' computations},
+ we should also provide a generic and methods.
+
+ qr.R(qr(x)) may differ for the "same" matrix, depending on it being
+ sparse or dense:
+ "qr.R() may differ from qr.R() because of permutations"
+
+ This is not really acceptable and currently influences rcond() as well.
+
+ eigen() should become generic, and get a method at least for diagonal,
+ but also for symmetric > dsyMatrix [LAPACK dsyev() uses UPLO !],
+ but also simply for dgeMatrix (without going via tradition matrices).
+ What about Sparse? There's fillin, but it may still be sensible, e.g.
+ mlist < list(1, 2:3, diag(x=5:3), 27, cbind(1,3:6), 100:101)
+ ee < eigen(tcrossprod(bdiag(lapply(mlist, as.matrix))))
+ Matrix( signif(ee$vectors, 3) )
+
+ facmul() has no single method defined; it looks like a good idea though
+ (instead of the infamous qr.qy, qr.qty,.... functions)
+
+ symmpart() and skewpart() for *sparse* matrices still use (x +/ t(x))/2
+ and could be made more efficient.
+ Consider going via asTuniq() or something very close to
+ .Arith.Csparse() in R/Ops.R
+
+ many setAs(*, "[dl]..Matrix") are still needed, as long as e.g.
+ replCmat() uses as_CspClass() and drop0(.) which itself call
+ as_CspClass() quite a bit. > try to replace these by
+ as(*, "CsparseMatrix"); forceSymmetric, etc.
+
+ implement fast diag() via calling new
+ src/Csparse.c's diag_tC_ptr()
+
+ add examples (and tests!) for update(, ..) and
+ Cholesky(......, Imult), also tests for hidden {hence no examples}
+ ldetL2up() { R/CHMfactor.R }
+
+ chol() gives "temporarily disabled"
+ but should give the *symbolic* factorization;
+ similarly Cholesky(.) is not enabled
+
+ writeMM(obj, file=stdout()) creates file "1" since file is silently
+ assumed to be a string, i.e. cannot be a connection.
+ An R (instead of C) version should be pretty simple, and would work with
+ connections automatically ["lsparse" become either "real" or
+ "pattern", "depending if they have NAs or not].