 pkg/TODO 2006/07/31 23:08:21 1339
+++ pkg/TODO 2008/04/23 11:23:50 2175
@@ 1,47 +1,8 @@

Modify mcmcsamp to return the deviance in the sample only when the
optional argument 'deviance' is TRUE (default FALSE).

Replace the use of the unexported replacement function LMEoptimize<
in lmer.R by the function LMEopt, which always returns NULL but
(in contradiction to the language specification) modifies its first argument.

Modify the oneargument form of the anova method for lmer objects (yet
 again) to calculate the F ratios. It is the df, not the ratio that
 is controversial.
Should there be an extractor function for the mean square error? If
 so, what should it be called?

Check for DimNames propagation in coercion and other operations.

 Revive the "lmer" and "glmer" classes. Slots like "family" should
 be in "glmer" only. Many methods for "lmer" can be simplified as,
 for example, they will always use the scale factor.

 Add a "pedigree" class and methods for it. Allow a pedigree
 argument in lmer.


 Sparse matrix methods can now be based on the CHOLMOD package. We
 will need to migrate from the current code to CHOLMODbased code
 using #ifdef USE_CHOLMOD. Some of the things to be done

  Move documentation from subdirectories of src to inst/doc
  Write utilities to create a cholmod_sparse pointer from a
 dgCMatrix or lgCMatrix (or zgCMatrix) object without copying and
 allocating.
  Start adding simple S4 methods (rcond, %*%, +, cbind, t).
+ Check for DimNames propagation in coercion and other operations.
 Report the problem in the Linux ldexp manual page. The second and
third calls in the Synopsis should be to ldexpf and ldexpl.
 [,] indexing: for sparse "works", but not yet for negative indices!

 consider moving alloc3Darray from ./src/Mutils.c to
 $(RSRC)/src/base/array.c



 provide methods for "dspMatrix" and "dppMatrix"!
 implement (more) methods for supporting "packed" (symmetric / triangular)
@@ 50,28 +11,22 @@
(have some dtr* <> dtp*)


 combine the C functions for multiplication by special forms and
solution wrt special forms by using a 'right' argument and a
'classed' argument.
[done with dgeMatrix_matrix_mm(); not yet for other classes;
and for _crossprod()]
 add more comprehensive examples / tests for Schur decomposition

 arithmetic for sparse matrices:
 o
 should return a sparse matrix for at least "+" and "*" , also %%,
 and "/" and "%/%" at least when the RHS is nonzero a scalar.
 Challenge: nice implementation (``common non0''; but Tsparse* is not uniq).


+
 "Math2" , "Math", "Arith":
keep triangular and symmetric Matrices when appropriate:
particularly desirable for "Math2": round(), signif()
+ For triangular matrices, more specifically make sure the four rules of
+ "triangular matrix algebra" (Golub+Van Loan 1996, 3.1.8, p.93) are
+ fulfilled; now(20080306) ok for Csparse; not yet for %*%
+
 "d" <> "l" coercion for all "[TCR]" sparse matrices is really trivial:
"d" > "l" : drops the 'x' slot
"l" > "d" : construct an 'x' slot of all '1'
@@ 84,35 +39,146 @@
for all "dsparse*" > "lsparse*" and vice versa.
How can one do this {in a documented way} ?
+ Think of constructing setAs(...) calls automatically in order to
+ basically enable all ``sensible'' as(fromMatrix, toMatrix) calls,
+ possibly using canCoerce(.)
+
+ setAs(, "[dln]Matrix") for in {Matrix or denseMatrix + sparseMatrix}
+
+ When we have a packed matrix, it's a waste to go through "full" to "sparse":
+ ==> implement
+ setAs("dspMatrix", "sparseMatrix")
+ setAs("dppMatrix", "sparseMatrix")
+ setAs("dtpMatrix", "sparseMatrix")
+ and the same for "lsp" , "ltp" and "nsp" , "ntp" !
+
 tcrossprod(x, y) : do provide methods for y != NULL
calling Lapack's DGEMM for "dense"
[200512xx: done for dgeMatrix at least]
 "ltTmatrix" :
 2) new("ltTMatrix", diag = "U", Dim = c(2:2, 2:2)) shows lack of ltTM*_validate

 BUGlet: Shouldn't lose factorization here:
h6 < Hilbert(6); chol(h6) ; str(h6) # has factor
str(H6 < as(h6, "dspMatrix")) # has lost factor
## and the same in a similar situation involving "dpo", "dpp"
 "Compare" for dgC & dgT  want to be able to do M[M > 10] etc

 Things like M[upper.tri(M)] are not really most useful for sparse
 matrices. > provide generic functions
 upperTriMatrix(), lowerTriMatrix() both with argument 'diag = TRUE'
 (which can be set to FALSE of course) which are used to extract a
 triangle from an arbitrary sparse matrix and return a "dtCMatrix".

 Factorizations: LU [ > solve() ] and Schur() for *sparse* Matrices.
  LUsparse is the purpose of all of ./src/UMFPACK/
 > ./inst/doc/UFsparse/*
 Note that UMFPACK is a bit slow compared to TAUCS etc, according to
 Gould et al.(2005)
 And then, possibly eigen(), svd(), qr(), ... for sparse
 {probably base some on the fast chol() of CHOLMOD}.
 ./man/lu.Rd notes that lu() is not yet available for sparse
+ Factorizations: LU done; also Schur() for *sparse* Matrices.
 20060415 : now with 'CSparse' { src/cs.c }, have at least solve( )
+ is.na() method for all our matrices [ ==> which(*, arr.ind=TRUE) might work ]
 nicer pedigree examples and tests
+ use .Call(Csparse_drop, M, tol) in more places,
+ both with 'tol = 0.' to drop "values that happen to be 0" and for
+ zapsmall() methods for Csparse*
+
+ implement .Call(Csparse_scale, ....) interfacing to cholmod_scale()
+ in src/CHOLMOD/Include/cholmod_matrixops.h : for another function
+ specifically for multiplying a cholmod_sparse object by a diagonal matrix.
+ Use it in %*% and [t]crossprod methods.
+
+ chol() and determinant() should ``work'': proper result or "good" error
+ message.
+
+ make sure *all* group methods have (maybe "bailout") setMethod for "Matrix".
+ e.g. zapsmall() fails "badly"
+
+ sum(): implement methods which work for *all* our matrices.
+
+ Implement expand(.) for the Cholesky() results
+ "dCHMsimpl" and "dCHMsuper"  currently have no *decent* way to get at
+ the matrix factors of the corresponding matrix factorization !!
+
+ rbind2(, ) does not work (e.g. , )
+
+ %*% {also in crossprod/tcrossprod} currently always
+ returns , since > Csparse_dense_prod > cholmod_sdmult
+ and that does only return dense.
+ When the sparse matrix is very sparse, i.e. has many rows with only zero
+ entries, it would make much sense to return sparse.
+
+ sparsesymmetric + diagonal should stay sparsesymmetric
+ (only stays sparse): Matrix(0, 4, 4) + Diagonal(4, 1:4)
+ > R/diagMatrix.R ('FIXME')
+ but also R/Ops.R to ensure spsym. + spsym. > spsym. etc
+
+ Diagonal(n) %*% A  too slow!! > ~/R/MM/Pkgex/Matrix/diagTamasex.R
+
+ ! loses symmetry, both for dense and sparse matrices.
+ !M where M is "sparseMatrix", currently always gives dense. This only
+ makes sense when M is ``really sparse''.
+
+ msy < as(matrix(c(2:1,1:2),2), "dsyMatrix"); str(msy)
+
+ shows that the Cholesky factorization is computed ``too quickly''.
+ Can be a big pain for largish matrices, when it is unneeded.
+
+ example(Cholesky, echo=FALSE) ; cm < chol(mtm); str(cm); str(mtm)
+
+ shows that chol() does not seems to make use of an already
+ present factorization and rather uses one with more '0' in x slot.
+
+ diag(m) < val currently automatically works via m[cbind(i,i)] < val
+ This (`[<` method) is now "smart" for diagonalMatrix, but needs also to
+ be for triangularMatrix, and probably also "dense*general*Matrix" since the
+ above currently goes via "matrix" and back instead of using the 'x' slot
+ directly; in particular, the triangular* "class property" is lost!
+
+ examples for solve( Cholesky(.), b, system = c("A", "LDLt"....))
+ probably rather in man/CHMfactorclass.Rd than man/Cholesky.Rd
+
+ LDL() looks relatively easy; via "tCsparse_diag()"
+ {diagonal entries of *triangular* Csparse}
+ > see comment in determinant() in R/dsCMatrix.R, will give
+ faster determinant
+
+ tr(A %*% B) {and even tr(A %*% B %*% C) ...} are also needed
+ frequently in some computations {conditional normal distr. ...}.
+ Since this can be done faster than by
+ sum(diag(A %*% B)) even for traditional matrices, e.g.
+ sum(A * t(B)) or {even faster for "full" mat}
+ crossprod(as.vector(A), as.vector(B))
+ and even more so for, e.g. %*%
+ {used in Soeren's 'gR' computations},
+ we should also provide a generic and methods.
+
+ qr.R(qr(x)) may differ for the "same" matrix, depending on it being
+ sparse or dense:
+ "qr.R() may differ from qr.R() because of permutations"
+
+ This is not really acceptable and currently influences rcond() as well.
+
+ eigen() should become generic, and get a method at least for diagonal,
+ but also for symmetric > dsyMatrix [LAPACK dsyev() uses UPLO !],
+ but also simply for dgeMatrix (without going via tradition matrices).
+ What about Sparse? There's fillin, but it may still be sensible, e.g.
+ mlist < list(1, 2:3, diag(x=5:3), 27, cbind(1,3:6), 100:101)
+ ee < eigen(tcrossprod(bdiag(lapply(mlist, as.matrix))))
+ Matrix( signif(ee$vectors, 3) )
+
+ facmul() has no single method defined; it looks like a good idea though
+ (instead of the infamous qr.qy, qr.qty,.... functions)
+
+ symmpart() and skewpart() for *sparse* matrices still use (x +/ t(x))/2
+ and could be made more efficient.
+ Consider going via asTuniq() or something very close to
+ .Arith.Csparse() in R/Ops.R
+
+ many setAs(*, "[dl]..Matrix") are still needed, as long as e.g.
+ replCmat() uses as_CspClass() and drop0(.) which itself call
+ as_CspClass() quite a bit. > try to replace these by
+ as(*, "CsparseMatrix"); forceSymmetric, etc.
+
+ implement fast diag() via calling new
+ src/Csparse.c's diag_tC_ptr()
+
+ add examples (and tests!) for update(, ..) and
+ Cholesky(......, Imult), also tests for hidden {hence no examples}
+ ldetL2up() { R/CHMfactor.R }
+
+ chol() gives "temporarily disabled"
+ but should give the *symbolic* factorization;
+ similarly Cholesky(.) is not enabled
+
+ writeMM(obj, file=stdout()) creates file "1" since file is silently
+ assumed to be a string, i.e. cannot be a connection.
+ An R (instead of C) version should be pretty simple, and would work with
+ connections automatically ["lsparse" become either "real" or
+ "pattern", "depending if they have NAs or not].