SCM

SCM Repository

[matrix] Annotation of /pkg/Matrix/TODO
ViewVC logotype

Annotation of /pkg/Matrix/TODO

Parent Directory Parent Directory | Revision Log Revision Log


Revision 2677 - (view) (download)

1 : mmaechler 2677 - Arm twisting ... and other convincing arguments by R core fellows:
2 :     ==> R/zzz.R --- do NOT redefine base, but define an S3 method
3 :     ------- for as.array() etc
4 :    
5 : maechler 2112 - Check for DimNames propagation in coercion and other operations.
6 :    
7 : bates 344 - Report the problem in the Linux ldexp manual page. The second and
8 :     third calls in the Synopsis should be to ldexpf and ldexpl.
9 : maechler 472
10 : maechler 634 - provide methods for "dspMatrix" and "dppMatrix"!
11 :    
12 : maechler 538 - implement (more) methods for supporting "packed" (symmetric / triangular)
13 : maechler 634 matrices; particularly something like pack() and unpack() [to/from our
14 :     classes from/to "numeric"] --- have already man/unpack.Rd but no method yet!
15 : maechler 538
16 : maechler 834 (have some dtr* <-> dtp*)
17 :    
18 : bates 645 - combine the C functions for multiplication by special forms and
19 :     solution wrt special forms by using a 'right' argument and a
20 :     'classed' argument.
21 : maechler 675 [done with dgeMatrix_matrix_mm(); not yet for other classes;
22 :     and for _crossprod()]
23 :    
24 : bates 1578 -----
25 : bates 712
26 : maechler 935 - "Math2" , "Math", "Arith":
27 :     keep triangular and symmetric Matrices when appropriate:
28 :     particularly desirable for "Math2": round(), signif()
29 : maechler 956
30 : mmaechler 2582 - For triangular matrices, make sure the four rules of
31 :     "triangular matrix algebra" (Golub+Van Loan 1996, 3.1.8, p.93) are fulfilled.
32 :     - since 2008-03-06 ok for Csparse
33 :     - since 2010-07-23 ok for <dtr> %*% <dtr>
34 :     TODO: e.g. for <ltr> %*% <dtC>
35 : maechler 2120
36 : maechler 956 - "d" <-> "l" coercion for all "[TCR]" sparse matrices is really trivial:
37 :     "d" -> "l" : drops the 'x' slot
38 :     "l" -> "d" : construct an 'x' slot of all '1'
39 :     We currently have many of these conversions explicitly, e.g.
40 :     setAs("dsTMatrix", "lsTMatrix",
41 :     function(from) new("lsTMatrix", i = from@i, j = from@j, uplo = from@uplo,
42 :     Dim = from@Dim, Dimnames = from@Dimnames))
43 :     but I would rather want to automatically construct all these coercion
44 :     methods at once by a ``method constructor'', i.e.,
45 :     for all "dsparse*" -> "lsparse*" and vice versa.
46 :     How can one do this {in a documented way} ?
47 : maechler 1087
48 : maechler 2048 - Think of constructing setAs(...) calls automatically in order to
49 :     basically enable all ``sensible'' as(fromMatrix, toMatrix) calls,
50 :     possibly using canCoerce(.)
51 :    
52 :     - When we have a packed matrix, it's a waste to go through "full" to "sparse":
53 :     ==> implement
54 :     setAs("dspMatrix", "sparseMatrix")
55 :     setAs("dppMatrix", "sparseMatrix")
56 :     setAs("dtpMatrix", "sparseMatrix")
57 :     and the same for "lsp" , "ltp" and "nsp" , "ntp" !
58 :    
59 : maechler 1097 - tcrossprod(x, y) : do provide methods for y != NULL
60 :     calling Lapack's DGEMM for "dense"
61 : maechler 1109 [2005-12-xx: done for dgeMatrix at least]
62 :    
63 : maechler 1592 - Factorizations: LU done; also Schur() for *sparse* Matrices.
64 : maechler 1253
65 : maechler 1619 - use .Call(Csparse_drop, M, tol) in more places,
66 :     both with 'tol = 0.' to drop "values that happen to be 0" and for
67 :     zapsmall() methods for Csparse*
68 : maechler 1654
69 :     - implement .Call(Csparse_scale, ....) interfacing to cholmod_scale()
70 :     in src/CHOLMOD/Include/cholmod_matrixops.h : for another function
71 :     specifically for multiplying a cholmod_sparse object by a diagonal matrix.
72 :     Use it in %*% and [t]crossprod methods.
73 :    
74 : mmaechler 2193 - chol() should ``work'': proper result or "good" error message.
75 : maechler 1659
76 : maechler 1714 - make sure *all* group methods have (maybe "bail-out") setMethod for "Matrix".
77 :     e.g. zapsmall(<pMatrix>) fails "badly"
78 : maechler 1725
79 : maechler 2112 - rbind2(<sparse>, <dense>) does not work (e.g. <dgC>, <dge>)
80 : maechler 1799
81 : maechler 1833 - <sparse> %*% <dense> {also in crossprod/tcrossprod} currently always
82 :     returns <dense>, since --> Csparse_dense_prod --> cholmod_sdmult
83 :     and that does only return dense.
84 :     When the sparse matrix is very sparse, i.e. has many rows with only zero
85 :     entries, it would make much sense to return sparse.
86 :    
87 :     - ! <symmetricMatrix> loses symmetry, both for dense and sparse matrices.
88 :     !M where M is "sparseMatrix", currently always gives dense. This only
89 :     makes sense when M is ``really sparse''.
90 : maechler 1855
91 : maechler 2018 - example(Cholesky, echo=FALSE) ; cm <- chol(mtm); str(cm); str(mtm)
92 :    
93 : mmaechler 2493 shows that chol() does not seem to make use of an already
94 : maechler 2018 present factorization and rather uses one with more '0' in x slot.
95 :    
96 : maechler 2005 - diag(m) <- val currently automatically works via m[cbind(i,i)] <- val
97 : maechler 2115 This (`[<-` method) is now "smart" for diagonalMatrix, but needs also to
98 :     be for triangularMatrix, and probably also "dense*general*Matrix" since the
99 : maechler 2005 above currently goes via "matrix" and back instead of using the 'x' slot
100 : maechler 2115 directly; in particular, the triangular* "class property" is lost!
101 : maechler 2043
102 : mmaechler 2661 Note that 'diag(M[,-1]) <- val' is deadly slow (*) for large sparse M,
103 :     but that's because of the 2nd line assignment in the equivalent
104 :     tmpM <- `diag<-`(M[,-1], val)
105 :     M[,-1] <- tmpM
106 :     (*): gives *error* {about negative integer} when
107 :     prod(dim(M)) > .Machine$integer.max, e.g. for square (n x n) M when
108 :     n >= 46341 == ceiling(2^15.5)
109 :     This is "the same" as Ashley Ford's report (25 Feb 2010),
110 :     MM @ ~/R/MM/Pkg-ex/Matrix/nsp-2col-index-bug.R
111 : mmaechler 2207
112 : maechler 2043 - examples for solve( Cholesky(.), b, system = c("A", "LDLt"....))
113 :     probably rather in man/CHMfactor-class.Rd than man/Cholesky.Rd
114 : maechler 2072
115 : maechler 2137 - LDL(<CHMsimpl>) looks relatively easy; via "tCsparse_diag()"
116 :     {diagonal entries of *triangular* Csparse}
117 :     --> see comment in determinant(<dsC>) in R/dsCMatrix.R, will give
118 :     faster determinant
119 :    
120 : maechler 2072 - tr(A %*% B) {and even tr(A %*% B %*% C) ...} are also needed
121 :     frequently in some computations {conditional normal distr. ...}.
122 :     Since this can be done faster than by
123 :     sum(diag(A %*% B)) even for traditional matrices, e.g.
124 :     sum(A * t(B)) or {even faster for "full" mat}
125 :     crossprod(as.vector(A), as.vector(B))
126 :     and even more so for, e.g. <sparse> %*% <dense>
127 :     {used in Soeren's 'gR' computations},
128 :     we should also provide a generic and methods.
129 : mmaechler 2618 - diag(A %*% B) might look like a "generalization" of tr(A %*% B),
130 :     but as the above tricks show, is not really.
131 : mmaechler 2661 Still, it's well worth to provide diag.prod(A, B)
132 : maechler 2103
133 :     - qr.R(qr(x)) may differ for the "same" matrix, depending on it being
134 :     sparse or dense:
135 :     "qr.R(<sparse>) may differ from qr.R(<dense>) because of permutations"
136 :    
137 :     This is not really acceptable and currently influences rcond() as well.
138 : maechler 2105
139 : maechler 2106 - eigen() should become generic, and get a method at least for diagonal,
140 :     but also for symmetric -> dsyMatrix [LAPACK dsyev() uses UPLO !],
141 :     but also simply for dgeMatrix (without going via tradition matrices).
142 :     What about Sparse? There's fill-in, but it may still be sensible, e.g.
143 :     mlist <- list(1, 2:3, diag(x=5:3), 27, cbind(1,3:6), 100:101)
144 :     ee <- eigen(tcrossprod(bdiag(lapply(mlist, as.matrix))))
145 :     Matrix( signif(ee$vectors, 3) )
146 :    
147 :     - facmul() has no single method defined; it looks like a good idea though
148 :     (instead of the infamous qr.qy, qr.qty,.... functions)
149 : maechler 2112
150 :     - symmpart() and skewpart() for *sparse* matrices still use (x +/- t(x))/2
151 :     and could be made more efficient.
152 :     Consider going via asTuniq() or something very close to
153 :     .Arith.Csparse() in R/Ops.R
154 : mmaechler 2486 For a traditional "matrix" object, we should speedup, using C code ..
155 : maechler 2112
156 : maechler 2115 - many setAs(*, "[dl]..Matrix") are still needed, as long as e.g.
157 :     replCmat() uses as_CspClass() and drop0(.) which itself call
158 :     as_CspClass() quite a bit. --> try to replace these by
159 :     as(*, "CsparseMatrix"); forceSymmetric, etc.
160 : maechler 2151
161 : maechler 2154 - implement fast diag(<triangularCsparse>) via calling new
162 :     src/Csparse.c's diag_tC_ptr()
163 :    
164 : maechler 2151 - add examples (and tests!) for update(<CHMfactor>, ..) and
165 :     Cholesky(......, Imult), also tests for hidden {hence no examples}
166 :     ldetL2up() { R/CHMfactor.R }
167 : maechler 2154
168 :     - chol(<nsCMatrix>) gives "temporarily disabled"
169 :     but should give the *symbolic* factorization;
170 :     similarly Cholesky(.) is not enabled
171 : mmaechler 2175
172 :     - writeMM(obj, file=stdout()) creates file "1" since file is silently
173 :     assumed to be a string, i.e. cannot be a connection.
174 :     An R (instead of C) version should be pretty simple, and would work with
175 :     connections automatically ["lsparse" become either "real" or
176 :     "pattern", "depending if they have NAs or not].
177 : mmaechler 2196
178 : mmaechler 2269 - <diagMatrix> o <ddenseMatrix> still works via sparse in some cases, but
179 :     could return <diagMatrix> in the same cases where <diagMatrix> o <numeric> does.
180 : mmaechler 2207
181 :     - look at solve.QP.compact() in \pkg{quadprog} and how to do that using
182 :     our sparse matrices. Maybe this needs to be re-implemented using CHOLMOD
183 :     routines.
184 :    
185 : mmaechler 2236 - We allow "over-allocated" (i,x)-slots for CsparseMatrix objects,
186 :     as per Csparse_validate() and the tests in tests/validObj.R. This is as
187 :     in CHOLMOD/CSparse, where nzmax (>= .@p[n]) corresponds to length(.@i),
188 :     and makes sense e.g. for M[.,.] <- v assignments which could allocate in
189 :     chunks and would not need to re-allocate anything in many cases.
190 :     HOWEVER, replCmat() in R/Csparse.R is still far from making use of that.
191 :    
192 : mmaechler 2271 - advertize rbind2() / cbind2() and (rather?) rBind() / cBind()
193 : mmaechler 2239 ------ -----
194 :     in all vignettes / talks / ... !!
195 :     People erronously try rbind/cbind see that they don't work and then
196 :     reinvent the wheel!
197 : dmbates 2277
198 : mmaechler 2341 --> Consider using the new 'dotMethods' functionality to define
199 :     cbind() and rbind() versions that work with Matrix.
200 : mmaechler 2500 The "Rmpfr" package does that now.
201 : mmaechler 2341
202 : mmaechler 2345 - In all(M1 == M2) for sparse large matrices M1, M2 (e.g. M2 <- M1 !),
203 :     the intermediate 'M1 == M2' typically is dense, hence potentially using
204 :     humongous amount of memory.
205 :     We should/could devise something like allCompare(M1, M2, `==`)
206 :     which would remain sparse in all its computations.
207 :    
208 : dmbates 2277 --------
209 :    
210 :     - Reconsider the linkages in the include files for the SuiteSparse
211 :     packages. It may be better simply to add all the src/<nm>/Include
212 :     directories to the include path for all compilations. I don't think
213 :     there is a big overhead. Right now we need to modify the include
214 :     file src/SPQR/Include/SuiteSparseQR_C.h so that it does not expect
215 :     to have src/UFsparse and src/CHOLMOD/Include on the include path.
216 :     Maybe just those two should be added to the include path.
217 : mmaechler 2341
218 : mmaechler 2490 - (systematically check that LAPACK-calling functions check for
219 :     0-dimensional input themselves; LAPACK gives an integer error code)
220 : mmaechler 2497
221 :     - the f[,5762] <- thisCol # now ...
222 :     line in tests/indexing.R uses very large objects unnecessarily;
223 :     Improve replTmat() in R/Tsparse.R, making use of new "abIndex"
224 :     vectors.
225 : mmaechler 2538
226 :     - {IS THIS CURRENT?}
227 :     Sept. 2009:
228 :     Subject: chol2inv() |-> solve(<CHMfactor>)
229 :    
230 :     when testing and documenting chol2inv(),
231 :     I found that it's pretty simple to also define a method for
232 :     "CHMfactor" objects, namely simply the solve(*, Diagonal(.) "A")
233 :     method.
234 :     This is not particularly exciting, and also does *not*, I think
235 :     help for defining a chol2inv() method for *sparse* (upper)
236 :     triangular matrices.
237 : mmaechler 2553
238 : mmaechler 2592 - sort(<sparseVector>, partial=..), needed, for mean(*, trim = .) or median().
239 : mmaechler 2553 Note that defining xtfrm() does not "help" (as sort() then goes via dense
240 :     index). See "mean" in R/Matrix.R
241 : mmaechler 2572
242 :     - rcond(<sparseMatrix>) - for square currently goes via *dense* -- BAD --
243 :     can we go via qr() in any case?
244 :     In some cases, e.g. lmer()'s "Lambda" (block triangular, small blocks)
245 :     rcond(L) := 1 / (norm(L) * norm(solve(L)))
246 :     is simple {and remains sparse, as solve(L) is still block triangular}
247 : mmaechler 2661
248 :     - How can we ensure that inst/include/cholmod.h remains
249 : mmaechler 2666 correct and equivalent to src/CHOLMOD/Include/cholmod_core.h and siblings ???
250 :     {currently need to do this manually (Emacs M-x compare-windows) for the typedefs}
251 :    
252 :     - finalize and activate the new *unused* code in src/sparseVector.c

R-Forge@R-project.org
ViewVC Help
Powered by ViewVC 1.0.0  
Thanks to:
Vienna University of Economics and Business University of Wisconsin - Madison Powered By FusionForge