SCM

SCM Repository

[matrix] Annotation of /pkg/Matrix/TODO
ViewVC logotype

Annotation of /pkg/Matrix/TODO

Parent Directory Parent Directory | Revision Log Revision Log


Revision 2773 - (view) (download)

1 : mmaechler 2712 ##-*- mode: org -*-
2 : mmaechler 2677
3 : mmaechler 2712 - Look at Paul Bailey's problem -- CHOLMOD error (even seg.fault for him)
4 :     --> ~/R/MM/Pkg-ex/Matrix/sparseOrderedLogit.R
5 :    
6 :     - BunchKaufman()'s result is not really useful yet {but it is used on C
7 :     level e.g. for solve(<dsyMatrix>). Should define expand() method or
8 :     similar, see man/BunchKaufman-methods.Rd and R/dsyMatrix.R (at end).
9 :    
10 :     ----------------------------------------
11 :    
12 :     - Investigate the "band changing (and getting) ideas 'band<-' etc,
13 : mmaechler 2739 from Jeremy D Silver, per posts to R-devel on Aug.26,2011
14 :     {MM: ~/R/MM/Pkg-ex/Matrix/bands-Jeremy_Silver-ex.R }
15 : mmaechler 2712
16 : mmaechler 2725 - cbind2() / rbind2() for sparseMatrices: dimnames propagation should
17 :     happen in C, see R/bind2.R and src/Csparse.c (Csparse_horzcat etc).
18 :    
19 : maechler 2112 - Check for DimNames propagation in coercion and other operations.
20 :    
21 : bates 344 - Report the problem in the Linux ldexp manual page. The second and
22 :     third calls in the Synopsis should be to ldexpf and ldexpl.
23 : maechler 472
24 : maechler 634 - provide methods for "dspMatrix" and "dppMatrix"!
25 :    
26 : maechler 538 - implement (more) methods for supporting "packed" (symmetric / triangular)
27 : maechler 634 matrices; particularly something like pack() and unpack() [to/from our
28 :     classes from/to "numeric"] --- have already man/unpack.Rd but no method yet!
29 : maechler 538
30 : maechler 834 (have some dtr* <-> dtp*)
31 :    
32 : bates 645 - combine the C functions for multiplication by special forms and
33 :     solution wrt special forms by using a 'right' argument and a
34 :     'classed' argument.
35 : maechler 675 [done with dgeMatrix_matrix_mm(); not yet for other classes;
36 :     and for _crossprod()]
37 :    
38 : bates 1578 -----
39 : bates 712
40 : maechler 935 - "Math2" , "Math", "Arith":
41 :     keep triangular and symmetric Matrices when appropriate:
42 :     particularly desirable for "Math2": round(), signif()
43 : maechler 956
44 : mmaechler 2582 - For triangular matrices, make sure the four rules of
45 :     "triangular matrix algebra" (Golub+Van Loan 1996, 3.1.8, p.93) are fulfilled.
46 :     - since 2008-03-06 ok for Csparse
47 :     - since 2010-07-23 ok for <dtr> %*% <dtr>
48 :     TODO: e.g. for <ltr> %*% <dtC>
49 : maechler 2120
50 : maechler 956 - "d" <-> "l" coercion for all "[TCR]" sparse matrices is really trivial:
51 :     "d" -> "l" : drops the 'x' slot
52 :     "l" -> "d" : construct an 'x' slot of all '1'
53 :     We currently have many of these conversions explicitly, e.g.
54 :     setAs("dsTMatrix", "lsTMatrix",
55 :     function(from) new("lsTMatrix", i = from@i, j = from@j, uplo = from@uplo,
56 :     Dim = from@Dim, Dimnames = from@Dimnames))
57 :     but I would rather want to automatically construct all these coercion
58 :     methods at once by a ``method constructor'', i.e.,
59 :     for all "dsparse*" -> "lsparse*" and vice versa.
60 :     How can one do this {in a documented way} ?
61 : maechler 1087
62 : maechler 2048 - Think of constructing setAs(...) calls automatically in order to
63 :     basically enable all ``sensible'' as(fromMatrix, toMatrix) calls,
64 :     possibly using canCoerce(.)
65 :    
66 :     - When we have a packed matrix, it's a waste to go through "full" to "sparse":
67 :     ==> implement
68 :     setAs("dspMatrix", "sparseMatrix")
69 :     setAs("dppMatrix", "sparseMatrix")
70 :     setAs("dtpMatrix", "sparseMatrix")
71 :     and the same for "lsp" , "ltp" and "nsp" , "ntp" !
72 :    
73 : maechler 1097 - tcrossprod(x, y) : do provide methods for y != NULL
74 :     calling Lapack's DGEMM for "dense"
75 : maechler 1109 [2005-12-xx: done for dgeMatrix at least]
76 :    
77 : maechler 1592 - Factorizations: LU done; also Schur() for *sparse* Matrices.
78 : maechler 1253
79 : maechler 1619 - use .Call(Csparse_drop, M, tol) in more places,
80 :     both with 'tol = 0.' to drop "values that happen to be 0" and for
81 :     zapsmall() methods for Csparse*
82 : maechler 1654
83 :     - implement .Call(Csparse_scale, ....) interfacing to cholmod_scale()
84 :     in src/CHOLMOD/Include/cholmod_matrixops.h : for another function
85 :     specifically for multiplying a cholmod_sparse object by a diagonal matrix.
86 :     Use it in %*% and [t]crossprod methods.
87 :    
88 : mmaechler 2193 - chol() should ``work'': proper result or "good" error message.
89 : maechler 1659
90 : maechler 1714 - make sure *all* group methods have (maybe "bail-out") setMethod for "Matrix".
91 :     e.g. zapsmall(<pMatrix>) fails "badly"
92 : maechler 1725
93 : maechler 2112 - rbind2(<sparse>, <dense>) does not work (e.g. <dgC>, <dge>)
94 : maechler 1799
95 : maechler 1833 - <sparse> %*% <dense> {also in crossprod/tcrossprod} currently always
96 :     returns <dense>, since --> Csparse_dense_prod --> cholmod_sdmult
97 :     and that does only return dense.
98 :     When the sparse matrix is very sparse, i.e. has many rows with only zero
99 :     entries, it would make much sense to return sparse.
100 :    
101 :     - ! <symmetricMatrix> loses symmetry, both for dense and sparse matrices.
102 :     !M where M is "sparseMatrix", currently always gives dense. This only
103 :     makes sense when M is ``really sparse''.
104 : maechler 1855
105 : maechler 2018 - example(Cholesky, echo=FALSE) ; cm <- chol(mtm); str(cm); str(mtm)
106 :    
107 : mmaechler 2493 shows that chol() does not seem to make use of an already
108 : maechler 2018 present factorization and rather uses one with more '0' in x slot.
109 :    
110 : maechler 2005 - diag(m) <- val currently automatically works via m[cbind(i,i)] <- val
111 : maechler 2115 This (`[<-` method) is now "smart" for diagonalMatrix, but needs also to
112 :     be for triangularMatrix, and probably also "dense*general*Matrix" since the
113 : maechler 2005 above currently goes via "matrix" and back instead of using the 'x' slot
114 : maechler 2115 directly; in particular, the triangular* "class property" is lost!
115 : mmaechler 2688 [current ??]
116 : maechler 2043
117 : mmaechler 2688 - The "[<-" now uses src/t_Csparse_subassign.c and no longer explodes
118 :     memory. *However* it is still too slow when the replacment region is large.
119 : mmaechler 2207
120 : mmaechler 2688
121 : maechler 2043 - examples for solve( Cholesky(.), b, system = c("A", "LDLt"....))
122 :     probably rather in man/CHMfactor-class.Rd than man/Cholesky.Rd
123 : maechler 2072
124 : maechler 2137 - LDL(<CHMsimpl>) looks relatively easy; via "tCsparse_diag()"
125 :     {diagonal entries of *triangular* Csparse}
126 :     --> see comment in determinant(<dsC>) in R/dsCMatrix.R, will give
127 :     faster determinant
128 :    
129 : maechler 2072 - tr(A %*% B) {and even tr(A %*% B %*% C) ...} are also needed
130 :     frequently in some computations {conditional normal distr. ...}.
131 :     Since this can be done faster than by
132 :     sum(diag(A %*% B)) even for traditional matrices, e.g.
133 :     sum(A * t(B)) or {even faster for "full" mat}
134 :     crossprod(as.vector(A), as.vector(B))
135 :     and even more so for, e.g. <sparse> %*% <dense>
136 :     {used in Soeren's 'gR' computations},
137 :     we should also provide a generic and methods.
138 : mmaechler 2618 - diag(A %*% B) might look like a "generalization" of tr(A %*% B),
139 :     but as the above tricks show, is not really.
140 : mmaechler 2684 Still, it's well worth to provide diag.prod(A, B):
141 : maechler 2103
142 : mmaechler 2684 Well, if A %*% B is square, diag(A %*% B) === colSums(t(A) * B)
143 :     and we should probably teach people about that !
144 :    
145 : maechler 2103 - qr.R(qr(x)) may differ for the "same" matrix, depending on it being
146 :     sparse or dense:
147 :     "qr.R(<sparse>) may differ from qr.R(<dense>) because of permutations"
148 :    
149 :     This is not really acceptable and currently influences rcond() as well.
150 : maechler 2105
151 : maechler 2106 - eigen() should become generic, and get a method at least for diagonal,
152 :     but also for symmetric -> dsyMatrix [LAPACK dsyev() uses UPLO !],
153 :     but also simply for dgeMatrix (without going via tradition matrices).
154 :     What about Sparse? There's fill-in, but it may still be sensible, e.g.
155 :     mlist <- list(1, 2:3, diag(x=5:3), 27, cbind(1,3:6), 100:101)
156 :     ee <- eigen(tcrossprod(bdiag(lapply(mlist, as.matrix))))
157 :     Matrix( signif(ee$vectors, 3) )
158 :    
159 :     - facmul() has no single method defined; it looks like a good idea though
160 :     (instead of the infamous qr.qy, qr.qty,.... functions)
161 : maechler 2112
162 :     - symmpart() and skewpart() for *sparse* matrices still use (x +/- t(x))/2
163 :     and could be made more efficient.
164 :     Consider going via asTuniq() or something very close to
165 :     .Arith.Csparse() in R/Ops.R
166 : mmaechler 2486 For a traditional "matrix" object, we should speedup, using C code ..
167 : maechler 2112
168 : maechler 2115 - many setAs(*, "[dl]..Matrix") are still needed, as long as e.g.
169 :     replCmat() uses as_CspClass() and drop0(.) which itself call
170 :     as_CspClass() quite a bit. --> try to replace these by
171 :     as(*, "CsparseMatrix"); forceSymmetric, etc.
172 : maechler 2151
173 : maechler 2154 - implement fast diag(<triangularCsparse>) via calling new
174 :     src/Csparse.c's diag_tC_ptr()
175 :    
176 : maechler 2151 - add examples (and tests!) for update(<CHMfactor>, ..) and
177 :     Cholesky(......, Imult), also tests for hidden {hence no examples}
178 :     ldetL2up() { R/CHMfactor.R }
179 : maechler 2154
180 :     - chol(<nsCMatrix>) gives "temporarily disabled"
181 :     but should give the *symbolic* factorization;
182 :     similarly Cholesky(.) is not enabled
183 : mmaechler 2175
184 :     - writeMM(obj, file=stdout()) creates file "1" since file is silently
185 :     assumed to be a string, i.e. cannot be a connection.
186 :     An R (instead of C) version should be pretty simple, and would work with
187 :     connections automatically ["lsparse" become either "real" or
188 :     "pattern", "depending if they have NAs or not].
189 : mmaechler 2196
190 : mmaechler 2269 - <diagMatrix> o <ddenseMatrix> still works via sparse in some cases, but
191 :     could return <diagMatrix> in the same cases where <diagMatrix> o <numeric> does.
192 : mmaechler 2207
193 :     - look at solve.QP.compact() in \pkg{quadprog} and how to do that using
194 :     our sparse matrices. Maybe this needs to be re-implemented using CHOLMOD
195 :     routines.
196 :    
197 : mmaechler 2236 - We allow "over-allocated" (i,x)-slots for CsparseMatrix objects,
198 :     as per Csparse_validate() and the tests in tests/validObj.R. This is as
199 :     in CHOLMOD/CSparse, where nzmax (>= .@p[n]) corresponds to length(.@i),
200 :     and makes sense e.g. for M[.,.] <- v assignments which could allocate in
201 :     chunks and would not need to re-allocate anything in many cases.
202 :     HOWEVER, replCmat() in R/Csparse.R is still far from making use of that.
203 :    
204 : mmaechler 2271 - advertize rbind2() / cbind2() and (rather?) rBind() / cBind()
205 : mmaechler 2239 ------ -----
206 :     in all vignettes / talks / ... !!
207 :     People erronously try rbind/cbind see that they don't work and then
208 :     reinvent the wheel!
209 : dmbates 2277
210 : mmaechler 2341 --> Consider using the new 'dotMethods' functionality to define
211 :     cbind() and rbind() versions that work with Matrix.
212 : mmaechler 2500 The "Rmpfr" package does that now.
213 : mmaechler 2341
214 : mmaechler 2345 - In all(M1 == M2) for sparse large matrices M1, M2 (e.g. M2 <- M1 !),
215 :     the intermediate 'M1 == M2' typically is dense, hence potentially using
216 :     humongous amount of memory.
217 :     We should/could devise something like allCompare(M1, M2, `==`)
218 :     which would remain sparse in all its computations.
219 :    
220 : dmbates 2277 --------
221 :    
222 :     - Reconsider the linkages in the include files for the SuiteSparse
223 :     packages. It may be better simply to add all the src/<nm>/Include
224 :     directories to the include path for all compilations. I don't think
225 :     there is a big overhead. Right now we need to modify the include
226 :     file src/SPQR/Include/SuiteSparseQR_C.h so that it does not expect
227 :     to have src/UFsparse and src/CHOLMOD/Include on the include path.
228 :     Maybe just those two should be added to the include path.
229 : mmaechler 2341
230 : mmaechler 2490 - (systematically check that LAPACK-calling functions check for
231 :     0-dimensional input themselves; LAPACK gives an integer error code)
232 : mmaechler 2497
233 : mmaechler 2688 - the f[,5762] <- thisCol now go via Csparse_subassign() call ...
234 :     [ in tests/indexing.R ].
235 :     Still would be nice to be able to use abIndex (see replTmat in R/Tsparse.R)
236 : mmaechler 2538
237 :     - {IS THIS CURRENT?}
238 :     Sept. 2009:
239 :     Subject: chol2inv() |-> solve(<CHMfactor>)
240 :    
241 :     when testing and documenting chol2inv(),
242 :     I found that it's pretty simple to also define a method for
243 :     "CHMfactor" objects, namely simply the solve(*, Diagonal(.) "A")
244 :     method.
245 :     This is not particularly exciting, and also does *not*, I think
246 :     help for defining a chol2inv() method for *sparse* (upper)
247 :     triangular matrices.
248 : mmaechler 2553
249 : mmaechler 2592 - sort(<sparseVector>, partial=..), needed, for mean(*, trim = .) or median().
250 : mmaechler 2553 Note that defining xtfrm() does not "help" (as sort() then goes via dense
251 :     index). See "mean" in R/Matrix.R
252 : mmaechler 2572
253 :     - rcond(<sparseMatrix>) - for square currently goes via *dense* -- BAD --
254 :     can we go via qr() in any case?
255 :     In some cases, e.g. lmer()'s "Lambda" (block triangular, small blocks)
256 :     rcond(L) := 1 / (norm(L) * norm(solve(L)))
257 :     is simple {and remains sparse, as solve(L) is still block triangular}
258 : mmaechler 2661
259 :     - How can we ensure that inst/include/cholmod.h remains
260 : mmaechler 2666 correct and equivalent to src/CHOLMOD/Include/cholmod_core.h and siblings ???
261 :     {currently need to do this manually (Emacs M-x compare-windows) for the typedefs}
262 :    
263 : mmaechler 2688 - finalize and activate the new *unused* code in src/t_sparseVector.c
264 : mmaechler 2773
265 :     - check all uses of alloca()/Alloca() in src/*.[ch]
266 :     ensuring that the *size* allocated cannot grow with the
267 :     vector/matrix/nnzero sizes of the input.
268 :     [see the change needed in svn r2770 in src/dtCMatrix.c !]
269 :    

root@r-forge.r-project.org
ViewVC Help
Powered by ViewVC 1.0.0  
Thanks to:
Vienna University of Economics and Business Powered By FusionForge