[#6625] Global Parameter For VM Allocation To Prevent "Error: cannot allocate vector of size x Gb"

Date:
2019-04-29 16:34 Priority:
3

State:
Closed

Submitted by:
JAMES Bowery (jabowery) Assigned to:
Nobody (None)

Product:
None Operating System:
None

Component:
None

Summary:
Global Parameter For VM Allocation To Prevent "Error: cannot allocate vector of size x Gb"

Detailed description
When using packages that, in turn, use the Matrix package, it is often difficult to estimate the maximum RAM allocation. This results in possibly days of computation being discarded with the error: Error: cannot allocate vector of size x Gb With the understanding that VM can result in paging and corresponding dramatic slowdown, nevertheless a global parameter set at the application level to use VM rather than RAM allocation would avoid this kind of hard failure. The main benefit of this is that the computation can continue while other options for completing the computation successfully are being explored, such as starting the computation in the cloud on a machine with more RAM, without terminating the computation already in progress on the chance that it will complete before the higher capacity computation completes.

Comments:

Message
Date: 2023-06-27 17:50 Sender: Mikael Jagan Closing now, absent further information ...
Date: 2022-08-21 22:34 Sender: Mikael Jagan Please provide a minimal example demonstrating the problem, preferably relying only on 'Matrix' and base R. Where feasible, we try to allocate up front so that such errors are thrown early. It's possible that we can do better in your use case, but it's hard to know without an example... As for your specific proposal of a "global parameter", can you elaborate? I.e., can you provide some code suggesting an implementation? Note that 'Matrix', much like base R, is designed to provide "basic" functionality, which others can extend, wrap, etc. as they please (see, e.g., 'MatrixExtra'). So, you might consider encapsulating your "feature" in a package of its own.

Field	Old Value	Date	By
close_date	None	2023-06-27 17:50	jaganmn
status_id	Open	2023-06-27 17:50	jaganmn