Forum: help

RE: unwanted temporary grids saved on AppData/Local/Temp [ Reply ]
By: Javier Fajardo on 2015-01-15 15:33

Dear all,

I also had the same problem a few weeks ago. I have a suggestion that was useful for me, which is a bit different to Frank's approach. I thought I could share my code, and maybe it will be useful for someone.
This is useful for multispecies modeling or parallel modeling.

Instead of creating a directory with a random name for temporary files later to be removed, I place those files under a folder with the species name. This way, it is easy to selectively remove the tmp files only for that species when its modeling is done.
To do this, inside the loop for multispecies modeling, after myRespName has been defined (the species that is being modelled), I create the folder using myRespName and tell raster to store there tmp files:

> dir.create (file.path("./",myRespName), showWarnings = FALSE)
> dir.create (file.path("./",myRespName,"/Temp"), showWarnings = FALSE)
> rasterOptions(tmpdir = paste("./",myRespName,"/Temp", sep=""))

Actually, the first folder I create has the very same name that the folder Biomod2 automatically creates for each species in the working directory. I place there the raster temporary folder (Temp) and, at the end of the modeling, I remove the species folder after saving somewhere else what I need to keep.

> unlink(paste("./",myRespName,sep=""), recursive = TRUE)

(if you want to keep biomod2 files, you can remove only Temp instead of the complete myRespName folder)

In parallel execution, with this approach you only remove a species tmp files when the species modeling has finished, and you make sure not to affect tmp files for another species that are being modelled in parallel in a different core, whose tmp files are in a different folder (under its own name), not affecting their modeling.

Best Regards,
Javier

RE: unwanted temporary grids saved on AppData/Local/Temp [ Reply ] By: damien georges on 2015-01-15 15:12	[forum:41803]
Thanks Frank for this tricks! I think that is the most efficient and elegant way to deal with this tmp raster files issue for now. Concerning the plotting issue you have Mathias, I think that just updating the package to the last R-Forge version should fix it. Best, Damien.

RE: unwanted temporary grids saved on AppData/Local/Temp [ Reply ]
By: Frank Breiner on 2015-01-15 14:06

[forum:41800]

Hi all,

I combined both solutions of Damien and Matthias.

First, I created a temporary directory to store temporary files and used removeTmpFiles() to clean up the temporary directory to avoid running over quota on the cluster. This works very well for running multiple species in a loop or in parallel on a cluster.

dir.create('/tmp/daily/fbreiner')
rasterOptions(tmpdir ='/tmp/daily/fbreiner')

...
BIOMOD_Modeling()
BIOMOD_Projection()
...
removeTmpFiles(h=0.5)

This worked very well for me to handle disk space on the cluster.

Cheers,
Frank

RE: unwanted temporary grids saved on AppData/Local/Temp [ Reply ]
By: Mathias Kuemmerlen on 2015-01-15 10:58

[forum:41797]

Hi Damien,

I have a similar problem, than Olivier has stated.
Until now I have used:

removeTmpFiles()

from the raster package.
Would this do the same job?
Or is it better to proceed as you have indicated?

Also, since I've been using the new version "biomod2_3.1-64" you posted on the 4th of December 2014, I get some plots (x = Index, y = ef.tmp) with each projection I actually don't need. I would like to avoid them, as they probably take up some memory too. How can I leave them out?

Pura vida,

- Mathias

RE: unwanted temporary grids saved on AppData/Local/Temp [ Reply ]
By: damien georges on 2015-01-05 12:02

[forum:41771]

Hi Olivier,

The easiest way to deal with this is to tell to raster package to write temp file in a defined directory and to remove this directory at the end of the computation :

Here an example :

library(raster)

## define the name of a temp directory where raster tmp files will be stored
raster_tmp_dir <- "raster_tmp"

## create the directory
dir.create(raster_tmp_dir, showWarnings = F, recursive = T)

## set raster options
rasterOptions(tmpdir = raster_tmp_dir)

### do all your biomod2 calculcaltion
### ...
### ...

## remove the tmp dir
unlink(raster_tmp_dir, recursive = T, force = T)

Hope that helps,

Cheers and happy new year !
Damien.

unwanted temporary grids saved on AppData/Local/Temp [ Reply ]
By: Olivier Brönnimann on 2014-12-26 16:05

[forum:41770]

The projections of my models on a large area with a high resolution fail because my C: hard disk get full. I found out that tons of temporary grids (e.g. 2014-12-26-151652-2396-43796.grd) are saved in
C:/Users/<myname>/AppData/Local/Temp/R_raster_<myname>/R_raster_<myname>. When running biomod2 on my computer, this is not too much of a problem since I can remove these files easily after each run/species, but it's a problem when I work on a cluster.

Is there a way to tell biomod to remove these files automatically just after the projections?

cheers,
Olivier