SCM

[#1194] creating a ncdf file from a stack with named variables

Date:
2010-10-26 11:59
Priority:
3
State:
Open
Submitted by:
Mario Frasca (mariotomo)
Assigned to:
Nobody (None)
Product:
None
Operating System:
All
Component:
None
Summary:
creating a ncdf file from a stack with named variables

Detailed description
here at office I need a way to save a stack of images as net cdf, with specific names for the row and column coordinates (y, x) plus also for the "z" (in my case, "time") coordinate.
this is currently not supported by writeRaster(format="CDF", ...)
I would like to invoke the function like this:

writeRaster(result, fileLocations$outputFileName, format="CDF", overwrite=TRUE, xname="x", yname="y", zname="time")

also, in case each raster in the stack has an attribute "zvalue", I need these values in zdim, not the sequence 1:nlayers(x)

for the rest, this is very useful software and also very readable. no problem patching it myself!

thanks!

Comments:

Message  ↓
Date: 2010-11-01 15:57
Sender: Mario Frasca

a few examples of code that I had to alter, and I suspect there's a way to make it look simpler.

I read radar images which I have to crop.
I crop them creating a raster that I use as a shape:
shape <- crop(raster(radarbeelden, layer=1), extent(areaExtent[2, 'x'], areaExtent[1, 'x'], areaExtent[2, 'y'], areaExtent[1, 'y']))
shape[] <- 0

I need weighted inverse distance masks and I create them like this:
for(i in 1:nrow(descriptionData)) {
logdebug("computing inverse distance from %s (ground station %d)", descriptionData[i, 'location'], i, logger="fews.diagnostics")
temp <- setValues(shape, NA)
temp[cellIndices[i]] <- 1
temp <- setValues(shape, 1) / distance(temp)
temp[cellIndices[i]] <- 1
temp[cellIndices[-i]] <- 0
buffer <- c(buffer, temp)
}

(cellIndices[i] contains the index of the pixel relative to my i-th location in the cropped image).

previously I could write:
temp <- 1 / distance(temp)

now this shorter instruction gives error:
Error in function (classes, fdef, mtable) :
unable to find an inherited method for function "raster", for signature "RasterLayer"

with the `buffer` vector of rasters I create the brick...
previously I used a stack and I created it like this:
> raster(buffer)
now I use a brick and I write:
> do.call("brick", buffer)

accessing the timestamps is now a lot easier (and a huge lot more readable)!
radarbeelden@zvalue
instead of
sapply(attr(radarbeelden, 'layers'), attr, 'zvalue')

but what happened to a few basic functions `+` and `prod`... first I could write:
calibrated <- calibrated + prod(image, incidence, correctionFactor)
(the arguments in prod are rasters or just numeric)
now I must write this:
addendum <- image * incidence * setValues(shape, correctionFactor)
calibrated[] <- calibrated[] + addendum[]

with the due adjustments, it looks like working, but if you could make my programmer's life easier, I would be thankful!

MF

Date: 2010-11-01 15:16
Sender: Mario Frasca

Hi Robert,

I had patched the released version but I get funny images on a client application (fews / http://www.wldelft.nl/soft/fews/int/index.html), but the file looks ok if I open it in 'ncview'. to me, this seems related to order of writing information in the cdf file, as if fews was assuming an ordering while I'm respecting an other.

I'm now trying to drop my patched version and adopt the newest one. there are a few issues.

I can write a rasterStack created with the old version, but I can't create the rasterStack with the new version!

I am now trying to use a brick instead of a stack, but I am sort of stuck... can I send you the code I don't manage to get to work?

Date: 2010-10-29 17:12
Sender: Robert Hijmans

Mario

I am on US Pacific time.
When raster reads ncdf with something like "minutes since 1970-01-01T00:00:00+0000", it computes the actual data/time and puts that in the zvalue slot (at least when detected (when using the CF standards)). When writing the inverse would needs to be done. This would allow merging different data sets, but it is tricky stuff that perhaps needs to be redesigned.

Because you do the assignment yourself:
result@zvalue <- as.seconds(index(neerslagmeting))

and perhaps the seconds are probably correctly written to the file (coerced to numbers) this may not be an issue for you.

There were some issues with R-Forge this week. Seems better now.

Date: 2010-10-27 11:34
Sender: Mario Frasca

(forgot: the way I'm calling the writeRaster function)

result <- stack(vector_of_raster_objects)
attr(result, 'zvalue') <- as.seconds(index(neerslagmeting)) / 60
writeRaster(result, fileLocations$outputFileName, format="CDF", overwrite=TRUE, xname="x", yname="y", zname="time", xyunit="", zunit="minutes since EPOCH", varname="image_data", varunit="unit")

Date: 2010-10-27 11:32
Sender: Mario Frasca

adding a patch that I'm using for the released version.

Date: 2010-10-27 06:11
Sender: Mario Frasca

we are thinking in the same direction, this is nice. I have also added the names varname etc.
in my data, the zunit is "minutes since 1970-01-01T00:00:00+0000", not a date/time type.

I am using a stack, not a brick... and I'm having problems replacing the released raster 1.5-16 with the 'bleeding edge' svn version. so I'm basing my patch on the released package.

(in which time zone are you actually? this week I work from 6:30 to 15:15 UTC, from next week I shift to one hour later)

Date: 2010-10-27 05:23
Sender: Robert Hijmans

Some additional arguments:

library(raster)
r <- raster(system.file("external/test.grd",
package="raster"))
b <- brick(r, sqrt(r), r+10)
b@zvalue <- 8:10
nc <- writeRaster(b, filename='netCDF.nc', format="CDF", overwrite=TRUE, xname='x', yname='y', zname='time', zunit='months', varname='rain', varunit='mm', longname='Daily rainfall')
print(nc)
str(nc)

Date: 2010-10-26 21:46
Sender: Robert Hijmans

Some progress made, see example below. Perhaps it is a problem that zvalue has to be numeric. As these are often dates, I need to deal with that (setting the origin etc, but have not done that yet).

library(raster)
r <- raster(system.file("external/test.grd", package="raster"))
b <- brick(r, sqrt(r), r+10)
b@zvalue <- 8:10
nc <- writeRaster(b, filename='netCDF.nc', format="CDF", overwrite=TRUE, xname='x', yname='y', zname='time', varname='rain', longname='Daily rainfall', zunit='mm')
print(nc)
str(nc)

Attached Files:

Attachments:
Size Name Date By Download
9 KiBraster-1.5.16.patch-11942010-10-27 11:32mariotomoraster-1.5.16.patch-1194

Changes

Field Old Value Date By
File Added74: raster-1.5.16.patch-11942010-10-27 11:32mariotomo
Thanks to:
Vienna University of Economics and Business Powered By FusionForge