R help - Aug 2012 - [ncdf4] error converting GEIA data to netCDF

summary: I can successfully ncvar_put(...) data to a file, but when I
try to ncvar_get(...) the same data, I get
> Error in if (nc$var[[li]]$hasAddOffset) addOffset = nc$var[[li]]$addOffset
else addOffset = 0 :
>   argument is of length zero
How to fix or debug?

details:

R code @

https://github.com/TomRoche/GEIA_to_NetCDF

successfully (if crudely) uses R packages={ncdf4, maps, fields} to

+ extract data from a GEIA-distributed datafile

https://github.com/downloads/TomRoche/GEIA_to_netCDF/N2OOC90Y.1A

+ display the data (mostly successfully--the map's legend has problems
  which I'll attack later)

https://github.com/downloads/TomRoche/GEIA_to_netCDF/output.1.png

+ create a netCDF file using the data read from the GEIA file. (At
  least, after nc_sync(netcdf.file), the file `ncdump -h`s properly.)

However, I can only *put* the data to the netCDF file:
> ncvar_put(
> + nc=netcdf.file,
> + varid=emis.var,
> + vals=t(global.emis.mx),
> + start=c(1, 1, 1),
> + count=c(-1,-1, 1)) # -1 -> all data
When I try to *pull* the data *from* the netCDF I created,
> > target.data <- ncvar_get(
> +   nc=netcdf.file,
> +   varid=emis.var,
> +   # read all the data
> +   start=rep(1, emis.var$ndims),
> + #  count=rep(-1, emis.var$ndims))
I get
> Error in if (nc$var[[li]]$hasAddOffset) addOffset = nc$var[[li]]$addOffset
else addOffset = 0 :
>   argument is of length zero
And I get the same error if I try the minor variation(s)
> target.data <- ncvar_get(
+   nc=netcdf.file,
+ #  varid=emis.var,
+   varid=emis.var.name,
+   # read all the data
+   start=rep(1, emis.var$ndims),
+   count=c(-1, -1, 1))

But the data itself appears to be OK--at least, it virtualizes
properly (above). So I'm thinking I must just be missing something
simple, and hoping Someone Out There with fresh eyeballs can point to
my error(s).

(And, in case you're wondering:

- I'm not just ncvar_put'ing the data for the exercise: I want the
  GEIA data in netCDF format for subsequent use.

- I tried to find the GEIA data distributed in netCDF format, and
  asked around, but have gotten no responses.

) TIA, Tom Roche <Tom_Roche at pobox.com>

If I had to guess, you did not call nc_close or nc_sync.  In netcdf, the data is
actually really written until that is called.

-Roy

On Aug 27, 2012, at 8:31 PM, Tom Roche wrote:
> 
> summary: I can successfully ncvar_put(...) data to a file, but when I
> try to ncvar_get(...) the same data, I get
> 
>> Error in if (nc$var[[li]]$hasAddOffset) addOffset =
nc$var[[li]]$addOffset else addOffset = 0 :
>>  argument is of length zero
> 
> How to fix or debug?
> 
> details:
> 
> R code @
> 
> https://github.com/TomRoche/GEIA_to_NetCDF
> 
> successfully (if crudely) uses R packages={ncdf4, maps, fields} to
> 
> + extract data from a GEIA-distributed datafile
> 
> https://github.com/downloads/TomRoche/GEIA_to_netCDF/N2OOC90Y.1A
> 
> + display the data (mostly successfully--the map's legend has problems
>  which I'll attack later)
> 
> https://github.com/downloads/TomRoche/GEIA_to_netCDF/output.1.png
> 
> + create a netCDF file using the data read from the GEIA file. (At
>  least, after nc_sync(netcdf.file), the file `ncdump -h`s properly.)
> 
> However, I can only *put* the data to the netCDF file:
> 
>> ncvar_put(
>> + nc=netcdf.file,
>> + varid=emis.var,
>> + vals=t(global.emis.mx),
>> + start=c(1, 1, 1),
>> + count=c(-1,-1, 1)) # -1 -> all data
> 
> When I try to *pull* the data *from* the netCDF I created,
> 
>>> target.data <- ncvar_get(
>> +   nc=netcdf.file,
>> +   varid=emis.var,
>> +   # read all the data
>> +   start=rep(1, emis.var$ndims),
>> + #  count=rep(-1, emis.var$ndims))
> 
> I get
> 
>> Error in if (nc$var[[li]]$hasAddOffset) addOffset =
nc$var[[li]]$addOffset else addOffset = 0 :
>>  argument is of length zero
> 
> And I get the same error if I try the minor variation(s)
> 
>> target.data <- ncvar_get(
> +   nc=netcdf.file,
> + #  varid=emis.var,
> +   varid=emis.var.name,
> +   # read all the data
> +   start=rep(1, emis.var$ndims),
> +   count=c(-1, -1, 1))
> 
> But the data itself appears to be OK--at least, it virtualizes
> properly (above). So I'm thinking I must just be missing something
> simple, and hoping Someone Out There with fresh eyeballs can point to
> my error(s).
> 
> (And, in case you're wondering:
> 
> - I'm not just ncvar_put'ing the data for the exercise: I want the
>  GEIA data in netCDF format for subsequent use.
> 
> - I tried to find the GEIA data distributed in netCDF format, and
>  asked around, but have gotten no responses.
> 
> ) TIA, Tom Roche <Tom_Roche at pobox.com>
> 
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup at unidata.ucar.edu
> For list information or to unsubscribe,  visit:
http://www.unidata.ucar.edu/mailing_lists/
**********************
"The contents of this message do not reflect any position of the U.S.
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: Roy.Mendelssohn at noaa.gov (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice"
-MLK Jr.

Just looked at your code, and you do call nc_sync.  I am wondering if it works
if you call nc_close, reopen and then write. Perhaps the nc_sync is sufficent
when you have been in define mode in the nc_create call.

-Roy

On Aug 27, 2012, at 8:31 PM, Tom Roche wrote:
> 
> summary: I can successfully ncvar_put(...) data to a file, but when I
> try to ncvar_get(...) the same data, I get
> 
>> Error in if (nc$var[[li]]$hasAddOffset) addOffset =
nc$var[[li]]$addOffset else addOffset = 0 :
>>  argument is of length zero
> 
> How to fix or debug?
> 
> details:
> 
> R code @
> 
> https://github.com/TomRoche/GEIA_to_NetCDF
> 
> successfully (if crudely) uses R packages={ncdf4, maps, fields} to
> 
> + extract data from a GEIA-distributed datafile
> 
> https://github.com/downloads/TomRoche/GEIA_to_netCDF/N2OOC90Y.1A
> 
> + display the data (mostly successfully--the map's legend has problems
>  which I'll attack later)
> 
> https://github.com/downloads/TomRoche/GEIA_to_netCDF/output.1.png
> 
> + create a netCDF file using the data read from the GEIA file. (At
>  least, after nc_sync(netcdf.file), the file `ncdump -h`s properly.)
> 
> However, I can only *put* the data to the netCDF file:
> 
>> ncvar_put(
>> + nc=netcdf.file,
>> + varid=emis.var,
>> + vals=t(global.emis.mx),
>> + start=c(1, 1, 1),
>> + count=c(-1,-1, 1)) # -1 -> all data
> 
> When I try to *pull* the data *from* the netCDF I created,
> 
>>> target.data <- ncvar_get(
>> +   nc=netcdf.file,
>> +   varid=emis.var,
>> +   # read all the data
>> +   start=rep(1, emis.var$ndims),
>> + #  count=rep(-1, emis.var$ndims))
> 
> I get
> 
>> Error in if (nc$var[[li]]$hasAddOffset) addOffset =
nc$var[[li]]$addOffset else addOffset = 0 :
>>  argument is of length zero
> 
> And I get the same error if I try the minor variation(s)
> 
>> target.data <- ncvar_get(
> +   nc=netcdf.file,
> + #  varid=emis.var,
> +   varid=emis.var.name,
> +   # read all the data
> +   start=rep(1, emis.var$ndims),
> +   count=c(-1, -1, 1))
> 
> But the data itself appears to be OK--at least, it virtualizes
> properly (above). So I'm thinking I must just be missing something
> simple, and hoping Someone Out There with fresh eyeballs can point to
> my error(s).
> 
> (And, in case you're wondering:
> 
> - I'm not just ncvar_put'ing the data for the exercise: I want the
>  GEIA data in netCDF format for subsequent use.
> 
> - I tried to find the GEIA data distributed in netCDF format, and
>  asked around, but have gotten no responses.
> 
> ) TIA, Tom Roche <Tom_Roche at pobox.com>
> 
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup at unidata.ucar.edu
> For list information or to unsubscribe,  visit:
http://www.unidata.ucar.edu/mailing_lists/
**********************
"The contents of this message do not reflect any position of the U.S.
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: Roy.Mendelssohn at noaa.gov (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice"
-MLK Jr.

Hello,

The following works fine for me:

 > nc <- nc_open("~/GEIA_N2O_oceanic.nc")
 > emi_n2o <- ncvar_get(nc, 'emi_n2o', start=c(1,1,1),
count=c(-1,-1,1))

Regards,
Pascal



Le 28/08/2012 12:31, Tom Roche a ?crit :
>
> summary: I can successfully ncvar_put(...) data to a file, but when I
> try to ncvar_get(...) the same data, I get
>
>> Error in if (nc$var[[li]]$hasAddOffset) addOffset =
nc$var[[li]]$addOffset else addOffset = 0 :
>>    argument is of length zero
>
> How to fix or debug?
>
> details:
>
> R code @
>
> https://github.com/TomRoche/GEIA_to_NetCDF
>
> successfully (if crudely) uses R packages={ncdf4, maps, fields} to
>
> + extract data from a GEIA-distributed datafile
>
> https://github.com/downloads/TomRoche/GEIA_to_netCDF/N2OOC90Y.1A
>
> + display the data (mostly successfully--the map's legend has problems
>    which I'll attack later)
>
> https://github.com/downloads/TomRoche/GEIA_to_netCDF/output.1.png
>
> + create a netCDF file using the data read from the GEIA file. (At
>    least, after nc_sync(netcdf.file), the file `ncdump -h`s properly.)
>
> However, I can only *put* the data to the netCDF file:
>
>> ncvar_put(
>> + nc=netcdf.file,
>> + varid=emis.var,
>> + vals=t(global.emis.mx),
>> + start=c(1, 1, 1),
>> + count=c(-1,-1, 1)) # -1 -> all data
>
> When I try to *pull* the data *from* the netCDF I created,
>
>>> target.data <- ncvar_get(
>> +   nc=netcdf.file,
>> +   varid=emis.var,
>> +   # read all the data
>> +   start=rep(1, emis.var$ndims),
>> + #  count=rep(-1, emis.var$ndims))
>
> I get
>
>> Error in if (nc$var[[li]]$hasAddOffset) addOffset =
nc$var[[li]]$addOffset else addOffset = 0 :
>>    argument is of length zero
>
> And I get the same error if I try the minor variation(s)
>
>> target.data <- ncvar_get(
> +   nc=netcdf.file,
> + #  varid=emis.var,
> +   varid=emis.var.name,
> +   # read all the data
> +   start=rep(1, emis.var$ndims),
> +   count=c(-1, -1, 1))
>
> But the data itself appears to be OK--at least, it virtualizes
> properly (above). So I'm thinking I must just be missing something
> simple, and hoping Someone Out There with fresh eyeballs can point to
> my error(s).
>
> (And, in case you're wondering:
>
> - I'm not just ncvar_put'ing the data for the exercise: I want the
>    GEIA data in netCDF format for subsequent use.
>
> - I tried to find the GEIA data distributed in netCDF format, and
>    asked around, but have gotten no responses.
>
> ) TIA, Tom Roche <Tom_Roche at pobox.com>
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

Tom Roche Mon, 27 Aug 2012 23:31:23 -0400
>> summary: I can successfully ncvar_put(...) data to a file, but when
>> I try to ncvar_get(...) the same data
in the one and only datavar
>> I get
>> > Error in if (nc$var[[li]]$hasAddOffset) addOffset =
nc$var[[li]]$addOffset else addOffset = 0 :
>> >   argument is of length zero
David W. Pierce Mon, 27 Aug 2012 21:35:35 -0700
> If you want to create a file and write to it, and then read back in from
that same file, close it first and then reopen it.
So nc_sync is not enough--good to know. But ...

Unfortunately I do that (nc_close then nc_open) in the latest code @

https://github.com/TomRoche/GEIA_to_NetCDF

(direct link to relevant file
https://github.com/TomRoche/GEIA_to_netCDF/blob/master/GEIA.to.netCDF.r

) but no fix--same error. Any other suggestions?

your assistance is appreciated, Tom Roche <Tom_Roche at pobox.com>