R help - Sep 2016 - Using apply on a three dimensional matrix and passing multiple arguments to user defined function

Dear R-user,

I have a three-dimensional matrix of atmospheric data. The first two dimensions
are spatial (lon and lat) and the third is time, such that

dim(data) <- c(nlon,nlat,ntime)

I wish to apply a land sea mask data which is a matrix of "0" and
"1" if dim(nlon,nlat)

dim(lsmask) <- c(nlon,nlat)

I wish to set all of the elements in the two-dimensional array of data[,,ntime]
for every 1:length(ntime).

I could do this in a loop:

for (i in 1:ntime){
    data[,,i][which(lsmask == 0)] <- NA
}

I would like to do this using apply, but I need to pass two variables to the
function in apply (data and lsmask), where data is a two-dimensional array.

I tried:

mask <- function(x,y) {x[which(y==0)] <- NA}

masked_data <- apply(data,c(1,2),mask,y=lsmask)

but I get back a vector of dim(nlon,nlat) populated with NA.

Any clues as to what I am missing?

Thanks in advance for you help.

Kind regards,
Justin



--
Justin Peter
Research Fellow
International Centre for Applied Climate Sciences,
University of Southern Queensland
West St, Toowoomba, QLD, 4350
Australia

Email: justin.peter at usq.edu.au<mailto:justin.peter at usq.edu.au>
Ph: +61 (0) 7 4631 1181
Fax: +61 (0) 7 4631 5581
Mob: +61 (0)474 774 107




_____________________________________________________________
This email (including any attached files) is confidential and is for the
intended recipient(s) only. If you received this email by mistake, please, as a
courtesy, tell the sender, then delete this email.

The views and opinions are the originator's and do not necessarily reflect
those of the University of Southern Queensland. Although all reasonable
precautions were taken to ensure that this email contained no viruses at the
time it was sent we accept no liability for any losses arising from its receipt.

The University of Southern Queensland is a registered provider of education with
the Australian Government.
(CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 )


	[[alternative HTML version deleted]]

Justin,

I don't think you can get the apply() function to return an array.  You
could use lapply() instead, and then use simplify2array() to convert the
list of matrices to an array.  Also, in your mask() function you don't need
the which() and you should return the x.  See my example with toy data
below.

# toy data
nlon <- 2
nlat <- 4
ntime <- 3
data <- array(1:(nlon*nlat*ntime), dim=c(nlon, nlat, ntime))
lsmask <- array(sample(0:1, size=nlon*nlat, replace=TRUE), dim=c(nlon,
nlat))

# newly defined function
mask <- function(x, y) {
  x[y==0] <- NA
  x
}

# doit
data2 <- simplify2array(lapply(1:ntime, function(i) mask(data[, , i],
lsmask)))


You may prefer to stick with the for() loop approach (for clarity or
simplicity or ...)  When I ramped up the toy data to much larger
dimensions, the lapply() approach was only slightly faster than the for()
loop approach on my PC.

data3 <- data
data3[ , , i] <- mask(data3[ , , i], lsmask)

Jean




On Tue, Sep 6, 2016 at 11:33 PM, Justin Peter <Justin.Peter at usq.edu.au>
wrote:
> Dear R-user,
>
> I have a three-dimensional matrix of atmospheric data. The first two
> dimensions are spatial (lon and lat) and the third is time, such that
>
> dim(data) <- c(nlon,nlat,ntime)
>
> I wish to apply a land sea mask data which is a matrix of "0" and
"1" if
> dim(nlon,nlat)
>
> dim(lsmask) <- c(nlon,nlat)
>
> I wish to set all of the elements in the two-dimensional array of
> data[,,ntime] for every 1:length(ntime).
>
> I could do this in a loop:
>
> for (i in 1:ntime){
>     data[,,i][which(lsmask == 0)] <- NA
> }
>
> I would like to do this using apply, but I need to pass two variables to
> the function in apply (data and lsmask), where data is a two-dimensional
> array.
>
> I tried:
>
> mask <- function(x,y) {x[which(y==0)] <- NA}
>
> masked_data <- apply(data,c(1,2),mask,y=lsmask)
>
> but I get back a vector of dim(nlon,nlat) populated with NA.
>
> Any clues as to what I am missing?
>
> Thanks in advance for you help.
>
> Kind regards,
> Justin
>
>
>
> --
> Justin Peter
> Research Fellow
> International Centre for Applied Climate Sciences,
> University of Southern Queensland
> West St, Toowoomba, QLD, 4350
> Australia
>
> Email: justin.peter at usq.edu.au<mailto:justin.peter at usq.edu.au>
> Ph: +61 (0) 7 4631 1181
> Fax: +61 (0) 7 4631 5581
> Mob: +61 (0)474 774 107
>
>
>
>
> _____________________________________________________________
> This email (including any attached files) is confidential and is for the
> intended recipient(s) only. If you received this email by mistake, please,
> as a courtesy, tell the sender, then delete this email.
>
> The views and opinions are the originator's and do not necessarily
reflect
> those of the University of Southern Queensland. Although all reasonable
> precautions were taken to ensure that this email contained no viruses at
> the time it was sent we accept no liability for any losses arising from its
> receipt.
>
> The University of Southern Queensland is a registered provider of
> education with the Australian Government.
> (CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 )
>
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
	[[alternative HTML version deleted]]

Should be working, so the devil is in the details you are not showing. Please
provide a reproducible (self-contained) example [1] and post in plain text
rather than HTML formatted email to avoid code corruption.

[1] http://adv-r.had.co.nz/Reproducibility.html
-- 
Sent from my phone. Please excuse my brevity.

On September 6, 2016 9:33:35 PM PDT, Justin Peter <Justin.Peter at
usq.edu.au> wrote:
>Dear R-user,
>
>I have a three-dimensional matrix of atmospheric data. The first two
>dimensions are spatial (lon and lat) and the third is time, such that
>
>dim(data) <- c(nlon,nlat,ntime)
>
>I wish to apply a land sea mask data which is a matrix of "0" and
"1"
>if dim(nlon,nlat)
>
>dim(lsmask) <- c(nlon,nlat)
>
>I wish to set all of the elements in the two-dimensional array of
>data[,,ntime] for every 1:length(ntime).
>
>I could do this in a loop:
>
>for (i in 1:ntime){
>    data[,,i][which(lsmask == 0)] <- NA
>}
>
>I would like to do this using apply, but I need to pass two variables
>to the function in apply (data and lsmask), where data is a
>two-dimensional array.
>
>I tried:
>
>mask <- function(x,y) {x[which(y==0)] <- NA}
>
>masked_data <- apply(data,c(1,2),mask,y=lsmask)
>
>but I get back a vector of dim(nlon,nlat) populated with NA.
>
>Any clues as to what I am missing?
>
>Thanks in advance for you help.
>
>Kind regards,
>Justin
>
>
>
>--
>Justin Peter
>Research Fellow
>International Centre for Applied Climate Sciences,
>University of Southern Queensland
>West St, Toowoomba, QLD, 4350
>Australia
>
>Email: justin.peter at usq.edu.au<mailto:justin.peter at usq.edu.au>
>Ph: +61 (0) 7 4631 1181
>Fax: +61 (0) 7 4631 5581
>Mob: +61 (0)474 774 107
>
>
>
>
>_____________________________________________________________
>This email (including any attached files) is confidential and is for
>the intended recipient(s) only. If you received this email by mistake,
>please, as a courtesy, tell the sender, then delete this email.
>
>The views and opinions are the originator's and do not necessarily
>reflect those of the University of Southern Queensland. Although all
>reasonable precautions were taken to ensure that this email contained
>no viruses at the time it was sent we accept no liability for any
>losses arising from its receipt.
>
>The University of Southern Queensland is a registered provider of
>education with the Australian Government.
>(CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 )
>
>
>	[[alternative HTML version deleted]]
>
>______________________________________________
>R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

Hi,

Jean's example with lapply works fine.

However, if you still want to use apply, I think this works.
One observation first. You were passing c(1,2) as second argument to 
apply, in your code. And that is what makes you have lots of NAs as a 
result, since your function is being applied twice, by rows and columns 
(first and second dimensions) respectively.
Use:
     masked_data <- apply(data,3,mask,y=lsmask)
     # but now masked_data has dim(nlon*nlat,ntime), so change it
     dim(masked_data) <- dim(data)

The apply goes over the third dimension (second parameter '3'), so it 
takes every nlot*nlat matrix as first argument for function mask.
I think it should work.

Regards,

*Luisfo Chiroque*
/PhD Student | PhD Candidate
IMDEA Networks Institute/
http://fourier.networks.imdea.org/people/~luis_nunez/ 
<http://fourier.networks.imdea.org/people/%7Eluis_nunez/>

On 09/07/2016 03:17 PM, Adams, Jean wrote:
> Justin,
>
> I don't think you can get the apply() function to return an array.  You
> could use lapply() instead, and then use simplify2array() to convert the
> list of matrices to an array.  Also, in your mask() function you don't
need
> the which() and you should return the x.  See my example with toy data
> below.
>
> # toy data
> nlon <- 2
> nlat <- 4
> ntime <- 3
> data <- array(1:(nlon*nlat*ntime), dim=c(nlon, nlat, ntime))
> lsmask <- array(sample(0:1, size=nlon*nlat, replace=TRUE), dim=c(nlon,
> nlat))
>
> # newly defined function
> mask <- function(x, y) {
>    x[y==0] <- NA
>    x
> }
>
> # doit
> data2 <- simplify2array(lapply(1:ntime, function(i) mask(data[, , i],
> lsmask)))
>
>
> You may prefer to stick with the for() loop approach (for clarity or
> simplicity or ...)  When I ramped up the toy data to much larger
> dimensions, the lapply() approach was only slightly faster than the for()
> loop approach on my PC.
>
> data3 <- data
> data3[ , , i] <- mask(data3[ , , i], lsmask)
>
> Jean
>
>
>
>
> On Tue, Sep 6, 2016 at 11:33 PM, Justin Peter <Justin.Peter at
usq.edu.au>
> wrote:
>
>> Dear R-user,
>>
>> I have a three-dimensional matrix of atmospheric data. The first two
>> dimensions are spatial (lon and lat) and the third is time, such that
>>
>> dim(data) <- c(nlon,nlat,ntime)
>>
>> I wish to apply a land sea mask data which is a matrix of "0"
and "1" if
>> dim(nlon,nlat)
>>
>> dim(lsmask) <- c(nlon,nlat)
>>
>> I wish to set all of the elements in the two-dimensional array of
>> data[,,ntime] for every 1:length(ntime).
>>
>> I could do this in a loop:
>>
>> for (i in 1:ntime){
>>      data[,,i][which(lsmask == 0)] <- NA
>> }
>>
>> I would like to do this using apply, but I need to pass two variables
to
>> the function in apply (data and lsmask), where data is a
two-dimensional
>> array.
>>
>> I tried:
>>
>> mask <- function(x,y) {x[which(y==0)] <- NA}
>>
>> masked_data <- apply(data,c(1,2),mask,y=lsmask)
>>
>> but I get back a vector of dim(nlon,nlat) populated with NA.
>>
>> Any clues as to what I am missing?
>>
>> Thanks in advance for you help.
>>
>> Kind regards,
>> Justin
>>
>>
>>
>> --
>> Justin Peter
>> Research Fellow
>> International Centre for Applied Climate Sciences,
>> University of Southern Queensland
>> West St, Toowoomba, QLD, 4350
>> Australia
>>
>> Email: justin.peter at usq.edu.au<mailto:justin.peter at
usq.edu.au>
>> Ph: +61 (0) 7 4631 1181
>> Fax: +61 (0) 7 4631 5581
>> Mob: +61 (0)474 774 107
>>
>>
>>
>>
>> _____________________________________________________________
>> This email (including any attached files) is confidential and is for
the
>> intended recipient(s) only. If you received this email by mistake,
please,
>> as a courtesy, tell the sender, then delete this email.
>>
>> The views and opinions are the originator's and do not necessarily
reflect
>> those of the University of Southern Queensland. Although all reasonable
>> precautions were taken to ensure that this email contained no viruses
at
>> the time it was sent we accept no liability for any losses arising from
its
>> receipt.
>>
>> The University of Southern Queensland is a registered provider of
>> education with the Australian Government.
>> (CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 )
>>
>>
>>          [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/
>> posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

	[[alternative HTML version deleted]]