R devel - Oct 2016 - lapply on long vector fails

I report here that, in RStudio in Data Scientist Workbench,
lapply(raw(2^31), function(x) NULL)
failed after not so long time.
> res <- lapply(raw(2^31), function(x) NULL)
Error in FUN(X[[i]], ...) : long vectors not supported yet:
memory.c:1652
> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux stretch/sid

locale:
 [1] LC_CTYPE=en_US.UTF-8      
 [2] LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8      
 [8] LC_NAME=C                 
 [9] LC_ADDRESS=C              
[10] LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8
[12] LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets 
[6] methods   base     

other attached packages:
[1] SparkR_1.6.1

loaded via a namespace (and not attached):
[1] tools_3.3.1


However, the code that implements 'lapply', function 'do_lapply'
in apply.c, seems to support long vectors.

The error message points to memory.c:1652. I don't understand the code
there.


A different case:
gc()
after
system.time(vector("list", 2^31))
gave an error with message pointing to memory.c at different line. Subsequent
gc()
didn't give error.
> system.time(vector("list", 2^31))
   user  system elapsed 
  3.104  15.608  18.711 
> gc()
Error in gc() : long vectors not supported yet:
memory.c:1121
> gc()
         used (Mb) gc trigger    (Mb)   max used    (Mb)
Ncells 445496 23.8     750400    40.1     592000    31.7
Vcells 667725  5.1 2062440028 15735.2 2148153146 16389.2

Thanks.  Fixed in R-devel in r71606.

Best,

luke

On Sat, 29 Oct 2016, Suharto Anggono Suharto Anggono via R-devel wrote:
> I report here that, in RStudio in Data Scientist Workbench,
> lapply(raw(2^31), function(x) NULL)
> failed after not so long time.
>
>> res <- lapply(raw(2^31), function(x) NULL)
> Error in FUN(X[[i]], ...) : long vectors not supported yet: memory.c:1652
>> sessionInfo()
> R version 3.3.1 (2016-06-21)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Debian GNU/Linux stretch/sid
>
> locale:
> [1] LC_CTYPE=en_US.UTF-8
> [2] LC_NUMERIC=C
> [3] LC_TIME=en_US.UTF-8
> [4] LC_COLLATE=en_US.UTF-8
> [5] LC_MONETARY=en_US.UTF-8
> [6] LC_MESSAGES=en_US.UTF-8
> [7] LC_PAPER=en_US.UTF-8
> [8] LC_NAME=C
> [9] LC_ADDRESS=C
> [10] LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8
> [12] LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets
> [6] methods   base
>
> other attached packages:
> [1] SparkR_1.6.1
>
> loaded via a namespace (and not attached):
> [1] tools_3.3.1
>
>
> However, the code that implements 'lapply', function
'do_lapply' in apply.c, seems to support long vectors.
>
> The error message points to memory.c:1652. I don't understand the code
there.
>
>
> A different case:
> gc()
> after
> system.time(vector("list", 2^31))
> gave an error with message pointing to memory.c at different line.
Subsequent
> gc()
> didn't give error.
>
>> system.time(vector("list", 2^31))
>   user  system elapsed
>  3.104  15.608  18.711
>> gc()
> Error in gc() : long vectors not supported yet: memory.c:1121
>> gc()
>         used (Mb) gc trigger    (Mb)   max used    (Mb)
> Ncells 445496 23.8     750400    40.1     592000    31.7
> Vcells 667725  5.1 2062440028 15735.2 2148153146 16389.2
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu