Jorge Cimentada
2017-Aug-06 13:17 UTC
[R] Crash when installing heavy packages in remote server with R and Rstudio server
Hi, I've been trying to install some very heavy packages from Github (~ 100MB - 300 MB) on my remote server using Rstudio server and I keep getting some crashes. These packages are pretty much datasets. I try doing it with this code and get the error below.> install.packages("devtools")> devtools::install_github("pbiecek/PISA2000lite")Downloading GitHub repo pbiecek/PISA2000lite at master from URL https://api.github.com/repos/pbiecek/PISA2000lite/zipball/master Installing PISA2000lite '/usr/lib/R/bin/R' --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL \ '/tmp/Rtmp1GoUVG/devtools76a72d179d7/pbiecek-PISA2000lite-54f4765' \ --library='/home/cimentadaj/R/x86_64-pc-linux-gnu-library/3.4' --install-tests * installing *source* package ?PISA2000lite? ... ** data *** moving datasets to lazyload DB Killed Installation failed: Command failed (137) The package actually gets installed any(grepl("PISA2012lite", installed.packages()[, 1])) [1] TRUE But I once I load the library it's as if none of the datasets are available. I've used these packages for a long time on my local computer so I'm sure it's not the packages. The server has enough space to handle the packages, so it's not memory either. Below you can find the specs of the R session and the server R version 3.4.1 (2017-06-30) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 16.04.3 LTS Matrix products: default BLAS: /usr/lib/libblas/libblas.so.3.6.0 LAPACK: /usr/lib/lapack/liblapack.so.3.6.0 locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 [4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C [10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] httr_1.2.1 compiler_3.4.1 R6_2.2.2 tools_3.4.1 withr_2.0.0 curl_2.8.1 [7] memoise_1.1.0 git2r_0.19.0 digest_0.6.12 devtools_1.13.3 The disk info: Filesystem Size Used Avail Use% Mounted on udev 981M 0 981M 0% /dev tmpfs 201M 21M 180M 11% /run /dev/vda1 39G 9.1G 30G 24% / tmpfs 1001M 840K 1000M 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 1001M 0 1001M 0% /sys/fs/cgroup /dev/vda15 105M 3.4M 102M 4% /boot/efi tmpfs 201M 0 201M 0% /run/user/1000 And server specs from Digital Ocean: 2 GB Memory / 40 GB Disk / LON1 - Ubuntu 16.04.3 x64 from I posted the question in this <https://stackoverflow.com/questions/45522145/crash-when-installing-heavy-packages-in-remote-server-with-rstudio-server?noredirect=1#comment78011638_45522145> stackoverflow thread and actually found out that the problem is not from Rstudio server in itself because I get the exact same error with: R CMD INSTALL PISA2000lite_1.0.tar.gz Any idea why this might be occurring? ----------------------------------- Jorge Cimentada *https://cimentadaj.github.io/ <https://cimentadaj.github.io/>* [[alternative HTML version deleted]]
Duncan Murdoch
2017-Aug-06 18:57 UTC
[R] Crash when installing heavy packages in remote server with R and Rstudio server
On 06/08/2017 9:17 AM, Jorge Cimentada wrote:> Hi, > > I've been trying to install some very heavy packages from Github (~ 100MB - > 300 MB) on my remote server using Rstudio server and I keep getting some > crashes. These packages are pretty much datasets.Based on your description, this is RStudio Server specific. You'll need to write to their tech support for help. This list is for R issues. Duncan Murdoch> > I try doing it with this code and get the error below. > >> install.packages("devtools") > >> devtools::install_github("pbiecek/PISA2000lite") > Downloading GitHub repo pbiecek/PISA2000lite at master > from URL https://api.github.com/repos/pbiecek/PISA2000lite/zipball/master > Installing PISA2000lite > '/usr/lib/R/bin/R' --no-site-file --no-environ --no-save --no-restore > --quiet CMD INSTALL \ > '/tmp/Rtmp1GoUVG/devtools76a72d179d7/pbiecek-PISA2000lite-54f4765' \ > --library='/home/cimentadaj/R/x86_64-pc-linux-gnu-library/3.4' > --install-tests > > * installing *source* package ?PISA2000lite? ... > ** data > *** moving datasets to lazyload DB > Killed > Installation failed: Command failed (137) > > > > The package actually gets installed > any(grepl("PISA2012lite", installed.packages()[, 1])) > > [1] TRUE > > > But I once I load the library it's as if none of the datasets are > available. I've used these packages for a long time on my local computer so > I'm sure it's not the packages. The server has enough space to handle the > packages, so it's not memory either. Below you can find the specs of the R > session and the server > > > R version 3.4.1 (2017-06-30) > > Platform: x86_64-pc-linux-gnu (64-bit) > > Running under: Ubuntu 16.04.3 LTS > > Matrix products: default > > BLAS: /usr/lib/libblas/libblas.so.3.6.0 > > LAPACK: /usr/lib/lapack/liblapack.so.3.6.0 > > > locale: > > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C > LC_TIME=en_US.UTF-8 > > [4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 > LC_MESSAGES=en_US.UTF-8 > > [7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C > > > [10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 > LC_IDENTIFICATION=C > > > attached base packages: > > [1] stats graphics grDevices utils datasets methods base > > > loaded via a namespace (and not attached): > > [1] httr_1.2.1 compiler_3.4.1 R6_2.2.2 tools_3.4.1 > withr_2.0.0 curl_2.8.1 > > [7] memoise_1.1.0 git2r_0.19.0 digest_0.6.12 devtools_1.13.3 > > > The disk info: > > > Filesystem Size Used Avail Use% Mounted on > > udev 981M 0 981M 0% /dev > > tmpfs 201M 21M 180M 11% /run > > /dev/vda1 39G 9.1G 30G 24% / > > tmpfs 1001M 840K 1000M 1% /dev/shm > > tmpfs 5.0M 0 5.0M 0% /run/lock > > tmpfs 1001M 0 1001M 0% /sys/fs/cgroup > > /dev/vda15 105M 3.4M 102M 4% /boot/efi > > tmpfs 201M 0 201M 0% /run/user/1000 > > And server specs from Digital Ocean: > 2 GB Memory / 40 GB Disk / LON1 - Ubuntu 16.04.3 x64 from > > I posted the question in this > <https://stackoverflow.com/questions/45522145/crash-when-installing-heavy-packages-in-remote-server-with-rstudio-server?noredirect=1#comment78011638_45522145> > stackoverflow thread and actually found out that the problem is not from > Rstudio server in itself because I get the exact same error with: > R CMD INSTALL PISA2000lite_1.0.tar.gz > > Any idea why this might be occurring? > > ----------------------------------- > > Jorge Cimentada > > *https://cimentadaj.github.io/ <https://cimentadaj.github.io/>* > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >
Duncan Murdoch
2017-Aug-06 19:04 UTC
[R] Crash when installing heavy packages in remote server with R and Rstudio server
On 06/08/2017 2:57 PM, Duncan Murdoch wrote:> On 06/08/2017 9:17 AM, Jorge Cimentada wrote: >> Hi, >> >> I've been trying to install some very heavy packages from Github (~ 100MB - >> 300 MB) on my remote server using Rstudio server and I keep getting some >> crashes. These packages are pretty much datasets. > > Based on your description, this is RStudio Server specific. You'll need > to write to their tech support for help. This list is for R issues.Oops, I see below that you found it happens with plain R as well. Was that tested on your server machine as well? It looks as though it has some execution time limit (or disk or memory limit) and you're exceeding it. So you should talk to your sysadmin. Duncan Murdoch> > Duncan Murdoch > >> >> I try doing it with this code and get the error below. >> >>> install.packages("devtools") >> >>> devtools::install_github("pbiecek/PISA2000lite") >> Downloading GitHub repo pbiecek/PISA2000lite at master >> from URL https://api.github.com/repos/pbiecek/PISA2000lite/zipball/master >> Installing PISA2000lite >> '/usr/lib/R/bin/R' --no-site-file --no-environ --no-save --no-restore >> --quiet CMD INSTALL \ >> '/tmp/Rtmp1GoUVG/devtools76a72d179d7/pbiecek-PISA2000lite-54f4765' \ >> --library='/home/cimentadaj/R/x86_64-pc-linux-gnu-library/3.4' >> --install-tests >> >> * installing *source* package ?PISA2000lite? ... >> ** data >> *** moving datasets to lazyload DB >> Killed >> Installation failed: Command failed (137) >> >> >> >> The package actually gets installed >> any(grepl("PISA2012lite", installed.packages()[, 1])) >> >> [1] TRUE >> >> >> But I once I load the library it's as if none of the datasets are >> available. I've used these packages for a long time on my local computer so >> I'm sure it's not the packages. The server has enough space to handle the >> packages, so it's not memory either. Below you can find the specs of the R >> session and the server >> >> >> R version 3.4.1 (2017-06-30) >> >> Platform: x86_64-pc-linux-gnu (64-bit) >> >> Running under: Ubuntu 16.04.3 LTS >> >> Matrix products: default >> >> BLAS: /usr/lib/libblas/libblas.so.3.6.0 >> >> LAPACK: /usr/lib/lapack/liblapack.so.3.6.0 >> >> >> locale: >> >> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C >> LC_TIME=en_US.UTF-8 >> >> [4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 >> LC_MESSAGES=en_US.UTF-8 >> >> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C >> >> >> [10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 >> LC_IDENTIFICATION=C >> >> >> attached base packages: >> >> [1] stats graphics grDevices utils datasets methods base >> >> >> loaded via a namespace (and not attached): >> >> [1] httr_1.2.1 compiler_3.4.1 R6_2.2.2 tools_3.4.1 >> withr_2.0.0 curl_2.8.1 >> >> [7] memoise_1.1.0 git2r_0.19.0 digest_0.6.12 devtools_1.13.3 >> >> >> The disk info: >> >> >> Filesystem Size Used Avail Use% Mounted on >> >> udev 981M 0 981M 0% /dev >> >> tmpfs 201M 21M 180M 11% /run >> >> /dev/vda1 39G 9.1G 30G 24% / >> >> tmpfs 1001M 840K 1000M 1% /dev/shm >> >> tmpfs 5.0M 0 5.0M 0% /run/lock >> >> tmpfs 1001M 0 1001M 0% /sys/fs/cgroup >> >> /dev/vda15 105M 3.4M 102M 4% /boot/efi >> >> tmpfs 201M 0 201M 0% /run/user/1000 >> >> And server specs from Digital Ocean: >> 2 GB Memory / 40 GB Disk / LON1 - Ubuntu 16.04.3 x64 from >> >> I posted the question in this >> <https://stackoverflow.com/questions/45522145/crash-when-installing-heavy-packages-in-remote-server-with-rstudio-server?noredirect=1#comment78011638_45522145> >> stackoverflow thread and actually found out that the problem is not from >> Rstudio server in itself because I get the exact same error with: >> R CMD INSTALL PISA2000lite_1.0.tar.gz >> >> Any idea why this might be occurring? >> >> ----------------------------------- >> >> Jorge Cimentada >> >> *https://cimentadaj.github.io/ <https://cimentadaj.github.io/>* >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> >