similar to: Suggestion: Create On-Disk Dataframes

It is not needed. There is a large community of developer using SparkR. https://spark.apache.org/docs/latest/sparkr.html It does exactly what you want. On 3 September 2017 at 20:38, Juan Telleria <jtelleriar at gmail.com> wrote: > Dear R Developers, > > I would like to suggest the creation of a new S4 object class for On-Disk > data.frames which do not fit in RAM memory, which

On 4 September 2017 at 11:35, Suzen, Mehmet wrote: | It is not needed. There is a large community of developer using SparkR. | https://spark.apache.org/docs/latest/sparkr.html | It does exactly what you want. I hope you are not going to mail a sparkr commercial to this list every day. As the count is now at two, this may be an excellent good time to stop it. Dirk --

Hola. Últimamente ha habido en la lista varios hilos sobre análisis de grandes volúmenes de datos con R. Las alternativas que se han mencionado son: - Usar una máquina más potente, vía Amazon Web Services, por ejemplo - Paralelización con openMp - h2o y su paquete para R, - Paquete sparklyr como wrapper de los algoritmos de spark, Y por supuesto, utilizar muestreo o incluso si tenemos

Merece la pena aprender python para Big data con Spark o usando la libreria que acaba de salir para R es suficiente? Qué creeis? [[alternative HTML version deleted]]

Buenas a todos. Ya sé que con sparkR o sparklyr puedo leer fácilmente ficheros con formato parquet, pero ¿hay alguna forma de leerlos sin tener que arrancar spark? Mi situación es que tengo unos ficheros en formato parquet en s3 y quiero leerlos desde una instancia pequeñita de amazon EC2 que quiero mantener sin instalarle spark. Estoy bicheando la librería https://github.com/cloudyr/aws.s3 y va

Hola Carlos. spark_read_parquet es de sparklyr y necesita un sparkcontext inicializado para leer el fichero de parquet. El mié., 4 oct. 2017 22:11, Carlos Ortega <cof en qualityexcellence.es> escribió: > Hola José Luis, > > ¿Has probado directamente con "dplyr"?... > > spark_read_parquet >

Dear R Developers, In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have the innodb_buffer_pool_size or the key_buffer_size for setting the maximum amount of RAM which can be used by a Server Instance: ?Would it be possible to create an R Configuration Variable which fixes the maximum amount of RAM memory to be used as Commit / Dynamic Memory Allocation? Thank you. Juan

Hola Mª Luz, Te cuento un poco mi visión: Lo primero de todo es tener claro qué quiero hacer exactamente en paralelo, se me ocurren 3 escenarios: (1) Aplicar un modelo en este caso SVM sobre unos datos muy grandes y por eso necesito hadoop/spark (2) Realizar muchos modelos SVM sobre datos pequeños (por ejemplo uno por usuario) y por eso necesito hadoop/spark para parelilizar estos procesos

Dear R Developers, Adding to source() base function a Timer which indicates the execution time of the source code would be a very well welcome feature, and in my opinion not difficult to implement as an additional funtion argument. The source(timing = TRUE) function shall execute internally the following code for each statement: old <- Sys.time() # get start time at the beginning of source()

Hola buenas, me preguntaba si alguno usa hadoop Spark en su día día y si me podíais recomendar un buen curso para empezar. Estuve en la charla de meetup de madrid hace unos meses de Rspark y estuvo bien, ahora me preguntaba si es posible profundizar. Pero me gustaría tener recomendaciones de cualquier material que podáis recomendar, cursos de coursera que hayais hecho, libros que hayais leido,

Hi, On a fresh install of R on mac os x 10.6.4 (snow leopard) RSQLite did not install while running biocLite() $ R R version 2.11.1 (2010-05-31) Copyright (C) 2010 The R Foundation for Statistical Computing ISBN 3-900051-07-0 [...] Loading required package: utils BioC_mirror = http://www.bioconductor.org Change using chooseBioCmirror(). [Previously saved workspace restored] >

Dear R Community Members, I would like to add to one article I have written the best Graphical User Interfaces the R programming language has. For the moment I know: A) Rstudio. B) R Tools for Visual Studio. C) Open Analytics Architect. Are there others worth to mention? Thank you. Kind regards, Juan Telleria [[alternative HTML version deleted]]

Hi All, I tried to install AnnotationDBI like so: source("http://bioconductor.org/biocLite.R") biocLite("AnnotationDbi") and got this error: .... Loading required package: RSQLite Error in dyn.load(file, ...) : unable to load shared library '/RHEL3/local/lib64/R/library/ RSQLite/libs/RSQLite.so': /RHEL3/local/lib64/R/library/RSQLite/libs/RSQLite.so: undefined

Please let me know where i have to type below thing to RSQLite database get installed.Please let me know the solution.Thanks in advance RSQLite -- Embedding the SQLite engine in R (The RSQLite package includes a recent copy of the SQLite distribution from http://www.sqlite.org.) Installation ------------ There are 3 alternatives for installation: 1. Simple installation: R CMD

[Sent offlist accidentally] What concerns me first and foremost is that the licensure would have to be ironclad (including for commercial use like vanilla R now) as well as ensuring that R remains completely FLOSS. Anything ?added? to R has to be a no-strings-attached gift to R. Also, I would think that it would have to play nice with existing workflows (like OpenBLAS instead of MKL) unless

Dear R Developers, First of all, I would like to thank you Jeroen Ooms for taking the binary Window Builds from Duncan. I firmly believe that the R Community will benefit a lot from his work. However, the debate I would like to open is about if some of Microsoft R Open Code shall be ported from R Open to Mainstream R. There are some beneficts in R Open such as multithreaded performance:

Dear all, I am trying to install RSQLite package since I want to install "sqldf", and I used >> install.packages("RSQLite") first, which gave Error message as below: make: *** [RS-DBI.o] Error 1 chmod: cannot access `/usr/lib/R/library/RSQLite/libs/*': No such file or directory ERROR: compilation failed for package 'RSQLite' ** Removing

Hi, I am trying to install RSQLite package on my Fedora workstation. I tried to install other packages as well, but each time I got the same error messages saying "compilation error" and "non zero exit status". Do I have to specify "lib="? I never specified the library path before when I was using Fedora Core 6. Warning in install.packages("RSQLite")

Hace poco me puse al tema del big data y la verdad es que een este campo la decisión parece clara. Python ahora mismo está un paso por delante de R, aunque sparklyR puede igualar la contienda. Pero lo que me empieza a preocupar es que parece que si nos alejamos del Big Data, Python tb está ganando adeptos a pasos agigantados. ¿Está perdiendo R la batalla? [[alternative HTML version deleted]]

But by statement in the source file, I mean, for knowing during the execution how much time is taking, without having to wait till the end. 2017-12-21 13:06 GMT+01:00 I?aki ?car <i.ucar86 at gmail.com>: > 2017-12-21 12:46 GMT+01:00 Juan Telleria <jtelleriar at gmail.com>: > > Dear R Developers, > > > > Adding to source() base function a Timer which indicates the