search for: bigdata

...ce, even though brick usage is heavily skewed right now (most bricks are at 87% capacity, but recently added ones are at 18-28%). Any ideas what might be causing this? The only references to rebalance memory leaks I could find were related to 3.2.x, not 3.3.1. gluster volume info: Volume Name: bigdata Type: Distributed-Replicate Volume ID: 56498956-7b4b-4ee3-9d2b-4c8cfce26051 Status: Started Number of Bricks: 25 x 2 = 50 Transport-type: tcp Bricks: Brick1: ml43:/mnt/donottouch/localb Brick2: ml44:/mnt/donottouch/localb Brick3: ml43:/mnt/donottouch/localc Brick4: ml44:/mnt/donottouch/localc Brick...

...43 13773 2.7GB 4285256 28574 > stopped > ml47 37618 241.3MB 4266889 24916 > stopped which prompted me to look at the rebalance log: > [2012-11-30 11:06:12.533580] W > [client3_1-fops.c:258:client3_1_mknod_cbk] 0-bigdata-client-12: remote > operation failed: File exists. Path: > /foo/data/onemil/dataset/bar/f8old/baz/85/m_85_269615212_b91ff3077e.t7 > (00000000-0000-0000-0000-000000000000) > [2012-11-30 11:06:12.533657] E [dht-common.c:1911:dht_getxattr] > 0-bigdata-dht: layout is NULL > [2012-...

Si, Bueno en realidad, estamos en un curso de bigdata y estan haciendo mucha scosas con Java, pero yo intento pasarlo a Python tood, ya uqe me parece un lenguaje mucho mas simple y más parecido a R Y si, lo que quiero principalmente es aplicar machine learning sobre conjuntos de datos enormes. Alguna recomendación? ____________________________...

Aunque es un poco offtopic, creeis necesario aprender java para temas de bigdata o con python es suficiente Gracias Jesús [[alternative HTML version deleted]]

...rt?a package called 'Parent' and contains a function that calls out to an object in the namespace of 'Parent'. When this function is called without attaching 'Child' to the search path, the function in 'Parent' cannot be found. Here an example from the manual of the?bigdata?package, but the problem is very widespread: x = matrix(rnorm(50*80),50,80) beta = c(3,2,1.5,rep(0,77)) y = rnorm(50) + x%*%beta z1 = bigdata::lasso.stars(x,y) The example fails because lasso.stars depends on 'glmnet' which is not loaded until?bigdata?is attached. The only way to be able...

On Mon, Nov 25, 2013 at 05:32:23PM +0100, Gonzalo Aguilar Delgado wrote: > Hello, > > I've found a problem running libguestfs on my system. > > The system has /var mounted with a symbolic link to /mnt/bigdata/var > > > > root@red1:/var/log/nova# ls -ld /mnt/bigdata/var > drwxrwxr-x 15 root users 4096 jul 7 17:55 /mnt/bigdata/var > root@red1:/var/log/nova# ls -ld /var > lrwxrwxrwx 1 root root 16 oct 30 2012 /var -> /mnt/bigdata/var > > > Any help here? This is a b...

Hi all, I'm currently programming my first complete package in S4. (thanks to Christophe Genolini for the nice introduction he wrote). I have an object "Data" with a number of slots. One of those slots is "meteo". Now "Meteo" is on itself a class with again a number of slots (like rainfall, temperature,..., you get the picture). I defined the slot

Hi, I am using version 2.1-1 of the ff package. I have a data set with 80 million rows and I need to create a new ffdf object, subseting by values in one of the original ffdf's columns. Here is my code: bigData <- read.table.ffdf(file="/data/demodata/data/smallData.txt", next.rows=1e5, head=TRUE, sep="|") dim(bigData) N <- nrow(bigData);N select <- ff( vmode='logical', length=N, update=FALSE) for (i in chunk(bigData, from=1, to=N, by=1e5)) select[i] <- ifelse(bigD...

Buenas, ¿Alguien ha hecho algun curso sobre bigdata que merezca la pena? La verdad es quequiero adquirir competencias sobre el manejo de grandes volumenes de información, hadoop, spark, etc y no se muy bien que estudios cursar. He estado mirando, pero veo que las escuelas de negocio lo orientan a big data + business analytics y no se si pueden pasar...

Hi, since bigglm doesn't have update, I was wondering how to achieve something like (similar to the example in ff package manual using biglm): first <- TRUE ffrowapply ({ if (first) { first <- FALSE fit <- bigglm(eqn, as.data.frame(bigdata[i1:i2,,drop=FALSE]), chunksize = 10000, family = binomial()) } else { fit <- update(fit, as.data.frame(bigdata[i1:i2,,drop=FALSE]), chunksize = 10000, family = binomial()) } }, X=bigdata, VERBOSE = TRUE, BATCHSIZE = nmax) Many thanks. Yuesheng [[alternative HTML version deleted]]

Hi, as in the subject, is it possible to call on of the analysis pass present in the LLVM (lib/ folder) within a tool pass (tools/ folder)? Thanks. Best, Simone

Hi there I'm working with ncdf data. I have different dataset for 4 runs, 4 seasons and 3 timeslices (48 datasets in total). The datasets have the following dimensions: 96 longitudes, 48 latitudes and 30 time steps. To read all of them in, I wrote the following loop: runs <- c("03","04","05","06") years <- c(1851,1961,2061) seasons <-

On Tue, Nov 26, 2013 at 01:06:58PM +0100, Gonzalo Aguilar Delgado wrote: > Starting /init script ... > initctl: Unable to connect to Upstart: Failed to connect to socket > /com/ubuntu/upstart: Connection refused > /etc/init.d/udev: 290: /etc/init.d/udev: > /lib/udev/create_static_nodes: not found > * Starting the hotplug events dispatcher udevd > ...done. >

Is this on Ubuntu? You probably need to add this patch: https://github.com/libguestfs/libguestfs/commit/ec6f9ee542e0e9ad0109a7522e83ea07b2f8a002 Rich. Hi, Yes it is. I'm trying to find where to put this, but I'm unable to find it. Do I have to compile it?

Hello, Is it possible to execute functions (outside the ui and server shiny environments) after reading data using reactiveFileReader() ? For example, I'd like to fit a linear model on data read using reactiveFileReader() outside ui/server. library(shiny) library(dplyr) bigData <- reactiveFileReader(1000, NULL, 'data.csv', read.csv) fit <- lm(y ~., data = bigData()) ui <- function() { } server <- function(input, output) { } Thank you, Lars. [[alternative HTML version deleted]]

Hello, I've found a problem running libguestfs on my system. The system has /var mounted with a symbolic link to /mnt/bigdata/var root@red1:/var/log/nova# ls -ld /mnt/bigdata/var drwxrwxr-x 15 root users 4096 jul 7 17:55 /mnt/bigdata/var root@red1:/var/log/nova# ls -ld /var lrwxrwxrwx 1 root root 16 oct 30 2012 /var -> /mnt/bigdata/var Any help here? this is test tool output: libguestfs-test-tool ******...

Hi listers, I am trying to arrange my data and I didn't find any information how to do it! I have a data with 3 variables: X Y Z 1-I would like to multiplicate de information of X according to the number I have for my Y variable... 2-Then I want to identify with a dicotomic variable by the number according my variable Z from X... I can do the first part by... z<-rep(x,y) But I don't

...a that using these classes to represent data can avoid copying that's not needed, while retaining the standard R functional semantics, or close to that. For a quick look, see ?LocalReferenceClasses. Here is the idea. We imagine that our object has components/slots/attributes/fields "BigData", say, and "twiddle". With normal R evaluation, replacing "twiddle" in the object will cause internal duplication of the whole thing, in the very likely case that we pass some object, myX say as argument x to a function. As soon as the evaluator sees a replacement func...

(Quisiera aclarar antes de nada que hay total libertad de usar las herramientas analíticas que se deseen, por supuesto R, y que se va a poner a disposición de los participantes una plataforma BigData de Telefónica -cosmos). ¿Puede BigData reformar tu ciudad? ¡Entra en el Datathon para el bien social de la Campus Party, en Londres, del 2 al 7 de Septiembre 2013! Regístrate en: http://new.dynamicinsights.telefonica.com/765/enter-the-datathon Datos, Premios y reuniones previas aquí: http://new...

Hi, I want to perform some analysis on subsets of huge data files. There are 20 of the files and I want to select the same subsets of each one (each subset is a chunk of 1500 or so consecutive rows from several million). To save time and processing power is there a method to tell R to *only* read in these rows, rather than reading in the entire dataset then selecting subsets and deleting the