Hi all, I have a function written by me that read a matrix (data frame) from a txt with 4 million of rows and 13 columns. The think is my function works with an input matrix of 100x13 and now I tried to execute my function with the big "input file" and it is running form the moment two hours... There is a way to know (how much time could it cost?) The second question is... I want to buy a new computer...to threat files like this (and make on the if....for....loops and so on)... I need a computer with high RAM or to "speed" this executing time I need other "technical hardware items".... I´m sure my programmation can be simplied but anyway I hope someone can give me his/her opinion. Many Thaks in advance. [[alternative HTML version deleted]]
Hi all,
My file that worked with a thin matrix (with few rows and colums) finally
made an error but I can´t comprehed why?
valor ausente donde TRUE/FALSE es necesario (absent value TRUE/FALSE is
necessary?)
Error en if (data[i, j] - last1[1, j] != 0) data2[i, j] = 0 else { :
You recommed to part the file of 4 Million rows x 13 colums?
2011/6/29 Trying To learn again <tryingtolearnagain@gmail.com>
> Hi all,
>
> I have a function written by me that read a matrix (data frame) from a txt
> with 4 million of rows and 13 columns.
>
> The think is my function works with an input matrix of 100x13 and now I
> tried to execute my function with the big "input file" and it is
running
> form the moment two hours...
>
> There is a way to know (how much time could it cost?)
>
> The second question is...
>
> I want to buy a new computer...to threat files like this (and make on the
> if....for....loops and so on)... I need a computer with high RAM or to
> "speed" this executing time I need other "technical hardware
items"....
>
> I´m sure my programmation can be simplied but anyway I hope someone can
> give me his/her opinion.
>
> Many Thaks in advance.
>
[[alternative HTML version deleted]]
What type of computer do you have now: operating system, memory. Here is how long it took for me to read in a file with 4M lines and 13 columns of numerics on each line:> system.time(x <- scan('/temp/large.txt', what = 0))Read 52614432 items user system elapsed 23.67 0.67 24.39> str(x)num [1:52614432] 1 1 1 1 1 1 1 1 1 1 ...> > object.size(x)420915496 bytes So how are you reading it in? Is your system paging? It should not take 2 hours. On Wed, Jun 29, 2011 at 4:43 PM, Trying To learn again <tryingtolearnagain at gmail.com> wrote:> Hi all, > > I have a function written by me that read a ?matrix (data frame) from a txt > with 4 million of rows and 13 columns. > > The think is my function works with an input matrix of 100x13 and now I > tried to execute my function with the big "input file" and it is running > form the moment two hours... > > There is a way to know (how much time could it cost?) > > The second question is... > > I want to buy a new computer...to threat files like this (and make on the > if....for....loops and so on)... I need a computer with high RAM or to > "speed" this executing time I need other "technical hardware items".... > > I?m sure my programmation can be simplied but anyway I hope someone can give > me his/her opinion. > > Many Thaks in advance. > > ? ? ? ?[[alternative HTML version deleted]] > > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > >-- Jim Holtman Data Munger Guru What is the problem that you are trying to solve?