Hi, My name is Alexandre Khelifa and I have been using R at my work for about 2 years. I have been working on a project when we do Monte Carlo Simulation and it involves a lot of calculations. I am currently using R x64.3.0.1 and used to work on a 4GB machine. However, the calculation time was very long (about 2 weeks), and the IT team and I decided to add more memory and to make it a 8GB virtual machine. I also added the following line in my code: * * - *options(java.parameters = "-Xmx8192m") * I re-did the same calculations (4GB vs. 8GB) but did not see a significant increase in the calculation time, so I was wondering if I did anything wrong and/or what would be the best solution to increase this time. Thanks a lot for your help and more generally for building such an amazing (and free) tool. Let me know if you have any questions. Regards, Alexandre Khelifa [[alternative HTML version deleted]]
On Mon, Oct 21, 2013 at 4:12 PM, Alexandre Khelifa <akhelifa at logitech.com> wrote:> Hi, > > My name is Alexandre Khelifa and I have been using R at my work for about 2 > years. I have been working on a project when we do Monte Carlo Simulation > and it involves a lot of calculations. > > I am currently using R x64.3.0.1 and used to work on a 4GB machine. > However, the calculation time was very long (about 2 weeks), and the IT > team and I decided to add more memory and to make it a 8GB virtual machine. > I also added the following line in my code: > * > * > > - *options(java.parameters = "-Xmx8192m") * > > > I re-did the same calculations (4GB vs. 8GB) but did not see a significant > increase in the calculation time, so I was wondering if I did anything > wrong and/or what would be the best solution to increase this time.If you and your team travel Earth-to-Space round trip at 87% the speed of light, your computations would *physically* become roughly twice as fast (you must leave computer behind). Alternatively, look at help(package="parallel"). But what really makes a difference is to find and replace bottle necks in your code by profiling it, cf. help("Rprof"). I'd go with the latter if you haven't already done so. /Henrik> > Thanks a lot for your help and more generally for building such an amazing > (and free) tool. > Let me know if you have any questions. > > Regards, > > Alexandre Khelifa > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
I would start with taking a subset of the data (definitely some that would run in less than 10 minutes) and use the profiler "Rprof" to see where time is being spent. you can use the the task monitor (if on windows) to see how much memory you are using; it sounds like you did not need the extra memory. You might see if you can partition your data so you can run multiple versions of R and then merge the results. Anything that takes more than a half hour, for me, is looked into to see where the problems are. For example dataframes arevexpensive to access and conversion to matrices is one way to speed it up. the is where the profiler helps. Sent from my iPad On Oct 21, 2013, at 19:12, Alexandre Khelifa <akhelifa at logitech.com> wrote:> Hi, > > My name is Alexandre Khelifa and I have been using R at my work for about 2 > years. I have been working on a project when we do Monte Carlo Simulation > and it involves a lot of calculations. > > I am currently using R x64.3.0.1 and used to work on a 4GB machine. > However, the calculation time was very long (about 2 weeks), and the IT > team and I decided to add more memory and to make it a 8GB virtual machine. > I also added the following line in my code: > * > * > > - *options(java.parameters = "-Xmx8192m") * > > > I re-did the same calculations (4GB vs. 8GB) but did not see a significant > increase in the calculation time, so I was wondering if I did anything > wrong and/or what would be the best solution to increase this time. > > Thanks a lot for your help and more generally for building such an amazing > (and free) tool. > Let me know if you have any questions. > > Regards, > > Alexandre Khelifa > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
The advice given is sensible. For a timing study see http://rwiki.sciviews.org/doku.php?id=tips:rqcasestudy We found that for optimization calculations, putting the objective function calculation or parts thereof in Fortran was helpful. But we kept those routines pretty small -- less than a page -- and we just called them to evaluate things, avoiding passing around information back and forth to R. JN On 13-10-22 06:00 AM, r-help-request at r-project.org wrote:> Message: 58 > Date: Tue, 22 Oct 2013 05:47:15 -0400 > From: Jim Holtman <jholtman at gmail.com> > To: Alexandre Khelifa <akhelifa at logitech.com> > Cc: "r-help at r-project.org" <r-help at r-project.org> > Subject: Re: [R] R - How to "physically" Increase Speed > Message-ID: <73D989DA-B6B3-421D-838C-903DA34350F2 at gmail.com> > Content-Type: text/plain; charset=us-ascii > > I would start with taking a subset of the data (definitely some that would run in less than 10 minutes) and use the profiler "Rprof" to see where time is being spent. you can use the the task monitor (if on windows) to see how much memory you are using; it sounds like you did not need the extra memory. > > You might see if you can partition your data so you can run multiple versions of R and then merge the results. > > Anything that takes more than a half hour, for me, is looked into to see where the problems are. For example dataframes arevexpensive to access and conversion to matrices is one way to speed it up. the is where the profiler helps. >