Dear group I have two R sessions ?running on Ubuntu 14.0x server , and I found that my program will take too long time to be finished( months...!), I used top command and found that ??cpu usage is 21.3%.? the server is Enterprise SP-64 - 64G E5-1630v3 SoftRaid 2x2 TB Server . 6 core how can I speed the program, kindly I need tutorial or book chapter that helps.. thanks in advance Ragia
Hi Ragia, Improving the efficiency of a program usually requires detailed analysis of what it is doing and how those operations can be performed more rapidly. That is to say, without knowing what the program is supposed to accomplish and how it is doing it now, very little help can be provided. One thing you might look for is "disk-thrashing" where your storage media (e.g. hard disk) is being accessed continuously. This usually indicates that the program is swapping out data to the disk, which is typically slow compared to processing it in memory. Jim On Tue, Mar 15, 2016 at 8:32 AM, Ragia . <ragia11 at hotmail.com> wrote:> > > > > Dear group > I have two R sessions running on Ubuntu 14.0x server , and I found that my program will take too long time to be finished( months...!), I used top command and found that cpu usage is 21.3%. > > the server is Enterprise SP-64 - 64G E5-1630v3 SoftRaid 2x2 TB Server . 6 core > > how can I speed the program, kindly I need tutorial or book chapter that helps.. > thanks in advance > Ragia > > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
thanks for answering the program is simulation it runs on data ( wish is taken from file to memory once) ?but it runs about 10k times and in each mad some calculations ? hope this clear the problem,? Ragia? ----------------------------------------> Date: Tue, 15 Mar 2016 08:45:56 +1100 > Subject: Re: [R] How to speed up R program > From: drjimlemon at gmail.com > To: ragia11 at hotmail.com > CC: r-help at r-project.org > > Hi Ragia, > Improving the efficiency of a program usually requires detailed > analysis of what it is doing and how those operations can be performed > more rapidly. That is to say, without knowing what the program is > supposed to accomplish and how it is doing it now, very little help > can be provided. One thing you might look for is "disk-thrashing" > where your storage media (e.g. hard disk) is being accessed > continuously. This usually indicates that the program is swapping out > data to the disk, which is typically slow compared to processing it in > memory. > > Jim > > > On Tue, Mar 15, 2016 at 8:32 AM, Ragia . <ragia11 at hotmail.com> wrote: >> >> >> >> >> Dear group >> I have two R sessions running on Ubuntu 14.0x server , and I found that my program will take too long time to be finished( months...!), I used top command and found that cpu usage is 21.3%. >> >> the server is Enterprise SP-64 - 64G E5-1630v3 SoftRaid 2x2 TB Server . 6 core >> >> how can I speed the program, kindly I need tutorial or book chapter that helps.. >> thanks in advance >> Ragia >> >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code.
Hi Ragia, If time is really a big problem and you have a lot of datas and you want to use all the cores of the processor, you should use FORTRAN for your calculations. But this is only possible with a real FORTRAN developer and will take some times. I am an old freelance (61) and I began computing in 1974 with FORTRAN ! I have been hired by a large company in France to translate calculations to FORTRAN. First try : we went from 20 minutes to less than 5 seconds. So it's worth. BUT it is complicated. R is a very permissive and 'near the human' language (I work with hydrological engineers and they can build large R programs) at the opposite, FORTRAN can only do calculations but it does them very very quickly and you must be aware of the hardware on witch you are working and you must take care of integer / real, real4 and real8, .... It's REALLY a computer language. 'modern' FORTRAN are able to use all the cores of the processor and then to have 100% cpu usage. It's INTEL OMP, MPI library, ... but it is another big step. So it's another way but a difficult one. Jean in France PS I spoke of Fortran, but good results can be done with C : C calculations are a little slower than FORTRAN but C possibilities are a lot more 'great' than FORTRAN. -------------- next part -------------- Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free.
You can start with ?Rprof which can help you find out what steps in your calculations are taking the most time. Depending on what, exactly, you're doing, look for places where you're using a data frame when a matrix would serve the purpose. Data frames have more overhead than matrices. Sarah Goslee's suggestion to avoid growing in place is a good one. -Don -- Don MacQueen Lawrence Livermore National Laboratory 7000 East Ave., L-627 Livermore, CA 94550 925-423-1062 On 3/14/16, 2:32 PM, "R-help on behalf of Ragia ." <r-help-bounces at r-project.org on behalf of ragia11 at hotmail.com> wrote:> > > > >Dear group >I have two R sessions running on Ubuntu 14.0x server , and I found that >my program will take too long time to be finished( months...!), I used >top command and found that cpu usage is 21.3%. > >the server is Enterprise SP-64 - 64G E5-1630v3 SoftRaid 2x2 TB Server . 6 >core > >how can I speed the program, kindly I need tutorial or book chapter that >helps.. >thanks in advance >Ragia > > >______________________________________________ >R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide >http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code.