Happy New Year all! I am looking for a good solution for keeping record of my experiments - could you please help me? My work is about analysing data... My current work-flow: 1. Everyday my bosses give me some small steps/tasks for analysing data - which are parts of one bigger/whole project. 2. Everyday I send tens of emails to bosses/colleagues to report my findings in each step. 3. Bosses/colleagues often respond to my findings in real-time and suggest new experiments/steps and ask "what-if" questions. 4. I often have to manually copy and paste the results from R console and put them into an Excel and decorate a bit and send out. 5. Every one week and 2 weeks, we need to present to more senior bosses with more nice-looking presentations which is a summary of our findings in those 1-2 weeks. It's this time that is most chaotic because my colleagues and I have to dig into all the hundreds of emails in the past 1-2 weeks and copy and paste and organize those data again and make a nice overall summary for presentation... 6. As I am a hard-working guy, I myself often run my own random/ad-hoc experiments using out-of-work time and whenever I have interesting findings, I will send to immediate bosses and colleagues to seek their comments. 7. All these experiments are in fact variations of different versions/ideas of one big/whole project. Lets say in one big project bosses/colleagues and I have come up with a few big ideas, then we have a few sub-projects: MyProjectIdea1 MyProjectIdea2 ... MyProjectIdeaN And each idea has a few variations, mostly are for answering "what-if" questions by varying the parameters here and there ... For example: MyProjectIdea1_Variation1_WhatIfParam1ChangedTo1.2? ... ... etc. 8. Most experiments run tens of minutes to many hours... and some of them have to run on Linux, and some others can be run on Windows. Fortunately we have universal paths accessible on both Windows and Linux, so those won't be problem... 9. Because of the time-consuming nature of these experiments, I also save the images as "rData" whenever I can. However, it's necessary to keep track of the context where these data were generated. Otherwise even the records of these images won't help recall the scenario we have run... --------------------------------------------------- Keeping track of these changes and all kinds of "what-if"s now becomes increasingly a problem for me. Some times in order to respond to a query, although I have done it before already, but because I didn't keep record and save the result, or even though I have saved the memory image yet I am not completely sure about the "cleanness" of the results/data,I have to redo it and wait for another few hours. Is there a way that I can manage these whole processes better and be more productive? I have been digging and thinking about this for while and I guess Sweave is the right way to go? The problem for Sweave is that it's hard to make Latex generated pdf appealing to business managers... so if I keep records in Sweave/Latex for my own record/benefit (that's already a big benefit)... I still need to somehow manually copy/paste the data from Sweave/Latex/pdf into Word/Excel/Powerpoint in order to make a nice presentation... I know there are some Open Office and Word version of Sweave... the problem is that I couldn't find many demonstrations on these topics and my question is: are they good and can they fulfill what we needed? Your thoughts are greatly appreciated! Thanks a lot! [[alternative HTML version deleted]]
Have you seen r2wd? http://www.r-bloggers.com/exporting-r-output-to-ms-word-with-r2wd-an-example-session/ On Jan 1, 2012, at 10:50 AM, Michael <comtech.usa at gmail.com> wrote:> Happy New Year all! > > I am looking for a good solution for keeping record of my experiments - > could you please help me? > > My work is about analysing data... My current work-flow: > > 1. Everyday my bosses give me some small steps/tasks for analysing data - > which are parts of one bigger/whole project. > 2. Everyday I send tens of emails to bosses/colleagues to report my > findings in each step. > 3. Bosses/colleagues often respond to my findings in real-time and suggest > new experiments/steps and ask "what-if" questions. > 4. I often have to manually copy and paste the results from R console and > put them into an Excel and decorate a bit and send out. > 5. Every one week and 2 weeks, we need to present to more senior bosses > with more nice-looking presentations which is a summary of our findings in > those 1-2 weeks. It's this time that is most chaotic because my colleagues > and I have to dig into all the hundreds of emails in the past 1-2 weeks and > copy and paste and organize those data again and make a nice overall > summary for presentation... > 6. As I am a hard-working guy, I myself often run my own random/ad-hoc > experiments using out-of-work time and whenever I have interesting > findings, I will send to immediate bosses and colleagues to seek their > comments. > 7. All these experiments are in fact variations of different versions/ideas > of one big/whole project. Lets say in one big project bosses/colleagues and > I have come up with a few big ideas, then we have a few sub-projects: > > MyProjectIdea1 > MyProjectIdea2 > ... > MyProjectIdeaN > > And each idea has a few variations, mostly are for answering "what-if" > questions by varying the parameters here and there ... > For example: > > MyProjectIdea1_Variation1_WhatIfParam1ChangedTo1.2? > ... > ... > etc. > > 8. Most experiments run tens of minutes to many hours... and some of them > have to run on Linux, and some others can be run on Windows. Fortunately we > have universal paths accessible on both Windows and Linux, so those won't > be problem... > > 9. Because of the time-consuming nature of these experiments, I also save > the images as "rData" whenever I can. However, it's necessary to keep track > of the context where these data were generated. Otherwise even the records > of these images won't help recall the scenario we have run... > > --------------------------------------------------- > > Keeping track of these changes and all kinds of "what-if"s now becomes > increasingly a problem for me. > > Some times in order to respond to a query, although I have done it before > already, but because I didn't keep record and save the result, or even > though I have saved the memory image yet I am not completely sure about the > "cleanness" of the results/data,I have to redo it and wait for another few > hours. > > Is there a way that I can manage these whole processes better and be more > productive? > > I have been digging and thinking about this for while and I guess Sweave is > the right way to go? > > The problem for Sweave is that it's hard to make Latex generated pdf > appealing to business managers... so if I keep records in Sweave/Latex for > my own record/benefit (that's already a big benefit)... I still need to > somehow manually copy/paste the data from Sweave/Latex/pdf into > Word/Excel/Powerpoint in order to make a nice presentation... > > I know there are some Open Office and Word version of Sweave... the problem > is that I couldn't find many demonstrations on these topics and my question > is: are they good and can they fulfill what we needed? > > Your thoughts are greatly appreciated! > > Thanks a lot! > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
I am not at my desk but you might search the CRAN web site for "reproducible research". Respectfully, Frank Lawrence -----Original Message----- From: Michael <comtech.usa at gmail.com> Sender: r-help-bounces at r-project.orgDate: Sun, 1 Jan 2012 09:50:24 To: r-help<R-help at stat.math.ethz.ch> Subject: [R] R report generator (for Word)? Happy New Year all! I am looking for a good solution for keeping record of my experiments - could you please help me? My work is about analysing data... My current work-flow: 1. Everyday my bosses give me some small steps/tasks for analysing data - which are parts of one bigger/whole project. 2. Everyday I send tens of emails to bosses/colleagues to report my findings in each step. 3. Bosses/colleagues often respond to my findings in real-time and suggest new experiments/steps and ask "what-if" questions. 4. I often have to manually copy and paste the results from R console and put them into an Excel and decorate a bit and send out. 5. Every one week and 2 weeks, we need to present to more senior bosses with more nice-looking presentations which is a summary of our findings in those 1-2 weeks. It's this time that is most chaotic because my colleagues and I have to dig into all the hundreds of emails in the past 1-2 weeks and copy and paste and organize those data again and make a nice overall summary for presentation... 6. As I am a hard-working guy, I myself often run my own random/ad-hoc experiments using out-of-work time and whenever I have interesting findings, I will send to immediate bosses and colleagues to seek their comments. 7. All these experiments are in fact variations of different versions/ideas of one big/whole project. Lets say in one big project bosses/colleagues and I have come up with a few big ideas, then we have a few sub-projects: MyProjectIdea1 MyProjectIdea2 ... MyProjectIdeaN And each idea has a few variations, mostly are for answering "what-if" questions by varying the parameters here and there ... For example: MyProjectIdea1_Variation1_WhatIfParam1ChangedTo1.2? ... ... etc. 8. Most experiments run tens of minutes to many hours... and some of them have to run on Linux, and some others can be run on Windows. Fortunately we have universal paths accessible on both Windows and Linux, so those won't be problem... 9. Because of the time-consuming nature of these experiments, I also save the images as "rData" whenever I can. However, it's necessary to keep track of the context where these data were generated. Otherwise even the records of these images won't help recall the scenario we have run... --------------------------------------------------- Keeping track of these changes and all kinds of "what-if"s now becomes increasingly a problem for me. Some times in order to respond to a query, although I have done it before already, but because I didn't keep record and save the result, or even though I have saved the memory image yet I am not completely sure about the "cleanness" of the results/data,I have to redo it and wait for another few hours. Is there a way that I can manage these whole processes better and be more productive? I have been digging and thinking about this for while and I guess Sweave is the right way to go? The problem for Sweave is that it's hard to make Latex generated pdf appealing to business managers... so if I keep records in Sweave/Latex for my own record/benefit (that's already a big benefit)... I still need to somehow manually copy/paste the data from Sweave/Latex/pdf into Word/Excel/Powerpoint in order to make a nice presentation... I know there are some Open Office and Word version of Sweave... the problem is that I couldn't find many demonstrations on these topics and my question is: are they good and can they fulfill what we needed? Your thoughts are greatly appreciated! Thanks a lot! [[alternative HTML version deleted]] ______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Hi Michael, I like Sweave and LaTeX, but I can appreciate the difficulty using it with collaborators. What about something similar using HTML? Certainly integrates to any webpages nicely. There are two packages I think do this nicely, one is the R2HTML package (on CRAN). Another one that is not on CRAN yet, but I think has a lot of potential is the knitr package. You can find it on github. I am not personally familiar with any good ways to integrate R with MS Office products. Cheers, Josh On Sun, Jan 1, 2012 at 7:50 AM, Michael <comtech.usa at gmail.com> wrote:> Happy New Year all! > > I am looking for a good solution for keeping record of my experiments - > could you please help me? > > My work is about analysing data... My current work-flow: > > 1. Everyday my bosses give me some small steps/tasks for analysing data - > which are parts of one bigger/whole project. > 2. Everyday I send tens of emails to bosses/colleagues to report my > findings in each step. > 3. Bosses/colleagues often respond to my findings in real-time and suggest > new experiments/steps and ask "what-if" questions. > 4. I often have to manually copy and paste the results from R console and > put them into an Excel and decorate a bit and send out. > 5. Every one week and 2 weeks, we need to present to more senior bosses > with more nice-looking presentations which is a summary of our findings in > those 1-2 weeks. It's this time that is most chaotic because my colleagues > and I have to dig into all the hundreds of emails in the past 1-2 weeks and > copy and paste and organize those data again and make a nice overall > summary for presentation... > 6. As I am a hard-working guy, I myself often run my own random/ad-hoc > experiments using out-of-work time and whenever I have interesting > findings, I will send to immediate bosses and colleagues to seek their > comments. > 7. All these experiments are in fact variations of different versions/ideas > of one big/whole project. Lets say in one big project bosses/colleagues and > I have come up with a few big ideas, then we have a few sub-projects: > > MyProjectIdea1 > MyProjectIdea2 > ... > MyProjectIdeaN > > And each idea has a few variations, mostly are for answering "what-if" > questions by varying the parameters here and there ... > For example: > > MyProjectIdea1_Variation1_WhatIfParam1ChangedTo1.2? > ... > ... > etc. > > 8. Most experiments run tens of minutes to many hours... and some of them > have to run on Linux, and some others can be run on Windows. Fortunately we > have universal paths accessible on both Windows and Linux, so those won't > be problem... > > 9. Because of the time-consuming nature of these experiments, I also save > the images as "rData" whenever I can. However, it's necessary to keep track > of the context where these data were generated. Otherwise even the records > of these images won't help recall the scenario we have run... > > --------------------------------------------------- > > Keeping track of these changes and all kinds of "what-if"s now becomes > increasingly a problem for me. > > Some times in order to respond to a query, although I have done it before > already, but because I didn't keep record and save the result, or even > though I have saved the memory image yet I am not completely sure about the > "cleanness" of the results/data,I have to redo it and wait for another few > hours. > > Is there a way that I can manage these whole processes better and be more > productive? > > I have been digging and thinking about this for while and I guess Sweave is > the right way to go? > > The problem for Sweave is that it's hard to make Latex generated pdf > appealing to business managers... so if I keep records in Sweave/Latex for > my own record/benefit (that's already a big benefit)... I still need to > somehow manually copy/paste the data from Sweave/Latex/pdf into > Word/Excel/Powerpoint in order to make a nice presentation... > > I know there are some Open Office and Word version of Sweave... the problem > is that I couldn't find many demonstrations on these topics and my question > is: are they good and can they fulfill what we needed? > > Your thoughts are greatly appreciated! > > Thanks a lot! > > ? ? ? ?[[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.-- Joshua Wiley Ph.D. Student, Health Psychology Programmer Analyst II, Statistical Consulting Group University of California, Los Angeles https://joshuawiley.com/
Dear Michael, On Sun, Jan 1, 2012 at 10:50 AM, Michael <comtech.usa at gmail.com> wrote:> Happy New Year all![snip]> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.Nobody's yet mentioned that the latest version of Emacs Org mode (http://orgmode.org/) ships with an ODT (OpenDocument Text) exporter. This means that a person can now use the same single plain text file to integrate R code (and/or 30+ other languages) to process: - HTML: ?C-c C-e h- LaTeX: C-c C-e l- PDF: ? C-c C-e p ?(via pdflatex)- ODT: ? C-c C-e o- ...etc. You get syntax highlighting, integrated math formulae, code tangling,... there's even an elementary table editor for simple spreadsheet operations. And again, all exported formats originate from the same plain text file. MS-Word can definitely read HTML, and when I last checked (long ago) there existed plugins for MS_Word to read .odt. ?Of course, LibreOffice can convert ODT (and HTML) to MS-Word if necessary. GoogleDocs does an OK job converting back and forth, too, and if this option is available to you there are some pretty cool Google collaborative tools. On the flipside, Emacs in general is a tough pill to swallow, and that's an understatement. ?Two parting thoughts:1) Org mode is similar to, but not identical with, Sweave.2) Org mode originated as an organization/outlining tool, so it has all sorts of tricks for "Getting Things Done", such as TODO lists, agendas, capturing-archiving, calendar integration, time tracking,... ?These could possibly address some of the other issues you mentioned. Good luck, and Happy New Year.Jay ************************************* G. Jay Kerns, Ph.D. Mathematics and Statistics Youngstown State University http://people.ysu.edu/~gkerns/
On Sun, 1 Jan 2012 09:50:24 -0600 Michael <comtech.usa at gmail.com> wrote:> Happy New Year all! > > I am looking for a good solution for keeping record of my experiments > - could you please help me? >If you are working from scripts, which is a very good way to standardize procedures as a work flow and analysis develop, I would suggest checking sink() and cat(). Used properly these commands can be used to capture screen output to text files. These can then be opened and formatted in Word or Office or Emacs or ... I use this method to capture results of analyses of archaeological data. Use the device() command to capture graphics to jpeg files, pdfs or other graphic formats. If you develop a script of your analysis, you will have a file that will load data, carry out sequenced procedures, shoot the results to a text file designated in sink() or cat(), and the results can then be integrated into a word document. Curiously, there are not many books about R that explicitly address questions like capturing intermediate or final output from an analysis to usable text. The volumes I've found most useful are Modern Applied Statistics with S, R Cookbook, and R in Action (these are not ordered in order of usefulness). None go into anything like detail about the process of actually producing report-quality output (except figures), perhaps because the assumption is that the commands above will be used to capture the statistical output for a report, which will then dealt with externally to R. JWD
On 01/01/12 15:50, Michael wrote:> Happy New Year all! > > I am looking for a good solution for keeping record of my experiments - > could you please help me? > > My work is about analysing data... My current work-flow: > > 1. Everyday my bosses give me some small steps/tasks for analysing data - > which are parts of one bigger/whole project. > 2. Everyday I send tens of emails to bosses/colleagues to report my > findings in each step. > 3. Bosses/colleagues often respond to my findings in real-time and suggest > new experiments/steps and ask "what-if" questions. > 4. I often have to manually copy and paste the results from R console and > put them into an Excel and decorate a bit and send out. > 5. Every one week and 2 weeks, we need to present to more senior bosses > with more nice-looking presentations which is a summary of our findings in > those 1-2 weeks. It's this time that is most chaotic because my colleagues > and I have to dig into all the hundreds of emails in the past 1-2 weeks and > copy and paste and organize those data again and make a nice overall > summary for presentation... > 6. As I am a hard-working guy, I myself often run my own random/ad-hoc > experiments using out-of-work time and whenever I have interesting > findings, I will send to immediate bosses and colleagues to seek their > comments. > 7. All these experiments are in fact variations of different versions/ideas > of one big/whole project. Lets say in one big project bosses/colleagues and > I have come up with a few big ideas, then we have a few sub-projects: > > MyProjectIdea1 > MyProjectIdea2 > ... > MyProjectIdeaN > > And each idea has a few variations, mostly are for answering "what-if" > questions by varying the parameters here and there ... > For example: > > MyProjectIdea1_Variation1_WhatIfParam1ChangedTo1.2? > ... > ... > etc. > > 8. Most experiments run tens of minutes to many hours... and some of them > have to run on Linux, and some others can be run on Windows. Fortunately we > have universal paths accessible on both Windows and Linux, so those won't > be problem... > > 9. Because of the time-consuming nature of these experiments, I also save > the images as "rData" whenever I can. However, it's necessary to keep track > of the context where these data were generated. Otherwise even the records > of these images won't help recall the scenario we have run... > > --------------------------------------------------- > > Keeping track of these changes and all kinds of "what-if"s now becomes > increasingly a problem for me. > > Some times in order to respond to a query, although I have done it before > already, but because I didn't keep record and save the result, or even > though I have saved the memory image yet I am not completely sure about the > "cleanness" of the results/data,I have to redo it and wait for another few > hours. > > Is there a way that I can manage these whole processes better and be more > productive? > > I have been digging and thinking about this for while and I guess Sweave is > the right way to go?I would agree that Sweave is a good way to organise these kind of analysis that you do repetitively. I would also recomend you look at some of the caching methods available (CacheSweave http://cran.r-project.org/web/packages/cacheSweave/index.html ) will allow you to skip chunks of the analysis that haven't changed since the last run.> The problem for Sweave is that it's hard to make Latex generated pdf > appealing to business managers... so if I keep records in Sweave/Latex for > my own record/benefit (that's already a big benefit)... I still need to > somehow manually copy/paste the data from Sweave/Latex/pdf into > Word/Excel/Powerpoint in order to make a nice presentation... > > I know there are some Open Office and Word version of Sweave... the problem > is that I couldn't find many demonstrations on these topics and my question > is: are they good and can they fulfill what we needed? >For Open Office have a look at the odfWeave package ( http://cran.r-project.org/web/packages/odfWeave/index.html ) that functions in a similar way to Sweave but starts with and produces an odt Open Doc text file. I have used it in the past but not recently, although I vaguely remember seeing a post recently that the current version doesn't compile on Windows, so that may curtail it's effectiveness. regards, Paul.