search for: summar

Is there a text mining/ NLP package in R that could do text summarization? For example, take a huge text as input and provide a summary of the text. In package tm, summarization is defined more as high frequency terms which is not what I want. I actually want a summary of what is present in the huge volume of text. Any help on a R package would be helpful. Than...

Hi All, I'm using the Hmisc summarize function and used list instead of llist to provide the by variables. It generated an error message. Is this a bug, or do I misunderstand how Hmisc works with lists? The program below demonstrates the error message. Thanks, Bob x<-1:8 group <- c(1,1,1,1,2,2,2,2) gender<- c(1,2...

...s the below dataset. I have a dataset that has stratum and plot identifier. Within each plot there is variable (Top) stating the number of measurments that should be used to to calculate the mean to the largest "top" elements within one of the vectors (X). I would like to process this summary statistic by groups. At this stage I have been trying to use the "summarize" function within the Hmisc library but I am getting the following error "Error in eval(expr, envir, enclos) : numeric 'envir' arg not of length one In addition: Warning message: no finite argument...

...first three observations have variance sigma^2). Best, Wolfgang -----Original Message----- From: R-devel [mailto:r-devel-bounces at r-project.org] On Behalf Of Arie ten Cate Sent: Saturday, 07 October, 2017 9:36 To: r-devel at r-project.org Subject: [Rd] Discourage the weights= option of lm with summarized data In the Details section of lm (linear models) in the Reference manual, it is suggested to use the weights= option for summarized data. This must be discouraged rather than encouraged. The motivation for this is as follows. With summarized data the standard errors get smaller with increasi...

Hi all, I'd like to use the Hmisc::summarize function, but it uses a function (FUN) of a single vector argument to create the statistical summaries. Consider an easy case: I'd like to compute the correlation between two variables in my dataframe, grouped according to other variables in the same dataframe. For exemple, consider the fo...

In the Details section of lm (linear models) in the Reference manual, it is suggested to use the weights= option for summarized data. This must be discouraged rather than encouraged. The motivation for this is as follows. With summarized data the standard errors get smaller with increasing numbers of observations. However, the standard errors in lm do not get smaller when for instance all weights are multiplied with th...

...nning. Arie On Sun, Oct 8, 2017 at 4:38 PM, Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote: > Ah, I think you are referring to this part from ?lm: > > "(including the case that there are w_i observations equal to y_i and the data have been summarized)" > > I see; indeed, I don't think this is what 'weights' should be used for (the other part before that is correct). Sorry, I misunderstood the point you were trying to make. > > Best, > Wolfgang > > -----Original Message----- > From: R-devel [mailto:r-...

Dear R Users, I am trying to write a wrapper around summarize and xYplot from Hmisc and am having trouble understanding how to pass arguments from the function I am writing to the nested functions. There must be a way, but I have not been able to figure it out. An example is below. Any advice would be greatly appreciated. Thanks, Dan # some ex...

Ah, I think you are referring to this part from ?lm: "(including the case that there are w_i observations equal to y_i and the data have been summarized)" I see; indeed, I don't think this is what 'weights' should be used for (the other part before that is correct). Sorry, I misunderstood the point you were trying to make. Best, Wolfgang -----Original Message----- From: R-devel [mailto:r-devel-bounces at r-project.org] On Be...

I was using summarize() in a data set in which one of the levels of the by variable was "". The summary statistic was consistently off by one level and the "" level was not in the output data frame. I tried to report it as a bug, but I could not log into the Hmisc bug reporting website to do so....

...d a number of dependent variables. From all of these dependent variables I would like to calculate mean, standard deviation and quantiles. With the function FUN I get all the means and stdev that I want but quantiles are only calculated for the first of the dependent variables (column 8 in the summarize command). What do I have to do differently in order to get all the quantiles that I want? Thanks, Kim sgldm2 <- read.table("E:/analysistemp/060412_test_data2.txt", header=T) attach(sgldm2) names(sgldm2) FUN <- function(x)c(Mean=mean(x,na.rm=TRUE), STDEV=sd(x,na...

Hi All I have a vector x containing 2 levels x = c(1,1,1,0,0,0,0,1,1,0,0,0,1,1,0) I would like to derive the following summarization Level Count 1 3 0 4 1 2 0 3 1 2 0 1 I have generated an inelegant solution using lags and loops but feel sure that there must be a better approach. If anyone has any thoughts I would be very grateful if you would share them. Thanks and best regards Pete -- View this message in context:...

...rybody who makes a package for the first time thinks "I can change anything!" and then runs into this same question. Has anybody written out information on how a package can override functions in R base in the R 2.14 (mandatory NAMESPACE era)? Suppose I want to alphabetize variables in a summary.data.frame, or return the standard deviation with the mean in summary output. I'm pasting in a working example below. It has new "summary.factor" method. It also has a function summarize that I might like to use in place of summary.data.frame. How would my new methods "drop o...

AFAIR, it is a little more subtle than that. If you have replication weights, then the estimates are right, it is "just" that the SE from summary.lm() are wrong. Somehow, the text should reflect this. It is of some importance when you put glm() into the mix, because you can in fact get correct results from things like y <- c(0,1) w <- c(49,51) glm(y~1, weights=w, family=binomial) -pd > On 9 Oct 2017, at 07:58 , Arie ten Cate...

...?weights? being inversely proportional to the variances); (B) or equivalently, when the elements of ?weights? are positive integers w_i, that each response y_i is the mean of w_i unit-weight observations (C) (including the case that there are w_i observations equal to y_i and the data have been summarized). (D) However, in the latter case, notice that within-group variation is not used. Therefore, the sigma estimate and residual degrees of freedom may be suboptimal; (E) in the case of replication weights, even wrong. (F) Hence, standard errors and analysis of variance tables should be trea...

Hi folks, I wrote a quick script to extract performance one-liners from Rails logs. (Didn''t want to futz with syslog for http://rails-analyzer.rubyforge.org/ .) Reads output from :info or :debug log levels (the defaults). Usage: # summarize < development.log # summarize < production.log Output looks like: (w/FULL_URL set false) 123.23.23.123 2006-05-05 10:59:42 | r 0.0--00 0% | db 0.00812 29% | 0.02726 36 r/s | 302 /some/uri 123.23.23.123 2006-05-05 10:59:43 | r 0.02059 55% | db 0.01246 33% | 0.03697 27 r/s | 200...

....51 and w <- 100 you get the same result. Arie On Mon, Oct 9, 2017 at 5:22 PM, peter dalgaard <pdalgd at gmail.com> wrote: > AFAIR, it is a little more subtle than that. > > If you have replication weights, then the estimates are right, it is "just" that the SE from summary.lm() are wrong. Somehow, the text should reflect this. > > It is of some importance when you put glm() into the mix, because you can in fact get correct results from things like > > y <- c(0,1) > w <- c(49,51) > glm(y~1, weights=w, family=binomial) > > -pd > >&g...

...? being inversely proportional to the variances); or equivalently, when the elements of ?weights? are positive integers w_i, that each response y_i is the mean of w_i unit-weight observations (including the case that there are w_i observations equal to y_i and the data have been summarized). However, in the latter case, notice that within-group variation is not used. Therefore, the sigma estimate and residual degrees of freedom may be suboptimal; in the case of replication weights, even wrong. Hence, standard errors and analysis of variance tables should be...

Hi folks, I'm trying to figure out how to get summarized data based on multiple columns. However, instead of giving summaries for every combination of categorical columns, I want it for each value of each categorical column regardless of the other columns. I could do this with three different commands, but i'm wondering if there's a more el...

I need some help summarizing complex data frames (small example below): m1_1 m2_1 m3_1 m1_2 m2_2 m3_2 i1 1 1 1 2 2 2 i1 2 1 1 2 2 2 i2 2 2 1 2 2 2 For an arbitrary number of columns (say m1 ?. m199) where the column names have variable patterns, an...