R help - Feb 2016 - Excluding "small data" from plot.

To R-help users:

I want to use ggplot two plot summary statistics on the frequency of
letters from
a page of text. My data frame has four columns:

(1) The line number [1 to 30]
(2) The letter [a to z]
(3) The frequency of the letter [assuming there is 80 letters per line]
(4) The factor 'type': bad or good (purely artificial factor)

I want to achieve the following plot:

(a) Bar plot with an x-axis to be the letters and the y-axis the sum of
30 letter frequencies from each line of each letter.
(b) Split each bar (for a letter) into two bars for 'good' and
'bad' types.
(c) Display the union of the top 8 most frequency used letters for both types
'good' and 'bad'.

By point (c) I mean: if a,e,f,h,i,t,s,r are the most frequent letter of type
'good' and a,e,f,h,i,m,l,p are the most frequent letter of type
'bad'. Then
I would like my plot to feature the letters a,e,f,h,i,t,s,r,m,l,p.

Here is my code:

# There will be 30 lines and we want to record the frequency of each letter
# on each line.

lines <- c(rep(1:30, each=26))
letter <- c(rep(letters, times=30))

# We have taken the letter frequencies from
# http://www.math.cornell.edu/~mec/2003-2004/cryptography/subs/frequencies.html

freq <- c(8.12, 1.49, 2.71, 4.32, 12.02, 2.30, 2.03, 5.92, 7.31, 0.10, 0.69,
3.98, 2.61, 6.95, 7.68, 1.82, 0.11, 6.02, 6.28, 9.10, 2.88, 1.11, 2.09, 0.17,
2.11, 0.07)
freq <- freq/100


# We assume each line contains 80 letters and change the seed for each line
# for variability.

letterfreq <- integer()
for (i in 1:30) {
    set.seed(i)
    s<-data.frame(sample(letters, size = 80, replace = TRUE, prob = freq))
    names(s) <- "ltr"
    s$ltr <- factor(s$ltr, levels = letters)
    frq<-as.data.frame(table(s))
    letterfreq <- append(letterfreq, frq$Freq)
}

ltrfreq <- data.frame(lines, letter, letterfreq)

# Add an artificial factor column _type_: good/bad. So each pair
# (week, letter) has type 'good' or 'bad' with equal
probability.
# Set the seed for reproducibility.

set.seed(999)
ltrfreq$type <-  factor(sample(c("good","bad"), size =
780, replace = TRUE,
    prob = c(0.5,0.5)))


# Here is the plot I want but this includes all 26 letters.

ggplot(ltrfreq,aes(x=factor(letter),y=letterfreq, fill=type), color=type) +
  stat_summary(fun.y=sum,position=position_dodge(),geom="bar")

Best regards,
Kieran.

Hi

There is probably better solution but I would aggregate

lll <- aggregate(ltrfreq$letterfreq, list(ltrfreq$letter, ltrfreq$type), sum)

order

ooo<-order(lll$x, decreasing=T)

split and sapply
lll<- lll[ooo,]
lll <- lapply(split(lll, lll$Group.2), head, 8)

and do.call rbind to get 8 letters for good and bad plotting.

lll<-do.call(rbind, lll)
library(ggplot2)
ggplot(lll,aes(x=Group.1,y=x, fill=Group.2), color=Group.2) +
+  stat_summary(fun.y=sum,position=position_dodge(),geom="bar")


Cheers
Petr

> -----Original Message-----
> From: R-help [mailto:r-help-bounces at r-project.org] On Behalf Of Kieran
> Sent: Wednesday, February 17, 2016 10:20 AM
> To: R-help at r-project.org
> Subject: [R] Excluding "small data" from plot.
>
> To R-help users:
>
> I want to use ggplot two plot summary statistics on the frequency of
> letters from
> a page of text. My data frame has four columns:
>
> (1) The line number [1 to 30]
> (2) The letter [a to z]
> (3) The frequency of the letter [assuming there is 80 letters per line]
> (4) The factor 'type': bad or good (purely artificial factor)
>
> I want to achieve the following plot:
>
> (a) Bar plot with an x-axis to be the letters and the y-axis the sum of
> 30 letter frequencies from each line of each letter.
> (b) Split each bar (for a letter) into two bars for 'good' and
'bad'
> types.
> (c) Display the union of the top 8 most frequency used letters for both
> types
> 'good' and 'bad'.
>
> By point (c) I mean: if a,e,f,h,i,t,s,r are the most frequent letter of
> type
> 'good' and a,e,f,h,i,m,l,p are the most frequent letter of type
'bad'.
> Then
> I would like my plot to feature the letters a,e,f,h,i,t,s,r,m,l,p.
>
> Here is my code:
>
> # There will be 30 lines and we want to record the frequency of each
> letter
> # on each line.
>
> lines <- c(rep(1:30, each=26))
> letter <- c(rep(letters, times=30))
>
> # We have taken the letter frequencies from
> # http://www.math.cornell.edu/~mec/2003-
> 2004/cryptography/subs/frequencies.html
>
> freq <- c(8.12, 1.49, 2.71, 4.32, 12.02, 2.30, 2.03, 5.92, 7.31, 0.10,
> 0.69,
> 3.98, 2.61, 6.95, 7.68, 1.82, 0.11, 6.02, 6.28, 9.10, 2.88, 1.11, 2.09,
> 0.17,
> 2.11, 0.07)
> freq <- freq/100
>
>
> # We assume each line contains 80 letters and change the seed for each
> line
> # for variability.
>
> letterfreq <- integer()
> for (i in 1:30) {
>     set.seed(i)
>     s<-data.frame(sample(letters, size = 80, replace = TRUE, prob >
freq))
>     names(s) <- "ltr"
>     s$ltr <- factor(s$ltr, levels = letters)
>     frq<-as.data.frame(table(s))
>     letterfreq <- append(letterfreq, frq$Freq)
> }
>
> ltrfreq <- data.frame(lines, letter, letterfreq)
>
> # Add an artificial factor column _type_: good/bad. So each pair
> # (week, letter) has type 'good' or 'bad' with equal
probability.
> # Set the seed for reproducibility.
>
> set.seed(999)
> ltrfreq$type <-  factor(sample(c("good","bad"), size
= 780, replace > TRUE,
>     prob = c(0.5,0.5)))
>
>
> # Here is the plot I want but this includes all 26 letters.
>
> ggplot(ltrfreq,aes(x=factor(letter),y=letterfreq, fill=type),
> color=type) +
>   stat_summary(fun.y=sum,position=position_dodge(),geom="bar")
>
> Best regards,
> Kieran.
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.
________________________________
Tento e-mail a jak?koliv k n?mu p?ipojen? dokumenty jsou d?v?rn? a jsou ur?eny
pouze jeho adres?t?m.
Jestli?e jste obdr?el(a) tento e-mail omylem, informujte laskav? neprodlen? jeho
odes?latele. Obsah tohoto emailu i s p??lohami a jeho kopie vyma?te ze sv?ho
syst?mu.
Nejste-li zam??len?m adres?tem tohoto emailu, nejste opr?vn?ni tento email
jakkoliv u??vat, roz?i?ovat, kop?rovat ?i zve?ej?ovat.
Odes?latel e-mailu neodpov?d? za eventu?ln? ?kodu zp?sobenou modifikacemi ?i
zpo?d?n?m p?enosu e-mailu.

V p??pad?, ?e je tento e-mail sou??st? obchodn?ho jedn?n?:
- vyhrazuje si odes?latel pr?vo ukon?it kdykoliv jedn?n? o uzav?en? smlouvy, a
to z jak?hokoliv d?vodu i bez uveden? d?vodu.
- a obsahuje-li nab?dku, je adres?t opr?vn?n nab?dku bezodkladn? p?ijmout;
Odes?latel tohoto e-mailu (nab?dky) vylu?uje p?ijet? nab?dky ze strany p??jemce
s dodatkem ?i odchylkou.
- trv? odes?latel na tom, ?e p??slu?n? smlouva je uzav?ena teprve v?slovn?m
dosa?en?m shody na v?ech jej?ch n?le?itostech.
- odes?latel tohoto emailu informuje, ?e nen? opr?vn?n uzav?rat za spole?nost
??dn? smlouvy s v?jimkou p??pad?, kdy k tomu byl p?semn? zmocn?n nebo p?semn?
pov??en a takov? pov??en? nebo pln? moc byly adres?tovi tohoto emailu p??padn?
osob?, kterou adres?t zastupuje, p?edlo?eny nebo jejich existence je adres?tovi
?i osob? j?m zastoupen? zn?m?.

This e-mail and any documents attached to it may be confidential and are
intended only for its intended recipients.
If you received this e-mail by mistake, please immediately inform its sender.
Delete the contents of this e-mail with all attachments and its copies from your
system.
If you are not the intended recipient of this e-mail, you are not authorized to
use, disseminate, copy or disclose this e-mail in any manner.
The sender of this e-mail shall not be liable for any possible damage caused by
modifications of the e-mail or by delay with transfer of the email.

In case that this e-mail forms part of business dealings:
- the sender reserves the right to end negotiations about entering into a
contract in any time, for any reason, and without stating any reasoning.
- if the e-mail contains an offer, the recipient is entitled to immediately
accept such offer; The sender of this e-mail (offer) excludes any acceptance of
the offer on the part of the recipient containing any amendment or variation.
- the sender insists on that the respective contract is concluded only upon an
express mutual agreement on all its aspects.
- the sender of this e-mail informs that he/she is not authorized to enter into
any contracts on behalf of the company except for cases in which he/she is
expressly authorized to do so in writing, and such authorization or power of
attorney is submitted to the recipient or the person represented by the
recipient, or the existence of such authorization is known to the recipient of
the person represented by the recipient.

Hi

Please cc your mails to r help. Users are around the world and you could have
response oslo from others.

You need to put NA in a missing good or bad letter. I would probably do merging
instead of do.call but there may be other options.

Something like that.
> l1<-sample(letters[1:10], 5)
> l2<-sample(letters[1:10], 5)
> x1<-data.frame(l1, x=rnorm(5))
> x2<-data.frame(l2)
> x1
  l1          x
1  d  0.7948186
2  b  1.3076922
3  j  1.8305538
4  i -0.1934532
5  e -1.5182885
> x2
  l2
1  f
2  j
3  h
4  a
5  i
> merge(x1, x2, by.x="l1",by.y="l2", all=TRUE)
  l1          x
1  b  1.3076922
2  d  0.7948186
3  e -1.5182885
4  i -0.1934532
5  j  1.8305538
6  a         NA
7  f         NA
8  h         NA

Or you can use some clever combination of intersect and setdiff

?intersect

to find which letters are missing in each group.

Cheers
Petr

> -----Original Message-----
> From: Kieran [mailto:kroberts012 at gmail.com]
> Sent: Wednesday, February 17, 2016 7:37 PM
> To: PIKAL Petr
> Subject: Re: [R] Excluding "small data" from plot.
>
> Hi Petr,
>
> Thanks for you reply. This almost does what I need but there is one
> problem.
>
> Before running
>
> lll <- do.call(rbind, lll),
>
> lll is a list with two data sets: the top 8 most frequently used
> letters of bad type [e,n,a,t,o,i,h,r] and the top 8 mostly frequently
> used letters of good type [e,t,i,a,s,o,r,h]. These two groups of
> letters are almost the same (as one would expect). They differ in two
> places: n (is bad) and s (is good).
>
> After running the do.call function with rbind I get a union of these
> rows. But then when I run
>
> ggplot(lll,aes(x=Group.1,y=x, fill=Group.2), color=Group.2) +
>     stat_summary(fun.y=sum,position=position_dodge(),geom="bar")
>
> For n on the x-axis I get two bars of the same colour red (=bad) For s
> on the x-axis I get two bars of the same colour blue (=good)
>
> But I really want is the 'good' value for n next to the bad value
for
> 'n'.  And the same for 's'. This task seems more
complicated.
>
> Best,
> Kieran.
>
> On 17 February 2016 at 13:49, PIKAL Petr <petr.pikal at precheza.cz>
> wrote:
> > Hi
> >
> > There is probably better solution but I would aggregate
> >
> > lll <- aggregate(ltrfreq$letterfreq, list(ltrfreq$letter,
> > ltrfreq$type), sum)
> >
> > order
> >
> > ooo<-order(lll$x, decreasing=T)
> >
> > split and sapply
> > lll<- lll[ooo,]
> > lll <- lapply(split(lll, lll$Group.2), head, 8)
> >
> > and do.call rbind to get 8 letters for good and bad plotting.
> >
> > lll<-do.call(rbind, lll)
> > library(ggplot2)
> > ggplot(lll,aes(x=Group.1,y=x, fill=Group.2), color=Group.2) +
> > + 
stat_summary(fun.y=sum,position=position_dodge(),geom="bar")
> >
> >
> > Cheers
> > Petr
> >
> >
> >> -----Original Message-----
> >> From: R-help [mailto:r-help-bounces at r-project.org] On Behalf Of
> >> Kieran
> >> Sent: Wednesday, February 17, 2016 10:20 AM
> >> To: R-help at r-project.org
> >> Subject: [R] Excluding "small data" from plot.
> >>
> >> To R-help users:
> >>
> >> I want to use ggplot two plot summary statistics on the frequency
of
> >> letters from a page of text. My data frame has four columns:
> >>
> >> (1) The line number [1 to 30]
> >> (2) The letter [a to z]
> >> (3) The frequency of the letter [assuming there is 80 letters per
> >> line]
> >> (4) The factor 'type': bad or good (purely artificial
factor)
> >>
> >> I want to achieve the following plot:
> >>
> >> (a) Bar plot with an x-axis to be the letters and the y-axis the
sum
> >> of 30 letter frequencies from each line of each letter.
> >> (b) Split each bar (for a letter) into two bars for 'good'
and 'bad'
> >> types.
> >> (c) Display the union of the top 8 most frequency used letters for
> >> both types 'good' and 'bad'.
> >>
> >> By point (c) I mean: if a,e,f,h,i,t,s,r are the most frequent
letter
> >> of type 'good' and a,e,f,h,i,m,l,p are the most frequent
letter of
> >> type 'bad'.
> >> Then
> >> I would like my plot to feature the letters a,e,f,h,i,t,s,r,m,l,p.
> >>
> >> Here is my code:
> >>
> >> # There will be 30 lines and we want to record the frequency of
each
> >> letter # on each line.
> >>
> >> lines <- c(rep(1:30, each=26))
> >> letter <- c(rep(letters, times=30))
> >>
> >> # We have taken the letter frequencies from #
> >> http://www.math.cornell.edu/~mec/2003-
> >> 2004/cryptography/subs/frequencies.html
> >>
> >> freq <- c(8.12, 1.49, 2.71, 4.32, 12.02, 2.30, 2.03, 5.92,
7.31,
> >> 0.10, 0.69, 3.98, 2.61, 6.95, 7.68, 1.82, 0.11, 6.02, 6.28, 9.10,
> >> 2.88, 1.11, 2.09, 0.17, 2.11, 0.07) freq <- freq/100
> >>
> >>
> >> # We assume each line contains 80 letters and change the seed for
> >> each line # for variability.
> >>
> >> letterfreq <- integer()
> >> for (i in 1:30) {
> >>     set.seed(i)
> >>     s<-data.frame(sample(letters, size = 80, replace = TRUE,
prob > >> freq))
> >>     names(s) <- "ltr"
> >>     s$ltr <- factor(s$ltr, levels = letters)
> >>     frq<-as.data.frame(table(s))
> >>     letterfreq <- append(letterfreq, frq$Freq) }
> >>
> >> ltrfreq <- data.frame(lines, letter, letterfreq)
> >>
> >> # Add an artificial factor column _type_: good/bad. So each pair #
> >> (week, letter) has type 'good' or 'bad' with equal
probability.
> >> # Set the seed for reproducibility.
> >>
> >> set.seed(999)
> >> ltrfreq$type <- 
factor(sample(c("good","bad"), size = 780, replace
> > >> TRUE,
> >>     prob = c(0.5,0.5)))
> >>
> >>
> >> # Here is the plot I want but this includes all 26 letters.
> >>
> >> ggplot(ltrfreq,aes(x=factor(letter),y=letterfreq, fill=type),
> >> color=type) +
> >>  
stat_summary(fun.y=sum,position=position_dodge(),geom="bar")
> >>
> >> Best regards,
> >> Kieran.
> >>
> >> ______________________________________________
> >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more,
see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide http://www.R-project.org/posting-
> >> guide.html and provide commented, minimal, self-contained,
> >> reproducible code.
> >
> > ________________________________
> > Tento e-mail a jak?koliv k n?mu p?ipojen? dokumenty jsou d?v?rn? a
> jsou ur?eny pouze jeho adres?t?m.
> > Jestli?e jste obdr?el(a) tento e-mail omylem, informujte laskav?
> neprodlen? jeho odes?latele. Obsah tohoto emailu i s p??lohami a jeho
> kopie vyma?te ze sv?ho syst?mu.
> > Nejste-li zam??len?m adres?tem tohoto emailu, nejste opr?vn?ni tento
> email jakkoliv u??vat, roz?i?ovat, kop?rovat ?i zve?ej?ovat.
> > Odes?latel e-mailu neodpov?d? za eventu?ln? ?kodu zp?sobenou
> modifikacemi ?i zpo?d?n?m p?enosu e-mailu.
> >
> > V p??pad?, ?e je tento e-mail sou??st? obchodn?ho jedn?n?:
> > - vyhrazuje si odes?latel pr?vo ukon?it kdykoliv jedn?n? o uzav?en?
> smlouvy, a to z jak?hokoliv d?vodu i bez uveden? d?vodu.
> > - a obsahuje-li nab?dku, je adres?t opr?vn?n nab?dku bezodkladn?
> p?ijmout; Odes?latel tohoto e-mailu (nab?dky) vylu?uje p?ijet? nab?dky
> ze strany p??jemce s dodatkem ?i odchylkou.
> > - trv? odes?latel na tom, ?e p??slu?n? smlouva je uzav?ena teprve
> v?slovn?m dosa?en?m shody na v?ech jej?ch n?le?itostech.
> > - odes?latel tohoto emailu informuje, ?e nen? opr?vn?n uzav?rat za
> spole?nost ??dn? smlouvy s v?jimkou p??pad?, kdy k tomu byl p?semn?
> zmocn?n nebo p?semn? pov??en a takov? pov??en? nebo pln? moc byly
> adres?tovi tohoto emailu p??padn? osob?, kterou adres?t zastupuje,
> p?edlo?eny nebo jejich existence je adres?tovi ?i osob? j?m zastoupen?
> zn?m?.
> >
> > This e-mail and any documents attached to it may be confidential and
> are intended only for its intended recipients.
> > If you received this e-mail by mistake, please immediately inform its
> sender. Delete the contents of this e-mail with all attachments and its
> copies from your system.
> > If you are not the intended recipient of this e-mail, you are not
> authorized to use, disseminate, copy or disclose this e-mail in any
> manner.
> > The sender of this e-mail shall not be liable for any possible damage
> caused by modifications of the e-mail or by delay with transfer of the
> email.
> >
> > In case that this e-mail forms part of business dealings:
> > - the sender reserves the right to end negotiations about entering
> into a contract in any time, for any reason, and without stating any
> reasoning.
> > - if the e-mail contains an offer, the recipient is entitled to
> immediately accept such offer; The sender of this e-mail (offer)
> excludes any acceptance of the offer on the part of the recipient
> containing any amendment or variation.
> > - the sender insists on that the respective contract is concluded
> only upon an express mutual agreement on all its aspects.
> > - the sender of this e-mail informs that he/she is not authorized to
> enter into any contracts on behalf of the company except for cases in
> which he/she is expressly authorized to do so in writing, and such
> authorization or power of attorney is submitted to the recipient or the
> person represented by the recipient, or the existence of such
> authorization is known to the recipient of the person represented by
> the recipient.
________________________________
Tento e-mail a jak?koliv k n?mu p?ipojen? dokumenty jsou d?v?rn? a jsou ur?eny
pouze jeho adres?t?m.
Jestli?e jste obdr?el(a) tento e-mail omylem, informujte laskav? neprodlen? jeho
odes?latele. Obsah tohoto emailu i s p??lohami a jeho kopie vyma?te ze sv?ho
syst?mu.
Nejste-li zam??len?m adres?tem tohoto emailu, nejste opr?vn?ni tento email
jakkoliv u??vat, roz?i?ovat, kop?rovat ?i zve?ej?ovat.
Odes?latel e-mailu neodpov?d? za eventu?ln? ?kodu zp?sobenou modifikacemi ?i
zpo?d?n?m p?enosu e-mailu.

V p??pad?, ?e je tento e-mail sou??st? obchodn?ho jedn?n?:
- vyhrazuje si odes?latel pr?vo ukon?it kdykoliv jedn?n? o uzav?en? smlouvy, a
to z jak?hokoliv d?vodu i bez uveden? d?vodu.
- a obsahuje-li nab?dku, je adres?t opr?vn?n nab?dku bezodkladn? p?ijmout;
Odes?latel tohoto e-mailu (nab?dky) vylu?uje p?ijet? nab?dky ze strany p??jemce
s dodatkem ?i odchylkou.
- trv? odes?latel na tom, ?e p??slu?n? smlouva je uzav?ena teprve v?slovn?m
dosa?en?m shody na v?ech jej?ch n?le?itostech.
- odes?latel tohoto emailu informuje, ?e nen? opr?vn?n uzav?rat za spole?nost
??dn? smlouvy s v?jimkou p??pad?, kdy k tomu byl p?semn? zmocn?n nebo p?semn?
pov??en a takov? pov??en? nebo pln? moc byly adres?tovi tohoto emailu p??padn?
osob?, kterou adres?t zastupuje, p?edlo?eny nebo jejich existence je adres?tovi
?i osob? j?m zastoupen? zn?m?.

This e-mail and any documents attached to it may be confidential and are
intended only for its intended recipients.
If you received this e-mail by mistake, please immediately inform its sender.
Delete the contents of this e-mail with all attachments and its copies from your
system.
If you are not the intended recipient of this e-mail, you are not authorized to
use, disseminate, copy or disclose this e-mail in any manner.
The sender of this e-mail shall not be liable for any possible damage caused by
modifications of the e-mail or by delay with transfer of the email.

In case that this e-mail forms part of business dealings:
- the sender reserves the right to end negotiations about entering into a
contract in any time, for any reason, and without stating any reasoning.
- if the e-mail contains an offer, the recipient is entitled to immediately
accept such offer; The sender of this e-mail (offer) excludes any acceptance of
the offer on the part of the recipient containing any amendment or variation.
- the sender insists on that the respective contract is concluded only upon an
express mutual agreement on all its aspects.
- the sender of this e-mail informs that he/she is not authorized to enter into
any contracts on behalf of the company except for cases in which he/she is
expressly authorized to do so in writing, and such authorization or power of
attorney is submitted to the recipient or the person represented by the
recipient, or the existence of such authorization is known to the recipient of
the person represented by the recipient.