trece por ciento
2010-Feb-01 16:40 UTC
[R] Import fixed-format ascii file with mixed record types
I need to import several ascii files in fixed format with two different record types. The data comes from European Labor Force Surveys, wich is a household survey. The first record type is for people over 16 years, and the second much sorter is for people aged 15 or less (this record has a filler with several blanks to get the same record length). The files tipically have 160000 records, with 176 characters per record, the data is numeric, corresponding to 102 variables, mostly integers (seven variables have two decimals). My opertating system is Windows XP. My questions: 1. Wich do you think is the best way to import the files into R? 2. Could you give me any references or examples? Thanking you in advance, Hug [[alternative HTML version deleted]]
David Winsemius
2010-Feb-01 18:01 UTC
[R] Import fixed-format ascii file with mixed record types
On Feb 1, 2010, at 11:40 AM, trece por ciento wrote:> I need to import several ascii files in fixed format with two > different record types. The data comes from European Labor Force > Surveys, wich is a household survey. The first record type is for > people over 16 years, and the second much sorter is for people aged > 15 or less (this record has a filler with several blanks to get the > same record length). > The files tipically have 160000 records, with 176 characters per > record, the data is numeric, corresponding to 102 variables, mostly > integers (seven variables have two decimals). My opertating system > is Windows XP. > My questions: > 1. Wich do you think is the best way to import the files into R??read.fwf> 2. Could you give me any references or examples?There are examples in the help page.> Thanking you in advance, > Hug > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.David Winsemius, MD Heritage Laboratories West Hartford, CT
trece por ciento
2010-Feb-01 19:33 UTC
[R] Import fixed-format ascii file with mixed record types
Thanks David, but can read.fwf cope with different record types? For example, if recordtype is the 4th character, I could have: 011125678 ---> This is record Type 1 011136779 ---> This is record Type 1 011124943 ---> This is record Type 1 011286711 ---> This is record Type 2 011234872 ---> This is record Type 2 011135628 ---> This is record Type 1 So, how can I tell read.fwf to take the correct type into account? Thanks again, Hug --- On Mon, 2/1/10, David Winsemius <dwinsemius at comcast.net> wrote: From: David Winsemius <dwinsemius at comcast.net> Subject: Re: [R] Import fixed-format ascii file with mixed record types To: "trece por ciento" <el13porciento at yahoo.com> Cc: r-help at r-project.org Date: Monday, February 1, 2010, 12:01 PM On Feb 1, 2010, at 11:40 AM, trece por ciento wrote:> I need to import several ascii files in fixed format with two different record types. The data comes from European Labor Force Surveys, wich is a household survey. The first record type is for people over 16 years, and the second much sorter is for people aged 15 or less (this record has a filler with several blanks to get the same record length). > The files tipically have 160000 records, with 176 characters per record, the data is numeric, corresponding to 102 variables, mostly integers (seven variables have two decimals). My opertating system is Windows XP. > My questions: > 1. Wich do you think is the best way to import the files into R??read.fwf> 2. Could you give me any references or examples?There are examples in the help page.> Thanking you in advance, > Hug > > > > > ??? [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.David Winsemius, MD Heritage Laboratories West Hartford, CT
David Winsemius
2010-Feb-01 19:54 UTC
[R] Import fixed-format ascii file with mixed record types
On Feb 1, 2010, at 2:33 PM, trece por ciento wrote:> Thanks David, but can read.fwf cope with different record types?I do not see any facility for that goal.> For example, if recordtype is the 4th character, I could have: > > 011125678 ---> This is record Type 1 > 011136779 ---> This is record Type 1 > 011124943 ---> This is record Type 1 > 011286711 ---> This is record Type 2 > 011234872 ---> This is record Type 2 > 011135628 ---> This is record Type 1 > > So, how can I tell read.fwf to take the correct type into account?It would be easier on the readers of the list if you would offer a more complete description of the problem. Problems which are unfolded bit by bit are rather annoying to some. Is there a website or document that describes this data resource? Perhaps an alternate strategy that used a database interface would be more appropriate? -- David.> Thanks again, > Hug > > --- On Mon, 2/1/10, David Winsemius <dwinsemius at comcast.net> wrote: > > From: David Winsemius <dwinsemius at comcast.net> > Subject: Re: [R] Import fixed-format ascii file with mixed record > types > To: "trece por ciento" <el13porciento at yahoo.com> > Cc: r-help at r-project.org > Date: Monday, February 1, 2010, 12:01 PM > > > On Feb 1, 2010, at 11:40 AM, trece por ciento wrote: > >> I need to import several ascii files in fixed format with two >> different record types. The data comes from European Labor Force >> Surveys, wich is a household survey. The first record type is for >> people over 16 years, and the second much sorter is for people aged >> 15 or less (this record has a filler with several blanks to get the >> same record length). >> The files tipically have 160000 records, with 176 characters per >> record, the data is numeric, corresponding to 102 variables, mostly >> integers (seven variables have two decimals). My opertating system >> is Windows XP. >> My questions: >> 1. Wich do you think is the best way to import the files into R? > > > ?read.fwf > >> 2. Could you give me any references or examples? > > There are examples in the help page. > >> Thanking you in advance, >> Hug >> >> >> >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > David Winsemius, MD > Heritage Laboratories > West Hartford, CT > > > > >David Winsemius, MD Heritage Laboratories West Hartford, CT
David Winsemius
2010-Feb-01 20:23 UTC
[R] Import fixed-format ascii file with mixed record types
On Feb 1, 2010, at 2:33 PM, trece por ciento wrote:> Thanks David, but can read.fwf cope with different record types? > For example, if recordtype is the 4th character, I could have: > > 011125678 ---> This is record Type 1 > 011136779 ---> This is record Type 1 > 011124943 ---> This is record Type 1 > 011286711 ---> This is record Type 2 > 011234872 ---> This is record Type 2 > 011135628 ---> This is record Type 1 > > So, how can I tell read.fwf to take the correct type into account?You may need to separate the line-types first. If the numbers of lines are not too large then this would exemplify a strategy: > txt <- "011125678 + 011136779 + 011124943 + 011286711 + 011234872 + 011135628" > substr(readLines(textConnection(txt)), 4,4) [1] "1" "1" "1" "2" "2" "1" > file1 <- readLines(textConnection(txt)) [substr(readLines(textConnection(txt)), 4,4) == "1"] > file2 <- readLines(textConnection(txt)) [substr(readLines(textConnection(txt)), 4,4) == "2"] > file1 [1] "011125678" "011136779" "011124943" "011135628" > file2 [1] "011286711" "011234872" Then these text objects could be processed with read.fwf(textConnection(file1) and the same for file2. -- David.> Thanks again, > Hug > > --- On Mon, 2/1/10, David Winsemius <dwinsemius at comcast.net> wrote: > > From: David Winsemius <dwinsemius at comcast.net> > Subject: Re: [R] Import fixed-format ascii file with mixed record > types > To: "trece por ciento" <el13porciento at yahoo.com> > Cc: r-help at r-project.org > Date: Monday, February 1, 2010, 12:01 PM > > > On Feb 1, 2010, at 11:40 AM, trece por ciento wrote: > >> I need to import several ascii files in fixed format with two >> different record types. The data comes from European Labor Force >> Surveys, wich is a household survey. The first record type is for >> people over 16 years, and the second much sorter is for people aged >> 15 or less (this record has a filler with several blanks to get the >> same record length). >> The files tipically have 160000 records, with 176 characters per >> record, the data is numeric, corresponding to 102 variables, mostly >> integers (seven variables have two decimals). My opertating system >> is Windows XP. >> My questions: >> 1. Wich do you think is the best way to import the files into R? > > > ?read.fwf > >> 2. Could you give me any references or examples? > > There are examples in the help page. > >> Thanking you in advance, >> Hug >> >> >> >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > David Winsemius, MD > Heritage Laboratories > West Hartford, CT > > > > >David Winsemius, MD Heritage Laboratories West Hartford, CT
trece por ciento
2010-Feb-01 20:25 UTC
[R] Import fixed-format ascii file with mixed record types
David, In this link there is an excel file with the data dictionary ftp://www.ine.es/temas/epa/disereg_epa0509.zip If I understand the second part of the answer: do you mean to process the files, for example with MS Access, and then importing the result? (any examples?) Regards, Hug --- On Mon, 2/1/10, David Winsemius <dwinsemius at comcast.net> wrote:> From: David Winsemius <dwinsemius at comcast.net> > Subject: Re: [R] Import fixed-format ascii file with mixed record types > To: "trece por ciento" <el13porciento at yahoo.com> > Cc: r-help at r-project.org > Date: Monday, February 1, 2010, 1:54 PM > > On Feb 1, 2010, at 2:33 PM, trece por ciento wrote: > > > Thanks David, but can read.fwf cope with different > record types? > > I do not see any facility for that goal. > > > For example, if recordtype is the 4th character, I > could have: > > > > 011125678 ---> This is record Type 1 > > 011136779 ---> This is record Type 1 > > 011124943 ---> This is record Type 1 > > 011286711 ---> This is record Type 2 > > 011234872 ---> This is record Type 2 > > 011135628 ---> This is record Type 1 > > > > So, how can I tell read.fwf to take the correct type > into account? > > It would be easier on the readers of the list if you would > offer a more complete description of the problem. Problems > which are unfolded bit by bit are rather annoying to some. > Is there a website or document that describes this data > resource? Perhaps an alternate strategy that used a database > interface would be more appropriate? > > --David. > > > > Thanks again, > > Hug > > > > --- On Mon, 2/1/10, David Winsemius <dwinsemius at comcast.net> > wrote: > > > > From: David Winsemius <dwinsemius at comcast.net> > > Subject: Re: [R] Import fixed-format ascii file with > mixed record types > > To: "trece por ciento" <el13porciento at yahoo.com> > > Cc: r-help at r-project.org > > Date: Monday, February 1, 2010, 12:01 PM > > > > > > On Feb 1, 2010, at 11:40 AM, trece por ciento wrote: > > > >> I need to import several ascii files in fixed > format with two different record types. The data comes from > European Labor Force Surveys, wich is a household survey. > The first record type is for people over 16 years, and the > second much sorter is for people aged 15 or less (this > record has a filler with several blanks to get the same > record length). > >> The files tipically have 160000 records, with 176 > characters per record, the data is numeric, corresponding to > 102 variables, mostly integers (seven variables have two > decimals). My opertating system is Windows XP. > >> My questions: > >> 1. Wich do you think is the best way to import the > files into R? > > > > > > ?read.fwf > > > >> 2. Could you give me any references or examples? > > > > There are examples in the help page. > > > >> Thanking you in advance, > >> Hug > >> > >> > >> > >> > >>? ???[[alternative HTML version > deleted]] > >> > >> ______________________________________________ > >> R-help at r-project.org > mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-help > >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > >> and provide commented, minimal, self-contained, > reproducible code. > > > > David Winsemius, MD > > Heritage Laboratories > > West Hartford, CT > > > > > > > > > > > > David Winsemius, MD > Heritage Laboratories > West Hartford, CT > >
Jim Holtman
2010-Feb-01 22:43 UTC
[R] Import fixed-format ascii file with mixed record types
you probably need to split your file into several files according to the format. you can use something like perl or do it within R by doing readline and then separating the lines. you can use grep, split and textConnection to do most of the work. What is the problem you are trying to solve? Sent from my iPhone. On Feb 1, 2010, at 14:33, trece por ciento <el13porciento at yahoo.com> wrote:> Thanks David, but can read.fwf cope with different record types? > For example, if recordtype is the 4th character, I could have: > > 011125678 ---> This is record Type 1 > 011136779 ---> This is record Type 1 > 011124943 ---> This is record Type 1 > 011286711 ---> This is record Type 2 > 011234872 ---> This is record Type 2 > 011135628 ---> This is record Type 1 > > So, how can I tell read.fwf to take the correct type into account? > Thanks again, > Hug > > --- On Mon, 2/1/10, David Winsemius <dwinsemius at comcast.net> wrote: > > From: David Winsemius <dwinsemius at comcast.net> > Subject: Re: [R] Import fixed-format ascii file with mixed record > types > To: "trece por ciento" <el13porciento at yahoo.com> > Cc: r-help at r-project.org > Date: Monday, February 1, 2010, 12:01 PM > > > On Feb 1, 2010, at 11:40 AM, trece por ciento wrote: > >> I need to import several ascii files in fixed format with two >> different record types. The data comes from European Labor Force >> Surveys, wich is a household survey. The first record type is for >> people over 16 years, and the second much sorter is for people aged >> 15 or less (this record has a filler with several blanks to get the >> same record length). >> The files tipically have 160000 records, with 176 characters per >> record, the data is numeric, corresponding to 102 variables, mostly >> integers (seven variables have two decimals). My opertating system >> is Windows XP. >> My questions: >> 1. Wich do you think is the best way to import the files into R? > > > ?read.fwf > >> 2. Could you give me any references or examples? > > There are examples in the help page. > >> Thanking you in advance, >> Hug >> >> >> >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > David Winsemius, MD > Heritage Laboratories > West Hartford, CT > > > > > > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.