Hi, I am using Spark and the Sparklyr library in R. I have a file with several lines. For example A B C awer.ttp.net Code 554 abcd.ttp.net Code 747 asdf.ttp.net Part 554 xyz.ttp.net Part 747 I want to split just column A of the table and I want a new row added to the table D, with values awe, abcd, asdf, and xyz. I am trying to use a command in the sparkly library to do that. If that?s not possible, can you please show me another way to do it. Thanks, Amrith [[alternative HTML version deleted]]
> On Jun 22, 2017, at 11:22 AM, Amrith Deepak <adeepak at apple.com> wrote: > > Hi, > > I am using Spark and the Sparklyr library in R. > > I have a file with several lines. For example > > A B C > awer.ttp.net Code 554 > abcd.ttp.net Code 747 > asdf.ttp.net Part 554 > xyz.ttp.net Part 747 > I want to split just column A of the table and I want a new row added to the table D, with values awe, abcd, asdf, and xyz. I am trying to use a command in the sparkly library to do that. If that?s not possible, can you please show me another way to do it. >Something along lines of: dfrm$D <- sapply( strsplit( dfrm$A, "\\.") , "[[", 1)> Thanks, > Amrith > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.David Winsemius Alameda, CA, USA
This function won?t work with objects in spark as you can?t do a dfda$a in spark as it?s not stored as a local variable. Thanks, Amrith> On Jun 22, 2017, at 4:15 PM, David Winsemius <dwinsemius at comcast.net> wrote: > > >> On Jun 22, 2017, at 11:22 AM, Amrith Deepak <adeepak at apple.com> wrote: >> >> Hi, >> >> I am using Spark and the Sparklyr library in R. >> >> I have a file with several lines. For example >> >> A B C >> awer.ttp.net Code 554 >> abcd.ttp.net Code 747 >> asdf.ttp.net Part 554 >> xyz.ttp.net Part 747 >> I want to split just column A of the table and I want a new row added to the table D, with values awe, abcd, asdf, and xyz. I am trying to use a command in the sparkly library to do that. If that?s not possible, can you please show me another way to do it. >> > > Something along lines of: > > dfrm$D <- sapply( strsplit( dfrm$A, "\\.") , "[[", 1) > > > >> Thanks, >> Amrith >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > David Winsemius > Alameda, CA, USA > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
Rows are horizontal, columns are vertical. You really need to spend some time with an R tutorial. dta <- read.table( "yourfile", header=TRUE, as.is=TRUE ) dta2 <- dta dta2$D <- c( "awe", "abcd", "asdf", "xyz" ) dta2 <- dta2[ , c( "A", "D" ) ] -- Sent from my phone. Please excuse my brevity. On June 22, 2017 11:22:57 AM PDT, Amrith Deepak <adeepak at apple.com> wrote:>Hi, > >I am using Spark and the Sparklyr library in R. > >I have a file with several lines. For example > >A B C >awer.ttp.net Code 554 >abcd.ttp.net Code 747 >asdf.ttp.net Part 554 >xyz.ttp.net Part 747 >I want to split just column A of the table and I want a new row added >to the table D, with values awe, abcd, asdf, and xyz. I am trying to >use a command in the sparkly library to do that. If that?s not >possible, can you please show me another way to do it. > > >Thanks, >Amrith > [[alternative HTML version deleted]] > >______________________________________________ >R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide >http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code.
Sorry, I don?t think you understand my problem. dta = spark_read_csv(sc,?data_tbl?,?extension/file",delimiter = "|") dta is just a pointer to spark where the data is stored. I am using sparklyr to run this on spark. I?m not running it locally so I can?t use the $. I know how to do this locally in R. Thanks, Amrith> On Jun 22, 2017, at 4:52 PM, Jeff Newmiller <jdnewmil at dcn.davis.ca.us> wrote: > > Rows are horizontal, columns are vertical. > > You really need to spend some time with an R tutorial. > > dta <- read.table( "yourfile", header=TRUE, as.is=TRUE ) > dta2 <- dta > dta2$D <- c( "awe", "abcd", "asdf", "xyz" ) > dta2 <- dta2[ , c( "A", "D" ) ] > -- > Sent from my phone. Please excuse my brevity. > > On June 22, 2017 11:22:57 AM PDT, Amrith Deepak <adeepak at apple.com> wrote: >> Hi, >> >> I am using Spark and the Sparklyr library in R. >> >> I have a file with several lines. For example >> >> A B C >> awer.ttp.net Code 554 >> abcd.ttp.net Code 747 >> asdf.ttp.net Part 554 >> xyz.ttp.net Part 747 >> I want to split just column A of the table and I want a new row added >> to the table D, with values awe, abcd, asdf, and xyz. I am trying to >> use a command in the sparkly library to do that. If that?s not >> possible, can you please show me another way to do it. >> >> >> Thanks, >> Amrith >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.[[alternative HTML version deleted]]