Dear Rui, THanks for your reply...The point is the loop is a scraping code, and in your examples you have assumed that the body acts on i, the loop variable. Can you adapt your code to JUST PRINT the loop variable i ? By the by, I think I have stumbled upon the answer: The lapply() caches the result, and prints the output of the function in question immediately after printing the final i. The i's get printed serially, as the function progresses....> lapply(1:4,function(x){print(x);Sys.sleep(x^2);x^2})[1] 1 [1] 2 [1] 3 [1] 4 [[1]] [1] 1 [[2]] [1] 4 [[3]] [1] 9 [[4]] [1] 16 Here x^2 's print only after 4 is printed on the console.... tHanks anyways for your reply.... THanking you, Yours sincerely, AKSHAY M KULKARNI ________________________________ From: Rui Barradas <ruipbarradas at sapo.pt> Sent: Tuesday, November 8, 2022 12:24 AM To: akshay kulkarni <akshay_e4 at hotmail.com>; R help Mailing list <r-help at r-project.org> Subject: Re: [R] print and lapply.... ?s 18:33 de 07/11/2022, akshay kulkarni escreveu:> Dear Rui, > Actually, I am replacing a big for loop by the lapply() function, and report the progress: > > lapply(TP, function(i) { BODY; print(i)}) > > Can you please adjust your solution in this light? > > THanking you, > Yours sincerely, > AKSHAY M KULKARNI > ________________________________ > From: Rui Barradas <ruipbarradas at sapo.pt> > Sent: Monday, November 7, 2022 11:59 PM > To: akshay kulkarni <akshay_e4 at hotmail.com>; R help Mailing list <r-help at r-project.org> > Subject: Re: [R] print and lapply.... > > ?s 17:17 de 07/11/2022, akshay kulkarni escreveu: >> Dear members, >> I have the following code and output: >> >>> TP <- 1:4 >>> lapply(TP,function(x){print(x);x^2}) >> [1] 1 >> [1] 2 >> [1] 3 >> [1] 4 >> [[1]] >> [1] 1 >> >> [[2]] >> [1] 4 >> >> [[3]] >> [1] 9 >> >> [[4]] >> [1] 16 >> >> How do I make the print function output x along with x^2, i.e not at the beginning but before each of x^2? >> >> Many thanks in advance.... >> >> THanking you, >> Yours sincerely >> AKSHAY M KULKARNI >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > Hello, > > Here are two options, with ?cat and with ?message. > > > TP <- 1:4 > lapply(TP, function(x){ > cat("x =", x, "x^2 =", x^2, "\n") > }) > > lapply(TP, function(x){ > msg <- paste("x =", x, "x^2 =", x^2) > message(msg) > }) > > > Hope this helps, > > Rui Barradas > > >Hello, What do you want the lapply loop to return? If you have a BODY doing computations, do you want the lapply to return those values and report the progress? I have chosen cat or message over print because - cat returns invisible(NULL), - message returns invisible() - print returns a value, what it prints. Can you adapt the code below to your use case? TP <- 1:4 lapply(TP, function(x, verbose = TRUE){ # BODY y <- rnorm(100, mean = x) # show progress if(verbose) cat("x =", x, "x^2 =", x^2, "\n") #return value c(x = x, mean = mean(y)) }) lapply(TP, function(x, verbose = TRUE){ # BODY y <- rnorm(100, mean = x) # show progress if(verbose) { msg <- paste("x =", x, "x^2 =", x^2) message(msg) } #return value c(x = x, mean = mean(y)) }) Hope this helps, Rui Barradas [[alternative HTML version deleted]]
" The lapply() caches the result, and prints the output of the function in question immediately after printing the final i. " I don't believe you understand how lists or lapply works. You seem to be trying to intuit from empirical behavior. Bad idea, if so. You need to read the docs or spend time with suitable tutorials. Consider: lapply produces a **list,** each component of which is the result of applying a function, FUN, to the components of its first argument (here 1:4), which is itself a list. Results are **not** cached in the sense you seem to think they are. The list object that is produced by the function is **only** printed if you ask it to be (via cat, print, or whatever) or, by default, if the result is unassigned. The default printing can be turned off via the invisible() function. So:> res <- lapply(TP, function(x){+ print(x) + x^2} + ) [1] 1 [1] 2 [1] 3 [1] 4 [1] 5 ## Nothing printed, but the results are the components of **list** res, which can be printed on the console by print(res) or simply:> res ## note the list syntax below[[1]] [1] 1 [[2]] [1] 4 [[3]] [1] 9 [[4]] [1] 16 [[5]] [1] 25 Now can you explain what happens here:> invisible(lapply(TP, function(x){+ print(x) + x^2} + )) [1] 1 [1] 2 [1] 3 [1] 4 [1] 5> ## results not printed; nothing assigned. Where can you find them? (See below)or here:> lapply(TP, function(x){+ print(x) + x^2} + ) [1] 1 [1] 2 [1] 3 [1] 4 [1] 5 [[1]] [1] 1 [[2]] [1] 4 [[3]] [1] 9 [[4]] [1] 16 [[5]] [1] 25 ## How could you recover the results of the lapply and not just look at the console printout if the results have not been assigned? ## Hint: See ?.Last.value -- Bert On Mon, Nov 7, 2022 at 11:22 AM akshay kulkarni <akshay_e4 at hotmail.com> wrote:> > Dear Rui, > THanks for your reply...The point is the loop is a scraping code, and in your examples you have assumed that the body acts on i, the loop variable. Can you adapt your code to JUST PRINT the loop variable i ? > > By the by, I think I have stumbled upon the answer: The lapply() caches the result, and prints the output of the function in question immediately after printing the final i. The i's get printed serially, as the function progresses.... > > > lapply(1:4,function(x){print(x);Sys.sleep(x^2);x^2}) > [1] 1 > [1] 2 > [1] 3 > [1] 4 > [[1]] > [1] 1 > > [[2]] > [1] 4 > > [[3]] > [1] 9 > > [[4]] > [1] 16 > > Here x^2 's print only after 4 is printed on the console.... > > tHanks anyways for your reply.... > > THanking you, > Yours sincerely, > AKSHAY M KULKARNI > ________________________________ > From: Rui Barradas <ruipbarradas at sapo.pt> > Sent: Tuesday, November 8, 2022 12:24 AM > To: akshay kulkarni <akshay_e4 at hotmail.com>; R help Mailing list <r-help at r-project.org> > Subject: Re: [R] print and lapply.... > > ?s 18:33 de 07/11/2022, akshay kulkarni escreveu: > > Dear Rui, > > Actually, I am replacing a big for loop by the lapply() function, and report the progress: > > > > lapply(TP, function(i) { BODY; print(i)}) > > > > Can you please adjust your solution in this light? > > > > THanking you, > > Yours sincerely, > > AKSHAY M KULKARNI > > ________________________________ > > From: Rui Barradas <ruipbarradas at sapo.pt> > > Sent: Monday, November 7, 2022 11:59 PM > > To: akshay kulkarni <akshay_e4 at hotmail.com>; R help Mailing list <r-help at r-project.org> > > Subject: Re: [R] print and lapply.... > > > > ?s 17:17 de 07/11/2022, akshay kulkarni escreveu: > >> Dear members, > >> I have the following code and output: > >> > >>> TP <- 1:4 > >>> lapply(TP,function(x){print(x);x^2}) > >> [1] 1 > >> [1] 2 > >> [1] 3 > >> [1] 4 > >> [[1]] > >> [1] 1 > >> > >> [[2]] > >> [1] 4 > >> > >> [[3]] > >> [1] 9 > >> > >> [[4]] > >> [1] 16 > >> > >> How do I make the print function output x along with x^2, i.e not at the beginning but before each of x^2? > >> > >> Many thanks in advance.... > >> > >> THanking you, > >> Yours sincerely > >> AKSHAY M KULKARNI > >> > >> [[alternative HTML version deleted]] > >> > >> ______________________________________________ > >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > >> https://stat.ethz.ch/mailman/listinfo/r-help > >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > >> and provide commented, minimal, self-contained, reproducible code. > > Hello, > > > > Here are two options, with ?cat and with ?message. > > > > > > TP <- 1:4 > > lapply(TP, function(x){ > > cat("x =", x, "x^2 =", x^2, "\n") > > }) > > > > lapply(TP, function(x){ > > msg <- paste("x =", x, "x^2 =", x^2) > > message(msg) > > }) > > > > > > Hope this helps, > > > > Rui Barradas > > > > > > > Hello, > > > What do you want the lapply loop to return? If you have a BODY doing > computations, do you want the lapply to return those values and report > the progress? > > I have chosen cat or message over print because > > - cat returns invisible(NULL), > - message returns invisible() > - print returns a value, what it prints. > > Can you adapt the code below to your use case? > > > > TP <- 1:4 > lapply(TP, function(x, verbose = TRUE){ > # BODY > y <- rnorm(100, mean = x) > > # show progress > if(verbose) > cat("x =", x, "x^2 =", x^2, "\n") > > #return value > c(x = x, mean = mean(y)) > }) > > lapply(TP, function(x, verbose = TRUE){ > # BODY > y <- rnorm(100, mean = x) > > # show progress > if(verbose) { > msg <- paste("x =", x, "x^2 =", x^2) > message(msg) > } > > #return value > c(x = x, mean = mean(y)) > }) > > > > Hope this helps, > > Rui Barradas > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
?s 19:22 de 07/11/2022, akshay kulkarni escreveu:> Dear Rui, > THanks for your reply...The point is the loop is a scraping code, and in your examples you have assumed that the body acts on i, the loop variable. Can you adapt your code to JUST PRINT the loop variable i ? > > By the by, I think I have stumbled upon the answer: The lapply() caches the result, and prints the output of the function in question immediately after printing the final i. The i's get printed serially, as the function progresses.... > >> lapply(1:4,function(x){print(x);Sys.sleep(x^2);x^2}) > [1] 1 > [1] 2 > [1] 3 > [1] 4 > [[1]] > [1] 1 > > [[2]] > [1] 4 > > [[3]] > [1] 9 > > [[4]] > [1] 16 > > Here x^2 's print only after 4 is printed on the console.... > > tHanks anyways for your reply.... > > THanking you, > Yours sincerely, > AKSHAY M KULKARNI > ________________________________ > From: Rui Barradas <ruipbarradas at sapo.pt> > Sent: Tuesday, November 8, 2022 12:24 AM > To: akshay kulkarni <akshay_e4 at hotmail.com>; R help Mailing list <r-help at r-project.org> > Subject: Re: [R] print and lapply.... > > ?s 18:33 de 07/11/2022, akshay kulkarni escreveu: >> Dear Rui, >> Actually, I am replacing a big for loop by the lapply() function, and report the progress: >> >> lapply(TP, function(i) { BODY; print(i)}) >> >> Can you please adjust your solution in this light? >> >> THanking you, >> Yours sincerely, >> AKSHAY M KULKARNI >> ________________________________ >> From: Rui Barradas <ruipbarradas at sapo.pt> >> Sent: Monday, November 7, 2022 11:59 PM >> To: akshay kulkarni <akshay_e4 at hotmail.com>; R help Mailing list <r-help at r-project.org> >> Subject: Re: [R] print and lapply.... >> >> ?s 17:17 de 07/11/2022, akshay kulkarni escreveu: >>> Dear members, >>> I have the following code and output: >>> >>>> TP <- 1:4 >>>> lapply(TP,function(x){print(x);x^2}) >>> [1] 1 >>> [1] 2 >>> [1] 3 >>> [1] 4 >>> [[1]] >>> [1] 1 >>> >>> [[2]] >>> [1] 4 >>> >>> [[3]] >>> [1] 9 >>> >>> [[4]] >>> [1] 16 >>> >>> How do I make the print function output x along with x^2, i.e not at the beginning but before each of x^2? >>> >>> Many thanks in advance.... >>> >>> THanking you, >>> Yours sincerely >>> AKSHAY M KULKARNI >>> >>> [[alternative HTML version deleted]] >>> >>> ______________________________________________ >>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >>> https://stat.ethz.ch/mailman/listinfo/r-help >>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >>> and provide commented, minimal, self-contained, reproducible code. >> Hello, >> >> Here are two options, with ?cat and with ?message. >> >> >> TP <- 1:4 >> lapply(TP, function(x){ >> cat("x =", x, "x^2 =", x^2, "\n") >> }) >> >> lapply(TP, function(x){ >> msg <- paste("x =", x, "x^2 =", x^2) >> message(msg) >> }) >> >> >> Hope this helps, >> >> Rui Barradas >> >> >> > Hello, > > > What do you want the lapply loop to return? If you have a BODY doing > computations, do you want the lapply to return those values and report > the progress? > > I have chosen cat or message over print because > > - cat returns invisible(NULL), > - message returns invisible() > - print returns a value, what it prints. > > Can you adapt the code below to your use case? > > > > TP <- 1:4 > lapply(TP, function(x, verbose = TRUE){ > # BODY > y <- rnorm(100, mean = x) > > # show progress > if(verbose) > cat("x =", x, "x^2 =", x^2, "\n") > > #return value > c(x = x, mean = mean(y)) > }) > > lapply(TP, function(x, verbose = TRUE){ > # BODY > y <- rnorm(100, mean = x) > > # show progress > if(verbose) { > msg <- paste("x =", x, "x^2 =", x^2) > message(msg) > } > > #return value > c(x = x, mean = mean(y)) > }) > > > > Hope this helps, > > Rui Barradas > >Hello, No, the x^2 are not printed after the i's. The x^2 are the function's return values. The function prints the i's, then returns x^2. As for your problem, it is now more clerar. I would write a function accepting a url to take care of scraping and call it in the lapply loop. The progress report can be in the loop, like below. This is a complete working example, scraping the Wikipedia list of countries by GDP. The urls are in a list (it's always the same, I'm not complicating things) and in a real scraping function I would wrap tryCatch around it, just in case. First the function, then the urls list, then the lapply loop. library(rvest) scrape <- function(url) { page <- read_html(url) gdp <- page |> html_element(".wikitable") |> html_table() |> as.data.frame() names(gdp) <- unlist(gdp[1,, drop = TRUE]) gdp <- gdp[-1,] row.names(gdp) <- NULL #return value gdp } wiki_gdp_url <- "https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal)" urls_list <- list(wiki_gdp_url, wiki_gdp_url) TP <- seq_along(urls_list) TP # [1] 1 2 df_list <- lapply(TP, \(i) { URL <- urls_list[[i]] data <- scrape(URL) # show progress message("iteration: ", i) #return value data }) str(df_list) Hope this helps, Rui Barradas