On Thu, 4 Nov 2021, Ben Tupper wrote:> The help for problems() shows that the expected argument has a default > value of .Last.value. If you don't provide the input argument, it just > uses the last thing your R session evaluated. That's great if you run > problems() right after your issues arises. But you have inserted > stop_for_problems() before you get a chance to run problems(). So, in your > case, if you want to inspect the problems associated with x, you should > provide x explicitly ala problems(x).Ben, I've isolated one data file to import by removing both stop_for_problems() and problems(): library(tidyverse) cor_stage <- read_csv("../data/cor-stage.csv", col_names = TRUE, col_types = list ( site_nbr = col_character(), year = col_integer(), mon = col_integer(), day = col_integer(), hr = col_double(), min = col_double(), ft = col_integer()) ) The gzipped data file can be downloaded from <https://tinyurl.com/5usc8wbz>. R imports the file but when I look at it the last column has not been imported and problems() doesn't return them:> source('import2.r') > cor_stage# A tibble: 415,903 ? 8 site_nbr year mon day hr min tz ft <chr> <int> <int> <int> <dbl> <dbl> <chr> <int> 1 14171600 2009 10 23 0 0 PDT NA 2 14171600 2009 10 23 0 15 PDT NA 3 14171600 2009 10 23 0 30 PDT NA 4 14171600 2009 10 23 0 45 PDT NA 5 14171600 2009 10 23 1 0 PDT NA 6 14171600 2009 10 23 1 15 PDT NA 7 14171600 2009 10 23 1 30 PDT NA 8 14171600 2009 10 23 1 45 PDT NA 9 14171600 2009 10 23 2 0 PDT NA 10 14171600 2009 10 23 2 15 PDT NA # ? with 415,893 more rows Warning message: One or more parsing issues, see `problems()` for details> problems() >There are 20 .csv files; a few import properly the rest don't. I've not before needed to import this many data files for a project but using read.csv() hasn't failed to import the data column, either. Thanks, Rich
On 04/11/2021 16:02, Rich Shepard wrote:> On Thu, 4 Nov 2021, Ben Tupper wrote: > > ????????????????????? ) > > The gzipped data file can be downloaded from > <https://tinyurl.com/5usc8wbz>.This seems to work fine for me: library(readr) cor_stage_file <- "cor-stage.csv" cor_stage <- read_csv(cor_stage_file, ????????????????????? col_types = list(site_nbr = col_integer(), ?????????????????????????????????????? year = col_character(), ?????????????????????????????????????? mon = col_character(), ?????????????????????????????????????? day = col_character(), ?????????????????????????????????????? hr = col_character(), ?????????????????????????????????????? min = col_character(), ?????????????????????????????????????? tz = col_character(), ?????????????????????????????????????? ft = col_double())) # Add a proper datetime column date_string <- with(cor_stage, paste(year, mon, day, sep="-")) time_string <- with(cor_stage, paste(hr, min, "00", sep=":")) cor_stage['date_time'] <- as.POSIXct(paste(date_string, time_string, cor_stage$tz)) Why are you importing the last "ft" column as an integer when it's clearly decimal data? -- Micha Silver Ben Gurion Univ. Sde Boker, Remote Sensing Lab cell: +972-523-665918