Roy Mendelssohn - NOAA Federal
2024-Sep-26 14:10 UTC
[R] Problem with converting grib file to excel
Hi Javad: I know a lot about reading GRIB files, I work with them all the time. But if you don?t make the file available, or point me to where I can download it, there is not much I can do. Thanks, -Roy> On Sep 25, 2024, at 9:41?PM, javad bayat <j.bayat194 at gmail.com> wrote: > > Dear all; > Many thanks for your responses. Actually it is not completely a GIS file, it is a data file which stores meteorological data of a specific region. But the site allows downloading with grib format and as I searched to read this type of file in R, I found the Raster Package. > In python it is possible to do this using cdsapi and xarray library, but I am not familiar with python. > Sincerely > > On Thu, Sep 26, 2024 at 2:33?AM Roy Mendelssohn - NOAA Federal via R-help <r-help at r-project.org <mailto:r-help at r-project.org>> wrote: >> At least for me the dataset file did not come through. I will look at it if it can be made available. It does look like the finial step of reading the data into raster failed, so then did the rest of th commands. >> >> -Roy >> >> >> > On Sep 25, 2024, at 3:24 PM, CALUM POLWART <polc1410 at gmail.com <mailto:polc1410 at gmail.com>> wrote: >> > >> > Noticeable lack of silence in the group on this one. >> > >> > I've not got time to test currently. But my experience of geo location >> > files - they often had more than 2 dimensional data. In other words you >> > might have a boundary of a region as an object with long and lat for maybe >> > 100 data points making up the region. So 200 pieces of data. All held as a >> > list or something similar in a single "cell" as excel would refer to it. >> > >> > My gut feeling is that's likely to make export to excel difficult without >> > data carpentry first? >> > >> > On Tue, 24 Sep 2024, 21:26 Bert Gunter, <bgunter.4567 at gmail.com <mailto:bgunter.4567 at gmail.com>> wrote: >> > >> >> You might try posting on r-sig-geo if you don't get a satisfactory >> >> response here. I assume there's a lot of expertise there on handling >> >> raster-type data. >> >> >> >> Cheers, >> >> Bert >> >> >> >> On Mon, Sep 23, 2024 at 11:31?PM javad bayat <j.bayat194 at gmail.com <mailto:j.bayat194 at gmail.com>> wrote: >> >>> >> >>> Dear R users; >> >>> I have downloaded a grib file format (Met.grib) and I want to export its >> >>> data to excel file. Also I want to do some mathematic on some columns. >> >> But >> >>> I got error. I would be more than happy if anyone can help me to do >> >> this. I >> >>> have provided the codes and the Met.grib file in this email. >> >>> Sincerely yours >> >>> >> >>> # Load the necessary libraries >> >>>> library(raster) # For reading GRIB files >> >>>> library(dplyr) # For data manipulation >> >>>> library(lubridate) # For date manipulation >> >>>> library(openxlsx) # For writing Excel files >> >>> >> >>> # Specify the file paths >> >>>> grib_file_path <- "C:/Users/Omrab_Lab/Downloads/Met.grib" >> >>>> excel_file_path <- "C:/Users/Omrab_Lab/Downloads/Met_updated.xlsx" >> >>> >> >>> # Open the GRIB file >> >>>> raster_data <- stack(grib_file_path) >> >>> >> >>> # Check the names of the layers to identify which ones to extract >> >>>> layer_names <- names(raster_data) >> >>>> print(layer_names) # Prints >> >>> >> >>> >> >>>> # Extract layers based on layer names - adjust as necessary >> >>>> t2m <- raster_data[[grep("t2m", layer_names)]] >> >>>> d2m <- raster_data[[grep("d2m", layer_names)]] >> >>>> tcc <- raster_data[[grep("tcc", layer_names)]] >> >>>> valid_time <- raster_data[[grep("valid_time", layer_names)]] >> >>>> t2m >> >>> class : RasterStack >> >>> nlayers : 0 >> >>> >> >>>> # Check if the raster layers are loaded correctly >> >>>> if (is.null(t2m) || is.null(d2m) || is.null(tcc) || >> >> is.null(valid_time)) >> >>> { >> >>> + stop("One or more raster layers could not be loaded. Please check >> >> the >> >>> layer names.") >> >>> + } >> >>> >> >>>> # Convert raster values to vectors >> >>>> t2m_values <- values(t2m) >> >>> Error in dimnames(x) <- dn : >> >>> length of 'dimnames' [2] not equal to array extent >> >>>> d2m_values <- values(d2m) >> >>> Error in dimnames(x) <- dn : >> >>> length of 'dimnames' [2] not equal to array extent >> >>>> tcc_values <- values(tcc) >> >>> Error in dimnames(x) <- dn : >> >>> length of 'dimnames' [2] not equal to array extent >> >>>> valid_time_values <- values(valid_time) >> >>> Error in dimnames(x) <- dn : >> >>> length of 'dimnames' [2] not equal to array extent >> >>> >> >>> # Check for NA values and dimensions >> >>> if (any(is.na <http://is.na/>(t2m_values)) || any(is.na <http://is.na/>(d2m_values)) || any(is.na <http://is.na/> >> >> (tcc_values)) >> >>> || any(is.na <http://is.na/>(valid_time_values))) { >> >>> warning("One or more layers contain NA values. These will be removed.") >> >>> } >> >>> >> >>> # Create the data frame, ensuring no NA values are included >> >>> df <- data.frame( >> >>> t2m = t2m_values, >> >>> d2m = d2m_values, >> >>> tcc = tcc_values, >> >>> valid_time = valid_time_values, >> >>> stringsAsFactors = FALSE >> >>> ) >> >>> >> >>> # Remove rows with NA values >> >>> df <- na.omit(df) >> >>> >> >>> # Convert temperatures from Kelvin to Celsius >> >>> df$t2m <- df$t2m - 273.15 >> >>> df$d2m <- df$d2m - 273.15 >> >>> >> >>> # Calculate relative humidity >> >>> calculate_relative_humidity <- function(t2m, d2m) { >> >>> es <- 6.112 * exp((17.67 * t2m) / (t2m + 243.5)) >> >>> e <- 6.112 * exp((17.67 * d2m) / (d2m + 243.5)) >> >>> rh <- (e / es) * 100 >> >>> return(rh) >> >>> } >> >>> df$RH <- calculate_relative_humidity(df$t2m, df$d2m) >> >>> >> >>> # Convert valid_time from numeric to POSIXct assuming it's in seconds >> >> since >> >>> the epoch >> >>> df$valid_time <- as.POSIXct(df$valid_time, origin = "1970-01-01") >> >>> >> >>> # Extract year, month, day, and hour from valid_time >> >>> df$Year <- year(df$valid_time) >> >>> df$Month <- month(df$valid_time) >> >>> df$Day <- day(df$valid_time) >> >>> df$Hour <- hour(df$valid_time) >> >>> >> >>> # Select only the desired columns >> >>> df_selected <- df %>% select(Year, Month, Day, Hour, tcc, t2m, RH) >> >>> >> >>> # Save the updated DataFrame to an Excel file >> >>> write.xlsx(df_selected, excel_file_path, row.names = FALSE) >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Best Regards >> >>> Javad Bayat >> >>> M.Sc. Environment Engineering >> >>> Alternative Mail: bayat194 at yahoo.com <mailto:bayat194 at yahoo.com> >> >>> >> >>> [[alternative HTML version deleted]] >> >>> >> >>> ______________________________________________ >> >>> R-help at r-project.org <mailto:R-help at r-project.org> mailing list -- To UNSUBSCRIBE and more, see >> >>> https://stat.ethz.ch/mailman/listinfo/r-help >> >>> PLEASE do read the posting guide >> >> https://www.R-project.org/posting-guide.html <https://www.r-project.org/posting-guide.html> >> >>> and provide commented, minimal, self-contained, reproducible code. >> >> >> >> ______________________________________________ >> >> R-help at r-project.org <mailto:R-help at r-project.org> mailing list -- To UNSUBSCRIBE and more, see >> >> https://stat.ethz.ch/mailman/listinfo/r-help >> >> PLEASE do read the posting guide >> >> https://www.R-project.org/posting-guide.html <https://www.r-project.org/posting-guide.html> >> >> and provide commented, minimal, self-contained, reproducible code. >> >> >> > >> > [[alternative HTML version deleted]] >> > >> > ______________________________________________ >> > R-help at r-project.org <mailto:R-help at r-project.org> mailing list -- To UNSUBSCRIBE and more, see >> > https://stat.ethz.ch/mailman/listinfo/r-help >> > PLEASE do read the posting guide https://www.R-project.org/posting-guide.html <https://www.r-project.org/posting-guide.html> >> > and provide commented, minimal, self-contained, reproducible code. >> >> ********************** >> "The contents of this message do not reflect any position of the U.S. Government or NOAA." >> ********************** >> Roy Mendelssohn >> Supervisory Operations Research Analyst >> NOAA/NMFS >> Environmental Research Division >> Southwest Fisheries Science Center >> ***Note new street address*** >> 110 McAllister Way >> Santa Cruz, CA 95060 >> Phone: (831)-420-3666 >> Fax: (831) 420-3980 >> e-mail: Roy.Mendelssohn at noaa.gov <mailto:Roy.Mendelssohn at noaa.gov> www: https://www.pfeg.noaa.gov/ >> >> "Old age and treachery will overcome youth and skill." >> "From those who have been given much, much will be expected" >> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr. >> >> ______________________________________________ >> R-help at r-project.org <mailto:R-help at r-project.org> mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide https://www.R-project.org/posting-guide.html <https://www.r-project.org/posting-guide.html> >> and provide commented, minimal, self-contained, reproducible code. > > > -- > Best Regards > Javad Bayat > M.Sc. Environment Engineering > Alternative Mail: bayat194 at yahoo.com <mailto:bayat194 at yahoo.com>[[alternative HTML version deleted]]
Dear Roy, Sorry for my mistake, I thought I have uploaded the grib file. I really apologise for that. I will send it on Saturday. Thank you very much. On Thu, 26 Sept 2024, 17:40 Roy Mendelssohn - NOAA Federal, < roy.mendelssohn at noaa.gov> wrote:> Hi Javad: > > I know a lot about reading GRIB files, I work with them all the time. > But if you don?t make the file available, or point me to where I can > download it, there is not much I can do. > > Thanks, > > -Roy > > On Sep 25, 2024, at 9:41?PM, javad bayat <j.bayat194 at gmail.com> wrote: > > Dear all; > Many thanks for your responses. Actually it is not completely a GIS file, > it is a data file which stores meteorological data of a specific region. > But the site allows downloading with grib format and as I searched to read > this type of file in R, I found the Raster Package. > In python it is possible to do this using cdsapi and xarray library, but I > am not familiar with python. > Sincerely > > On Thu, Sep 26, 2024 at 2:33?AM Roy Mendelssohn - NOAA Federal via R-help < > r-help at r-project.org> wrote: > >> At least for me the dataset file did not come through. I will look at it >> if it can be made available. It does look like the finial step of reading >> the data into raster failed, so then did the rest of th commands. >> >> -Roy >> >> >> > On Sep 25, 2024, at 3:24 PM, CALUM POLWART <polc1410 at gmail.com> wrote: >> > >> > Noticeable lack of silence in the group on this one. >> > >> > I've not got time to test currently. But my experience of geo location >> > files - they often had more than 2 dimensional data. In other words you >> > might have a boundary of a region as an object with long and lat for >> maybe >> > 100 data points making up the region. So 200 pieces of data. All held >> as a >> > list or something similar in a single "cell" as excel would refer to it. >> > >> > My gut feeling is that's likely to make export to excel difficult >> without >> > data carpentry first? >> > >> > On Tue, 24 Sep 2024, 21:26 Bert Gunter, <bgunter.4567 at gmail.com> wrote: >> > >> >> You might try posting on r-sig-geo if you don't get a satisfactory >> >> response here. I assume there's a lot of expertise there on handling >> >> raster-type data. >> >> >> >> Cheers, >> >> Bert >> >> >> >> On Mon, Sep 23, 2024 at 11:31?PM javad bayat <j.bayat194 at gmail.com> >> wrote: >> >>> >> >>> Dear R users; >> >>> I have downloaded a grib file format (Met.grib) and I want to export >> its >> >>> data to excel file. Also I want to do some mathematic on some columns. >> >> But >> >>> I got error. I would be more than happy if anyone can help me to do >> >> this. I >> >>> have provided the codes and the Met.grib file in this email. >> >>> Sincerely yours >> >>> >> >>> # Load the necessary libraries >> >>>> library(raster) # For reading GRIB files >> >>>> library(dplyr) # For data manipulation >> >>>> library(lubridate) # For date manipulation >> >>>> library(openxlsx) # For writing Excel files >> >>> >> >>> # Specify the file paths >> >>>> grib_file_path <- "C:/Users/Omrab_Lab/Downloads/Met.grib" >> >>>> excel_file_path <- "C:/Users/Omrab_Lab/Downloads/Met_updated.xlsx" >> >>> >> >>> # Open the GRIB file >> >>>> raster_data <- stack(grib_file_path) >> >>> >> >>> # Check the names of the layers to identify which ones to extract >> >>>> layer_names <- names(raster_data) >> >>>> print(layer_names) # Prints >> >>> >> >>> >> >>>> # Extract layers based on layer names - adjust as necessary >> >>>> t2m <- raster_data[[grep("t2m", layer_names)]] >> >>>> d2m <- raster_data[[grep("d2m", layer_names)]] >> >>>> tcc <- raster_data[[grep("tcc", layer_names)]] >> >>>> valid_time <- raster_data[[grep("valid_time", layer_names)]] >> >>>> t2m >> >>> class : RasterStack >> >>> nlayers : 0 >> >>> >> >>>> # Check if the raster layers are loaded correctly >> >>>> if (is.null(t2m) || is.null(d2m) || is.null(tcc) || >> >> is.null(valid_time)) >> >>> { >> >>> + stop("One or more raster layers could not be loaded. Please >> check >> >> the >> >>> layer names.") >> >>> + } >> >>> >> >>>> # Convert raster values to vectors >> >>>> t2m_values <- values(t2m) >> >>> Error in dimnames(x) <- dn : >> >>> length of 'dimnames' [2] not equal to array extent >> >>>> d2m_values <- values(d2m) >> >>> Error in dimnames(x) <- dn : >> >>> length of 'dimnames' [2] not equal to array extent >> >>>> tcc_values <- values(tcc) >> >>> Error in dimnames(x) <- dn : >> >>> length of 'dimnames' [2] not equal to array extent >> >>>> valid_time_values <- values(valid_time) >> >>> Error in dimnames(x) <- dn : >> >>> length of 'dimnames' [2] not equal to array extent >> >>> >> >>> # Check for NA values and dimensions >> >>> if (any(is.na(t2m_values)) || any(is.na(d2m_values)) || any(is.na >> >> (tcc_values)) >> >>> || any(is.na(valid_time_values))) { >> >>> warning("One or more layers contain NA values. These will be >> removed.") >> >>> } >> >>> >> >>> # Create the data frame, ensuring no NA values are included >> >>> df <- data.frame( >> >>> t2m = t2m_values, >> >>> d2m = d2m_values, >> >>> tcc = tcc_values, >> >>> valid_time = valid_time_values, >> >>> stringsAsFactors = FALSE >> >>> ) >> >>> >> >>> # Remove rows with NA values >> >>> df <- na.omit(df) >> >>> >> >>> # Convert temperatures from Kelvin to Celsius >> >>> df$t2m <- df$t2m - 273.15 >> >>> df$d2m <- df$d2m - 273.15 >> >>> >> >>> # Calculate relative humidity >> >>> calculate_relative_humidity <- function(t2m, d2m) { >> >>> es <- 6.112 * exp((17.67 * t2m) / (t2m + 243.5)) >> >>> e <- 6.112 * exp((17.67 * d2m) / (d2m + 243.5)) >> >>> rh <- (e / es) * 100 >> >>> return(rh) >> >>> } >> >>> df$RH <- calculate_relative_humidity(df$t2m, df$d2m) >> >>> >> >>> # Convert valid_time from numeric to POSIXct assuming it's in seconds >> >> since >> >>> the epoch >> >>> df$valid_time <- as.POSIXct(df$valid_time, origin = "1970-01-01") >> >>> >> >>> # Extract year, month, day, and hour from valid_time >> >>> df$Year <- year(df$valid_time) >> >>> df$Month <- month(df$valid_time) >> >>> df$Day <- day(df$valid_time) >> >>> df$Hour <- hour(df$valid_time) >> >>> >> >>> # Select only the desired columns >> >>> df_selected <- df %>% select(Year, Month, Day, Hour, tcc, t2m, RH) >> >>> >> >>> # Save the updated DataFrame to an Excel file >> >>> write.xlsx(df_selected, excel_file_path, row.names = FALSE) >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Best Regards >> >>> Javad Bayat >> >>> M.Sc. Environment Engineering >> >>> Alternative Mail: bayat194 at yahoo.com >> >>> >> >>> [[alternative HTML version deleted]] >> >>> >> >>> ______________________________________________ >> >>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> >>> https://stat.ethz.ch/mailman/listinfo/r-help >> >>> PLEASE do read the posting guide >> >> https://www.R-project.org/posting-guide.html >> <https://www.r-project.org/posting-guide.html> >> >>> and provide commented, minimal, self-contained, reproducible code. >> >> >> >> ______________________________________________ >> >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> >> https://stat.ethz.ch/mailman/listinfo/r-help >> >> PLEASE do read the posting guide >> >> https://www.R-project.org/posting-guide.html >> <https://www.r-project.org/posting-guide.html> >> >> and provide commented, minimal, self-contained, reproducible code. >> >> >> > >> > [[alternative HTML version deleted]] >> > >> > ______________________________________________ >> > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> > https://stat.ethz.ch/mailman/listinfo/r-help >> > PLEASE do read the posting guide >> https://www.R-project.org/posting-guide.html >> <https://www.r-project.org/posting-guide.html> >> > and provide commented, minimal, self-contained, reproducible code. >> >> ********************** >> "The contents of this message do not reflect any position of the U.S. >> Government or NOAA." >> ********************** >> Roy Mendelssohn >> Supervisory Operations Research Analyst >> NOAA/NMFS >> Environmental Research Division >> Southwest Fisheries Science Center >> ***Note new street address*** >> 110 McAllister Way >> Santa Cruz, CA 95060 >> Phone: (831)-420-3666 >> Fax: (831) 420-3980 >> e-mail: Roy.Mendelssohn at noaa.gov www: https://www.pfeg.noaa.gov/ >> >> "Old age and treachery will overcome youth and skill." >> "From those who have been given much, much will be expected" >> "the arc of the moral universe is long, but it bends toward justice" -MLK >> Jr. >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> https://www.R-project.org/posting-guide.html >> <https://www.r-project.org/posting-guide.html> >> and provide commented, minimal, self-contained, reproducible code. >> > > > -- > Best Regards > Javad Bayat > M.Sc. Environment Engineering > Alternative Mail: bayat194 at yahoo.com > > >[[alternative HTML version deleted]]