I'm puzzled. It doesn't seem to take long on MacOS, neither under 4.3.x. or 4.5.0. The tarballs for BH are like 14 MB and the Windows .zip is 21 MB, which is large, but not that large? I'm somewhat suspicious about the "package updating" techniques that some people seeme to be using. It sounds like you are copying package directories from an older version, then using update.packages(). A number of things could go wrong if you do that across an API change. At the very least, you need update.packages(checkBuilt=TRUE) (and probably you also want ask=FALSE), but I would prefer a safer strategy of just installing what that you need into a fresh install of R. I.e. something like this: In the old R do> pkglist <- rownames(subset(as.data.frame(installed.packages()), !(Priority %in% c("base","recommended")))) > save(pkglist, file="~/tmp/4.4pkglist")Install new version, then> load("~/tmp/4.4pkglist") > system.time(install.packages(pkglist))--- Please select a CRAN mirror for use in this session --- trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/abind_1.4-8.tgz' trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/ada_2.0-5.tgz' tryi....... ......ng URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/zoo_1.8-14.tgz' The downloaded binary packages are in /var/folders/h0/hzzhnnfd1gx7399sx_cwlhpw0000gn/T//RtmpQDl3Tj/downloaded_packages user system elapsed 4.574 5.235 33.213 Warning message: package ?fEcofin? is not available for this version of R A version of this package for your version of R might be available elsewhere, see the ideas at https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages> length(pkglist)[1] 252 (most of the time was spent waiting for XQuartz and Tcl/Tk to fire up so that I could choose the CRAN mirror.) Actually, BH was not on the list, but> system.time(install.packages("BH"))trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/BH_1.87.0-1.tgz' Content type 'application/x-gzip' length 13900146 bytes (13.3 MB) =================================================downloaded 13.3 MB The downloaded binary packages are in /var/folders/h0/hzzhnnfd1gx7399sx_cwlhpw0000gn/T//RtmpQDl3Tj/downloaded_packages user system elapsed 1.054 2.340 5.091 -pd> On 23 Apr 2025, at 23:33 , Evan Cooch <evan.cooch at gmail.com> wrote: > > So, I decided to update from 4.4.3 -> 4.5.0. Updating all packages in > one shot (selecting all of the offered updates) failed, on all my > machines (Linux, Windows - lots of RAM, lots of CPU speed). So, tried > the updates a few at a time. Didn't take me long to discover that the > 'culprit' was the BH packages. For some reason, it takes a *long* time > for it to download/update or download/install. Even installing from a > local download took a long time. So long, that on my Windows machines, > the Rgui simply stopped responding. At all. Had to kill the process > manually, and try again. > > So, in case anyone else runs into this (I've replicated the problem on 5 > different machines - 3 Windows, 2 Linux) > > 1\ if BH is included in a slew of updates you're trying to apply all at > once, it might clobber everything because its choking on BH, kill RGui > on my Windows machines. Less of an issue on my Linux boxes (good old > CLI), but still... > > 2\ even BH alone requires some patience. On a high-end machine (24 > threads, 64 Gb RAM, etc etc) took almost 8 minutes to download/update. > Everything else (even big suckers like terra) took only seconds on said > machine. But BH? > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide https://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.-- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
Good advice, but that approach (as you describe it) is not something I (or anyone I know) has needed to follow for past updates to R, and apparently is/was only? a consideration for BH. As per OP, every other package updated just fine "the old way" (which you correctly describe as what most of us do -- copy old /library folder to 'new' /library folder, and simply update everything). On 4/24/2025 7:32 AM, peter dalgaard wrote:> I'm puzzled. It doesn't seem to take long on MacOS, neither under 4.3.x. or 4.5.0. > > The tarballs for BH are like 14 MB and the Windows .zip is 21 MB, which is large, but not that large? > > I'm somewhat suspicious about the "package updating" techniques that some people seeme to be using. It sounds like you are copying package directories from an older version, then using update.packages(). A number of things could go wrong if you do that across an API change. > > At the very least, you need update.packages(checkBuilt=TRUE) (and probably you also want ask=FALSE), but I would prefer a safer strategy of just installing what that you need into a fresh install of R. I.e. something like this: > > In the old R do >> pkglist <- rownames(subset(as.data.frame(installed.packages()), !(Priority %in% c("base","recommended")))) >> save(pkglist, file="~/tmp/4.4pkglist") > Install new version, then >> load("~/tmp/4.4pkglist") >> system.time(install.packages(pkglist)) > --- Please select a CRAN mirror for use in this session --- > trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/abind_1.4-8.tgz' > trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/ada_2.0-5.tgz' > tryi....... > ......ng URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/zoo_1.8-14.tgz' > > The downloaded binary packages are in > /var/folders/h0/hzzhnnfd1gx7399sx_cwlhpw0000gn/T//RtmpQDl3Tj/downloaded_packages > user system elapsed > 4.574 5.235 33.213 > Warning message: > package ?fEcofin? is not available for this version of R > > A version of this package for your version of R might be available elsewhere, > see the ideas at > https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages >> length(pkglist) > [1] 252 > > (most of the time was spent waiting for XQuartz and Tcl/Tk to fire up so that I could choose the CRAN mirror.) > > > Actually, BH was not on the list, but > >> system.time(install.packages("BH")) > trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/BH_1.87.0-1.tgz' > Content type 'application/x-gzip' length 13900146 bytes (13.3 MB) > =================================================> downloaded 13.3 MB > > > The downloaded binary packages are in > /var/folders/h0/hzzhnnfd1gx7399sx_cwlhpw0000gn/T//RtmpQDl3Tj/downloaded_packages > user system elapsed > 1.054 2.340 5.091 > > -pd > > >> On 23 Apr 2025, at 23:33 , Evan Cooch<evan.cooch at gmail.com> wrote: >> >> So, I decided to update from 4.4.3 -> 4.5.0. Updating all packages in >> one shot (selecting all of the offered updates) failed, on all my >> machines (Linux, Windows - lots of RAM, lots of CPU speed). So, tried >> the updates a few at a time. Didn't take me long to discover that the >> 'culprit' was the BH packages. For some reason, it takes a *long* time >> for it to download/update or download/install. Even installing from a >> local download took a long time. So long, that on my Windows machines, >> the Rgui simply stopped responding. At all. Had to kill the process >> manually, and try again. >> >> So, in case anyone else runs into this (I've replicated the problem on 5 >> different machines - 3 Windows, 2 Linux) >> >> 1\ if BH is included in a slew of updates you're trying to apply all at >> once, it might clobber everything because its choking on BH, kill RGui >> on my Windows machines. Less of an issue on my Linux boxes (good old >> CLI), but still... >> >> 2\ even BH alone requires some patience. On a high-end machine (24 >> threads, 64 Gb RAM, etc etc) took almost 8 minutes to download/update. >> Everything else (even big suckers like terra) took only seconds on said >> machine. But BH? >> >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guidehttps://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code.[[alternative HTML version deleted]]
[oops, forgot to copy r-help, which could be rather important to do, so resending] There's a difference between "getting away with in the past" and "safe to do".... I just had another conversation (off-line) with someone getting bitten by the change to md5sum(), and I'm 99% sure it happened because copying the old library overwrote the new tools package with the old one, an updated package tried to call tools::md5sum(bytes=....), and things blew up. So even though people have apparently gotten away with copying the old folder in the past, they probably actually created a bastardized installation, where some system packages was actually from a previous version (which of course is inconsequential if and only if they weren't actually changed by the update...). - pd> On 24 Apr 2025, at 14:31 , Evan Cooch <evan.cooch at gmail.com> wrote: > > Good advice, but that approach (as you describe it) is not something I (or anyone I know) has needed to follow for past updates to R, and apparently is/was only a consideration for BH. As per OP, every other package updated just fine "the old way" (which you correctly describe as what most of us do -- copy old /library folder to 'new' /library folder, and simply update everything). > > On 4/24/2025 7:32 AM, peter dalgaard wrote: >> I'm puzzled. It doesn't seem to take long on MacOS, neither under 4.3.x. or 4.5.0. >> >> The tarballs for BH are like 14 MB and the Windows .zip is 21 MB, which is large, but not that large? >> >> I'm somewhat suspicious about the "package updating" techniques that some people seeme to be using. It sounds like you are copying package directories from an older version, then using update.packages(). A number of things could go wrong if you do that across an API change. >> >> At the very least, you need update.packages(checkBuilt=TRUE) (and probably you also want ask=FALSE), but I would prefer a safer strategy of just installing what that you need into a fresh install of R. I.e. something like this: >> >> In the old R do >> >>> pkglist <- rownames(subset(as.data.frame(installed.packages()), !(Priority %in% c("base","recommended")))) >>> save(pkglist, file="~/tmp/4.4pkglist") >>> >> Install new version, then >> >>> load("~/tmp/4.4pkglist") >>> system.time(install.packages(pkglist)) >>> >> --- Please select a CRAN mirror for use in this session --- >> trying URL ' >> https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/abind_1.4-8.tgz >> ' >> trying URL ' >> https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/ada_2.0-5.tgz >> ' >> tryi....... >> ......ng URL ' >> https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/zoo_1.8-14.tgz >> ' >> >> The downloaded binary packages are in >> /var/folders/h0/hzzhnnfd1gx7399sx_cwlhpw0000gn/T//RtmpQDl3Tj/downloaded_packages >> user system elapsed >> 4.574 5.235 33.213 >> Warning message: >> package ?fEcofin? is not available for this version of R >> >> A version of this package for your version of R might be available elsewhere, >> see the ideas at >> >> https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages >> >> >>> length(pkglist) >>> >> [1] 252 >> >> (most of the time was spent waiting for XQuartz and Tcl/Tk to fire up so that I could choose the CRAN mirror.) >> >> >> Actually, BH was not on the list, but >> >> >>> system.time(install.packages("BH")) >>> >> trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.5/BH_1.87.0-1.tgz >> ' >> Content type 'application/x-gzip' length 13900146 bytes (13.3 MB) >> =================================================>> downloaded 13.3 MB >> >> >> The downloaded binary packages are in >> /var/folders/h0/hzzhnnfd1gx7399sx_cwlhpw0000gn/T//RtmpQDl3Tj/downloaded_packages >> user system elapsed >> 1.054 2.340 5.091 >> >> -pd >> >> >> >>> On 23 Apr 2025, at 23:33 , Evan Cooch <evan.cooch at gmail.com> >>> wrote: >>> >>> So, I decided to update from 4.4.3 -> 4.5.0. Updating all packages in >>> one shot (selecting all of the offered updates) failed, on all my >>> machines (Linux, Windows - lots of RAM, lots of CPU speed). So, tried >>> the updates a few at a time. Didn't take me long to discover that the >>> 'culprit' was the BH packages. For some reason, it takes a *long* time >>> for it to download/update or download/install. Even installing from a >>> local download took a long time. So long, that on my Windows machines, >>> the Rgui simply stopped responding. At all. Had to kill the process >>> manually, and try again. >>> >>> So, in case anyone else runs into this (I've replicated the problem on 5 >>> different machines - 3 Windows, 2 Linux) >>> >>> 1\ if BH is included in a slew of updates you're trying to apply all at >>> once, it might clobber everything because its choking on BH, kill RGui >>> on my Windows machines. Less of an issue on my Linux boxes (good old >>> CLI), but still... >>> >>> 2\ even BH alone requires some patience. On a high-end machine (24 >>> threads, 64 Gb RAM, etc etc) took almost 8 minutes to download/update. >>> Everything else (even big suckers like terra) took only seconds on said >>> machine. But BH? >>> >>> >>> [[alternative HTML version deleted]] >>> >>> ______________________________________________ >>> >>> R-help at r-project.org >>> mailing list -- To UNSUBSCRIBE and more, see >>> >>> https://stat.ethz.ch/mailman/listinfo/r-help >>> >>> PLEASE do read the posting guide >>> https://www.R-project.org/posting-guide.html >>> >>> and provide commented, minimal, self-contained, reproducible code. >>> >-- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com