Hi Jan,
The reason, I suspect without speaking for R-core, is that by design you
should not be specifying package dependencies as additional packages to
install. install.packages already does this for you, as it did in the
construct of a repository code that I provided previously in the thread.
You should be *only* doing
install.packages(<pkg in question>, repos = *)
Then everything happens automatically via extremely well tested very mature
code.
I (still) don't understand why you'd need to pass install.packages the
vector of dependencies yourself, as that is counter to install.packages'
core design.
Does that make sense?
Best,
~G
On Fri, Oct 28, 2022 at 12:18 PM Jan Gorecki <j.gorecki at wit.edu.pl>
wrote:
> Gabriel,
>
> I am trying to design generic solution that could be applied to
> arbitrary package. Therefore I went with the latter solution you
> proposed.
> If we wouldn't have to exclude base packages, then its a 3 liner
>
> file.copy("DESCRIPTION", file.path(tdir<-tempdir(),
"PACKAGES"));
> db<-available.packages(paste0("file://", tdir));
> utils::install.packages(tools::package_dependencies("pkgname",
db,
> which="most")[[1L]])
>
> As you noticed, we still have to filter out base packages. Otherwise
> it won't be a robust utility that can be used in CI. Therefore we have
> to add a call to tools:::.get_standard_package_names() which is an
> internal function (as of now). Not only complicating the call but also
> putting the functionality outside of safe use.
>
> Considering above, don't you agree that the following one liner could
> nicely address the problem? The problem that hundreds/thousands of
> packages are now addressing in their CI scripts by using a third party
> packages.
>
> utils::install.packages(packages.dcf("DESCRIPTION",
which="most"))
>
> It is hard to me to understand why R members don't consider this basic
> functionality to be part of base R. Possibly they just don't need it
> themselves. Yet isn't this sufficient that hundreds/thousands of
> packages does need this functionality?
>
> Best regards,
> Jan
>
> On Mon, Oct 17, 2022 at 8:39 AM Jan Gorecki <j.gorecki at wit.edu.pl>
wrote:
> >
> > Gabriel and Simon
> >
> > I completely agree with what you are saying.
> > The thing is that obtaining recursive deps, all/most whatever, is
> already well supported in core R. What is missing is just this single
> functionality I am requesting.
> >
> > If you will look into the branch you can see there is mirror.packages
> function meant to mirror a slice of CRAN. It is doing exactly what you
> described: package_dependencies; to obtain recursive deps, then download
> all, etc.
> > I would love to have this function provided by core R as well, but we
> need to start somewhere.
> >
> > There are other use cases as well.
> > For example CI, where one wants to install all/most dependencies and
> then run R CMD check. Then we don't worry about recursive deps are they
> will be resolved automatically.
> > I don't think it's reasonable to force users to use 3rd party
packages
> to handle such a common and simple use case. Otherwise one has to hard code
> deps in CI script. Not robust at all.
> >
> > packages.dcf and repos.dcf makes all that way easier, and are solid
base
> for building customized orchestration like mirroring slice of CRAN.
> >
> > Best regards
> > Jan
> >
> > On Sun, Oct 16, 2022, 01:31 Simon Urbanek <simon.urbanek at
r-project.org>
> wrote:
> >>
> >> Jan,
> >>
> >> I think using a single DCF as input is not very practical and
would not
> be useful in the context you describe (creating self contained repos) since
> they typically concern a list of packages, but essentially splitting out
> the part of install.packages() which determines which files will be pulled
> from where would be very useful as it would be trivial to use it to create
> repository (what we always do in corporate environments) instead of
> installing the packages. I suspect that install packages is already too
> complex so instead of adding a flag to install.packages one could move that
> functionality into a separate function - we all do that constantly for the
> sites we manage, so it would be certainly something worthwhile.
> >>
> >> Cheers,
> >> Simon
> >>
> >>
> >> > On Oct 15, 2022, at 7:14 PM, Jan Gorecki <j.gorecki at
wit.edu.pl>
> wrote:
> >> >
> >> > Hi Gabriel,
> >> >
> >> > It's very nice usage you provided here. Maybe instead of
adding new
> >> > function we could extend packages_depenedncies then? To
accept file
> path to
> >> > dsc file.
> >> >
> >> > What about repos.dcf? Maybe additional repositories could be
an
> attribute
> >> > attached to returned character vector.
> >> >
> >> > The use case is to, for a given package sources, obtain its
> dependencies,
> >> > so one can use that for installing them/mirroring CRAN
subset, or
> whatever.
> >> > The later is especially important for a production
environment where
> one
> >> > wants to have fixed version of packages, and mirroring
relevant
> subset of
> >> > CRAN is the most simple, and IMO reliable, way to manage such
> environment.
> >> >
> >> > Regards
> >> > Jan
> >> >
> >> > On Fri, Oct 14, 2022, 23:34 Gabriel Becker <gabembecker at
gmail.com>
> wrote:
> >> >
> >> >> Hi Jan and Jan,
> >> >>
> >> >> Can you explain a little more what exactly you want the
> non-recursive,
> >> >> non-version aware dependencies from an individual package
for?
> >> >>
> >> >> Either way package_dependencies will do this for you*
with a little
> >> >> "aggressive convincing". It wants output from
available.packages,
> but who
> >> >> really cares what it wants? It's a function and we
are people :)
> >> >>
> >> >>> library(tools)
> >> >>> db <-
read.dcf("~/gabe/checkedout/rtables_clean/DESCRIPTION")
> >> >>> package_dependencies("rtables", db, which =
intersect(c("Depends",
> >> >> "Suggests", "Imports",
"LinkingTo"), colnames(db)))
> >> >> $rtables
> >> >> [1] "methods" "magrittr"
"formatters" "dplyr" "tibble"
> >> >> [6] "tidyr" "testthat"
"xml2" "knitr" "rmarkdown"
> >> >> [11] "flextable" "officer"
"stats" "htmltools" "grid"
> >> >>
> >> >>
> >> >> The only gotcha that I see immediately is that
"LinkingTo" isn't
> always
> >> >> there (whereas it is with real output from
available.packages). If
> you
> >> >> know your package doesn't have that (or that it does)
at call time ,
> this
> >> >> becomes a one-liner:
> >> >>
> >> >> package_dependencies("rtables", db >
>> >>
read.dcf("~/gabe/checkedout/rtables_clean/DESCRIPTION"), which >
>> >> c("Depends", "Suggests",
"Imports"))
> >> >> $rtables
> >> >> [1] "methods" "magrittr"
"formatters" "dplyr" "tibble"
> >> >> [6] "tidyr" "testthat"
"xml2" "knitr" "rmarkdown"
> >> >> [11] "flextable" "officer"
"stats" "htmltools" "grid"
> >> >>
> >> >> You can also trick it a slightly different way by giving
it what it
> >> >> actually wants
> >> >>
> >> >>> tdir <- tempdir()
> >> >>>
file.copy("~/gabe/checkedout/rtables_clean/DESCRIPTION",
> file.path(tdir,
> >> >> "PACKAGES"))
> >> >> [1] TRUE
> >> >>> avl <-
available.packages(paste0("file://", tdir))
> >> >>> library(tools)
> >> >>> package_dependencies("rtables", avl)
> >> >> $rtables
> >> >> [1] "methods" "magrittr"
"formatters" "stats" "htmltools"
> >> >> [6] "grid"
> >> >>
> >> >>> package_dependencies("rtables", avl, which
= "all")
> >> >> $rtables
> >> >> [1] "methods" "magrittr"
"formatters" "stats" "htmltools"
> >> >> [6] "grid" "dplyr"
"tibble" "tidyr" "testthat"
> >> >> [11] "xml2" "knitr"
"rmarkdown" "flextable" "officer"
> >> >>
> >> >> So the only real benefits I see that we'd be picking
up here is
> automatic
> >> >> filtering by priority, and automatic extraction of the
package name
> from
> >> >> the DESCRIPTION file. I'm not sure either of those
warrant a new
> exported
> >> >> function that R-core has to maintain forever.
> >> >>
> >> >> Best,
> >> >> ~G
> >> >>
> >> >> * I haven't tested this across all OSes, but I
dont' know of any
> reason it
> >> >> wouldn't work generally.
> >> >>
> >> >> On Fri, Oct 14, 2022 at 2:33 PM Jan Gorecki <j.gorecki
at wit.edu.pl>
> wrote:
> >> >>
> >> >>> Hello Jan,
> >> >>>
> >> >>> Thanks for confirming about many packages reinventing
this missing
> >> >>> functionality.
> >> >>> packages.dcf was not meant handle versions. It just
extracts names
> of
> >> >>> dependencies... Yes, such a simple thing, yet missing
in base R.
> >> >>>
> >> >>> Versions of packages can be controlled when setting
up R pkgs repo.
> This
> >> >>> is
> >> >>> how I used to handle it. Making a CRAN subset mirror
of fixed
> version
> >> >>> pkgs.
> >> >>> BTW. function for that is also included in mentioned
branch. I am
> just not
> >> >>> proposing it, to increase the chance of having at
least this simple,
> >> >>> missing, functionality merged.
> >> >>>
> >> >>> Best
> >> >>> Jan
> >> >>>
> >> >>> On Fri, Oct 14, 2022, 15:14 Jan Net?k <netikja at
gmail.com> wrote:
> >> >>>
> >> >>>> Hello Jan,
> >> >>>>
> >> >>>> I have seen many packages that implemented
dependencies
> "extraction" on
> >> >>>> their own for internal purposes and today I was
doing exactly that
> for
> >> >>>> mine. It's not a big deal using read.dcf on
DESCRIPTION. It was
> >> >>> sufficient
> >> >>>> for me, but I had to take care of some \n chars
(the overall
> returned
> >> >>> value
> >> >>>> has some rough edges, in my opinion). However,
the function from
> the
> >> >>> branch
> >> >>>> seems to not care about version requirements,
which are crucial
> for me.
> >> >>>> Maybe that is something to reconsider before
merging.
> >> >>>>
> >> >>>> Best,
> >> >>>> Jan
> >> >>>>
> >> >>>> p? 14. 10. 2022 v 2:27 odes?latel Jan Gorecki
<
> j.gorecki at wit.edu.pl>
> >> >>>> napsal:
> >> >>>>
> >> >>>>> Dear R devs,
> >> >>>>>
> >> >>>>> I would like to raise a request for a simple
helper function.
> >> >>>>> Utility function to extract package
dependencies from DESCRIPTION
> file.
> >> >>>>>
> >> >>>>> I do think that tools package is better
place, for such a
> fundamental
> >> >>>>> functionality, than community packages.
> >> >>>>>
> >> >>>>> tools pkg seems perfect fit (having already
great function
> >> >>>>> write_PACKAGES).
> >> >>>>>
> >> >>>>> Functionality I am asking for is already in R
svn repository since
> >> >>> 2016,
> >> >>>>> in
> >> >>>>> a branch tools4pkgs. Function is called
'packages.dcf'.
> >> >>>>> Another one 'repos.dcf' would be a
good functional complementary
> to it.
> >> >>>>>
> >> >>>>> Those two simple helper functions really
makes it easier for
> >> >>> organizations
> >> >>>>> to glue together usage of their own R
packages repos and CRAN
> repo in a
> >> >>>>> smooth way. That could possibly help to
offload CRAN from new
> >> >>> submissions.
> >> >>>>>
> >> >>>>> gh mirror link for easy preview:
> >> >>>>>
> >> >>>>>
> >> >>>
>
https://github.com/wch/r-source/blob/tools4pkgs/src/library/tools/R/packages.R#L419
> >> >>>>>
> >> >>>>> Regards
> >> >>>>> Jan Gorecki
> >> >>>>>
> >> >>>>> [[alternative HTML version deleted]]
> >> >>>>>
> >> >>>>>
______________________________________________
> >> >>>>> R-devel at r-project.org mailing list
> >> >>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
> >> >>>>>
> >> >>>>
> >> >>>
> >> >>> [[alternative HTML version deleted]]
> >> >>>
> >> >>> ______________________________________________
> >> >>> R-devel at r-project.org mailing list
> >> >>> https://stat.ethz.ch/mailman/listinfo/r-devel
> >> >>>
> >> >>
> >> >
> >> > [[alternative HTML version deleted]]
> >> >
> >> > ______________________________________________
> >> > R-devel at r-project.org mailing list
> >> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >> >
> >>
>
[[alternative HTML version deleted]]