Gabriel,
It is the most basic CI use case. One wants to install only
dependencies only of the package, and run R CMD check on the package.
Unless you say that installing the package and then running R CMD
check on that package is considered good practice. Then yes,
functionality I am asking about is not needed. Somehow I never thought
that this could be considered a good practice just by the fact that
installation of the package could already impact environment in which
check is taking place.
Best,
Jan
On Fri, Oct 28, 2022 at 8:42 PM Gabriel Becker <gabembecker at gmail.com>
wrote:>
> Hi Jan,
>
> The reason, I suspect without speaking for R-core, is that by design you
should not be specifying package dependencies as additional packages to install.
install.packages already does this for you, as it did in the construct of a
repository code that I provided previously in the thread. You should be *only*
doing
>
> install.packages(<pkg in question>, repos = *)
>
> Then everything happens automatically via extremely well tested very mature
code.
>
> I (still) don't understand why you'd need to pass install.packages
the vector of dependencies yourself, as that is counter to install.packages'
core design.
>
> Does that make sense?
>
> Best,
> ~G
>
> On Fri, Oct 28, 2022 at 12:18 PM Jan Gorecki <j.gorecki at
wit.edu.pl> wrote:
>>
>> Gabriel,
>>
>> I am trying to design generic solution that could be applied to
>> arbitrary package. Therefore I went with the latter solution you
>> proposed.
>> If we wouldn't have to exclude base packages, then its a 3 liner
>>
>> file.copy("DESCRIPTION", file.path(tdir<-tempdir(),
"PACKAGES"));
>> db<-available.packages(paste0("file://", tdir));
>>
utils::install.packages(tools::package_dependencies("pkgname", db,
>> which="most")[[1L]])
>>
>> As you noticed, we still have to filter out base packages. Otherwise
>> it won't be a robust utility that can be used in CI. Therefore we
have
>> to add a call to tools:::.get_standard_package_names() which is an
>> internal function (as of now). Not only complicating the call but also
>> putting the functionality outside of safe use.
>>
>> Considering above, don't you agree that the following one liner
could
>> nicely address the problem? The problem that hundreds/thousands of
>> packages are now addressing in their CI scripts by using a third party
>> packages.
>>
>> utils::install.packages(packages.dcf("DESCRIPTION",
which="most"))
>>
>> It is hard to me to understand why R members don't consider this
basic
>> functionality to be part of base R. Possibly they just don't need
it
>> themselves. Yet isn't this sufficient that hundreds/thousands of
>> packages does need this functionality?
>>
>> Best regards,
>> Jan
>>
>> On Mon, Oct 17, 2022 at 8:39 AM Jan Gorecki <j.gorecki at
wit.edu.pl> wrote:
>> >
>> > Gabriel and Simon
>> >
>> > I completely agree with what you are saying.
>> > The thing is that obtaining recursive deps, all/most whatever, is
already well supported in core R. What is missing is just this single
functionality I am requesting.
>> >
>> > If you will look into the branch you can see there is
mirror.packages function meant to mirror a slice of CRAN. It is doing exactly
what you described: package_dependencies; to obtain recursive deps, then
download all, etc.
>> > I would love to have this function provided by core R as well, but
we need to start somewhere.
>> >
>> > There are other use cases as well.
>> > For example CI, where one wants to install all/most dependencies
and then run R CMD check. Then we don't worry about recursive deps are they
will be resolved automatically.
>> > I don't think it's reasonable to force users to use 3rd
party packages to handle such a common and simple use case. Otherwise one has to
hard code deps in CI script. Not robust at all.
>> >
>> > packages.dcf and repos.dcf makes all that way easier, and are
solid base for building customized orchestration like mirroring slice of CRAN.
>> >
>> > Best regards
>> > Jan
>> >
>> > On Sun, Oct 16, 2022, 01:31 Simon Urbanek <simon.urbanek at
r-project.org> wrote:
>> >>
>> >> Jan,
>> >>
>> >> I think using a single DCF as input is not very practical and
would not be useful in the context you describe (creating self contained repos)
since they typically concern a list of packages, but essentially splitting out
the part of install.packages() which determines which files will be pulled from
where would be very useful as it would be trivial to use it to create repository
(what we always do in corporate environments) instead of installing the
packages. I suspect that install packages is already too complex so instead of
adding a flag to install.packages one could move that functionality into a
separate function - we all do that constantly for the sites we manage, so it
would be certainly something worthwhile.
>> >>
>> >> Cheers,
>> >> Simon
>> >>
>> >>
>> >> > On Oct 15, 2022, at 7:14 PM, Jan Gorecki <j.gorecki at
wit.edu.pl> wrote:
>> >> >
>> >> > Hi Gabriel,
>> >> >
>> >> > It's very nice usage you provided here. Maybe instead
of adding new
>> >> > function we could extend packages_depenedncies then? To
accept file path to
>> >> > dsc file.
>> >> >
>> >> > What about repos.dcf? Maybe additional repositories could
be an attribute
>> >> > attached to returned character vector.
>> >> >
>> >> > The use case is to, for a given package sources, obtain
its dependencies,
>> >> > so one can use that for installing them/mirroring CRAN
subset, or whatever.
>> >> > The later is especially important for a production
environment where one
>> >> > wants to have fixed version of packages, and mirroring
relevant subset of
>> >> > CRAN is the most simple, and IMO reliable, way to manage
such environment.
>> >> >
>> >> > Regards
>> >> > Jan
>> >> >
>> >> > On Fri, Oct 14, 2022, 23:34 Gabriel Becker
<gabembecker at gmail.com> wrote:
>> >> >
>> >> >> Hi Jan and Jan,
>> >> >>
>> >> >> Can you explain a little more what exactly you want
the non-recursive,
>> >> >> non-version aware dependencies from an individual
package for?
>> >> >>
>> >> >> Either way package_dependencies will do this for you*
with a little
>> >> >> "aggressive convincing". It wants output
from available.packages, but who
>> >> >> really cares what it wants? It's a function and
we are people :)
>> >> >>
>> >> >>> library(tools)
>> >> >>> db <-
read.dcf("~/gabe/checkedout/rtables_clean/DESCRIPTION")
>> >> >>> package_dependencies("rtables", db,
which = intersect(c("Depends",
>> >> >> "Suggests", "Imports",
"LinkingTo"), colnames(db)))
>> >> >> $rtables
>> >> >> [1] "methods" "magrittr"
"formatters" "dplyr" "tibble"
>> >> >> [6] "tidyr" "testthat"
"xml2" "knitr" "rmarkdown"
>> >> >> [11] "flextable" "officer"
"stats" "htmltools" "grid"
>> >> >>
>> >> >>
>> >> >> The only gotcha that I see immediately is that
"LinkingTo" isn't always
>> >> >> there (whereas it is with real output from
available.packages). If you
>> >> >> know your package doesn't have that (or that it
does) at call time , this
>> >> >> becomes a one-liner:
>> >> >>
>> >> >> package_dependencies("rtables", db >>
>> >>
read.dcf("~/gabe/checkedout/rtables_clean/DESCRIPTION"), which
>> >> >> c("Depends", "Suggests",
"Imports"))
>> >> >> $rtables
>> >> >> [1] "methods" "magrittr"
"formatters" "dplyr" "tibble"
>> >> >> [6] "tidyr" "testthat"
"xml2" "knitr" "rmarkdown"
>> >> >> [11] "flextable" "officer"
"stats" "htmltools" "grid"
>> >> >>
>> >> >> You can also trick it a slightly different way by
giving it what it
>> >> >> actually wants
>> >> >>
>> >> >>> tdir <- tempdir()
>> >> >>>
file.copy("~/gabe/checkedout/rtables_clean/DESCRIPTION",
file.path(tdir,
>> >> >> "PACKAGES"))
>> >> >> [1] TRUE
>> >> >>> avl <-
available.packages(paste0("file://", tdir))
>> >> >>> library(tools)
>> >> >>> package_dependencies("rtables", avl)
>> >> >> $rtables
>> >> >> [1] "methods" "magrittr"
"formatters" "stats" "htmltools"
>> >> >> [6] "grid"
>> >> >>
>> >> >>> package_dependencies("rtables", avl,
which = "all")
>> >> >> $rtables
>> >> >> [1] "methods" "magrittr"
"formatters" "stats" "htmltools"
>> >> >> [6] "grid" "dplyr"
"tibble" "tidyr" "testthat"
>> >> >> [11] "xml2" "knitr"
"rmarkdown" "flextable" "officer"
>> >> >>
>> >> >> So the only real benefits I see that we'd be
picking up here is automatic
>> >> >> filtering by priority, and automatic extraction of
the package name from
>> >> >> the DESCRIPTION file. I'm not sure either of
those warrant a new exported
>> >> >> function that R-core has to maintain forever.
>> >> >>
>> >> >> Best,
>> >> >> ~G
>> >> >>
>> >> >> * I haven't tested this across all OSes, but I
dont' know of any reason it
>> >> >> wouldn't work generally.
>> >> >>
>> >> >> On Fri, Oct 14, 2022 at 2:33 PM Jan Gorecki
<j.gorecki at wit.edu.pl> wrote:
>> >> >>
>> >> >>> Hello Jan,
>> >> >>>
>> >> >>> Thanks for confirming about many packages
reinventing this missing
>> >> >>> functionality.
>> >> >>> packages.dcf was not meant handle versions. It
just extracts names of
>> >> >>> dependencies... Yes, such a simple thing, yet
missing in base R.
>> >> >>>
>> >> >>> Versions of packages can be controlled when
setting up R pkgs repo. This
>> >> >>> is
>> >> >>> how I used to handle it. Making a CRAN subset
mirror of fixed version
>> >> >>> pkgs.
>> >> >>> BTW. function for that is also included in
mentioned branch. I am just not
>> >> >>> proposing it, to increase the chance of having at
least this simple,
>> >> >>> missing, functionality merged.
>> >> >>>
>> >> >>> Best
>> >> >>> Jan
>> >> >>>
>> >> >>> On Fri, Oct 14, 2022, 15:14 Jan Net?k <netikja
at gmail.com> wrote:
>> >> >>>
>> >> >>>> Hello Jan,
>> >> >>>>
>> >> >>>> I have seen many packages that implemented
dependencies "extraction" on
>> >> >>>> their own for internal purposes and today I
was doing exactly that for
>> >> >>>> mine. It's not a big deal using read.dcf
on DESCRIPTION. It was
>> >> >>> sufficient
>> >> >>>> for me, but I had to take care of some \n
chars (the overall returned
>> >> >>> value
>> >> >>>> has some rough edges, in my opinion).
However, the function from the
>> >> >>> branch
>> >> >>>> seems to not care about version requirements,
which are crucial for me.
>> >> >>>> Maybe that is something to reconsider before
merging.
>> >> >>>>
>> >> >>>> Best,
>> >> >>>> Jan
>> >> >>>>
>> >> >>>> p? 14. 10. 2022 v 2:27 odes?latel Jan Gorecki
<j.gorecki at wit.edu.pl>
>> >> >>>> napsal:
>> >> >>>>
>> >> >>>>> Dear R devs,
>> >> >>>>>
>> >> >>>>> I would like to raise a request for a
simple helper function.
>> >> >>>>> Utility function to extract package
dependencies from DESCRIPTION file.
>> >> >>>>>
>> >> >>>>> I do think that tools package is better
place, for such a fundamental
>> >> >>>>> functionality, than community packages.
>> >> >>>>>
>> >> >>>>> tools pkg seems perfect fit (having
already great function
>> >> >>>>> write_PACKAGES).
>> >> >>>>>
>> >> >>>>> Functionality I am asking for is already
in R svn repository since
>> >> >>> 2016,
>> >> >>>>> in
>> >> >>>>> a branch tools4pkgs. Function is called
'packages.dcf'.
>> >> >>>>> Another one 'repos.dcf' would be
a good functional complementary to it.
>> >> >>>>>
>> >> >>>>> Those two simple helper functions really
makes it easier for
>> >> >>> organizations
>> >> >>>>> to glue together usage of their own R
packages repos and CRAN repo in a
>> >> >>>>> smooth way. That could possibly help to
offload CRAN from new
>> >> >>> submissions.
>> >> >>>>>
>> >> >>>>> gh mirror link for easy preview:
>> >> >>>>>
>> >> >>>>>
>> >> >>>
https://github.com/wch/r-source/blob/tools4pkgs/src/library/tools/R/packages.R#L419
>> >> >>>>>
>> >> >>>>> Regards
>> >> >>>>> Jan Gorecki
>> >> >>>>>
>> >> >>>>> [[alternative HTML version
deleted]]
>> >> >>>>>
>> >> >>>>>
______________________________________________
>> >> >>>>> R-devel at r-project.org mailing list
>> >> >>>>>
https://stat.ethz.ch/mailman/listinfo/r-devel
>> >> >>>>>
>> >> >>>>
>> >> >>>
>> >> >>> [[alternative HTML version deleted]]
>> >> >>>
>> >> >>> ______________________________________________
>> >> >>> R-devel at r-project.org mailing list
>> >> >>> https://stat.ethz.ch/mailman/listinfo/r-devel
>> >> >>>
>> >> >>
>> >> >
>> >> > [[alternative HTML version deleted]]
>> >> >
>> >> > ______________________________________________
>> >> > R-devel at r-project.org mailing list
>> >> > https://stat.ethz.ch/mailman/listinfo/r-devel
>> >> >
>> >>