Dave Lange
2020-Jun-25 02:52 UTC
[R-sig-Debian] Update on docker Python:3 and adding R:4.x
My dockerfile: FROM python:3 WORKDIR /usr/src/app COPY requirements.txt ./ COPY languageserver-master ./ COPY radian-master ./ RUN pip3 install --no-cache-dir -r requirements.txt EXPOSE 8080 RUN apt-key adv --keyserver keys.gnupg.net --recv-key 'E19F5F87128899B192B1A2C2AD5F960A256A04AF' RUN echo "deb http://cloud.r-project.org/bin/linux/debian buster-cran40/">> /etc/apt/sources.listRUN apt update \ && apt-get install -y --no-install-recommends \ ca-certificates \ wget \ && rm -rf /var/lib/apt/lists/* ## Now install R ## RUN apt install -t buster-cran40 r-base ## CMD ["R"] CMD python On Wed, Jun 24, 2020 at 4:38 PM Dirk Eddelbuettel <edd at debian.org> wrote:> > On 24 June 2020 at 15:15, Dave Lange wrote: > | I continue to receive an error installing R via dockerfile on a buster > | image python:3. > | E: The value 'buster-cran40' is invalid for APT::Default-Release as such > a > | release is not available in the sources > > Do you have the Dockerfile in public repo we can look at? > > | My starting point is the debian buster based Python:3 image adding a > couple > > This is an R list so please tell us more about Python:3. What it is based > on? > > Hypothetically, could you just start from debian:buster, add python3 and > then > add the buster-cran40 repo by Johannes? > > | of python specific configurations and then using the commands in the R > | project documentation for installing R on Buster. I got slightly > different > | answers when I used apt versus apt-get. There were warnings about > | unverified sources solved by a reference to the certificate key. > > You generally must install a key to validate a repository. This could even > be > your error. > > In any event, this is all "academic". Maybe bring us the famous "MCVE": a > minimally complete verifiable example. Otherwise we have simply no idea > what > you may be doing. > > | It turns out building the python container and commenting the R commands > | out allowed me to manually step through my dockerfile lines. Its > repeatable > | that the R install fails with the error above when in the dockerfile. > | Running the commands manually allows the installation to finish > | successfully. I sense that docker is multithreaded and hits the "use > | buster-cran40" before it defines buster-cran40. Manually stepping through > > I doubt that. Docker is very carefully "layered". Each RUN command results > in > one layer on filesystem. You can build them one by one. There is no > concurrency as each subsequent RUN needs / depends upon previous ones. > > | the commands keeps the preferred order. At this point I'm happy with a > | repeatable process. > | > | It sounds like I have been re-inventing the wheel, which has been > | educational for me. If someone wants to change my starting point to > | something that already has stable/latest production for debian, Python3 > and > | R4 and will be updated for the future I would appreciate the head start. > > Should be easy. Look at the variety of Dockerfiles is maintain inside the > Rocker Project -- while most are based on Debian's testing release you can > still look at them (though note that some are also Ubuntu based) > > You could start at https://github.com/rocker-org/rocker but also look at > other repositories in the same org at GH. > > Dirk > > -- > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org >[[alternative HTML version deleted]]
Dirk Eddelbuettel
2020-Jun-25 03:54 UTC
[R-sig-Debian] Update on docker Python:3 and adding R:4.x
On 24 June 2020 at 19:52, Dave Lange wrote: | ## Now install R | ## RUN apt install -t buster-cran40 r-base It's just a guess but something I noticed working with one of my Dockerfiles is that the above does not work too well -- I think due to the independence of the 'layers' from each RUN command which I mentioned earlier. So when you I run apt install, I now always update first so maybe try RUN apt update && apt install -y r-base instead. (I also removed the -t buster-cran40 as the 'highest version should win'. That may or may not work. Adjust as needed.) Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
Johannes Ranke
2020-Jun-25 06:46 UTC
[R-sig-Debian] Update on docker Python:3 and adding R:4.x
Am Donnerstag, 25. Juni 2020, 04:52:37 CEST schrieb Dave Lange:> My dockerfile: >...> RUN apt update \ > && apt-get install -y --no-install-recommends \ > ca-certificates \ > wget \ > && rm -rf /var/lib/apt/lists/*Any reason why you remove the contents of /var/lib/apt/lists? On my system, this directory holds PGP signed information about each repository, so this may well be the reason why later buster-cran40 is said not to be available in the sources. Johannes> > ## Now install R > ## RUN apt install -t buster-cran40 r-base > > ## CMD ["R"] > CMD python > > On Wed, Jun 24, 2020 at 4:38 PM Dirk Eddelbuettel <edd at debian.org> wrote: > > On 24 June 2020 at 15:15, Dave Lange wrote: > > | I continue to receive an error installing R via dockerfile on a buster > > | > > | image python:3. > > | E: The value 'buster-cran40' is invalid for APT::Default-Release as such > > > > a > > > > | release is not available in the sources > > > > Do you have the Dockerfile in public repo we can look at? > > > > | My starting point is the debian buster based Python:3 image adding a > > > > couple > > > > This is an R list so please tell us more about Python:3. What it is based > > on? > > > > Hypothetically, could you just start from debian:buster, add python3 and > > then > > add the buster-cran40 repo by Johannes? > > > > | of python specific configurations and then using the commands in the R > > | project documentation for installing R on Buster. I got slightly > > > > different > > > > | answers when I used apt versus apt-get. There were warnings about > > | unverified sources solved by a reference to the certificate key. > > > > You generally must install a key to validate a repository. This could even > > be > > your error. > > > > In any event, this is all "academic". Maybe bring us the famous "MCVE": a > > minimally complete verifiable example. Otherwise we have simply no idea > > what > > you may be doing. > > > > | It turns out building the python container and commenting the R commands > > | out allowed me to manually step through my dockerfile lines. Its > > > > repeatable > > > > | that the R install fails with the error above when in the dockerfile. > > | Running the commands manually allows the installation to finish > > | successfully. I sense that docker is multithreaded and hits the "use > > | buster-cran40" before it defines buster-cran40. Manually stepping > > | through > > > > I doubt that. Docker is very carefully "layered". Each RUN command results > > in > > one layer on filesystem. You can build them one by one. There is no > > concurrency as each subsequent RUN needs / depends upon previous ones. > > > > | the commands keeps the preferred order. At this point I'm happy with a > > | repeatable process. > > | > > | It sounds like I have been re-inventing the wheel, which has been > > | educational for me. If someone wants to change my starting point to > > | something that already has stable/latest production for debian, Python3 > > > > and > > > > | R4 and will be updated for the future I would appreciate the head start. > > > > Should be easy. Look at the variety of Dockerfiles is maintain inside the > > Rocker Project -- while most are based on Debian's testing release you can > > still look at them (though note that some are also Ubuntu based) > > > > You could start at https://github.com/rocker-org/rocker but also look at > > other repositories in the same org at GH. > > > > Dirk > > > > -- > > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org > > [[alternative HTML version deleted]] > > _______________________________________________ > R-SIG-Debian mailing list > R-SIG-Debian at r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-debian
Johannes Ranke
2020-Jun-25 06:50 UTC
[R-sig-Debian] Update on docker Python:3 and adding R:4.x
Am Donnerstag, 25. Juni 2020, 05:54:03 CEST schrieb Dirk Eddelbuettel:> > RUN apt update && apt install -y r-base > > instead. (I also removed the -t buster-cran40 as the 'highest version > should win'. That may or may not work. Adjust as needed.)Removing -t buster-cran40 only installs R 4.0.x if there are no packages left on the system that depend on the lower r-api-35. Therefore this should be OK for a docker image which does not come with R. Johannes
Dave Lange
2020-Jun-27 19:26 UTC
[R-sig-Debian] Update on docker Python:3 and adding R:4.x
Thanks for all the feedback these comments will all go in my dockerfile for my next attempt. Dave On Wed, Jun 24, 2020 at 11:46 PM Johannes Ranke <johannes.ranke at jrwb.de> wrote:> Am Donnerstag, 25. Juni 2020, 04:52:37 CEST schrieb Dave Lange: > > My dockerfile: > > > ... > > > RUN apt update \ > > && apt-get install -y --no-install-recommends \ > > ca-certificates \ > > wget \ > > && rm -rf /var/lib/apt/lists/* > > Any reason why you remove the contents of /var/lib/apt/lists? On my > system, > this directory holds PGP signed information about each repository, so this > may > well be the reason why later buster-cran40 is said not to be available in > the > sources. > > Johannes > > > > > ## Now install R > > ## RUN apt install -t buster-cran40 r-base > > > > ## CMD ["R"] > > CMD python > > > > On Wed, Jun 24, 2020 at 4:38 PM Dirk Eddelbuettel <edd at debian.org> > wrote: > > > On 24 June 2020 at 15:15, Dave Lange wrote: > > > | I continue to receive an error installing R via dockerfile on a > buster > > > | > > > | image python:3. > > > | E: The value 'buster-cran40' is invalid for APT::Default-Release as > such > > > > > > a > > > > > > | release is not available in the sources > > > > > > Do you have the Dockerfile in public repo we can look at? > > > > > > | My starting point is the debian buster based Python:3 image adding a > > > > > > couple > > > > > > This is an R list so please tell us more about Python:3. What it is > based > > > on? > > > > > > Hypothetically, could you just start from debian:buster, add python3 > and > > > then > > > add the buster-cran40 repo by Johannes? > > > > > > | of python specific configurations and then using the commands in the > R > > > | project documentation for installing R on Buster. I got slightly > > > > > > different > > > > > > | answers when I used apt versus apt-get. There were warnings about > > > | unverified sources solved by a reference to the certificate key. > > > > > > You generally must install a key to validate a repository. This could > even > > > be > > > your error. > > > > > > In any event, this is all "academic". Maybe bring us the famous > "MCVE": a > > > minimally complete verifiable example. Otherwise we have simply no idea > > > what > > > you may be doing. > > > > > > | It turns out building the python container and commenting the R > commands > > > | out allowed me to manually step through my dockerfile lines. Its > > > > > > repeatable > > > > > > | that the R install fails with the error above when in the dockerfile. > > > | Running the commands manually allows the installation to finish > > > | successfully. I sense that docker is multithreaded and hits the "use > > > | buster-cran40" before it defines buster-cran40. Manually stepping > > > | through > > > > > > I doubt that. Docker is very carefully "layered". Each RUN command > results > > > in > > > one layer on filesystem. You can build them one by one. There is no > > > concurrency as each subsequent RUN needs / depends upon previous ones. > > > > > > | the commands keeps the preferred order. At this point I'm happy with > a > > > | repeatable process. > > > | > > > | It sounds like I have been re-inventing the wheel, which has been > > > | educational for me. If someone wants to change my starting point to > > > | something that already has stable/latest production for debian, > Python3 > > > > > > and > > > > > > | R4 and will be updated for the future I would appreciate the head > start. > > > > > > Should be easy. Look at the variety of Dockerfiles is maintain inside > the > > > Rocker Project -- while most are based on Debian's testing release you > can > > > still look at them (though note that some are also Ubuntu based) > > > > > > You could start at https://github.com/rocker-org/rocker but also > look at > > > other repositories in the same org at GH. > > > > > > Dirk > > > > > > -- > > > http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org > > > > [[alternative HTML version deleted]] > > > > _______________________________________________ > > R-SIG-Debian mailing list > > R-SIG-Debian at r-project.org > > https://stat.ethz.ch/mailman/listinfo/r-sig-debian > > > >[[alternative HTML version deleted]]