R SIG Fedora - Jun 2023 - Problem with R, staged installation for packages, and samba share

Unfortunately, I cannot reproduce the issue here. :( I mounted a smb
share, created a symlink from ~/R, and installed dplyr without any
issue (apart from the installation process taking ages). Fortunately,
this rules out our binary R package as responsible for the issue you
are experiencing, unless I'm missing something.

Issue aside, may I ask what is the purpose of putting the library in a
smb share? May I suggest [1]?

[1] https://cran.r-project.org/bin/linux/fedora/#additional-packages

I?aki

On Mon, 12 Jun 2023 at 16:48, Tom Callaway <spotrh at gmail.com>
wrote:
>
> The log of you running configure on your instance allows me to compare it
to the log of when we build it as a package (you don't need to provide that
one).
>
> Honestly, I have no idea right now _why_ pre-built R would fail to move
files to a cifs share when a non-root user can do it. I'm hoping maybe
something will jump out as to how it configures on your setup. We aren't
modifying R's source code at all, nor are we configuring it in a way that
would affect this as far as I know.
>
> Does this also happen with the older R binary RPMs? If you go back to 4.2.3
or 4.2.2, does it happen?
>
> ~spot
>
>
> On Mon, Jun 12, 2023 at 10:38?AM Alexandre Courtiol <alexandre.courtiol
at gmail.com> wrote:
>>
>> "is the staged installation enabled too in the cases you don't
see this issue?"
>> -> Yes, it should since we stick to default config.
>>
>> "Can you provide a log of the output (including the commandline
with any options passed) for configure?"
>> -> If we compile locally there is no issue despite not using any
option, and if we install R from binary, there is no configure log AFAIK, so I
am not sure how to meet this request.
>>
>> "can you provide the specific mount options for the cifs
share?"
>> ->
sec=krb5,multiuser,mfsymlinks,user=smbuser,domain=localdomain,_netdev,noauto,x-systemd.automount
>>
>> ++
>>
>> On Mon, 12 Jun 2023 at 16:11, Tom Callaway <spotrh at gmail.com>
wrote:
>>>
>>> Okay, so if it's not SELinux... perhaps something about how R
is being configured is different?
>>>
>>> Can you provide a log of the output (including the commandline with
any options passed) for configure?
>>>
>>> Also, can you provide the specific mount options for the cifs
share?
>>>
>>> ~spot
>>>
>>> On Mon, Jun 12, 2023 at 9:12?AM Alexandre Courtiol
<alexandre.courtiol at gmail.com> wrote:
>>>>
>>>> Thanks for the tip but nope: SELinux is off on that system...
>>>>
>>>> On Mon, 12 Jun 2023 at 13:24, I?aki Ucar <iucar at
fedoraproject.org> wrote:
>>>>>
>>>>> On Mon, 12 Jun 2023 at 13:19, Tom Callaway <spotrh at
gmail.com> wrote:
>>>>> >
>>>>> > Hmm, that's a weird one. Is SELinux on and
enforcing on that setup?
>>>>>
>>>>> I was going to bet on the same thing. :) But let me add: if
the answer
>>>>> is affirmative, and this doesn't happen with SELinux
disabled, then
>>>>> *the answer is NOT to disable SELinux*. Instead, let's
figure this
>>>>> out, because the answer is to set the proper labels or
permissions.
>>>>>
>>>>> I?aki
>>>>>
>>>>> >
>>>>> > ~spot
>>>>> >
>>>>> > On Mon, Jun 12, 2023, 6:55 AM Alexandre Courtiol <
>>>>> > alexandre.courtiol at gmail.com> wrote:
>>>>> >
>>>>> > > Dear Fedora-R enthusiasts,
>>>>> > >
>>>>> > > We are experiencing some issues with the binary
releases of R 4.3 for
>>>>> > > Fedora 37 & 38 when running it on our
infrastructure.
>>>>> > >
>>>>> > > The issue is that packages won't install,
unless the (default) staged
>>>>> > > installation process for packages is switched off
(--no-staged-install).
>>>>> > >
>>>>> > > It seems to be related to the fact that we are
storing R libraries on a
>>>>> > > drive mounted via samba share.
>>>>> > >
>>>>> > > Interestingly, installing R from sources does not
cause the issue.
>>>>> > > When using other OS (e.g. Arch) on the same
infrastructure, the problem
>>>>> > > also disappears.
>>>>> > >
>>>>> > > We are looking for a way out that would allow
users to install R packages
>>>>> > > as usual and that would ideally not force the
admin to install R from
>>>>> > > sources.
>>>>> > >
>>>>> > > The issue should be reproducible as follows:
>>>>> > >
>>>>> > > 1. Mount the share:
>>>>> > >
>>>>> > > $ mount -t cifs //server/share /mnt/share -o
...options...
>>>>> > >
>>>>> > > 2. Put R library onto the share:
>>>>> > >
>>>>> > > $ mkdir /mnt/share/R
>>>>> > > $ ln -s /mnt/share/R ~/R
>>>>> > >
>>>>> > > 3. Install a package that has dependencies (It
doesn't matter whether the
>>>>> > > dependencies actually have to be installed or
not. E.g. if you install all
>>>>> > > of the dependencies of 'dplyr' and then
install 'dplyr' itself in an extra
>>>>> > > call to 'install.packages', the problem
will still be triggered.):
>>>>> > >
>>>>> > > $ Rscript -e
"install.packages('dplyr', repos='https://cloud.r-project.org
>>>>> > > ')"
>>>>> > >
>>>>> > > 4. The installation will fail while trying to
move the package to its final
>>>>> > > location:
>>>>> > >
>>>>> > > mv: cannot move
>>>>> > >
'/mnt/share/R/x86_64-redhat-linux-gnu-library/4.3/00LOCK-dplyr/00new/dplyr'
>>>>> > > to
'/mnt/share/R/x86_64-redhat-linux-gnu-library/4.3/dplyr': Permission
>>>>> > > denied
>>>>> > > ERROR:   moving to final location failed
>>>>> > >
>>>>> > > 5. Moving the folder as described in the error
manually from the shell
>>>>> > > succeeds:
>>>>> > >
>>>>> > > $ mv
>>>>> > >
/mnt/share/R/x86_64-redhat-linux-gnu-library/4.3/00LOCK-dplyr/00new/dplyr
>>>>> > >
/mnt/share/R/x86_64-redhat-linux-gnu-library/4.3/dplyr
>>>>> > > $ # => works
>>>>> > >
>>>>> > > And if R is either compiled from source (taking
the usual
>>>>> > > "configure/make/make install" route) or
installed from Conda, the
>>>>> > > permission error at step 4 doesn't happen. We
also tested creating the
>>>>> > > binary RPM package on the same machine, and the
result was the same as
>>>>> > > installing the binary package from the repo (i.e.
error).
>>>>> > >
>>>>> > > ++
>>>>> > >
>>>>> > >
>>>>> > > --
>>>>> > > Alexandre Courtiol, www.datazoogang.de
>>>>> > >
>>>>> > >         [[alternative HTML version deleted]]
>>>>> > >
>>>>> > > _______________________________________________
>>>>> > > R-SIG-Fedora mailing list
>>>>> > > R-SIG-Fedora at r-project.org
>>>>> > >
https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
>>>>> > >
>>>>> >
>>>>> >         [[alternative HTML version deleted]]
>>>>> >
>>>>> > _______________________________________________
>>>>> > R-SIG-Fedora mailing list
>>>>> > R-SIG-Fedora at r-project.org
>>>>> > https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> I?aki ?car
>>>>
>>>>
>>>>
>>>> --
>>>> Alexandre Courtiol, www.datazoogang.de
>>
>>
>>
>> --
>> Alexandre Courtiol, www.datazoogang.de


-- 
I?aki ?car

Update:

On Mon, 12 Jun 2023 at 17:40, I?aki Ucar <iucar at fedoraproject.org>
wrote:
>
> Unfortunately, I cannot reproduce the issue here. :( I mounted a smb
> share, created a symlink from ~/R, and installed dplyr without any
> issue (apart from the installation process taking ages). Fortunately,
> this rules out our binary R package as responsible for the issue you
> are experiencing, unless I'm missing something.
The experiment above was done using kio-fuse to mount the folder. Now
I've been able to reproduce the error using a regular CIFS mount. I
reproduced this both with Fedora and Debian R binaries. Then I
executed the installation step by step, and I checked that, when the
mv command is invoked, manually running mv in the console does trigger
the error too. So again, I don't think there's any issue with R.

However, I've also managed to fix the issue by adding the nolease
option to the mount command. Hope it helps.

I?aki
>
> Issue aside, may I ask what is the purpose of putting the library in a
> smb share? May I suggest [1]?
>
> [1] https://cran.r-project.org/bin/linux/fedora/#additional-packages
>
> I?aki
>
> On Mon, 12 Jun 2023 at 16:48, Tom Callaway <spotrh at gmail.com>
wrote:
> >
> > The log of you running configure on your instance allows me to compare
it to the log of when we build it as a package (you don't need to provide
that one).
> >
> > Honestly, I have no idea right now _why_ pre-built R would fail to
move files to a cifs share when a non-root user can do it. I'm hoping maybe
something will jump out as to how it configures on your setup. We aren't
modifying R's source code at all, nor are we configuring it in a way that
would affect this as far as I know.
> >
> > Does this also happen with the older R binary RPMs? If you go back to
4.2.3 or 4.2.2, does it happen?
> >
> > ~spot
> >
> >
> > On Mon, Jun 12, 2023 at 10:38?AM Alexandre Courtiol
<alexandre.courtiol at gmail.com> wrote:
> >>
> >> "is the staged installation enabled too in the cases you
don't see this issue?"
> >> -> Yes, it should since we stick to default config.
> >>
> >> "Can you provide a log of the output (including the
commandline with any options passed) for configure?"
> >> -> If we compile locally there is no issue despite not using
any option, and if we install R from binary, there is no configure log AFAIK, so
I am not sure how to meet this request.
> >>
> >> "can you provide the specific mount options for the cifs
share?"
> >> ->
sec=krb5,multiuser,mfsymlinks,user=smbuser,domain=localdomain,_netdev,noauto,x-systemd.automount
> >>
> >> ++
> >>
> >> On Mon, 12 Jun 2023 at 16:11, Tom Callaway <spotrh at
gmail.com> wrote:
> >>>
> >>> Okay, so if it's not SELinux... perhaps something about
how R is being configured is different?
> >>>
> >>> Can you provide a log of the output (including the commandline
with any options passed) for configure?
> >>>
> >>> Also, can you provide the specific mount options for the cifs
share?
> >>>
> >>> ~spot
> >>>
> >>> On Mon, Jun 12, 2023 at 9:12?AM Alexandre Courtiol
<alexandre.courtiol at gmail.com> wrote:
> >>>>
> >>>> Thanks for the tip but nope: SELinux is off on that
system...
> >>>>
> >>>> On Mon, 12 Jun 2023 at 13:24, I?aki Ucar <iucar at
fedoraproject.org> wrote:
> >>>>>
> >>>>> On Mon, 12 Jun 2023 at 13:19, Tom Callaway <spotrh
at gmail.com> wrote:
> >>>>> >
> >>>>> > Hmm, that's a weird one. Is SELinux on and
enforcing on that setup?
> >>>>>
> >>>>> I was going to bet on the same thing. :) But let me
add: if the answer
> >>>>> is affirmative, and this doesn't happen with
SELinux disabled, then
> >>>>> *the answer is NOT to disable SELinux*. Instead,
let's figure this
> >>>>> out, because the answer is to set the proper labels or
permissions.
> >>>>>
> >>>>> I?aki
> >>>>>
> >>>>> >
> >>>>> > ~spot
> >>>>> >
> >>>>> > On Mon, Jun 12, 2023, 6:55 AM Alexandre Courtiol
<
> >>>>> > alexandre.courtiol at gmail.com> wrote:
> >>>>> >
> >>>>> > > Dear Fedora-R enthusiasts,
> >>>>> > >
> >>>>> > > We are experiencing some issues with the
binary releases of R 4.3 for
> >>>>> > > Fedora 37 & 38 when running it on our
infrastructure.
> >>>>> > >
> >>>>> > > The issue is that packages won't
install, unless the (default) staged
> >>>>> > > installation process for packages is
switched off  (--no-staged-install).
> >>>>> > >
> >>>>> > > It seems to be related to the fact that we
are storing R libraries on a
> >>>>> > > drive mounted via samba share.
> >>>>> > >
> >>>>> > > Interestingly, installing R from sources
does not cause the issue.
> >>>>> > > When using other OS (e.g. Arch) on the same
infrastructure, the problem
> >>>>> > > also disappears.
> >>>>> > >
> >>>>> > > We are looking for a way out that would
allow users to install R packages
> >>>>> > > as usual and that would ideally not force
the admin to install R from
> >>>>> > > sources.
> >>>>> > >
> >>>>> > > The issue should be reproducible as follows:
> >>>>> > >
> >>>>> > > 1. Mount the share:
> >>>>> > >
> >>>>> > > $ mount -t cifs //server/share /mnt/share -o
...options...
> >>>>> > >
> >>>>> > > 2. Put R library onto the share:
> >>>>> > >
> >>>>> > > $ mkdir /mnt/share/R
> >>>>> > > $ ln -s /mnt/share/R ~/R
> >>>>> > >
> >>>>> > > 3. Install a package that has dependencies
(It doesn't matter whether the
> >>>>> > > dependencies actually have to be installed
or not. E.g. if you install all
> >>>>> > > of the dependencies of 'dplyr' and
then install 'dplyr' itself in an extra
> >>>>> > > call to 'install.packages', the
problem will still be triggered.):
> >>>>> > >
> >>>>> > > $ Rscript -e
"install.packages('dplyr', repos='https://cloud.r-project.org
> >>>>> > > ')"
> >>>>> > >
> >>>>> > > 4. The installation will fail while trying
to move the package to its final
> >>>>> > > location:
> >>>>> > >
> >>>>> > > mv: cannot move
> >>>>> > >
'/mnt/share/R/x86_64-redhat-linux-gnu-library/4.3/00LOCK-dplyr/00new/dplyr'
> >>>>> > > to
'/mnt/share/R/x86_64-redhat-linux-gnu-library/4.3/dplyr': Permission
> >>>>> > > denied
> >>>>> > > ERROR:   moving to final location failed
> >>>>> > >
> >>>>> > > 5. Moving the folder as described in the
error manually from the shell
> >>>>> > > succeeds:
> >>>>> > >
> >>>>> > > $ mv
> >>>>> > >
/mnt/share/R/x86_64-redhat-linux-gnu-library/4.3/00LOCK-dplyr/00new/dplyr
> >>>>> > >
/mnt/share/R/x86_64-redhat-linux-gnu-library/4.3/dplyr
> >>>>> > > $ # => works
> >>>>> > >
> >>>>> > > And if R is either compiled from source
(taking the usual
> >>>>> > > "configure/make/make install"
route) or installed from Conda, the
> >>>>> > > permission error at step 4 doesn't
happen. We also tested creating the
> >>>>> > > binary RPM package on the same machine, and
the result was the same as
> >>>>> > > installing the binary package from the repo
(i.e. error).
> >>>>> > >
> >>>>> > > ++
> >>>>> > >
> >>>>> > >
> >>>>> > > --
> >>>>> > > Alexandre Courtiol, www.datazoogang.de
> >>>>> > >
> >>>>> > >         [[alternative HTML version deleted]]
> >>>>> > >
> >>>>> > >
_______________________________________________
> >>>>> > > R-SIG-Fedora mailing list
> >>>>> > > R-SIG-Fedora at r-project.org
> >>>>> > >
https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
> >>>>> > >
> >>>>> >
> >>>>> >         [[alternative HTML version deleted]]
> >>>>> >
> >>>>> > _______________________________________________
> >>>>> > R-SIG-Fedora mailing list
> >>>>> > R-SIG-Fedora at r-project.org
> >>>>> >
https://stat.ethz.ch/mailman/listinfo/r-sig-fedora
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> I?aki ?car
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Alexandre Courtiol, www.datazoogang.de
> >>
> >>
> >>
> >> --
> >> Alexandre Courtiol, www.datazoogang.de
>
>
>
> --
> I?aki ?car


-- 
I?aki ?car