llvm dev - Nov 2017 - Experiment on how to improve our temporary file handing.

On Mon, Nov 13, 2017 at 01:46:32PM -0800, Davide Italiano
wrote:
> On Mon, Nov 13, 2017 at 11:48 AM, Rafael Avila de Espindola
> <rafael.espindola at gmail.com> wrote:
> > Davide Italiano <davide.italiano at gmail.com> writes:
> >
> >>> I couldn't find any support for this on FreeBSD.
> >>>
> >>
> >> AFAIK FreeBSD supports some variant of /proc that should map Linux
> >> (although the mapping isn't 1:1).
> >> Does it lack support for this? Thanks for looking into this, BTW!
> >
> > O_TMPFILE is the main thing that seems to be missing.
> >
> 
> It doesn't seem unreasonable to ask FreeBSD to implement this (at
> least for ufs/zfs), maybe Ed/Kostik have thoughts about it.
Can you please explain, how exactly O_TMPFILE functionality you asking
for relates to the /proc ? Do you want to re-link O_TMPFILE-opened
inodes into named files ?

Konstantin Belousov <kostikbel at gmail.com> writes:
> On Mon, Nov 13, 2017 at 01:46:32PM -0800, Davide Italiano wrote:
>> On Mon, Nov 13, 2017 at 11:48 AM, Rafael Avila de Espindola
>> <rafael.espindola at gmail.com> wrote:
>> > Davide Italiano <davide.italiano at gmail.com> writes:
>> >
>> >>> I couldn't find any support for this on FreeBSD.
>> >>>
>> >>
>> >> AFAIK FreeBSD supports some variant of /proc that should map
Linux
>> >> (although the mapping isn't 1:1).
>> >> Does it lack support for this? Thanks for looking into this,
BTW!
>> >
>> > O_TMPFILE is the main thing that seems to be missing.
>> >
>> 
>> It doesn't seem unreasonable to ask FreeBSD to implement this (at
>> least for ufs/zfs), maybe Ed/Kostik have thoughts about it.
>
> Can you please explain, how exactly O_TMPFILE functionality you asking
> for relates to the /proc ? Do you want to re-link O_TMPFILE-opened
> inodes into named files ?
The relation to proc seems to be a linux peculiarity.

What we want in the end is to implement "-o foo" in lld with the
following guarantees:

1 foo is only created with the final content.
2 We don't leave a temporary file behind on a power failure.
3 Less critical but nice: if foo already exists, it is replaced
  atomically.

Using mkstemp + rename we get 1 and 3.

On linux we can use O_TMPFILE, but the only way for a non privileged
process to then give a name to that fd is:

linkat(AT_FDCWD, "/proc/self/fd/<num>", AT_FDCWD,
"destination",
       AT_SYMLINK_FOLLOW)

which has two problem: It requires proc to be mounted and it is not
atomic if the destination already exists.

A system with O_TMPFILE which allows some form of renameat to
atomically give that fd a name would be perfect.

Cheers,
Rafael

On Mon, Nov 13, 2017 at 03:29:32PM -0800, Rafael Avila de Espindola
wrote:
> Konstantin Belousov <kostikbel at gmail.com> writes:
> 
> > On Mon, Nov 13, 2017 at 01:46:32PM -0800, Davide Italiano wrote:
> >> On Mon, Nov 13, 2017 at 11:48 AM, Rafael Avila de Espindola
> >> <rafael.espindola at gmail.com> wrote:
> >> > Davide Italiano <davide.italiano at gmail.com> writes:
> >> >
> >> >>> I couldn't find any support for this on FreeBSD.
> >> >>>
> >> >>
> >> >> AFAIK FreeBSD supports some variant of /proc that should
map Linux
> >> >> (although the mapping isn't 1:1).
> >> >> Does it lack support for this? Thanks for looking into
this, BTW!
> >> >
> >> > O_TMPFILE is the main thing that seems to be missing.
> >> >
> >> 
> >> It doesn't seem unreasonable to ask FreeBSD to implement this
(at
> >> least for ufs/zfs), maybe Ed/Kostik have thoughts about it.
> >
> > Can you please explain, how exactly O_TMPFILE functionality you asking
> > for relates to the /proc ? Do you want to re-link O_TMPFILE-opened
> > inodes into named files ?
> 
> The relation to proc seems to be a linux peculiarity.
> 
> What we want in the end is to implement "-o foo" in lld with the
> following guarantees:
> 
> 1 foo is only created with the final content.
> 2 We don't leave a temporary file behind on a power failure.
> 3 Less critical but nice: if foo already exists, it is replaced
>   atomically.
> 
> Using mkstemp + rename we get 1 and 3.
> 
> On linux we can use O_TMPFILE, but the only way for a non privileged
> process to then give a name to that fd is:
> 
> linkat(AT_FDCWD, "/proc/self/fd/<num>", AT_FDCWD,
"destination",
>        AT_SYMLINK_FOLLOW)
> 
> which has two problem: It requires proc to be mounted and it is not
> atomic if the destination already exists.
This is quite different request from O_TMPFILE. The tmpfile can be
implemented in a day or two for UFS. The ability to enter the inode
referenced by a file descriptor, into the filesystem namespace at a
given location, was proposed several times and rejected, I believe. The
cause is that it potentially allows to gain additional access rights.

Imagine that a process obtained a file descriptor only opened for read,
e.g. by passing over unix domain socket. If it is possible to link
its inode, you can re-open it limited by access permissions on the
inode. Inode may have rw rights, but its current containing directory
disallowing the walk.

This is especially important for capabilities-based sandbox environments,
like capsicum.
> 
> A system with O_TMPFILE which allows some form of renameat to
> atomically give that fd a name would be perfect.
I believe that #2 is not that critical.