zfs discuss - Jun 2012 - History of EPERM for unlink() of directories on ZFS?

Does someone know the history which led to the EPERM for unlink() of
directories on ZFS? Why was this done this way, and not something like
allowing the unlink and execute it on the next scrub or remount?

Lionel

>Does someone know the history which led to the EPERM for unlink() of
>directories on ZFS? Why was this done this way, and not something like
>allowing the unlink and execute it on the next scrub or remount?

It''s not about the unlink(), it''s about the link() and
unlink().
But not allowing link & unlink, you force the filesystem to contain only 
trees and not graphs.

It also allows you to create directories were ".." points to a
directory
were the inode cannot be found, simply because it was just removed.

The support for link() on directories in ufs has always given issues
and would create problems fsck couldn''t fix.

To be honest, I think we should also remove this from all other
filesystems and I think ZFS was created this way because all modern
filesystems do it that way.

Casper

I don''t know the precise history, but I think its a mistake to permit
direct link() or unlink() of directories.  I do note that on BSD (MacOS at
least) unlink returns EPERM if the executing user is not superuser.  I do see
that the man page for unlink() says this on illumos:

                     The  named   file   is   a   directory   and
                     {PRIV_SYS_LINKDIR}  is  not  asserted in the
                     effective set of the calling process, or the
                     filesystem  implementation  does not support
                     unlink() or unlinkat() on directories.

I can''t imagine why you''d *ever* want to support unlink() of a
*directory* -- what''s the use case for it anyway (outside of filesystem
repair)?

Garrett D''Amore
garrett at damore.org



On Jun 25, 2012, at 2:23 AM, Lionel Cons wrote:
> Does someone know the history which led to the EPERM for unlink() of
> directories on ZFS? Why was this done this way, and not something like
> allowing the unlink and execute it on the next scrub or remount?
> 
> Lionel
> 
> 
> -------------------------------------------
> illumos-developer
> Archives: https://www.listbox.com/member/archive/182179/=now
> RSS Feed:
https://www.listbox.com/member/archive/rss/182179/21239177-c925e33f
> Modify Your Subscription:
https://www.listbox.com/member/?member_id=21239177&id_secret=21239177-4dba8197
> Powered by Listbox: http://www.listbox.com

The decision to not support link(2) of directories was very deliberate - it
is an abomination that never should have been allowed in the first place.
My guess is that the behavior of unlink(2) on directories is a direct
side-effect of that (if link isn''t supported, then why support
unlink?).
Also worth noting that ZFS also doesn''t let you open(2) directories and
read(2) from them, something (I believe) UFS does allow.

- Eric

On Mon, Jun 25, 2012 at 10:40 AM, Garrett D''Amore <garrett at
damore.org>wrote:
> I don''t know the precise history, but I think its a mistake to
permit
> direct link() or unlink() of directories.  I do note that on BSD (MacOS at
> least) unlink returns EPERM if the executing user is not superuser.  I do
> see that the man page for unlink() says this on illumos:
>
>                     The  named   file   is   a   directory   and
>                     {PRIV_SYS_LINKDIR}  is  not  asserted in the
>                     effective set of the calling process, or the
>                     filesystem  implementation  does not support
>                     unlink() or unlinkat() on directories.
>
> I can''t imagine why you''d *ever* want to support unlink()
of a *directory*
> -- what''s the use case for it anyway (outside of filesystem
repair)?
>
> Garrett D''Amore
> garrett at damore.org
>
>
>
> On Jun 25, 2012, at 2:23 AM, Lionel Cons wrote:
>
> > Does someone know the history which led to the EPERM for unlink() of
> > directories on ZFS? Why was this done this way, and not something like
> > allowing the unlink and execute it on the next scrub or remount?
> >
> > Lionel
> >
> >
> > -------------------------------------------
> > illumos-developer
> > Archives: https://www.listbox.com/member/archive/182179/=now
> > RSS Feed:
> https://www.listbox.com/member/archive/rss/182179/21239177-c925e33f
> > Modify Your Subscription: https://www.listbox.com/member/?&
> > Powered by Listbox: http://www.listbox.com
>
>
>
> -------------------------------------------
> illumos-developer
> Archives: https://www.listbox.com/member/archive/182179/=now
> RSS Feed:
> https://www.listbox.com/member/archive/rss/182179/21175057-f8151d0d
> Modify Your Subscription:
>
https://www.listbox.com/member/?member_id=21175057&id_secret=21175057-02786781
> Powered by Listbox: http://www.listbox.com
>


-- 
Eric Schrock
Delphix
http://blog.delphix.com/eschrock

275 Middlefield Road, Suite 50
Menlo Park, CA 94025
http://www.delphix.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20120625/6ffc0f29/attachment.html>

>The decision to not support link(2) of directories was very deliberate - it
>is an abomination that never should have been allowed in the first place.
>My guess is that the behavior of unlink(2) on directories is a direct
>side-effect of that (if link isn''t supported, then why support
unlink?).
>Also worth noting that ZFS also doesn''t let you open(2) directories
and
>read(2) from them, something (I believe) UFS does allow.
In the very beginning, mkdir(1) was a set-uid application; it used
"mknod" to make a directory and then created a link from
	newdir to newdir/.
and from
	"." to newdir/..

Traditionally, we was only allowed for the superuser and when
we added privileges a special privileges was added.

I think we should remove it for the other filesystems.

Casper

Eric Schrock <eric.schrock at delphix.com> wrote:
> The decision to not support link(2) of directories was very deliberate - it
> is an abomination that never should have been allowed in the first place.
> My guess is that the behavior of unlink(2) on directories is a direct
> side-effect of that (if link isn''t supported, then why support
unlink?).
> Also worth noting that ZFS also doesn''t let you open(2)
directories and
> read(2) from them, something (I believe) UFS does allow.
Link/unlink on directories is not a property of UFS.

UFS has been designed without that feature, but it has been added by AT&T
with
SVr4.

J?rg

-- 
 EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
       js at cs.tu-berlin.de                (uni)  
       joerg.schilling at fokus.fraunhofer.de (work) Blog:
http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

On Mon, Jun 25, 2012 at 11:19 AM, <Casper.Dik at oracle.com>
wrote:
>
>
> In the very beginning, mkdir(1) was a set-uid application; it used
> "mknod" to make a directory and then created a link from
>        newdir to newdir/.
> and from
>        "." to newdir/..
>
Interesting, guess you learn something new every day :-)

http://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd/mkdir.c

Thanks,

- Eric

-- 
Eric Schrock
Delphix
http://blog.delphix.com/eschrock

275 Middlefield Road, Suite 50
Menlo Park, CA 94025
http://www.delphix.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20120625/e7fd15be/attachment.html>

Eric Schrock <eric.schrock at delphix.com> wrote:
> On Mon, Jun 25, 2012 at 11:19 AM, <Casper.Dik at oracle.com> wrote:
> >
> >
> > In the very beginning, mkdir(1) was a set-uid application; it used
> > "mknod" to make a directory and then created a link from
> >        newdir to newdir/.
> > and from
> >        "." to newdir/..
> >
>
> Interesting, guess you learn something new every day :-)
>
> http://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd/mkdir.c
This was a nice way to become superuser those days.....

Just run a loop to make a directory in /tmp and run another program that tries 
to remove the directory and replace it by a hadlink to /etc/passwd. Mkdir(1) 
then did a "chown <you> /etc/passwd"... We tried this and it
took aprox.
30 minutes to become super user this way.

And BSD introduced the syscall mkdir(2) to fix this and this is is why UFS was 
not designed to support link(2) in directories.

BTW: to implement mkdir(2), there was a new struct dirtemplate in the kernel 
with the following comment:

/* 
 * A virgin directory (no blushing please). 
 */ 
struct dirtemplate mastertemplate = { 
        0, 12, 1, ".", 
        0, DIRBLKSIZ - 12, 2, ".." 
}; 


This is the first time where Sun verified not to have humor, as Sun removed 
that comment...

J?rg

-- 
 EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
       js at cs.tu-berlin.de                (uni)  
       joerg.schilling at fokus.fraunhofer.de (work) Blog:
http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

On 25 June 2012 11:33,  <Casper.Dik at oracle.com>
wrote:
>
>>Does someone know the history which led to the EPERM for unlink() of
>>directories on ZFS? Why was this done this way, and not something like
>>allowing the unlink and execute it on the next scrub or remount?
>
>
> It''s not about the unlink(), it''s about the link() and
unlink().
> But not allowing link & unlink, you force the filesystem to contain
only
> trees and not graphs.
>
> It also allows you to create directories were ".." points to a
directory
> were the inode cannot be found, simply because it was just removed.
>
> The support for link() on directories in ufs has always given issues
> and would create problems fsck couldn''t fix.
>
> To be honest, I think we should also remove this from all other
> filesystems and I think ZFS was created this way because all modern
> filesystems do it that way.
This may be wrong way to go if it breaks existing applications which
rely on this feature. It does break applications in our case.

Anyway, we''ve added this to the list of mandatory features and see
what we can procure with that.

Lionel

>> To be honest, I think we should also remove this from all other
>> filesystems and I think ZFS was created this way because all modern
>> filesystems do it that way.
>
>This may be wrong way to go if it breaks existing applications which
>rely on this feature. It does break applications in our case.
I don''t think this isn''t supported on most Linux filesystems
either.

What do you use it for?
>Anyway, we''ve added this to the list of mandatory features and see
>what we can procure with that.

Not much?

I''d suggest whether you can restructure your code and work without
this.

For one, it requires the application to run with root (older versions) or
with specific privileges which aren''t, e.g., available in non-global
zones.

Casper

On 26 June 2012 14:51,  <Casper.Dik at oracle.com>
wrote:
>
>>> To be honest, I think we should also remove this from all other
>>> filesystems and I think ZFS was created this way because all modern
>>> filesystems do it that way.
>>
>>This may be wrong way to go if it breaks existing applications which
>>rely on this feature. It does break applications in our case.
>
> I don''t think this isn''t supported on most Linux
filesystems either.
>
> What do you use it for?
>
>>Anyway, we''ve added this to the list of mandatory features and
see
>>what we can procure with that.
>
>
> Not much?
We''ve already asked our Netapp representative. She said it''s
not hard
to add that.
> I''d suggest whether you can restructure your code and work without
this.
It would require touching code for which we don''t have sources anymore
(people gone, too). It would also require to create hard links to the
results files directly, which means linking 15000+ files per directory
with a minimum of 30000 directories. Each day (this is CERN after
all).

The other way around would be to throw the SPARC machines away and go
with Netapp.
> For one, it requires the application to run with root (older versions) or
> with specific privileges which aren''t, e.g., available in
non-global zones.
The code runs with privileges anyway.

Lionel

>We''ve already asked our Netapp representative. She said
it''s not hard
>to add that.
And symlinks don''t work for this?  I''m amazed because
we''re talking about
the same file system.   Or is it that the code you have does the 
hardlinking?


If you want this rfo Oracle, you would need to talk to an Oracle 
representative and not in a mailing list (for illumos email will work I 
suppose)
>> I''d suggest whether you can restructure your code and work
without this.
>
>It would require touching code for which we don''t have sources
anymore
>(people gone, too). It would also require to create hard links to the
>results files directly, which means linking 15000+ files per directory
>with a minimum of 30000 directories. Each day (this is CERN after
>all).
I''m assuming then that it is the code for which you don''t have
the source
which does the hardlinking?

I''m still not sure why symlinks won''t work or for that matter
loopback
mounts.

Casper

On 06/26/12 05:46 AM, Lionel Cons wrote:
> On 25 June 2012 11:33,  <Casper.Dik at oracle.com> wrote:
>> To be honest, I think we should also remove this from all other
>> filesystems and I think ZFS was created this way because all modern
>> filesystems do it that way.
> 
> This may be wrong way to go if it breaks existing applications which
> rely on this feature. It does break applications in our case.
Existing applications rely on the ability to corrupt UFS filesystems?
Sounds horrible.

-- 
	-Alan Coopersmith-              alan.coopersmith at oracle.com
	 Oracle Solaris Engineering - http://blogs.oracle.com/alanc

On Tue, Jun 26, 2012 at 9:44 AM, Alan Coopersmith
<alan.coopersmith at oracle.com> wrote:
> On 06/26/12 05:46 AM, Lionel Cons wrote:
>> On 25 June 2012 11:33, ?<Casper.Dik at oracle.com> wrote:
>>> To be honest, I think we should also remove this from all other
>>> filesystems and I think ZFS was created this way because all modern
>>> filesystems do it that way.
>>
>> This may be wrong way to go if it breaks existing applications which
>> rely on this feature. It does break applications in our case.
>
> Existing applications rely on the ability to corrupt UFS filesystems?
> Sounds horrible.
My guess is that the OP just wants unlink() of an empty directory to
be the same as rmdir() of the same.  Or perhaps they want unlink() of
a non-empty directory to result in a recursive rm...  But if they
really want hardlinks to directories, then yeah, that''s horrible.

Nico
--

On Tue, Jun 26, 2012 at 10:41:14AM -0500, Nico Williams
wrote:
> On Tue, Jun 26, 2012 at 9:44 AM, Alan Coopersmith
> <alan.coopersmith at oracle.com> wrote:
> > On 06/26/12 05:46 AM, Lionel Cons wrote:
> >> On 25 June 2012 11:33, ?<Casper.Dik at oracle.com> wrote:
> >>> To be honest, I think we should also remove this from all
other
> >>> filesystems and I think ZFS was created this way because all
modern
> >>> filesystems do it that way.
> >>
> >> This may be wrong way to go if it breaks existing applications
which
> >> rely on this feature. It does break applications in our case.
> >
> > Existing applications rely on the ability to corrupt UFS filesystems?
> > Sounds horrible.
> 
> My guess is that the OP just wants unlink() of an empty directory to
> be the same as rmdir() of the same.  Or perhaps they want unlink() of
> a non-empty directory to result in a recursive rm...  But if they
> really want hardlinks to directories, then yeah, that''s horrible.
This all sounds like a good use for LD_PRELOAD and a tiny library
that intercepts and modernizes system calls.

-- 
-Gary Mills-		-refurb-		-Winnipeg, Manitoba, Canada-

On Tue, Jun 26, 2012 at 8:12 AM, Lionel Cons
<lionelcons1972 at googlemail.com> wrote:
> On 26 June 2012 14:51, ?<Casper.Dik at oracle.com> wrote:
> We''ve already asked our Netapp representative. She said
it''s not hard
> to add that.
Did NetApp tell you that they''ll add support for using the NFSv4 LINK
operation on source objects that are directories?!  I''d be extremely
surprised!  Or did they only tell you that they''ll add support for
using the NFSv4 REMOVE operation on non-empty directories?  The latter
is definitely feasible (although it could fail due to share deny OPENs
of files below, say, but hey).  The former is... not sane.
>> I''d suggest whether you can restructure your code and work
without this.
>
> It would require touching code for which we don''t have sources
anymore
> (people gone, too). It would also require to create hard links to the
> results files directly, which means linking 15000+ files per directory
> with a minimum of 30000 directories. Each day (this is CERN after
> all).
Oh, I see.  But you still don''t want hardlinks to directories!
Instead you might be able to use LD_PRELOAD to emulate the behavior
that the application wants.  The app is probably implementing
rename(), so just detect the sequence and map it to an actual
rename(2).
> The other way around would be to throw the SPARC machines away and go
> with Netapp.
So Solaris is just a fileserver here?

Nico
--

On Jun 26, 2012, at 4:46 PM, Lionel Cons wrote:
> On 25 June 2012 11:33,  <Casper.Dik at oracle.com> wrote:
>> 
>> 
>> To be honest, I think we should also remove this from all other
>> filesystems and I think ZFS was created this way because all modern
>> filesystems do it that way.
I agree with Casper here.  This is a historical accident that would be nice to
fix.
> 
> This may be wrong way to go if it breaks existing applications which
> rely on this feature. It does break applications in our case.
Really?!?  unlink on directories is an incredibly bad idea.  What is your
application?  Do you know why it is doing this?
> 
> Anyway, we''ve added this to the list of mandatory features and see
> what we can procure with that.
Rule out ZFS, and most other filesystems unless you are happy to have your
application run with elevated privilege.

	- Garrett
> 
> Lionel
> 
> 
> -------------------------------------------
> illumos-developer
> Archives: https://www.listbox.com/member/archive/182179/=now
> RSS Feed:
https://www.listbox.com/member/archive/rss/182179/21239177-c925e33f
> Modify Your Subscription:
https://www.listbox.com/member/?member_id=21239177&id_secret=21239177-4dba8197
> Powered by Listbox: http://www.listbox.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20120627/dd5c7dc6/attachment-0001.html>