zfs discuss - Jun 2010 - At what level does the “zfs” directory exist?

I?ve posted a query regarding the visibility of snapshots via CIFS here
(http://opensolaris.org/jive/thread.jspa?threadID=130577&tstart=0)
however, I?m beginning to suspect that it may be a more fundamental ZFS question
so I?m asking the same question here.

At what level does the ?zfs? directory exist?

If  the ?.zfs? subdirectory only exists as the direct child of the mount point
then can someone suggest how I can make it visible lower down without requiring
me (even if it were possible for 50k users) to make each users? home folder a
file system?

By way of a background, I?m looking at the possibility of hosting our students
personal file space on OpenSolaris since the capacities required go well beyond
my budget to keep investing in our NetApp kit.
So far I?ve managed to implement the same functionality however, the visibility
of the snapshots to allow self-service file restores is a real issue which may
prevent me for going forward on this platform.

I?d appreciate any suggestions.

Thanks
Michael.
-- 
This message posted from opensolaris.org

I have done a similar deployment,

However we gave each student their own ZFS filesystem. Each of which had a
.zfs directory in it.


On 16 June 2010 08:51, MichaelHoy <michael.hoy at unn.ac.uk> wrote:
> I?ve posted a query regarding the visibility of snapshots via CIFS here (
> http://opensolaris.org/jive/thread.jspa?threadID=130577&tstart=0)
> however, I?m beginning to suspect that it may be a more fundamental ZFS
> question so I?m asking the same question here.
>
> At what level does the ?zfs? directory exist?
>
> If  the ?.zfs? subdirectory only exists as the direct child of the mount
> point then can someone suggest how I can make it visible lower down without
> requiring me (even if it were possible for 50k users) to make each users?
> home folder a file system?
>
> By way of a background, I?m looking at the possibility of hosting our
> students personal file space on OpenSolaris since the capacities required
go
> well beyond my budget to keep investing in our NetApp kit.
> So far I?ve managed to implement the same functionality however, the
> visibility of the snapshots to allow self-service file restores is a real
> issue which may prevent me for going forward on this platform.
>
> I?d appreciate any suggestions.
>
> Thanks
> Michael.
> --
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100616/ec32360b/attachment.html>

David Markey wrote:
> 
> I have done a similar deployment,
> 
> However we gave each student their own ZFS filesystem. Each of which had
> a .zfs directory in it.
Don''t host 50k filesystems on a single pool. It''s more pain
than it''s
worth.

MichaelHoy wrote:
> I?ve posted a query regarding the visibility of snapshots via CIFS here
(http://opensolaris.org/jive/thread.jspa?threadID=130577&tstart=0)
> however, I?m beginning to suspect that it may be a more fundamental ZFS
question so I?m asking the same question here.
> 
> At what level does the ?zfs? directory exist?
> 
> If  the ?.zfs? subdirectory only exists as the direct child of the mount
point then can someone suggest how I can make it visible lower down without
requiring me (even if it were possible for 50k users) to make each users? home
folder a file system?
> 
> By way of a background, I?m looking at the possibility of hosting our
students personal file space on OpenSolaris since the capacities required go
well beyond my budget to keep investing in our NetApp kit.
> So far I?ve managed to implement the same functionality however, the
visibility of the snapshots to allow self-service file restores is a real issue
which may prevent me for going forward on this platform.
> 
> I?d appreciate any suggestions.
Do you only want to share the filesystem via CIFS? Have you had a look
at the shadow_copy2 extension for samba? It maps the snapshots so windows
can access them via "previous versions" from the explorers context
menu.

--Arne
> 
> Thanks
> Michael.

On 16/06/2010 09:11, Arne Jansen wrote:
> MichaelHoy wrote:
>    
>> I?ve posted a query regarding the visibility of snapshots via CIFS here
(http://opensolaris.org/jive/thread.jspa?threadID=130577&tstart=0)
>> however, I?m beginning to suspect that it may be a more fundamental ZFS
question so I?m asking the same question here.
>>
>> At what level does the ?zfs? directory exist?
>>
>> If  the ?.zfs? subdirectory only exists as the direct child of the
mount point then can someone suggest how I can make it visible lower down
without requiring me (even if it were possible for 50k users) to make each
users? home folder a file system?
>>
>> By way of a background, I?m looking at the possibility of hosting our
students personal file space on OpenSolaris since the capacities required go
well beyond my budget to keep investing in our NetApp kit.
>> So far I?ve managed to implement the same functionality however, the
visibility of the snapshots to allow self-service file restores is a real issue
which may prevent me for going forward on this platform.
>>
>> I?d appreciate any suggestions.
>>      
> Do you only want to share the filesystem via CIFS? Have you had a look
> at the shadow_copy2 extension for samba? It maps the snapshots so windows
> can access them via "previous versions" from the explorers
context menu.
>    
btw: the CIFS service supports Windows Shadow Copies out-of-the-box.

-- 
Robert Milkowski
http://milek.blogspot.com

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Arne Jansen
> 
> Don''t host 50k filesystems on a single pool. It''s more
pain than it''s
> worth.
I assume Michael has reached this conclusion due to factors which are not
necessary to discuss here.  He has a problem, asked for help.  The above is
not helpful.

Michael, I assume you reached this conclusion (must host 50k users on a
single system) due to some logical process and/or resource constraint,
right?

Did you consider the possibility of something like DFS, and/or shadow
copies, which are included in Windows Server, and allow for
"snapshots" and
load distribution across multiple servers, using the same UNC space?
\\domainname\users\eharvey could be hosted by any number of servers, and I
as a user, would have no idea and no care which one(s) were serving the
requests.

The problem with "previous versions" is that it''s only
available to windows
clients.  If you happen to be using OSX or Ubuntu or whatever, as your cifs
client, you won''t have access to previous versions (AFAIK).

So solaris & zfs certainly have a place here.  In some environments, windows
might be the better server solution.  In some environments, osol might be
better.  And by logical process, you might be stuck putting it all on a
single server.  Or the average workload might be small enough that a single
server is perfectly adequate.

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of MichaelHoy
> 
> If  the ?.zfs? subdirectory only exists as the direct child of the
> mount point then can someone suggest how I can make it visible lower
> down without requiring me (even if it were possible for 50k users) to
> make each users? home folder a file system?
I know other people have suggested previous versions.  This might get you
somewhere.

The solution I use in my environment is:  I always "share" (via cifs)
or "export" (via nfs) the root of the filesystem.  So the
".zfs" directory is always accessible by whatever client is mounting
it.

Assuming you have an automated script or group policy which pushes the
"mount" to all the clients, you should not have much difficulty in
just mounting peoples'' home directories with a longer UNC path. 
\\server\tank\home\user instead of \\server\home\user

Also, my users often forget ".zfs" and can''t remember where
to find snapshots.  I solve this in a ridiculously simple way, which I endorse:
	ln -s .zfs/snapshot snapshot
Believe it or not, it works equally well for cifs and nfs clients.

For unix/linux clients, I also endorse zhist.  Just simplifies and accelerates
finding snapshots.

> By way of a background, I?m looking at the possibility of hosting our
> students personal file space on OpenSolaris since the capacities
> required go well beyond my budget to keep investing in our NetApp kit.
I don''t know what other options you may have.  It''s either
continue investing in the Netapp, or make the home paths work somehow with ZFS. 
I don''t think there''s anything else out there.

Somebody might say "AFS."  And hairy though it may be, it''s
possible that it may benefit you.  I don''t know.  I know only one
thing:  Anybody who has used AFS either loves it or hates it.

> > From: zfs-discuss-bounces at opensolaris.org
> [mailto:zfs-discuss-
> > bounces at opensolaris.org] On Behalf Of Arne Jansen
> > 
> > Don''t host 50k filesystems on a single pool. It''s
> more pain than it''s
> > worth.
> 
> I assume Michael has reached this conclusion due to
> factors which are not
> necessary to discuss here.  He has a problem, asked
> for help.  The above is
> not helpful.
Actually it is a helpful advice. Arne talked about a single pool i.e. zpool, not
a single system.
It''s a known fact that many ZFS filesystems in one pool is bad for boot
time.

Have a look here:
http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide


> Michael, I assume you reached this conclusion (must
> host 50k users on a
> single system) due to some logical process and/or
> resource constraint,
> right?
> 
> Did you consider the possibility of something like
> DFS, and/or shadow
> copies, which are included in Windows Server, and
> allow for "snapshots" and
> load distribution across multiple servers, using the
> same UNC space?
> \\domainname\users\eharvey could be hosted by any
> number of servers, and I
> as a user, would have no idea and no care which
> one(s) were serving the
> requests.
> 
> The problem with "previous versions" is that it''s
> only available to windows
> clients.  If you happen to be using OSX or Ubuntu or
> whatever, as your cifs
> client, you won''t have access to previous versions
> (AFAIK).
> 
> So solaris & zfs certainly have a place here.  In
> some environments, windows
> might be the better server solution.  In some
> environments, osol might be
> better.  And by logical process, you might be stuck
> putting it all on a
> single server.  Or the average workload might be
> small enough that a single
> server is perfectly adequate.
> 
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discu
> ss
>
-- 
This message posted from opensolaris.org

Thanks for the responses.

I?m sorry if this is a bit long winded but I thought that explaining the context
might assist in describing the proposed solution.

I work for a University and we have an artificially high number of user
accounts. We have applicants, current students and those that have just left
(completed course/failed). All users get personal file space regardless of
whether they use it or not hence, the need to host the personal file space of
some 50k users.
Concurrency is actually only about 3k.

It been confirmed that the .zfs subdirectory only exists as the direct child of
the mount point.

The question was then... can someone suggest how I can make it visible lower
down without requiring me (even if it were possible for 50k users) to make each
users? home folder a file system?

First thing, it?s simply not practical to have so many file systems. I?d already
tested 5k and boot time was unacceptable, never mind the other inherent
implications of such a strategy. Therefore, access to Previous Versions via
Windows is out.

Having given consideration to the responses it?s essentially confirmed that that
I need to expose a higher-level structure via a UNC path to allow access to
.zfs.

Our student user accounts are prefixed with a letter.
Therefore I intend to have 26 volumes and within each volume, a directory per
user. The smbautohome works fine and allows the path for Home Directory
attribute in AD to be nice and short e.g. \\server\a123456.

The 26 top-level volumes will each need to be shared to allow the users to
explore the snapshots via drilling down into the path e.g.
\\server\prefixa\.zfs\snapshot\Monday\a124567\.

Not ideal since currently the snapshots are visible in their default home
folder.

Edward Ned Harvey made a very good suggestion in his post regarding creating a
symbolic link in their personal file space to drop them directly into the
snapshot folder. Nice and simple so it should work well.

As an aside and perhaps I should log this as a different post, does anyone have
any experience of making Windows see the assigned user quota value and not the
physical allocation to the volume?
When we allocate personal quotas (zfs set userquota at a1234567=5g pstuds/a) the
actual amount of space reported by Windows is that of the volume and not the
user.
I know the quotas are being enforced because Windows tells me there is not
enough space when I?ve consumed the user quota. It would be nice for the user to
see 5G and not say 50TB since it?ll raise expectations and confuse them.

Thanks again for the responses.

Michael.
-- 
This message posted from opensolaris.org

On 17/06/2010 09:18, MichaelHoy wrote:
> First thing, it?s simply not practical to have so many file systems. I?d
already tested 5k and boot time was unacceptable, never mind the other inherent
implications of such a strategy. Therefore, access to Previous Versions via
Windows is out.
>    
Previous Versions should work even if you have a one large filesystems 
with all users homes as directories within.

What Solaris/OpenSolaris version did you try for the 5k test?

-- 
Robert Milkowski
http://milek.blogspot.com

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of MichaelHoy
> 
> First thing, it?s simply not practical to have so many file systems.
> I?d already tested 5k and boot time was unacceptable, never mind the
> other inherent implications of such a strategy. Therefore, access to
> Previous Versions via Windows is out.
Forgive me for not seeing the connection here.  I get the idea that boot time is
unacceptably slow with 5k filesystems or higher.  But why does that make a
problem for Previous Versions for Windows?