zfs discuss - Oct 2009 - group and user quotas - a temporary hack?

I am one of the much blessed university users who wishes to provide home
directories and web space to thousands of users and is being bitten by
abysmal scaling behaviour of zfs, the overhead of creating thousands of zfs
files sytems in a pool can takes days to complete.  Sharing or unsharing
them can take hours.

The creation of user and group quotas allows me to create a functional
environment but I loose many key zfs features such as snapshot control of an
individual''s files and easy self service recovery also the flexability
in
setting share parametes on an individual''s filesystem.  It is not clear
to
me how snapshots interact with user and group quotas.  The latest ZFS admin
guide contains a single line mentioning snapshots and user quotas:

However, the user or group quota is applied when a clone or a snapshot
is
> created from a file system that has a user or group quota.
>
"applied to a clone" I understand what that means, "applied to a
snapshot" -
not so clear does it mean enforced on the original dataset?  If so how do
you delegate the ability for an individual to free blocks charged against
quota that are in filesystem wide snapshots?  I''m guessing you cannot.

I truly hope that effort is being made to resolve the underlying scaling
issues but I have not seen any light in the recent "next words" talk. 
If
anyone could point me to discussions where this issue is being driven
I''d
love to find out.


Regards,
Alastair
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20091005/faa336f4/attachment.html>

Alastair Neil wrote:
>     However, the user or group quota is applied when a clone or a
>     snapshot is created from a file system that has a user or group quota.
> 
> "applied to a clone" I understand what that means, "applied
to a
> snapshot" - not so clear does it mean enforced on the original
dataset?
Since snapshots can not be modified, you''re right that
"applying" user quotas
to snapshots doesn''t make any sense.  However, the user/group
quota/used is
recorded in the snapshot, and the values (including user quotas) are the 
values at the time the snapshot was taken.  Contrast that with other 
properties (eg, quota, exec), where the snapshot does not have its own value 
for those properties; it just applies the filesystem''s current value.

--matt

On Tue, Oct 20, 2009 at 12:12 PM, Matthew Ahrens <Matthew.Ahrens at
sun.com>wrote:
> Alastair Neil wrote:
>
>>    However, the user or group quota is applied when a clone or a
>>    snapshot is created from a file system that has a user or group
quota.
>>
>> "applied to a clone" I understand what that means,
"applied to a snapshot"
>> - not so clear does it mean enforced on the original dataset?
>>
>
> Since snapshots can not be modified, you''re right that
"applying" user
> quotas to snapshots doesn''t make any sense.  However, the
user/group
> quota/used is recorded in the snapshot, and the values (including user
> quotas) are the values at the time the snapshot was taken.  Contrast that
> with other properties (eg, quota, exec), where the snapshot does not have
> its own value for those properties; it just applies the
filesystem''s current
> value.
>
> --matt
>
So users quotas are not charged for blocks in the snapshot?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20091020/a26aaefc/attachment.html>

Alastair Neil wrote:
> 
> 
> On Tue, Oct 20, 2009 at 12:12 PM, Matthew Ahrens <Matthew.Ahrens at
sun.com
> <mailto:Matthew.Ahrens at sun.com>> wrote:
> 
>     Alastair Neil wrote:
> 
>            However, the user or group quota is applied when a clone or a
>            snapshot is created from a file system that has a user or
>         group quota.
> 
>         "applied to a clone" I understand what that means,
"applied to a
>         snapshot" - not so clear does it mean enforced on the original
>         dataset?
> 
> 
>     Since snapshots can not be modified, you''re right that
"applying"
>     user quotas to snapshots doesn''t make any sense.  However, the
>     user/group quota/used is recorded in the snapshot, and the values
>     (including user quotas) are the values at the time the snapshot was
>     taken.  Contrast that with other properties (eg, quota, exec), where
>     the snapshot does not have its own value for those properties; it
>     just applies the filesystem''s current value.
> 
>     --matt
> 
> 
> So users quotas are not charged for blocks in the snapshot?
That''s correct.  Snapshot blocks don''t contribute to the
filesystem''s space
referenced / refquota / userused / userquota.

--matt