zfs code - Apr 2009 - PSARC/2009/204 ZFS user/group quotas & space accounting

On Sat, Apr 18, 2009 at 06:05:56PM -0700, Matthew.Ahrens at sun.com
wrote:
> Author: Matthew Ahrens <Matthew.Ahrens at Sun.COM>
> Repository: /hg/onnv/onnv-gate
> Latest revision: f41cf682d0d3e3cf5c4ec17669b903ae621ef882
> Total changesets: 1
> Log message:
> PSARC/2009/204 ZFS user/group quotas & space accounting
Great to see this functionality integrated, but if I read the diff
properly, zfs allow/unallow command will now call Python scripts from
inside zfs utility? Is that right? This is quite important for me,
because if that''s true this means I''ve to remove delegations
from
FreeBSD, because we don''t have Python in base system.
> 6827260 assertion failed in arc_read(): hdr == pbuf->b_hdr
I see this assertion beeing removed from arc.c, can you give more
details on how the actual bug causing this assertion was fixed?
(We were seeing this panic on FreeBSD/ZFS.)

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!

Pawel Jakub Dawidek wrote:
> On Sat, Apr 18, 2009 at 06:05:56PM -0700, Matthew.Ahrens at sun.com wrote:
>> Author: Matthew Ahrens <Matthew.Ahrens at Sun.COM>
>> Repository: /hg/onnv/onnv-gate
>> Latest revision: f41cf682d0d3e3cf5c4ec17669b903ae621ef882
>> Total changesets: 1
>> Log message:
>> PSARC/2009/204 ZFS user/group quotas & space accounting
> 
> Great to see this functionality integrated, but if I read the diff
> properly, zfs allow/unallow command will now call Python scripts from
> inside zfs utility? Is that right? This is quite important for me,
> because if that''s true this means I''ve to remove
delegations from
> FreeBSD, because we don''t have Python in base system.
> 
Yes, this is true, both user quotas and delegations now use python
for their CLI processing.  Its likely that other ZFS CLI subcommands
will go this direction as well.  Its unfortunate that Python isn''t
available in the base FreeBSD... is it possible that you could bundle
it with the ZFS distro (or make it a dependency for the install)?
>> 6827260 assertion failed in arc_read(): hdr == pbuf->b_hdr
> 
> I see this assertion beeing removed from arc.c, can you give more
> details on how the actual bug causing this assertion was fixed?
> (We were seeing this panic on FreeBSD/ZFS.)
> 
We introduced the b_lock around the arc_read_nolock() call in arc_read
and also use this lock in arc_release(), see bug (6732083).  The other
reason you might hit this assertion is because of the assignment to
the local ''hdr'' variable before we hold the b_lock in
arc_read() -- Matt
removed both the assignment and the assertion.

-Mark

Mark Maybee wrote:
> Pawel Jakub Dawidek wrote:
>> On Sat, Apr 18, 2009 at 06:05:56PM -0700, Matthew.Ahrens at sun.com
wrote:
>>> Author: Matthew Ahrens <Matthew.Ahrens at Sun.COM>
>>> Repository: /hg/onnv/onnv-gate
>>> Latest revision: f41cf682d0d3e3cf5c4ec17669b903ae621ef882
>>> Total changesets: 1
>>> Log message:
>>> PSARC/2009/204 ZFS user/group quotas & space accounting
>>
>> Great to see this functionality integrated, but if I read the diff
>> properly, zfs allow/unallow command will now call Python scripts from
>> inside zfs utility? Is that right? This is quite important for me,
>> because if that''s true this means I''ve to remove
delegations from
>> FreeBSD, because we don''t have Python in base system.
>>
> Yes, this is true, both user quotas and delegations now use python
> for their CLI processing.  Its likely that other ZFS CLI subcommands
> will go this direction as well.  Its unfortunate that Python isn''t
> available in the base FreeBSD... is it possible that you could bundle
> it with the ZFS distro (or make it a dependency for the install)?
Wow, I always thought Solaris was the last to distribute new tools!

Yep.  In the short term, you could maintain the old allow/unallow C code. 
The kernel interfaces haven''t changed, and no new functionality has
been
added, so that should work fine.

You could also use the python code, but notify people that certain 
subcommands will only work once you install the python package.

--matt

On Tue, Apr 21, 2009 at 10:04:53AM -0600, Matthew Ahrens
wrote:
> Mark Maybee wrote:
> >Pawel Jakub Dawidek wrote:
> >>On Sat, Apr 18, 2009 at 06:05:56PM -0700, Matthew.Ahrens at sun.com
wrote:
> >>>Author: Matthew Ahrens <Matthew.Ahrens at Sun.COM>
> >>>Repository: /hg/onnv/onnv-gate
> >>>Latest revision: f41cf682d0d3e3cf5c4ec17669b903ae621ef882
> >>>Total changesets: 1
> >>>Log message:
> >>>PSARC/2009/204 ZFS user/group quotas & space accounting
> >>
> >>Great to see this functionality integrated, but if I read the diff
> >>properly, zfs allow/unallow command will now call Python scripts
from
> >>inside zfs utility? Is that right? This is quite important for me,
> >>because if that''s true this means I''ve to remove
delegations from
> >>FreeBSD, because we don''t have Python in base system.
> >>
> >Yes, this is true, both user quotas and delegations now use python
> >for their CLI processing.  Its likely that other ZFS CLI subcommands
> >will go this direction as well.  Its unfortunate that Python
isn''t
> >available in the base FreeBSD... is it possible that you could bundle
> >it with the ZFS distro (or make it a dependency for the install)?
It depends if Python is needed for zfs tools to compile. If not I think
it is fair to just print a warning that one needs to install Python to
be able to use particular commands.
> Wow, I always thought Solaris was the last to distribute new tools!
:) Well, at some point (during FreeBSD 4.x) we had perl in the base. It
was decided to kick it out from the base and never repeat that mistake:)

The reasons I gathered were as follows:
- Perl API was unstable (Python might be different...),
- It was a real PITA to keep it up-to-date in the base,
- OS release tends to be around much longer than perl/python versions
  and people always wanted to have the latest one; all in all people
  were installing newer version of perl and it was confusing to have two
  perl version - one (older) from the base and another from the ports
  collection,

I wish you to be more successful with Python:)
> Yep.  In the short term, you could maintain the old allow/unallow C code. 
> The kernel interfaces haven''t changed, and no new functionality
has been
> added, so that should work fine.
> 
> You could also use the python code, but notify people that certain 
> subcommands will only work once you install the python package.
Yes, that''s probably the best I can do.

Thank you both.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-code/attachments/20090422/dc5abf92/attachment.bin>

Pawel Jakub Dawidek wrote:
> The reasons I gathered were as follows:
> - Perl API was unstable (Python might be different...),
> - It was a real PITA to keep it up-to-date in the base,
> - OS release tends to be around much longer than perl/python versions
>   and people always wanted to have the latest one; all in all people
>   were installing newer version of perl and it was confusing to have two
>   perl version - one (older) from the base and another from the ports
>   collection,
> 
> I wish you to be more successful with Python:)
Given the packaging system for the OpenSolaris distribution is written 
in Python we know we will always have Python available :-)

--
Darren J Moffat

On Tue, Apr 21, 2009 at 7:04 PM, Matthew Ahrens <Matthew.Ahrens at
sun.com> wrote:
>
> Wow, I always thought Solaris was the last to distribute new tools!
BTW, why python 2.4 and not python 2.6 ?
I was under impression that when 2.6 was integrated the intention was
to migrate existing things to 2.6 and for new stuff to go directly to
2.6

-- 
Regards,
        Cyril