zfs discuss - Aug 2009 - Zfs improvements to compression in Solaris 10?

Are there any improvements in the Solaris 10 pipeline for how 
compression is implemented?

I changed my USB-based backup pool to use gzip compression (with 
default level 6) rather than the lzjb compression which was used 
before.  When lzjb compression was used, it would case the X11 session 
to become jerky and unresponsive while data was copied to the backup 
pool.  With gzip compression, the system just "goes away" for as long 
as eight seconds at a time.  It goes away for so long that the 
Perfmeter tool pops up a window saying that it can''t contact 
localhost.

I have done some simple testing to verify that the issue is not 
specific to the X11 server since this little test loop shows the (up 
to) 8 second delays in execution:

while true
do
   date
   sleep 1
done > times.txt

Does current OpenSolaris do better in this area?

Bob
--
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

You seem to be hitting :

http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6586537

The fix is available in OpenSolaris build 115 and later not for Solaris 10 yet.

--
Prabahar.

On Tue, Aug 04, 2009 at 10:08:37AM -0500, Bob Friesenhahn
wrote:
> Are there any improvements in the Solaris 10 pipeline for how compression 
> is implemented?
>
> I changed my USB-based backup pool to use gzip compression (with default 
> level 6) rather than the lzjb compression which was used before.  When 
> lzjb compression was used, it would case the X11 session to become jerky 
> and unresponsive while data was copied to the backup pool.  With gzip 
> compression, the system just "goes away" for as long as eight
seconds at a
> time.  It goes away for so long that the Perfmeter tool pops up a window 
> saying that it can''t contact localhost.
>
> I have done some simple testing to verify that the issue is not specific 
> to the X11 server since this little test loop shows the (up to) 8 second 
> delays in execution:
>
> while true
> do
>   date
>   sleep 1
> done > times.txt
>
> Does current OpenSolaris do better in this area?
>
> Bob
> --
> Bob Friesenhahn
> bfriesen at simple.dallas.tx.us,
http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

On Tue, 4 Aug 2009, Prabahar Jeyaram wrote:
> You seem to be hitting :
>
> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6586537
>
> The fix is available in OpenSolaris build 115 and later not for Solaris 10
yet.
It is interesting that this is a simple thread priority issue.  The 
system has a ton of available CPU but the higher priority compression 
thread seems to cause scheduling lockout.  The Perfmeter tool shows 
that compression is a very short-term spike in CPU. Of course since 
Perfmeter and other apps stop running, it might be missing some sample 
data.

I could put the X11 server into the real-time scheduling class but 
hate to think about what would happen as soon as Firefox visits a web 
site. :-)

Compression is only used for the intermittently-used backup pool so it 
would be a shame to reduce overall system performance for the rest of 
the time.

Do you know if this fix is planned to be integrated into a future 
Solaris 10 update?

Bob
--
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

Hi Bob,

Bob Friesenhahn wrote:
> On Tue, 4 Aug 2009, Prabahar Jeyaram wrote:
>
>> You seem to be hitting :
>>
>> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6586537
>>
>> The fix is available in OpenSolaris build 115 and later not for 
>> Solaris 10 yet.
>
> It is interesting that this is a simple thread priority issue.  The 
> system has a ton of available CPU but the higher priority compression 
> thread seems to cause scheduling lockout.  The Perfmeter tool shows 
> that compression is a very short-term spike in CPU. Of course since 
> Perfmeter and other apps stop running, it might be missing some sample 
> data.
>
> I could put the X11 server into the real-time scheduling class but 
> hate to think about what would happen as soon as Firefox visits a web 
> site. :-)
I can understand wanting to be careful about running Xorg real time, but 
what does this have to
do with firefox?  Firefox will still run in the interactive class.

max

On Tue, Aug 04, 2009 at 01:01:40PM -0500, Bob Friesenhahn
wrote:
> On Tue, 4 Aug 2009, Prabahar Jeyaram wrote:
>
>> You seem to be hitting :
>>
>> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6586537
>>
>> The fix is available in OpenSolaris build 115 and later not for Solaris
10 yet.
>
> It is interesting that this is a simple thread priority issue.  The  
> system has a ton of available CPU but the higher priority compression  
> thread seems to cause scheduling lockout.  The Perfmeter tool shows that 
> compression is a very short-term spike in CPU. Of course since Perfmeter 
> and other apps stop running, it might be missing some sample data.
>
> I could put the X11 server into the real-time scheduling class but hate to 
> think about what would happen as soon as Firefox visits a web site. :-)
>
> Compression is only used for the intermittently-used backup pool so it  
> would be a shame to reduce overall system performance for the rest of the 
> time.
>
> Do you know if this fix is planned to be integrated into a future Solaris 
> 10 update?
>
Yes. It is planned for S10U9. 

--
Prabahar.
> Bob
> --
> Bob Friesenhahn
> bfriesen at simple.dallas.tx.us,
http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

Prabahar Jeyaram wrote:
> You seem to be hitting :
>
> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6586537
>
> The fix is available in OpenSolaris build 115 and later not for Solaris 10
yet.
>
>   
Have you got more information on how was it fixed?

-- 
Robert Milkowski
http://milek.blogspot.com

Le 4 ao?t 09 ? 20:25, Prabahar Jeyaram a ?crit :
> On Tue, Aug 04, 2009 at 01:01:40PM -0500, Bob Friesenhahn wrote:
>> On Tue, 4 Aug 2009, Prabahar Jeyaram wrote:
>>
>>> You seem to be hitting :
>>>
>>> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6586537
>>>
>>> The fix is available in OpenSolaris build 115 and later not for  
>>> Solaris 10 yet.
>>
>> It is interesting that this is a simple thread priority issue.  The
>> system has a ton of available CPU but the higher priority compression
>> thread seems to cause scheduling lockout.  The Perfmeter tool shows  
>> that
>> compression is a very short-term spike in CPU. Of course since  
>> Perfmeter
>> and other apps stop running, it might be missing some sample data.
>>
>> I could put the X11 server into the real-time scheduling class but  
>> hate to
>> think about what would happen as soon as Firefox visits a web  
>> site. :-)
>>
>> Compression is only used for the intermittently-used backup pool so  
>> it
>> would be a shame to reduce overall system performance for the rest  
>> of the
>> time.
>>
>> Do you know if this fix is planned to be integrated into a future  
>> Solaris
>> 10 update?
>>
>
> Yes. It is planned for S10U9.

In the mean time, is there a patch available for Solaris 10?
I can''t find it on sunsolve.

Thanks,

Ga?tan

-- 
Ga?tan Lehmann
Biologie du D?veloppement et de la Reproduction
INRA de Jouy-en-Josas (France)
tel: +33 1 34 65 29 66    fax: 01 34 65 29 09
http://voxel.jouy.inra.fr  http://www.itk.org
http://www.mandriva.org  http://www.bepo.fr

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: Ceci est une signature ?lectronique PGP
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20091030/f7c02c18/attachment.bin>

On Fri, Oct 30, 2009 at 09:48:39AM +0100, Ga?tan Lehmann
wrote:
> 
> Le 4 ao?t 09 ? 20:25, Prabahar Jeyaram a ?crit :
> 
> >On Tue, Aug 04, 2009 at 01:01:40PM -0500, Bob Friesenhahn wrote:
> >>On Tue, 4 Aug 2009, Prabahar Jeyaram wrote:
> >>
> >>>You seem to be hitting :
> >>>
>
>>>http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6586537
> >>>
> >>>The fix is available in OpenSolaris build 115 and later not for
> >>>Solaris 10 yet.
> >>
> >>It is interesting that this is a simple thread priority issue.  The
> >>system has a ton of available CPU but the higher priority
compression
> >>thread seems to cause scheduling lockout.  The Perfmeter tool
> >>shows that
> >>compression is a very short-term spike in CPU. Of course since
> >>Perfmeter
> >>and other apps stop running, it might be missing some sample data.
> >>
> >>I could put the X11 server into the real-time scheduling class
> >>but hate to
> >>think about what would happen as soon as Firefox visits a web
> >>site. :-)
> >>
> >>Compression is only used for the intermittently-used backup pool
> >>so it
> >>would be a shame to reduce overall system performance for the
> >>rest of the
> >>time.
> >>
> >>Do you know if this fix is planned to be integrated into a future
> >>Solaris
> >>10 update?
> >>
> >
> >Yes. It is planned for S10U9.
> 
> 
> In the mean time, is there a patch available for Solaris 10?
NO. Not yet.
> I can''t find it on sunsolve.
> 
--
Prabahar.
> Thanks,
> 
> Ga?tan
> 
> -- 
> Ga?tan Lehmann
> Biologie du D?veloppement et de la Reproduction
> INRA de Jouy-en-Josas (France)
> tel: +33 1 34 65 29 66    fax: 01 34 65 29 09
> http://voxel.jouy.inra.fr  http://www.itk.org
> http://www.mandriva.org  http://www.bepo.fr
>

On Fri, 30 Oct 2009, Ga?tan Lehmann wrote:
>> 
>> Yes. It is planned for S10U9.
>
>
> In the mean time, is there a patch available for Solaris 10?
> I can''t find it on sunsolve.
Notice that the fix for this requires adding a new kernel scheduling 
class with default lower priority than user processes, but with the 
ability to raise the priority if user processes continue to hog all 
CPU.  This means that it requires more than a simple "zfs" fix.

Bob
--
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

There is no official patch yet. Please contact your Sun Support Channel if you
need
an interim patch for S10 till Official patch is released.

Thanks,
Prabahar.


On Fri, Oct 30, 2009 at 09:48:39AM +0100, Ga?tan Lehmann
wrote:
> 
> Le 4 ao?t 09 ? 20:25, Prabahar Jeyaram a ?crit :
> 
> >On Tue, Aug 04, 2009 at 01:01:40PM -0500, Bob Friesenhahn wrote:
> >>On Tue, 4 Aug 2009, Prabahar Jeyaram wrote:
> >>
> >>>You seem to be hitting :
> >>>
>
>>>http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6586537
> >>>
> >>>The fix is available in OpenSolaris build 115 and later not for
> >>>Solaris 10 yet.
> >>
> >>It is interesting that this is a simple thread priority issue.  The
> >>system has a ton of available CPU but the higher priority
compression
> >>thread seems to cause scheduling lockout.  The Perfmeter tool
> >>shows that
> >>compression is a very short-term spike in CPU. Of course since
> >>Perfmeter
> >>and other apps stop running, it might be missing some sample data.
> >>
> >>I could put the X11 server into the real-time scheduling class
> >>but hate to
> >>think about what would happen as soon as Firefox visits a web
> >>site. :-)
> >>
> >>Compression is only used for the intermittently-used backup pool
> >>so it
> >>would be a shame to reduce overall system performance for the
> >>rest of the
> >>time.
> >>
> >>Do you know if this fix is planned to be integrated into a future
> >>Solaris
> >>10 update?
> >>
> >
> >Yes. It is planned for S10U9.
> 
> 
> In the mean time, is there a patch available for Solaris 10?
> I can''t find it on sunsolve.
> 
> Thanks,
> 
> Ga?tan
> 
> -- 
> Ga?tan Lehmann
> Biologie du D?veloppement et de la Reproduction
> INRA de Jouy-en-Josas (France)
> tel: +33 1 34 65 29 66    fax: 01 34 65 29 09
> http://voxel.jouy.inra.fr  http://www.itk.org
> http://www.mandriva.org  http://www.bepo.fr
>