zfs discuss - Oct 2008 - Performance with tens of thousands of zfs filesystems

I have been reading this forum for a little while, and am interested in more
information about the performance of ZFS when creating large amounts of
filesystems.   We are considering using ZFS for the user''s home
folders, and this could potentially be 30''000 filesystems, and if using
snapshots that number would be multiplied by x snapshots as well.

I am a bit nervous after reading this forum, that the performance when getting
huge numbers of filesystems is not very good.  Using hours to boot the server,
and possibly weeks to make the filesystems seems "not right".

Any official input on how this will be in the upcoming release of Solaris 10?

Yours sincerly,
Morten-Christian Bernson
Solaris System Administrator
University of Bergen
Norway
mcb at uib.no
-- 
This message posted from opensolaris.org

On Tue, Oct 28, 2008 at 9:27 AM, Morten-Christian Bernson <mcb at uib.no>
wrote:
> I have been reading this forum for a little while, and am interested in
more information about the performance of ZFS when creating large amounts of
filesystems.   We are considering using ZFS for the user''s home
folders, and this could potentially be 30''000 filesystems, and if using
snapshots that number would be multiplied by x snapshots as well.
>
> I am a bit nervous after reading this forum, that the performance when
getting huge numbers of filesystems is not very good.  Using hours to boot the
server, and possibly weeks to make the filesystems seems "not right".
You''ll have to solve this issue by using multiple ZFS servers so that
the number of filesystems per server is a reasonably good fit with
ZFS'' current capabilities.  OTOH  ZFS is improving in this arena over
time and, even with multiple servers, ZFS will still provide a cost
effective solution.   In any case, with 5 ZFS servers, would it not be
better to have only 20% of your user community affected if a server
were to have a catastrophic failure?  (rhetorical question)
> Any official input on how this will be in the upcoming release of Solaris
10?
You won''t get any "official" input on Solaris (the commercial
product)
related topics on this list - which is dedicated to OpenSolaris.  But,
to give you something that resembles an answer, Sol10 Update 6 is
based on Build 88 AFAIR.  So looking at the features/performance of
that release will give you a good feel for Sol 10U6 .. the usual
disclaimers YMMV etc.

Regards,

-- 
Al Hopper  Logical Approach Inc,Plano,TX al at logical-approach.com
                   Voice: 972.379.2133 Timezone: US CDT
OpenSolaris Governing Board (OGB) Member - Apr 2005 to Mar 2007
http://www.opensolaris.org/os/community/ogb/ogb_2005-2007/

> You''ll have to solve this issue by using multiple ZFS
> servers so that the number of filesystems per server is a reasonably
> good fit with ZFS'' current capabilities.  OTOH  ZFS is improving
in
> this arena over time and, even with multiple servers, ZFS will still
> provide a cost effective solution.   In any case, with 5 ZFS
> servers, would it not be better to have only 20% of your user community
> affected if a server were to have a catastrophic failure?  (rhetorical
> question)
Of course we are going to have several servers, but we were hoping to be able to
share all the folders from one at a time.  We want to use the mirroring
capabilities in ZFS to take disk from two san''s in different buildings,
and mirror them up in the zfs pool.  We will also have a second server in
another building ready to do an import of the pool, highjack the IP and share
the files.  This way we would be safe in case of san failure, server failure and
problems in one of the buildings.

It all cooks down to if ZFS is capable of sharing this amount of filesystemes on
one server though.  If it takes a week to make the filesystems, and hours to
boot the server (an I guess importing the pool on the other server), that would
be a big drawback...
> You won''t get any "official" input on Solaris (the
> commercial product) related topics on this list - which is dedicated to
> OpenSolaris.  But, to give you something that resembles an answer, Sol10
> Update 6 is based on Build 88 AFAIR.  So looking at the
> features/performance of that release will give you a good feel for Sol 10U6
> .. the usual disclaimers YMMV etc.
I know OpenSolaris is not officially supported by SUN in any way, but most of
the development seems to be going on for OpenSolaris, and then gets implemented
into Solaris when it''s stable and ready.  This is why I wanted comments
on the status of this in this forum.  I was hoping for some reassuring that this
would be ok in the upcoming release of Solaris :)

Today it''s 30. Oct, and sun.com still says "Note: Solaris 10 10/08
availability is planned for the end of October.".  I am waiting in
anticipation, and can''t wait to test this out in the new release :)
-- 
This message posted from opensolaris.org

Morten-Christian Bernson wrote:
>> You''ll have to solve this issue by using multiple ZFS servers
so
>> that the number of filesystems per server is a reasonably good fit
>> with ZFS'' current capabilities.  OTOH  ZFS is improving in
this
>> arena over time and, even with multiple servers, ZFS will still 
>> provide a cost effective solution.   In any case, with 5 ZFS 
>> servers, would it not be better to have only 20% of your user
>> community affected if a server were to have a catastrophic failure?
>> (rhetorical question)
> 
> Of course we are going to have several servers, but we were hoping to
> be able to share all the folders from one at a time.  We want to use
> the mirroring capabilities in ZFS to take disk from two san''s in
> different buildings, and mirror them up in the zfs pool.  We will
> also have a second server in another building ready to do an import
> of the pool, highjack the IP and share the files.  This way we would
> be safe in case of san failure, server failure and problems in one of
> the buildings.
> 
> It all cooks down to if ZFS is capable of sharing this amount of
> filesystemes on one server though.  If it takes a week to make the
> filesystems, and hours to boot the server (an I guess importing the
> pool on the other server), that would be a big drawback...
> 
>> You won''t get any "official" input on Solaris (the
commercial
>> product) related topics on this list - which is dedicated to 
>> OpenSolaris.  But, to give you something that resembles an answer,
>> Sol10 Update 6 is based on Build 88 AFAIR.  So looking at the 
>> features/performance of that release will give you a good feel for
>> Sol 10U6 .. the usual disclaimers YMMV etc.
> 
> I know OpenSolaris is not officially supported by SUN in any way, but
> most of the development seems to be going on for OpenSolaris, and
> then gets implemented into Solaris when it''s stable and ready. 
This
> is why I wanted comments on the status of this in this forum.  I was
> hoping for some reassuring that this would be ok in the upcoming
> release of Solaris :)
	OpenSolaris (as a distribution) is ABSOLUTELY supported by Sun. Take a 
look at the datasheet from opensolaris.com and sun.com

http://www.opensolaris.com/opensolaris_datasheet.pdf
http://www.sun.com/service/opensolaris/index.jsp

-Phil

-- 
      ______
     /_____/\      /----------------------------------------------------\
    /____ \\ \     |    Phillip Wagstrom, Area System Support Engineer  |
   /_____\ \\ /    |----------------------------------------------------|
  /_____/ \/ / /   | Enterprise Services       | 7760 France Avenue So. |
._____/ /   \//\  | Sun Microsystems, Inc.    | Suite 950              |
\_____\//\   / /  | phillip.wagstrom at Sun.COM  | Bloomington, MN 55435  |
  \_____/ / /\ /   | Ph. 612.486.4610 / x66731 | FAX 612.486.4610       |
   \_____/ \\ \    \----------------------------------------------------/
    \_____\ \\
     \_____\/

On Thu, 30 Oct 2008, Phillip Wagstrom -- Area SSE wrote:
>
> 	OpenSolaris (as a distribution) is ABSOLUTELY supported by Sun. Take a
> look at the datasheet from opensolaris.com and sun.com
Does Sun now offer patches for OpenSolaris?  Note that Sun also 
"supports" Linux and Windows on their hardware.

Happyness is a fresh upgrade ...

Bob
=====================================Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

Bob Friesenhahn wrote:
> On Thu, 30 Oct 2008, Phillip Wagstrom -- Area SSE wrote:
>> 	OpenSolaris (as a distribution) is ABSOLUTELY supported by Sun. Take a
>> look at the datasheet from opensolaris.com and sun.com
> 
> Does Sun now offer patches for OpenSolaris? 
since when does "support" equate to "patches" (only)?
There''s much more to
support than just supplying (or even creating!) patches.

(oh, btw: wasn''t IPS created in part to get away from the whole patch
...
ermm ... issue? ;-)

Michael
-- 
Michael Schuster        http://blogs.sun.com/recursion
Recursion, n.: see ''Recursion''