ZFS is very exciting technology. While talking about filesystems causes most of my colleagues'' eyes to glaze over, this ZFS stuff keeps me up at night. I''m going to be building an Athlon box to host about 8 300GB drives or so in RAID-Z. My question is what kind of CPU load does ZFS take compared to a traditional filesystem? If I want to build a simple box for doing just NAS, what kind CPU power will i need to run ZFS sufficiently? Where on the processing power scale does the CPU start to become a bottleneck? What about RAM? I can''t imagine that a filesystem will have significant resource requirements, but ZFS just does so many cool things. Thanks, This message posted from opensolaris.org
>I''m going to be building an Athlon box to host about 8 300GB drives or >so in RAID-Z.>My question is what kind of CPU load does ZFS take compared to a >traditional filesystem?>If I want to build a simple box for doing just NAS, what kind CPU >power will i need to run ZFS suf ficiently? Where on the processing >power scale does the CPU start to become a bottleneck?You have soem choice in how much CPU you make ZFS use, e.g., you can turn on compression which makes CPU use worse.>What about RAM? I can''t imagine that a filesystem will have >significant resource requirements, bu t ZFS just does so many cool >things.Filesystems need RAM to do caching an dbuffering; but the thing ZFS needs more than physical memory is *virtual* memory. And the latter means that you need to run ZFS on a 64 bit CPU. Casper
As a rule of thumb, assume one 2Ghz Opteron for every 200MBytes/s, for the ZFS and NFS stack. Most of the time, even a 1.8Ghz Althon64 is more then enough to drive the stack, assuming you only have a single gigabit ethernet as the ingress. Casper''s point about 64-bit''s is very important. ZFS needs to map the cache into the kernel''s address space, which is very limited in the 32-bit world, thus the suggested requirement for 64-bit CPUs. Typically, more physical memory is a better gain than faster CPUs for low end file servers. Richard. On Tue, Jan 17, 2006 at 08:28:31AM +0100, Casper.Dik at sun.com wrote:> > >I''m going to be building an Athlon box to host about 8 300GB drives or > >so in RAID-Z. > > >My question is what kind of CPU load does ZFS take compared to a > >traditional filesystem? > > >If I want to build a simple box for doing just NAS, what kind CPU > >power will i need to run ZFS suf ficiently? Where on the processing > >power scale does the CPU start to become a bottleneck? > > You have soem choice in how much CPU you make ZFS use, e.g., you > can turn on compression which makes CPU use worse. > > >What about RAM? I can''t imagine that a filesystem will have > >significant resource requirements, bu t ZFS just does so many cool > >things. > > Filesystems need RAM to do caching an dbuffering; but the thing > ZFS needs more than physical memory is *virtual* memory. > > And the latter means that you need to run ZFS on a 64 bit CPU. > > Casper > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss-- :-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-: Richard Mc Dougall : Richard.McDougall at Sun.COM Performance and Availability : x31542 Engineering : http://devnull.eng Sun Microsystems Inc : +1 650 352 6438 :-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:
> As a rule of thumb, assume one 2Ghz Opteron for every > 200MBytes/s, > > Casper''s point about 64-bit''s is very important. ZFS > needs to map the > cache into the kernel''s address space, which is very > limited in the > 32-bit world, thus the suggested requirement for > 64-bit CPUs.Well, don''t tell my 7-year odl Pentium that! Performance isn''t great, but what did I expect with such an old system? Seems to run OK. Rainer This message posted from opensolaris.org
Kyle McDonald
2006-Jan-19 15:49 UTC
[zfs-discuss] ZFS on-disk changes from NV_29 to NV_31?
Hi all.
Is there any reason why exporting a pool from NV_29, and jumpstarting
(or live upgrading) to NV_31, and then importing the pool won''t work?
-Kyle
On Thu, Jan 19, 2006 at 10:49:05AM -0500, Kyle McDonald wrote:> Is there any reason why exporting a pool from NV_29, and jumpstarting > (or live upgrading) to NV_31, and then importing the pool won''t work?Not to my knowledge. Unless, of course, some of the disks get stomped on (or disappear) between the export and import. --Bill
Darren J Moffat
2006-Jan-19 16:29 UTC
[zfs-discuss] ZFS on-disk changes from NV_29 to NV_31?
On Thu, 2006-01-19 at 15:49, Kyle McDonald wrote:> Hi all. > > Is there any reason why exporting a pool from NV_29, and jumpstarting > (or live upgrading) to NV_31, and then importing the pool won''t work?I''ve done it and it does work. In fact I didn''t even bother to export and reimport the pool I just did the live upgrade with the file systems still mounted (since my home dir is on them). -- Darren J Moffat
Casper.Dik at Sun.COM
2006-Jan-19 17:19 UTC
[zfs-discuss] ZFS on-disk changes from NV_29 to NV_31?
>Is there any reason why exporting a pool from NV_29, and jumpstarting >(or live upgrading) to NV_31, and then importing the pool won''t work?Did that, no issues. Export/Import not necessary (except with a fresh install) Casper
Kyle McDonald
2006-Jan-19 17:50 UTC
[zfs-discuss] ZFS on-disk changes from NV_29 to NV_31?
Casper.Dik at sun.com wrote:>> Is there any reason why exporting a pool from NV_29, and jumpstarting >> (or live upgrading) to NV_31, and then importing the pool won''t work? >> > > Did that, no issues. Export/Import not necessary (except with a fresh > install) >Thanks everyone. I went the re-jumpstart route. The import ran perfectly. This stuff is great! keep up the good work! -Kyle
>> Casper''s point about 64-bit''s is very important. ZFS > needs to map the > cache into the kernel''s address space, which is very > limited in the > 32-bit world, thus the suggested requirement for > 64-bit CPUs. >I''m a newbie to ZFS. Can some explain this point a bit deeper. If I try to run ZFS on a 32-bit system will it just be slower or is the maximum storage pool size actually limited by the 32-but address space? This message posted from opensolaris.org
On Wed, May 03, 2006 at 07:39:30AM -0700, Tom Smith wrote:> > Casper''s point about 64-bit''s is very important. ZFS needs to map > > the cache into the kernel''s address space, which is very limited in > > the 32-bit world, thus the suggested requirement for 64-bit CPUs. > > I''m a newbie to ZFS. Can some explain this point a bit deeper. If I > try to run ZFS on a 32-bit system will it just be slower or is the > maximum storage pool size actually limited by the 32-but address > space?Only the cache size is limited by the 32-bit address space, thus (potentially) affecting performance. The amount of space in the storage pool is not limited. --matt
> > I''m a newbie to ZFS. Can some explain this point a bit deeper. If I > > try to run ZFS on a 32-bit system will it just be slower or is the > > maximum storage pool size actually limited by the 32-but address > > space? > > Only the cache size is limited by the 32-bit address space, thus > (potentially) affecting performance.And this cache is zfs''s "ARC cache"? If yes: Doesn''t the ARC cache allocate all its data from the kernel''s heap? And since the kernel''s heap is limited to 512 mbytes on 32-bit machines and maybe 50% of the heap could be allocated by the ARC cache - the rest is typically in use by other kernel code - this would result in an upper limit of ~ 256 mbytes of ARC cached data on any 32-bit machine (even when the 32-bit machine has 4 GB of physical memory installed)? This message posted from opensolaris.org