zfs discuss - Feb 2007 - Zfs best practice for 2U SATA iSCSI NAS

Is there a best practice guide for using zfs as a basic rackable small
storage solution?

I''m considering zfs with a 2U 12 disk Xeon based server system vs
something like a second hand FAS250.

Target enviroment is mixature of Xen or VI hosts via iSCSI and nfs/cifs.

Being able to take snapshots of running (or maybe paused) xen iscsi
vols and re-export then for cloning and remote backup replication is
important. the aspect I like about zfs is the offsite storage system
can also be generic hardware and thus much cheaper. Being able to run
a postgresql or mysql directly on the storage server has was postives
as well, although a generic storage appliance has a better admin
profile.

Some questions:
1. how stable is zfs? i''m tolarent to some sweat work to fix problems
but data loss is unacceptable
2. If drives need to be pulled and put into a new chasis does zfs
handle them having new device names and being out of order?
3. Is it possible to hot swap drives with raidz(2)
4. How does performance compare with ''brand name'' storage
systems?

Thanks
Nicholas

Hi Nicholas,

ZFS itself is very stable and very effective as fast FS in our
experience. If you browse the archives of the list you''ll see that NFS
performance is pretty acceptable, with some performance/RAM quirks
around small files:

http://www.opensolaris.org/jive/message.jspa?threadID=19858
http://www.opensolaris.org/jive/thread.jspa?threadID=18394

To my understanding  the iSCSI driver is undergoing significant
performance improvements...maybe someone close to this can help?


If by VI you are referring to VMware Infrastructure...you won''t get
any support from VMware if you''re using the iSCSI target on Solaris as
its not approved by them. Not that this is really a problem in my
experience as VMware tech support is pretty terrible anyway.

> Some questions:
> 1. how stable is zfs? i''m tolarent to some sweat work to fix
problems
> but data loss is unacceptable
We haven''t experienced any data loss, and have had some pretty nasty
things thrown at it (FC array rebooted unexpectedly).
> 2. If drives need to be pulled and put into a new chasis does zfs
> handle them having new device names and being out of order?
My understanding and experience here is yes. It''ll read the ZFS lables
off the drives/slice.
> 3. Is it possible to hot swap drives with raidz(2)
Depends on your underlying hardware. To my knowledge hot-swapping is
not dependent on the RAID-level at all.
> 4. How does performance compare with ''brand name'' storage
systems?
No clue if you''re referring to NetApp. Does anyone else know?

-J

On 2/18/07, Jason J. W. Williams <jasonjwwilliams at gmail.com> wrote:
> If by VI you are referring to VMware Infrastructure...you won''t
get
> any support from VMware if you''re using the iSCSI target on
Solaris as
> its not approved by them. Not that this is really a problem in my
> experience as VMware tech support is pretty terrible anyway.

Actually Virtual Iron, they have a nice system at the moment with live
migration of windows guest.

> Some questions:
> > 1. how stable is zfs? i''m tolarent to some sweat work to fix
problems
> > but data loss is unacceptable
>
> We haven''t experienced any data loss, and have had some pretty
nasty
> things thrown at it (FC array rebooted unexpectedly).


Nice.

Some other questions.

1. Has anyone run Xen hosts off zfs iscsi targets? Does offline migration
work?

2. What is the recommended version of Opensolaris to use at the moment that
has iscsi?  Is there a stable-like branch or is it better to stay on the N-1
update?

3. Which leads to: coming from Debian, how easy is system updates?  I
remember with OpenBSD system updates used to be a pain.

4. I assume that since Solaris can''t boot off zfs yet, that the best
option
is  two mirrored drives for the OS, and all the other drives in the pool.

5. Is there a recommended amount of system memory for (only) serving
storage? ie. is 4Gb sufficient for 10x500Gb SATA array, or will increasing
this make a big difference to performance. How about CPU? Would 2 Xeon 5130s
be sufficient for 20-40 iSCSI targets?

6. Is there a iscsi LUN limit?

Thanks
Nicholas
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070219/1f8345ac/attachment.html>

Hello Nicholas,

Monday, February 19, 2007, 11:31:50 AM, you wrote:

>

2. What is the recommended version of Opensolaris to use at the moment that has
iscsi?  Is there a stable-like branch or is it better to stay on the N-1 update?
 

3. Which leads to: coming from Debian, how easy is system updates?  I remember
with OpenBSD system updates used to be a pain.

4. I assume that since Solaris can''t boot off zfs yet, that the best
option is  two mirrored drives for the OS, and all the other drives in the
pool. 

5. Is there a recommended amount of system memory for (only) serving storage?
ie. is 4Gb sufficient for 10x500Gb SATA array, or will increasing this make a
big difference to performance. How about CPU? Would 2 Xeon 5130s be sufficient
for 20-40 iSCSI targets? 

2. if you are talking about iSCSI client it''s in a Solaris 10 release.

   IIRC correctly iSCSI server should be in U4 - otherwise use Nevada

3. you have to options to upgrade system - via patches or via upgrade.

   both are fairly easy

4. that''s what a lot of people are doing now. However even with zfsboot
code I guess lot of people

   will still mirror two drives for a system (but with zfs) and create separate
pool for all the other

   disks

5. there''s no simple answer to this question as it greatly depends on
workload and data.

   One thing you should keep in mind - Solaris *has* to boot in a 64bit mode if
you wan to

   use all that memory as a cache for zfs, so old x86 32bit CPU are not welcome

-- 

Best regards,

 Robert                            mailto:rmilkowski@task.gda.pl

                                       http://milek.blogspot.com


_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Hi Nicholas,
> Actually Virtual Iron, they have a nice system at the moment with live
> migration of windows guest.
Ah. We looked at them for some Windows DR. They do have a nice product.
> 3. Which leads to: coming from Debian, how easy is system updates?  I
> remember with OpenBSD system updates used to be a pain.
Not a pain, but coming from Debian/Gentoo not great either. Packaging
is one of the last areas that Solaris really needs an upgrade. You
might want to take a look at Nexenta, which is OpenSolaris with GNU
userland and apt-get. Works pretty well. Once installed you can update
it to Build 56 to get the iSCSI target.

-J

On 2/20/07, Jason J. W. Williams <jasonjwwilliams at gmail.com>
wrote:
>
>
> Ah. We looked at them for some Windows DR. They do have a nice product.

Just waiting for them to get iscsi and vlan support. Supposely sometime in
the next couple months.  Combined with zfs/iscsi it will make a very nice
small data center solution.


Not a pain, but coming from Debian/Gentoo not great either.
Packaging
> is one of the last areas that Solaris really needs an upgrade. You
> might want to take a look at Nexenta, which is OpenSolaris with GNU
> userland and apt-get. Works pretty well. Once installed you can update
> it to Build 56 to get the iSCSI target.

I''ve thought about this. How stable is it for just serving
(iscsi/nfs/cifs)
storage? What about when Zones are added with a db (pgdb, mydb) instance?

Nicholas
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070220/9f372568/attachment.html>

On 2/19/07, Robert Milkowski <rmilkowski at task.gda.pl> wrote:
> 5. there''s no simple answer to this question as it greatly depends
on
> workload and data.
>
>    One thing you should keep in mind - Solaris *has* to boot in a 64bit
> mode if you wan to
>
>    use all that memory as a cache for zfs, so old x86 32bit CPU are not
> welcome
>
Any rules of thumb?  ie 512Mb or 1024Mb per TB?

Nicholas
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070220/5a1d2ad2/attachment.html>

Hello Nicholas,

Tuesday, February 20, 2007, 12:55:05 AM, you wrote:

>

On 2/19/07, Robert Milkowski <rmilkowski@task.gda.pl> wrote: 

5. there''s no simple answer to this question as it greatly depends on
workload and data. 

   One thing you should keep in mind - Solaris *has* to boot in a 64bit mode if
you wan to

   use all that memory as a cache for zfs, so old x86 32bit CPU are not welcome

Any rules of thumb?  ie 512Mb or 1024Mb per TB?

Really there''s none - it''s so heavily dependent on data and
workload that someimes 512MB per TB would be even an overkill,

sometimes the difference between 4GB and 16GB is hard to notice, etc.

Generally if you can afford buy more memory. Or if you can check different
setups then

try them. Perhaps buy 2-4GB in such dimms so there''re still free ones
so you can easily add another dimms.

-- 

Best regards,

 Robert                            mailto:rmilkowski@task.gda.pl

                                       http://milek.blogspot.com


_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss