zfs discuss - Nov 2011 - Couple of questions about ZFS on laptops

Hello all,

   I am thinking about a new laptop. I see that there are
a number of higher-performance models (incidenatlly, they
are also marketed as "gamer" ones) which offer two SATA
2.5" bays and an SD flash card slot. Vendors usually
position the two-HDD bay part as either "get lots of
capacity with RAID0 over two HDDs, or get some capacity
and some performance by mixing one HDD with one SSD".
Some vendors go as far as suggesting a highest performance
with RAID0 over two SSDs.

   Now, if I were to use this for work with ZFS on an
OpenSolaris-descendant OS, and I like my data enough
to want it mirrored, but still I want an SSD performance
boost (i.e. to run VMs in real-time), I seem to have
a number of options:

1) Use a ZFS mirror of two SSDs
    - seems too pricey
2) Use a HDD with redundant data (copies=2 or mirroring
    over two partitions), and an SSD for L2ARC (+maybe ZIL)
    - possible unreliability if the only HDD breaks
3) Use a ZFS mirror of two HDDs
    - lowest performance
4) Use a ZFS mirror of two HDDs and an SD card for L2ARC.
    Perhaps add another "built-in flash card" with PCMCIA
    adapters for CF, etc.

Now, there is a couple of question points for me here.

One was raised in my recent questions about CF ports in a
Thumper. The general reply was that even high-performance
CF cards are aimed for "linear" RW patterns and may be
slower than HDDs for random access needed as L2ARCs, so
flash cards may actually lower the system performance.
I wonder if the same is the case with SD cards, and/or
if anyone encountered (and can advise) some CF/SD cards
with good random access performance (better than HDD
random IOPS). Perhaps an extra IO path can be beneficial
even if random performances are on the same scale - HDDs
would have less work anyway and can perform better with
their other tasks?

On another hand, how would current ZFS behave if someone
ejects an L2ARC device (flash card) and replaces it with
another unsuspecting card, i.e. one from a photo camera?
Would ZFS automatically replace the L2ARC device and
kill the photos, or would the cache be disabled with
no fatal implication for the pools nor for the other
card? Ultimately, when the ex-L2ARC card gets plugged
back in, would ZFS automagically attach it as the cache
device, or does this have to be done manually?


Second question regards single-HDD reliability: I can
do ZFS mirroring over two partitions/slices, or I can
configure "copies=2" for the datasets. Either way I
think I can get protection from bad blocks of whatever
nature, as long as the spindle spins. Can these two
methods be considered equivalent, or is one preferred
(and for what reason)?


Also, how do other list readers place and solve their
preferences with their OpenSolaris-based laptops? ;)

Thanks,
//Jim Klimov

On Tue, 8 Nov 2011, Jim Klimov wrote:
>
> Second question regards single-HDD reliability: I can
> do ZFS mirroring over two partitions/slices, or I can
> configure "copies=2" for the datasets. Either way I
> think I can get protection from bad blocks of whatever
> nature, as long as the spindle spins. Can these two
> methods be considered equivalent, or is one preferred
> (and for what reason)?
Using two partitions on the same disk seems to give you most of the 
headaches associated with more disks without much of the benefit.  If 
there is any minor issue, you will see zfs resilvering partitions and 
resilvering will be slow due to the drive heads flailing back and 
forth between partitions. There is also the issue that the block 
allocation is not likely to be very efficient in terms of head 
movement if two partitions are used.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

2011-11-08 23:36, Bob Friesenhahn wrote:
> On Tue, 8 Nov 2011, Jim Klimov wrote:
>>
>> Second question regards single-HDD reliability: I can
>> do ZFS mirroring over two partitions/slices, or I can
>> configure "copies=2" for the datasets. Either way I
>> think I can get protection from bad blocks of whatever
>> nature, as long as the spindle spins. Can these two
>> methods be considered equivalent, or is one preferred
>> (and for what reason)?
>
> Using two partitions on the same disk seems to give you most of the
> headaches associated with more disks without much of the benefit. If
> there is any minor issue, you will see zfs resilvering partitions and
> resilvering will be slow due to the drive heads flailing back and forth
> between partitions. There is also the issue that the block allocation is
> not likely to be very efficient in terms of head movement if two
> partitions are used.
Thanks, Bob, I figured so...
And would copies=2 save me from problems of data loss and/or
inefficient resilvering? Does all required data and metadata
get duplicated this way, so any broken sector can be amended?
I read on this list recently, that some metadata is already
"copies=2 or =3". To what extent?.. Should the trunk of the
ZFS block tree be expected always secured, even on one disk?

Thanks,
//Jim Klimov

On Wed, 9 Nov 2011, Jim Klimov wrote:
>
> Thanks, Bob, I figured so...
> And would copies=2 save me from problems of data loss and/or
> inefficient resilvering? Does all required data and metadata
> get duplicated this way, so any broken sector can be amended?
> I read on this list recently, that some metadata is already
> "copies=2 or =3". To what extent?.. Should the trunk of the
> ZFS block tree be expected always secured, even on one disk?
With only one disk partition in a vdev, then there will be no 
resilvering since there is nothing to resilver.  Metadata has always 
stored at least two copies.  It is always possible to lose the whole 
pool if the device does not work according to specification (or you 
drop the laptop on the ground).  Using copies=2 and doing a ''zfs 
scrub'' at least once after bulk data has been written should help 
avoid media read errors.  Zfs will still "resilver" blocks which 
failed to read as long as there is a redundant copy.

If you do want to increase reliability then you should mirror between 
disks, even if you feel that this will be slow.  It will still be 
faster (for reads) than using just one disk.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Jim Klimov
> 
> 1) Use a ZFS mirror of two SSDs
>     - seems too pricey
> 2) Use a HDD with redundant data (copies=2 or mirroring
>     over two partitions), and an SSD for L2ARC (+maybe ZIL)
>     - possible unreliability if the only HDD breaks
> 3) Use a ZFS mirror of two HDDs
>     - lowest performance
> 4) Use a ZFS mirror of two HDDs and an SD card for L2ARC.
>     Perhaps add another "built-in flash card" with PCMCIA
>     adapters for CF, etc.
The performance of a SSD or flash drive or SD card is almost entirely
dependent on the robustness/versatility of the built-in controller circuit.
You can rest assured that no SD card and no USB device is going to have
performance even remotely close to a decent SSD, except under the conditions
that are specifically optimized for that device.  The manufacturers, of
course, will publish their maximum specs, and the real world usage of the
device might be an order of magnitude lower.

A little while back, I performed an experiment - I went out and bought the
best rated, most expensive USB3 flash drives I could find, and I benchmarked
them against the cheapest USB2 hard drives I could find.  The hard drives
won by a clear margin, like 4x to 8x faster, except when running large
sequential "dd" to/from the raw flash device on the first boot - in
which
case the flash won by a small margin (like 10%)

Given your hardware limitations, the only way to go fast is to use a SSD,
and the only way to go fast with redundancy is to use a mirror of two
SSD''s.

If you don''t go for the SSD''s, then your HDD''s will
be the second fastest
option.  Do not put any SD card into the mix.  It will only hurt you.

> Second question regards single-HDD reliability: I can
> do ZFS mirroring over two partitions/slices, or I can
> configure "copies=2" for the datasets. Either way I
> think I can get protection from bad blocks of whatever
> nature, as long as the spindle spins. Can these two
> methods be considered equivalent, or is one preferred
> (and for what reason)?
I would opt for the copies=2 method, because it''s reconfigurable if you
want, and it''s designed to work within a single pool, so it more
closely
resembles your actual usage.  If you mirror across two partitions on the
same disk, there may be unintended performance consequences because nobody
expected you to do that when they wrote the code.

> Also, how do other list readers place and solve their
> preferences with their OpenSolaris-based laptops? ;)
I''m sorry to say, there is no ZFS-based OS and no laptop hardware that
I
consider to be a reliable combination.  Of course I haven''t tested them
all,
but I don''t believe in any of them because it''s
"unintended, uncharted,
untested, unsupported."  I think you''ll find the best support for
this
subject on the openindiana mailing lists.

After oracle acquired sun, most of the home users and laptop users left the
opensolaris mailing lists in favor of the openindiana lists.  The people
that remain here are primarily focused on enterprise and servers.

Some laptops have pc card and expresscard slots, and you can get an adapter for
sd card, so you could set up your os non mirrored and just set up home on a pair
of sd cards. Something like
http://www.amazon.com/Sandisk-SDAD109A11-Digital-Card-Express/dp/B000W3QLLW

I''ve done this in the past, variations of this, including using a
partition and a usb stick:

http://solarisdesktop.blogspot.com/2007/02/stick-to-zfs-or-laptop-with-mirrored.html
Wow, where did the time go, that was almost 5 years ago...

Anyway, i pretty much ditched carrying the laptop, the current one i have is too
heavy (m4400). But it does run really nicely sol11 and openindiana. The m4400 is
set up with 2 drives, not mirrored. I''m tempted to put a sandforce
based ssd for faster booting and better zfs perf for demos. Then i have an
sdcard and expresscard adapter for sd. This gives me 16gb mirrored for my
documents, which is plenty.

Francois
Sent from my iPad

On Nov 8, 2011, at 12:05 PM, Jim Klimov <jimklimov at cos.ru> wrote:
> Hello all,
> 
>  I am thinking about a new laptop. I see that there are
> a number of higher-performance models (incidenatlly, they
> are also marketed as "gamer" ones) which offer two SATA
> 2.5" bays and an SD flash card slot. Vendors usually
> position the two-HDD bay part as either "get lots of
> capacity with RAID0 over two HDDs, or get some capacity
> and some performance by mixing one HDD with one SSD".
> Some vendors go as far as suggesting a highest performance
> with RAID0 over two SSDs.
> 
>  Now, if I were to use this for work with ZFS on an
> OpenSolaris-descendant OS, and I like my data enough
> to want it mirrored, but still I want an SSD performance
> boost (i.e. to run VMs in real-time), I seem to have
> a number of options:
> 
> 1) Use a ZFS mirror of two SSDs
>   - seems too pricey
> 2) Use a HDD with redundant data (copies=2 or mirroring
>   over two partitions), and an SSD for L2ARC (+maybe ZIL)
>   - possible unreliability if the only HDD breaks
> 3) Use a ZFS mirror of two HDDs
>   - lowest performance
> 4) Use a ZFS mirror of two HDDs and an SD card for L2ARC.
>   Perhaps add another "built-in flash card" with PCMCIA
>   adapters for CF, etc.
> 
> Now, there is a couple of question points for me here.
> 
> One was raised in my recent questions about CF ports in a
> Thumper. The general reply was that even high-performance
> CF cards are aimed for "linear" RW patterns and may be
> slower than HDDs for random access needed as L2ARCs, so
> flash cards may actually lower the system performance.
> I wonder if the same is the case with SD cards, and/or
> if anyone encountered (and can advise) some CF/SD cards
> with good random access performance (better than HDD
> random IOPS). Perhaps an extra IO path can be beneficial
> even if random performances are on the same scale - HDDs
> would have less work anyway and can perform better with
> their other tasks?
> 
> On another hand, how would current ZFS behave if someone
> ejects an L2ARC device (flash card) and replaces it with
> another unsuspecting card, i.e. one from a photo camera?
> Would ZFS automatically replace the L2ARC device and
> kill the photos, or would the cache be disabled with
> no fatal implication for the pools nor for the other
> card? Ultimately, when the ex-L2ARC card gets plugged
> back in, would ZFS automagically attach it as the cache
> device, or does this have to be done manually?
> 
> 
> Second question regards single-HDD reliability: I can
> do ZFS mirroring over two partitions/slices, or I can
> configure "copies=2" for the datasets. Either way I
> think I can get protection from bad blocks of whatever
> nature, as long as the spindle spins. Can these two
> methods be considered equivalent, or is one preferred
> (and for what reason)?
> 
> 
> Also, how do other list readers place and solve their
> preferences with their OpenSolaris-based laptops? ;)
> 
> Thanks,
> //Jim Klimov
> 
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

On Nov 9, 2011, at 6:08 PM, Francois Dion wrote:
> Some laptops have pc card and expresscard slots, and you can get an adapter
for sd card, so you could set up your os non mirrored and just set up home on a
pair of sd cards. Something like
> http://www.amazon.com/Sandisk-SDAD109A11-Digital-Card-Express/dp/B000W3QLLW
> 
> I''ve done this in the past, variations of this, including using a
partition and a usb stick:
SDcard is suitable for boot *only* if it is connected via USB.  While the
drivers I wrote for SDHCI work fine for using media, you can''t boot off
it generally -- usually the laptop BIOS simply lacks the support needed to see
these.

It used to be that CompactFlash was a preferred option, but I think CF is
falling out of favor these days.

	- Garrett
> 
>
http://solarisdesktop.blogspot.com/2007/02/stick-to-zfs-or-laptop-with-mirrored.html
> Wow, where did the time go, that was almost 5 years ago...
> 
> Anyway, i pretty much ditched carrying the laptop, the current one i have
is too heavy (m4400). But it does run really nicely sol11 and openindiana. The
m4400 is set up with 2 drives, not mirrored. I''m tempted to put a
sandforce based ssd for faster booting and better zfs perf for demos. Then i
have an sdcard and expresscard adapter for sd. This gives me 16gb mirrored for
my documents, which is plenty.
> 
> Francois
> Sent from my iPad
> 
> On Nov 8, 2011, at 12:05 PM, Jim Klimov <jimklimov at cos.ru> wrote:
> 
>> Hello all,
>> 
>> I am thinking about a new laptop. I see that there are
>> a number of higher-performance models (incidenatlly, they
>> are also marketed as "gamer" ones) which offer two SATA
>> 2.5" bays and an SD flash card slot. Vendors usually
>> position the two-HDD bay part as either "get lots of
>> capacity with RAID0 over two HDDs, or get some capacity
>> and some performance by mixing one HDD with one SSD".
>> Some vendors go as far as suggesting a highest performance
>> with RAID0 over two SSDs.
>> 
>> Now, if I were to use this for work with ZFS on an
>> OpenSolaris-descendant OS, and I like my data enough
>> to want it mirrored, but still I want an SSD performance
>> boost (i.e. to run VMs in real-time), I seem to have
>> a number of options:
>> 
>> 1) Use a ZFS mirror of two SSDs
>>  - seems too pricey
>> 2) Use a HDD with redundant data (copies=2 or mirroring
>>  over two partitions), and an SSD for L2ARC (+maybe ZIL)
>>  - possible unreliability if the only HDD breaks
>> 3) Use a ZFS mirror of two HDDs
>>  - lowest performance
>> 4) Use a ZFS mirror of two HDDs and an SD card for L2ARC.
>>  Perhaps add another "built-in flash card" with PCMCIA
>>  adapters for CF, etc.
>> 
>> Now, there is a couple of question points for me here.
>> 
>> One was raised in my recent questions about CF ports in a
>> Thumper. The general reply was that even high-performance
>> CF cards are aimed for "linear" RW patterns and may be
>> slower than HDDs for random access needed as L2ARCs, so
>> flash cards may actually lower the system performance.
>> I wonder if the same is the case with SD cards, and/or
>> if anyone encountered (and can advise) some CF/SD cards
>> with good random access performance (better than HDD
>> random IOPS). Perhaps an extra IO path can be beneficial
>> even if random performances are on the same scale - HDDs
>> would have less work anyway and can perform better with
>> their other tasks?
>> 
>> On another hand, how would current ZFS behave if someone
>> ejects an L2ARC device (flash card) and replaces it with
>> another unsuspecting card, i.e. one from a photo camera?
>> Would ZFS automatically replace the L2ARC device and
>> kill the photos, or would the cache be disabled with
>> no fatal implication for the pools nor for the other
>> card? Ultimately, when the ex-L2ARC card gets plugged
>> back in, would ZFS automagically attach it as the cache
>> device, or does this have to be done manually?
>> 
>> 
>> Second question regards single-HDD reliability: I can
>> do ZFS mirroring over two partitions/slices, or I can
>> configure "copies=2" for the datasets. Either way I
>> think I can get protection from bad blocks of whatever
>> nature, as long as the spindle spins. Can these two
>> methods be considered equivalent, or is one preferred
>> (and for what reason)?
>> 
>> 
>> Also, how do other list readers place and solve their
>> preferences with their OpenSolaris-based laptops? ;)
>> 
>> Thanks,
>> //Jim Klimov
>> 
>> _______________________________________________
>> zfs-discuss mailing list
>> zfs-discuss at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss