zfs discuss - Sep 2010 - Configuration questions for Home File Server (CPU cores, dedup, checksum)?

I am working on a home file server.  After reading a wide range of blogs and
forums, I have a few questions that are still not clear to me....

1.  Is there a benefit in having quad core CPU (e.g. Athlon II X4 vs X2)? All of
the web blogs seem to suggest using lower-wattage dual core CPUs.  But; with the
recent advent of dedup, SHA256 checksum, etc., I am now wondering if opensolaris
is better served with quad core.

2.  For checksum on ZFS datasets, is there value in using Fletcher4 or SHA256
instead of just using the default (which Evil Tuning Guide says is Fletcher2,
others say it is Fletcher4)?  And, should I consider using different checksums
for different ZFS datasets (e.g. irreplaceable documents and digital photos vs.
backups from other machines).  I think the answer was that for ZFS volumes, the
checksum is only to detect an error and not for reconstructing data and so the
default was fine.  {Also, can you confirm the default is now Flectcher 4?}

3.  Should I consider using dedup if my server has only 8Gb of RAM?  Or, will
that not be enough to hold the DDT?  In which case, should I add L2ARC / ZIL or
am I better to just skip using dedup on a home file server?

Thanks,
Craig

----------------
My config:

[u]Use:[/u] Home File Server supporting PC and Mac machines -- audio/video
files, home directories (documents & photos), and backup images.  Will not
run any other services.

[u]Objective:[/u]  adequate performance for home use; maximize protection for
documents and photos; minimize administrative upkeep.

[u]Configuration:[/u]
OpenSolaris B134
AMD X2 260 / Asus MA4785TD-V EVO
8Gb ECC RAM (Crucial CT25672BA1339)
Intel EXPI9301CT NIC
LSI SAS9211-8i
Boot Disk:  WD Scorpio Black 250Gb (2-way mirror)
Media Pool:  WD Caviar Blue 640Gb (2-way mirror)
Mass Storage (File & Backup) Pool:  WD Carviar Blue 640Gb (9 disk Raid-Z3)
-- 
This message posted from opensolaris.org

Craig,

I''m sure the other home file server users will comment on your gear
and any possible benefit of a L2ARC or separate log device...

Use the default checksum which is fletcher4, I fixed the tuning guide 
reference, skip dedup for now. Keep things as simple as possible.

Thanks,

Cindy

On 09/07/10 14:58, Craig Stevenson wrote:
> I am working on a home file server.  After reading a wide range of blogs
and forums, I have a few questions that are still not clear to me....
> 
> 1.  Is there a benefit in having quad core CPU (e.g. Athlon II X4 vs X2)?
All of the web blogs seem to suggest using lower-wattage dual core CPUs.  But;
with the recent advent of dedup, SHA256 checksum, etc., I am now wondering if
opensolaris is better served with quad core.
> 
> 2.  For checksum on ZFS datasets, is there value in using Fletcher4 or
SHA256 instead of just using the default (which Evil Tuning Guide says is
Fletcher2, others say it is Fletcher4)?  And, should I consider using different
checksums for different ZFS datasets (e.g. irreplaceable documents and digital
photos vs. backups from other machines).  I think the answer was that for ZFS
volumes, the checksum is only to detect an error and not for reconstructing data
and so the default was fine.  {Also, can you confirm the default is now
Flectcher 4?}
> 
> 3.  Should I consider using dedup if my server has only 8Gb of RAM?  Or,
will that not be enough to hold the DDT?  In which case, should I add L2ARC /
ZIL or am I better to just skip using dedup on a home file server?
> 
> Thanks,
> Craig
> 
> ----------------
> My config:
> 
> [u]Use:[/u] Home File Server supporting PC and Mac machines -- audio/video
files, home directories (documents & photos), and backup images.  Will not
run any other services.
> 
> [u]Objective:[/u]  adequate performance for home use; maximize protection
for documents and photos; minimize administrative upkeep.
> 
> [u]Configuration:[/u]
> OpenSolaris B134
> AMD X2 260 / Asus MA4785TD-V EVO
> 8Gb ECC RAM (Crucial CT25672BA1339)
> Intel EXPI9301CT NIC
> LSI SAS9211-8i
> Boot Disk:  WD Scorpio Black 250Gb (2-way mirror)
> Media Pool:  WD Caviar Blue 640Gb (2-way mirror)
> Mass Storage (File & Backup) Pool:  WD Carviar Blue 640Gb (9 disk
Raid-Z3)

On 09/07/2010 03:58 PM, Craig Stevenson wrote:
> I am working on a home file server.  After reading a wide range of blogs
and forums, I have a few questions that are still not clear to me....
>
> 1.  Is there a benefit in having quad core CPU (e.g. Athlon II X4 vs X2)?
All of the web blogs seem to suggest using lower-wattage dual core CPUs.  But;
with the recent advent of dedup, SHA256 checksum, etc., I am now wondering if
opensolaris is better served with quad core.
With a big RAIDZ3, it''s well worth having extra cores. A scrub on my
eight-disk
RAIDZ2 uses about 60% of all four cores on my Athlon II X4 630.

With smaller pools, dual-core would be OK.

If you''re going to use dedup, you might want to go with an eight-disk
RAIDZ2 and
an SSD for L2ARC, instead of a nine-disk RAIDZ3.

On Tue, Sep  7 at 17:13, Russ Price wrote:
>On 09/07/2010 03:58 PM, Craig Stevenson wrote:
>> I am working on a home file server.  After reading a wide range of
>> blogs and forums, I have a few questions that are still not clear
>> to me....
>>
>> 1.  Is there a benefit in having quad core CPU (e.g. Athlon II X4
>> vs X2)? All of the web blogs seem to suggest using lower-wattage
>> dual core CPUs.  But; with the recent advent of dedup, SHA256
>> checksum, etc., I am now wondering if opensolaris is better served
>> with quad core.
>
>With a big RAIDZ3, it''s well worth having extra cores. A scrub on
my
>eight-disk RAIDZ2 uses about 60% of all four cores on my Athlon II X4 
>630.
How are you measuring using 60% across all four cores?

I kicked off a scrub just to see, and we''re scrubbing at 200MB/s (2
vdevs) and the CPU is 94% idle, 6% kernel, 0% IOWAIT.

zpool-tank is using 3.2% CPU as shown by ''ps aux | grep tank''

Am I missing something?


-- 
Eric D. Mudama
edmudama at mail.bounceswoosh.org

Craig,

3. I do not think you will get much dedupe on video, music and photos. I would
not bother. If you really wanted to know at some later stage, you could create a
new file system, enable dedupe, and copy your data (or a subset) into it just to
see. In my experience there is a significant CPU penalty as well. My four core
(1.86GHz xeons, 4 yrs old) box nearly maxes out when putting a lot of data into
a deduped file system.

-Scott
-- 
This message posted from opensolaris.org

On 09/07/2010 05:58 PM, Eric D. Mudama wrote:
> How are you measuring using 60% across all four cores?
>
> I kicked off a scrub just to see, and we''re scrubbing at 200MB/s
(2
> vdevs) and the CPU is 94% idle, 6% kernel, 0% IOWAIT.
>
> zpool-tank is using 3.2% CPU as shown by ''ps aux | grep
tank''
Whoops... I misspoke - it should have been about 23-25% per core. I''m
getting
old. :o) I am using gkrellm to watch the CPU usage.

In any case, a scrub uses wildly different amounts of CPU at different times, 
and sometimes it uses far less (particularly early in the process, at least on 
my specific RAIDZ2). On the other hand, dd''ing a 23 GB video file on
the RAIDZ2
to /dev/null will consistently get the 23-25% per core figure.

> 3. Should I consider using dedup if my server has only 8Gb of RAM? Or,
> will that not be enough to hold the DDT? In which case, should I add
> L2ARC / ZIL or am I better to just skip using dedup on a home file
> server?
As Cindy said, skip dedup for now. It''s not stable (enough). Try to
destroy a dataset with dedup enabled and lots of deduped data, and zfs may hang
for hours or days before it lets you back in.

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 97542685
roy at karlsbakk.net
http://blogg.karlsbakk.net/
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er
et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av
idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og
relevante synonymer p? norsk.

On 08/09/2010 00:41, Scott Meilicke wrote:
> Craig,
>
> 3. I do not think you will get much dedupe on video, music and photos. I
would not bother. If you really wanted to know at some later stage, you could
create a new file system, enable dedupe, and copy your data (or a subset) into
it just to see. In my experience there is a significant CPU penalty as well. My
four core (1.86GHz xeons, 4 yrs old) box nearly maxes out when putting a lot of
data into a deduped file system.
If you want to know what your potential savings for dedup are you can 
use zdb(1M) to calculate that:

# zdb -S <poolname>


-- 
Darren J Moffat

On Tue, September 7, 2010 15:58, Craig Stevenson wrote:
> 3.  Should I consider using dedup if my server has only 8Gb of RAM?  Or,
> will that not be enough to hold the DDT?  In which case, should I add
> L2ARC / ZIL or am I better to just skip using dedup on a home file server?
I would not consider using dedup in the current state of the code.  I hear
too many horror stories.

Also, why do you think you''d get much benefit?  It takes pretty big
blocks
of exact bit-for-bit duplication to actually trigger the code, and
you''re
not going to find them in compressed image (including motion picture /
video) or audio files, for example (the main things that take up much
space on most home servers).
-- 
David Dyer-Bennet, dd-b at dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info