zfs discuss - Jan 2010 - Performance of partition based SWAP vs. ZFS zvol SWAP

Has anyone done research into the performance of SWAP on the traditional
partitioned based SWAP device as compared to a SWAP area set up on ZFS
with a zvol?

 I can find no best practices for this issue. In the old days it was
considered important to separate the swap devices onto individual disks
(controllers)  and select the outer cylinder groups for the partition
(to gain some read speed).  How does this compare to creating a single
SWAP zvol within a rootpool and then mirroring the rootpool across two
separate disks?

Ray Licon
at&t, Global Systems and Applications
Office: 310-762-6968
Pager: 310-681-0140

Has anyone done research into the performance of SWAP on the traditional
partitioned based SWAP device as compared to a SWAP area set up on ZFS with a
zvol?

 I can find no best practices for this issue. In the old days it was considered
important to separate the swap devices onto individual disks (controllers)  and
select the outer cylinder groups for the partition (to gain some read speed). 
How does this compare to creating a single SWAP zvol within a rootpool and then
mirroring the rootpool across two separate disks?
-- 
This message posted from opensolaris.org

RayLicon wrote:
> Has anyone done research into the performance of SWAP on the traditional
partitioned based SWAP device as compared to a SWAP area set up on ZFS with a
zvol?
> 
>  I can find no best practices for this issue. In the old days it was
considered important to separate the swap devices onto individual disks
(controllers)  and select the outer cylinder groups for the partition (to gain
some read speed).  How does this compare to creating a single SWAP zvol within a
rootpool and then mirroring the rootpool across two separate disks?
Best practice nowadays is to design a system so it doesn''t need to
swap.
Then it doesn''t matter what the performance of the swap device is.

-- 
Andrew Gabriel

Ok ...

 Given that ... yes, we all know that swapping is bad (thanks for the
enlightenment).

 To Swap or not to Swap isn''t releated to this question, and besides,
even if you don''t page swap, other mechanisms can still claim swap
space, such as the tmp file system.

 The question is simple, "IF" you have to swap (for whatever reason),
then which of two alternatives is better (separate disk partitons on multiple
disks, or zvol ZFS stripes or mirrors - and why).

 If no one has any data on  this issue then fine, but I didn''t waste my
time posting to this site to get responses that  simply say -don''t swap
-- 
This message posted from opensolaris.org

LICON, RAY (ATTPB) wrote:
> Thanks for the reply.
> 
> In many situations, the hardware design isn''t up to me and budgets
tend
> to dictate everything these days. True, nobody wants to swap, but the
> question is "if" you had to -- what design serves you best.
Independent
> swap slices or putting it all under control of zfs.
It depends why you need to swap, i.e. why are you using more memory than
you have, and is your working set size bigger than memory (thrashing),
or is swapping likely to be just a once-off event or infrequently repeated?

You probably need to forget most of what you learned about swapping 25
years ago, when systems routinely swapped, and technology was very
different. Disks have got faster over that period, probably of the order
100 times faster. However, CPUs have got 100,000 times faster, so in
reality a disk looks to be 1000 times slower from the CPU''s standpoint
than it did 25 years ago. This means that CPU cycles lost due to
swapping will appear to have a proportionally much more dire effect on
performance than they did many years back.

There are lots more options available today than there were when systems
routinely swapped. A couple of examples that spring to mind...

ZFS has been explicitly designed to swap it''s own cache data, only we
don''t call it swapping - we call it an L2ARC or ReadZilla. So if you
have a system where the application is going to struggle with main
memory, you might configure ZFS to significantly reduce it''s memory
buffer (ARC), and instead give it an L2ARC on a fast solid state disk.
This might result in less performance degradation in some systems where
memory is short, depending heavily on the behaviour of the application.

If you do have to go with brute force old style swapping, then you might
want to invest in solid state disk swap devices, which will go some way
towards reducing the factor of 1000 I mentioned above. (Take note of
aligning swap to the 4k flash i/o boundaries.)

Probably lots of other possibilities too, given more than a couple of
minutes thought.

-- 
Andrew

On Jan 27, 2010, at 12:25 PM, RayLicon wrote:
> Ok ...
> 
> Given that ... yes, we all know that swapping is bad (thanks for the
enlightenment).
> 
> To Swap or not to Swap isn''t releated to this question, and
besides, even if you don''t page swap, other mechanisms can still claim
swap space, such as the tmp file system.
> 
> The question is simple, "IF" you have to swap (for whatever
reason), then which of two alternatives is better (separate disk partitons on
multiple disks, or zvol ZFS stripes or mirrors - and why).
Swap is striped across swap devices in 1 MB chunks. In other words, striping
under
the stripe won''t make much difference, especially if you are using ZFS
dynamic striping.
> If no one has any data on  this issue then fine, but I didn''t
waste my time posting to this site to get responses that  simply say
-don''t swap
The data is that 5 orders of magnitude is the latency difference between 
RAM and HDDs. Double the performance of your swap devices and you 
are still looking at 5 orders of magnitude.
 -- richard

>>>>> "ag" == Andrew Gabriel <Andrew.Gabriel at
Sun.COM> writes:
    ag> is your working set size bigger than memory (thrashing),

n...no, not...not exactly. :)

    ag> or is swapping likely to be just a once-off event or
    ag> infrequently repeated?

once-off!  or...well...repeated, every time the garbage collector runs.

    ag> You probably need to forget most of what you learned about
    ag> swapping 25 years ago, when systems routinely swapped, and
    ag> technology was very different.

yes, some Lisp machines had integrated swapper/garbagecollectors.  Now
we have sbrk() + gc.  dumb!

We used to not worry about overcommitting because refusing to
overcommit just meant some of the allocated swap space would never get
written.  It was a little bit foolish because the threat of thrashing
means, whenever swap''s involved, you''re basically
overcommitted, but
it let us feel better.  Now that we''re not using swap, failure to
overcommit seems rather wasteful.  At the very least you should allow
the ARC cache to grow into memory reserved for an allocation, then
boot the ARC out of it if the process actually writes to more than you
thought it would and you need to keep a commitment you thought you
wouldn''t.

    ag> solid state disk swap devices,

smart!  it might turn out to be good for ebooks and other
power-constrained devices, too, because DRAM uses battery: swapping to
conserve energy rather than RAM.  It might be worth tracking pages in
a more complicated way than we''re now doing if the goal is to evacuate
RAM and power it down, so maybe holding onto ancient swap wisdom and
code isn''t as helpful to this as it might seem.

The point, keep swap on ZFS so you can grow/shrink/delete it as
fashion changes, is good.  But the OP''s question still stands: does
ZFS swap perform almost as well as raw device swap, or is it worth
partitioning disks if you insist on actually using swap?  I guess no
one knows.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 304 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100127/fda6ad43/attachment.bin>

On Wed, 27 Jan 2010, RayLicon wrote:
>
> If no one has any data on this issue then fine, but I didn''t waste
> my time posting to this site to get responses that simply say
-don''t
> swap
Perhaps you can set up a test environment, measure this in a 
scientific way, and provide a formal summary for our edification?

Thanks,

Bob
--
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/