zfs discuss - Jun 2009 - ZFS mirrors with uncoordinated LUN snapshots (Amazon EBS)

I''m setting up OpenSolaris on Amazon EC2, and planning on using their
Elastic Block Store volumes to store a persistent ZFS zpool. I''m
inclined to make a mirror of two EBS volumes (essentially LUNs with snapshot
features and an API for mapping/unmapping them), for better data protection.
However, EC2 only lets you snapshot one volume at a time; there is no
consistency group feature for taking simultaneous snapshots of the volumes
comprising a zpool. Likewise, you can only map or unmap one volume at a time.

My question is this: how well can ZFS deal with the mirror devices getting out
of sync? For instance, if one or both of my EBS volumes are lost and I have to
restore from EBS snapshots, one volume will have a newer version of the data
than the other. Will ZFS be able to recognize this and safely resilver from the
newer device to the older?
-- 
This message posted from opensolaris.org

Clayton Wheeler wrote:
> I''m setting up OpenSolaris on Amazon EC2, and planning on using
their Elastic Block Store volumes to store a persistent ZFS zpool. I''m
inclined to make a mirror of two EBS volumes (essentially LUNs with snapshot
features and an API for mapping/unmapping them), for better data protection.
However, EC2 only lets you snapshot one volume at a time; there is no
consistency group feature for taking simultaneous snapshots of the volumes
comprising a zpool. Likewise, you can only map or unmap one volume at a time.
>   
Interesting.  Let us know how it works.
> My question is this: how well can ZFS deal with the mirror devices getting
out of sync? For instance, if one or both of my EBS volumes are lost and I have
to restore from EBS snapshots, one volume will have a newer version of the data
than the other. Will ZFS be able to recognize this and safely resilver from the
newer device to the older?
>   
Syncing is done to a transaction group. By default, txgs are sync''ed
every
5 or 30 seconds.

It would be relatively easy to setup a script which would notify EBS
to snap immediately after a txg commit completes. If the workload contains
a lot of sync writes, special care would be needed to design the system to
properly deal with the ZIL.
 -- richard

On Wed, Jun 10, 2009 at 14:18, Clayton Wheeler<no-reply at
opensolaris.org> wrote:
> I''m setting up OpenSolaris on Amazon EC2, and planning on using
their Elastic Block Store volumes to store a persistent ZFS zpool. I''m
inclined to make a mirror of two EBS volumes (essentially LUNs with snapshot
features and an API for mapping/unmapping them), for better data protection.
However, EC2 only lets you snapshot one volume at a time; there is no
consistency group feature for taking simultaneous snapshots of the volumes
comprising a zpool. Likewise, you can only map or unmap one volume at a time.
You could export the zpool (which will cause it to stop writing to
disk) and then take the snapshots.  This would prevent desynchronized
volumes entirely.
> My question is this: how well can ZFS deal with the mirror devices getting
out of sync? For instance, if one or both of my EBS volumes are lost and I have
to restore from EBS snapshots, one volume will have a newer version of the data
than the other. Will ZFS be able to recognize this and safely resilver from the
newer device to the older?
In my experience, ZFS doesn''t have problems with this kind of scenario
as long as you don''t touch the snapshots before feeding both of them
back into the system.  That is, if you snapshot volumes A and B at
different times, don''t make changes to both (producing A'' and
B'')
before putting them back in a zpool.  A and B will be able to form a
mirror, but A'' and B'' will not necessarily.

You should make sure that whatever application you are running forces
sync of its data to disk before taking the snapshots, but you probably
knew that already.

Will