zfs discuss - Nov 2010 - zpool import is this safe to use "-f" option in this case ?

Hi,

I have done the following (which is required for my case)

Created a zpool (smpool) on a device/LUN from an array (IBM 6K) on host1
created a array level snapshot of the device using "dscli" to another
device which is successful.
Now I make the snapshot device visible to another host (host2)

I tried "zpool import smpool". Got a warning message that host1 is
using this pool (might be the smpool metata data has stored this info) and asked
to use "-f"

When i tried zpool import with -f option, I am able to successfully import to
host2 and able to access all file systems and snapshots.

My query is in this scenario is always safe to use "-f" to import ??
would there be any issues ? 
Also I have observed that zpool import took some time to for successful
completion. Is there a way minimize "zpool import -f" operation time
??

Regards,
sridhar.
-- 
This message posted from opensolaris.org

sridhar,
> I have done the following (which is required for my case)
> 
> Created a zpool (smpool) on a device/LUN from an array (IBM 6K) on host1
> created a array level snapshot of the device using "dscli" to
another device which is successful.
> Now I make the snapshot device visible to another host (host2)
Even though the array is capable of taking device/LUN snapshots, this is a
non-standard mode of operation regarding the use of ZFS.

It raises concerns that if one had a problem using a ZFS in this manner, there
would be few Oracle or community users of ZFS that could assist. Even if the
alleged problem was not related to using ZFS with array based snapshots, usage
would always create a level of uncertainty.

Instead I would suggest using ZFS send / recv instead.
> I tried "zpool import smpool". Got a warning message that host1
is using this pool (might be the smpool metata data has stored this info) and
asked to use "-f"
> 
> When i tried zpool import with -f option, I am able to successfully import
to host2 and able to access all file systems and snapshots.
> 
> My query is in this scenario is always safe to use "-f" to import
??
In this scenario, it is safe to use "-f" with zpool import. 
> would there be any issues ? 
Prior to taking the next snapshot, one must be assured that the device/LUN on
host2 is returned to the "zpool export" state. Failure to do this
could cause zpool corruption, ZFS I/O failures, or even the possibility of a
system panic on host2.
> Also I have observed that zpool import took some time to for successful
completion. Is there a way minimize "zpool import -f" operation time
??
No.

- Jim
> 
> Regards,
> sridhar.
> -- 
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

On Wed, Nov 17, 2010 at 10:12 AM, Jim Dunham <james.dunham at
oracle.com>wrote:
> sridhar,
>
> > I have done the following (which is required for my case)
> >
> > Created a zpool (smpool) on a device/LUN from an array (IBM 6K) on
host1
> > created a array level snapshot of the device using "dscli"
to another
> device which is successful.
> > Now I make the snapshot device visible to another host (host2)
>
> Even though the array is capable of taking device/LUN snapshots, this is a
> non-standard mode of operation regarding the use of ZFS.
>
> It raises concerns that if one had a problem using a ZFS in this manner,
> there would be few Oracle or community users of ZFS that could assist. Even
> if the alleged problem was not related to using ZFS with array based
> snapshots, usage would always create a level of uncertainty.
>
> Instead I would suggest using ZFS send / recv instead.
>
>
That''s what we call FUD.  "It might be a problem if you use
someone else''s
feature that we duplicate".  If Oracle isn''t going to support
array-based
snapshots, come right out and say it.  You might as well pack up the cart
now though, there isn''t an enterprise array on the market that
doesn''t have
snapshots, and you will be the ONLY OS I''ve ever heard of even
suggesting
that array-based snapshots aren''t allowed.


> > would there be any issues ?
>
> Prior to taking the next snapshot, one must be assured that the device/LUN
> on host2 is returned to the "zpool export" state. Failure to do
this could
> cause zpool corruption, ZFS I/O failures, or even the possibility of a
> system panic on host2.
>
>
Really?  And how did you come to that conclusion?



OP: Yes, you do need to use a -f.  The zpool has a signature that is there
when the pool is imported (this is to keep an admin from accidentally
importing the pool to two different systems at the same time).  The only way
to clear it is to do a zpool export before taking the initial snapshot, or
doing the -f on import.  Jim here is doing a great job of spreading FUD, and
none of it is true.  What you''re doing should absolutely work, just
make
sure there is no I/O in flight when you take the original snapshot.

Either export the pool first (I would recommend this approach), shut the
system down, or just make sure you aren''t doing any writes when taking
the
array-based snapshot.

--Tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20101117/63a8a9e4/attachment.html>

Tim,
> 
> On Wed, Nov 17, 2010 at 10:12 AM, Jim Dunham <james.dunham at
oracle.com> wrote:
> sridhar,
> 
> > I have done the following (which is required for my case)
> >
> > Created a zpool (smpool) on a device/LUN from an array (IBM 6K) on
host1
> > created a array level snapshot of the device using "dscli"
to another device which is successful.
> > Now I make the snapshot device visible to another host (host2)
> 
> Even though the array is capable of taking device/LUN snapshots, this is a
non-standard mode of operation regarding the use of ZFS.
> 
> It raises concerns that if one had a problem using a ZFS in this manner,
there would be few Oracle or community users of ZFS that could assist. Even if
the alleged problem was not related to using ZFS with array based snapshots,
usage would always create a level of uncertainty.
> 
> Instead I would suggest using ZFS send / recv instead.
> 
> 
> That''s what we call FUD.  "It might be a problem if you use
someone else''s feature that we duplicate".  If Oracle
isn''t going to support array-based snapshots, come right out and say
it.  You might as well pack up the cart now though, there isn''t an
enterprise array on the market that doesn''t have snapshots, and you
will be the ONLY OS I''ve ever heard of even suggesting that array-based
snapshots aren''t allowed.
That''s not what I said... Non-standard mode of operation is not the
same thing as not supported. Using ZFS''s standard mode of operation
based on its built-in support for snapshots is well proven, well document
technology.
> 
>  
> > would there be any issues ?
> 
> Prior to taking the next snapshot, one must be assured that the device/LUN
on host2 is returned to the "zpool export" state. Failure to do this
could cause zpool corruption, ZFS I/O failures, or even the possibility of a
system panic on host2.
> 
> 
> Really?  And how did you come to that conclusion?  
As prior developer and project lead of host-based snapshot and replication
software on Solaris, I have first hand experience using ZFS with snapshots.

If while ZFS on node2 is accessing an instance of snapshot data, the array
updates the snapshot data, ZFS will see newly created CRCs created by node1.
These CRCs will be considered as metadata corruption, and depending on exactly
what ZFS was doing at the time the corruption was detected, the software attempt
some form of error recovery.
> OP: Yes, you do need to use a -f.  The zpool has a signature that is there
when the pool is imported (this is to keep an admin from accidentally importing
the pool to two different systems at the same time).  The only way to clear it
is to do a zpool export before taking the initial snapshot, or doing the -f on
import.  Jim here is doing a great job of spreading FUD, and none of it is true.
> 
> What you''re doing should absolutely work, just make sure there is
no I/O in flight when you take the original snapshot.
> 
> Either export the pool first (I would recommend this approach), shut the
system down, or just make sure you aren''t doing any writes when taking
the array-based snapshot.
These last two statements need clarification. 

ZFS is always on disk consistent, even in the context of using snapshots.
Therefore as far as ZFS is concerned, there is no need to assure that there are
no I/Os in flight, or that the storage pool is exported, or that the system is
shutdown, or that one is doing any writes.

Although ZFS is always on disk consistent, many applications are not filesystem
consistent. To be filesystem consistent, an application by design must issue
careful writes and/or synchronized filesystem operations. Not knowing this fact,
or lacking this functionality, a system admin will need to deploy some of the
work-arounds suggested above. The most important one not listed, is to stop or
pause those applications which are know not to be filesystem consistent.

- Jim
> 
> --Tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20101116/3137fe51/attachment-0001.html>

+1

When I did my stuff (with a major bank) two years ago, my reasoning was that we
(Sun, remember them?) had made huge capital out of the "always consistent
on disk" claim, and that we could be expected to stand by and honour that
promise.

But because this was a big bank, I felt that due dilligence required that I
should call up the ultimate authorities (the ZFS architects and implementors)
for confirmation that what I was intending was based on rock solid assumptions.
Having obtained those assurances, we went ahead and did implemented some
impressive, rule changing, business enabling stuff.

zfs send | recv is cool technology, but it is not the only show in town. The
downsides include: it''s slow; that it impacts the performance of the
sending system; that there''s no easy way to know if continuous sending
of incremental changes will be able to keep up with demand; etc

LUN snapshots are, by comparison, free at the point of use (no impact on the
sending system), and practically instant.

And of course, there''s nothing to stop both techniques being used
together (e.g. take a LUN snapshot, import the pool into another host, and do
the zfs send there, where it has no impact on the performance of the live
system).

And of course, there are independent, experienced, expert people of integrity
out there you can always hire to help you implement such schemes safely and
wisely.

Phil
www.harmanholistix.com


On 17 Nov 2010, at 00:19, Tim Cook <tim at cook.ms> wrote:
> 
> On Wed, Nov 17, 2010 at 10:12 AM, Jim Dunham <james.dunham at
oracle.com> wrote:
> sridhar,
> 
> > I have done the following (which is required for my case)
> >
> > Created a zpool (smpool) on a device/LUN from an array (IBM 6K) on
host1
> > created a array level snapshot of the device using "dscli"
to another device which is successful.
> > Now I make the snapshot device visible to another host (host2)
> 
> Even though the array is capable of taking device/LUN snapshots, this is a
non-standard mode of operation regarding the use of ZFS.
> 
> It raises concerns that if one had a problem using a ZFS in this manner,
there would be few Oracle or community users of ZFS that could assist. Even if
the alleged problem was not related to using ZFS with array based snapshots,
usage would always create a level of uncertainty.
> 
> Instead I would suggest using ZFS send / recv instead.
> 
> 
> That''s what we call FUD.  "It might be a problem if you use
someone else''s feature that we duplicate".  If Oracle
isn''t going to support array-based snapshots, come right out and say
it.  You might as well pack up the cart now though, there isn''t an
enterprise array on the market that doesn''t have snapshots, and you
will be the ONLY OS I''ve ever heard of even suggesting that array-based
snapshots aren''t allowed.
> 
>  
> > would there be any issues ?
> 
> Prior to taking the next snapshot, one must be assured that the device/LUN
on host2 is returned to the "zpool export" state. Failure to do this
could cause zpool corruption, ZFS I/O failures, or even the possibility of a
system panic on host2.
> 
> 
> Really?  And how did you come to that conclusion?  
> 
> 
> 
> OP: Yes, you do need to use a -f.  The zpool has a signature that is there
when the pool is imported (this is to keep an admin from accidentally importing
the pool to two different systems at the same time).  The only way to clear it
is to do a zpool export before taking the initial snapshot, or doing the -f on
import.  Jim here is doing a great job of spreading FUD, and none of it is true.
What you''re doing should absolutely work, just make sure there is no
I/O in flight when you take the original snapshot.
> 
> Either export the pool first (I would recommend this approach), shut the
system down, or just make sure you aren''t doing any writes when taking
the array-based snapshot.
> 
> --Tim
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20101117/489a0f61/attachment.html>

On Wed, Nov 17, 2010 at 2:56 PM, Jim Dunham <james.dunham at oracle.com>
wrote:
> Tim,
>
>
> On Wed, Nov 17, 2010 at 10:12 AM, Jim Dunham <james.dunham at
oracle.com>wrote:
>
>> sridhar,
>>
>> > I have done the following (which is required for my case)
>> >
>> > Created a zpool (smpool) on a device/LUN from an array (IBM 6K) on
host1
>> > created a array level snapshot of the device using
"dscli" to another
>> device which is successful.
>> > Now I make the snapshot device visible to another host (host2)
>>
>> Even though the array is capable of taking device/LUN snapshots, this
is a
>> non-standard mode of operation regarding the use of ZFS.
>>
>> It raises concerns that if one had a problem using a ZFS in this
manner,
>> there would be few Oracle or community users of ZFS that could assist.
Even
>> if the alleged problem was not related to using ZFS with array based
>> snapshots, usage would always create a level of uncertainty.
>>
>> Instead I would suggest using ZFS send / recv instead.
>>
>>
> That''s what we call FUD.  "It might be a problem if you use
someone else''s
> feature that we duplicate".  If Oracle isn''t going to support
array-based
> snapshots, come right out and say it.  You might as well pack up the cart
> now though, there isn''t an enterprise array on the market that
doesn''t have
> snapshots, and you will be the ONLY OS I''ve ever heard of even
suggesting
> that array-based snapshots aren''t allowed.
>
>
> That''s not what I said... Non-standard mode of operation is *not*
the same
> thing as not supported. Using ZFS''s standard mode of operation
based on its
> built-in support for snapshots is well proven, well document technology.
>

How is using an array based snapshot to create a copy of a filesystem
"non-standard"?  Non-standard to who?  Array based snapshots were
around
long-before ZFS was created.  It was proven and documented long before ZFS
was around as well.  Given your history in the industry, I know you
aren''t
so new to this game you didn''t already know that, so I''m not
really sure
what the purpose of "proven and documented" was, other than to try to
insinuate that other technologies are not.


>
>
>
>
>> > would there be any issues ?
>>
>> Prior to taking the next snapshot, one must be assured that the
device/LUN
>> on host2 is returned to the "zpool export" state. Failure to
do this could
>> cause zpool corruption, ZFS I/O failures, or even the possibility of a
>> system panic on host2.
>>
>>
> Really?  And how did you come to that conclusion?
>
>
> As prior developer and project lead of host-based snapshot and replication
> software on Solaris, I have first hand experience using ZFS with snapshots.
>
> If while ZFS on node2 is accessing an instance of snapshot data, the array
> updates the snapshot data, ZFS will see newly created CRCs created by
node1.
> These CRCs will be considered as metadata corruption, and depending on
> exactly what ZFS was doing at the time the corruption was detected, the
> software attempt some form of error recovery.
>
>
The array doesn''t "update the snapshot data". 
That''s the whole point of the
snapshot.  It''s point-in-time.  Either the snapshot exists as it was
taken,
or it''s deleted.  What array on the market changes blocks in a snapshot
that
are being presented out as a live filesystem to a host?  I''ve never
heard of
any such behavior, and that sort of behavior would be absolutely brain-dead.


OP: Yes, you do need to use a -f.  The zpool has a signature that is
there
> when the pool is imported (this is to keep an admin from accidentally
> importing the pool to two different systems at the same time).  The only
way
> to clear it is to do a zpool export before taking the initial snapshot, or
> doing the -f on import.  Jim here is doing a great job of spreading FUD,
and
> none of it is true.
>
>
> What you''re doing should absolutely work, just make sure there is
no I/O in
> flight when you take the original snapshot.
>
> Either export the pool first (I would recommend this approach), shut the
> system down, or just make sure you aren''t doing any writes when
taking the
> array-based snapshot.
>
>
> These last two statements need clarification.
>
> ZFS is always on disk consistent, even in the context of using snapshots.
> Therefore as far as ZFS is concerned, there is no need to assure that there
> are no I/Os in flight, or that the storage pool is exported, or that the
> system is shutdown, or that one is doing any writes.
>
>
Except when it isn''t.  Which is why zfs import -F was added to ZFS.  In
theory ZFS doesn''t need checkdisk and it didn''t need an import
-F because
it''s always consistent on disk.  In reality, that''s utterly
false as well.


> Although ZFS is always on disk consistent, many applications are not
> filesystem consistent. To be filesystem consistent, an application by
design
> must issue careful writes and/or synchronized filesystem operations. Not
> knowing this fact, or lacking this functionality, a system admin will need
> to deploy some of the work-arounds suggested above. The most important one
> not listed, is to stop or pause those applications which are know not to be
> filesystem consistent.
>
>
>
If the zpool is exported or the system is shutdown, there won''t be any
access from any applications

--Tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20101117/af858349/attachment.html>

Hi,

My understanding is ZFS itself is a great file system by combining fs/vm with
the numerous  feature added to it.

In the similar lines existing fs/vm and array snapshot are still in use and
customers is requesting similar kind of support for zfs.

So it would be very great help of getting similar interface to match the use
cases looking for along with the new features.

If a customer is having Solaris 9/10 with UFS /SVM stack, back applications run
in the mentioned method ( earlier thread and other threads).

If somebody moves from UFS/SVM stack to ZFS, customer expects the backup
application should run in the similar configuration (at least for now).

To match the requirements, zfs/zpool support is required. 

Off course down the line I am sure applications starts new methods provided by
zfs to support it.

Regards,
sridhar.
-- 
This message posted from opensolaris.org

On Tue, November 16, 2010 22:56, Jim Dunham wrote:
> Although ZFS is always on disk consistent, many applications are not
> filesystem consistent. To be filesystem consistent, an application by
> design must issue careful writes and/or synchronized filesystem
> operations. Not knowing this fact, or lacking this functionality, a system
> admin will need to deploy some of the work-arounds suggested above. The
> most important one not listed, is to stop or pause those applications
> which are know not to be filesystem consistent.
Windows has an API that ties into VSS/Shadow Copy where an application can
register itself, and so when a back up app is running, it can tell
everyone "please quiesce, we need to take a snapshot now".
Applications
can then create check points so that the snapshot will have consistent
data, and that snapshot is what is backed up.

It''d be useful for ZFS snapshots, but also for things like running in a
virtualized environment (VMware, LDoms, etc.) where the hosting platform
wants to create a checkpoint for the vDisks. Similarly for LUN snapshots
under EMC, NetApp, etc.

Currently such a mechanism / API does not exist in Solaris (or any Unix
AFAICT): do you know if an RFE has been filed for such a thing?

On Nov 17, 2010, at 3:06 AM, sridhar surampudi wrote:
> Hi,
> 
> My understanding is ZFS itself is a great file system by combining fs/vm
with the numerous  feature added to it.
> 
> In the similar lines existing fs/vm and array snapshot are still in use and
customers is requesting similar kind of support for zfs.
> 
> So it would be very great help of getting similar interface to match the
use cases looking for along with the new features.
> 
> If a customer is having Solaris 9/10 with UFS /SVM stack, back applications
run in the mentioned method ( earlier thread and other threads).
> 
> If somebody moves from UFS/SVM stack to ZFS, customer expects the backup
application should run in the similar configuration (at least for now).
I am not aware of any SVM method to do this.  Do you have a pointer
to documentation or source code?
 -- richard