zfs discuss - Jan 2010 - zpool destroy -f hangs system, now zpool import hangs system.

I have a zpool raidz1 array (called storage) that I created under snv_118.

I then created a zfs filesystem called storage/vmware which I shared
out via iscsi.



I then deleted the vmware filesystem, using ''zpool destroy -f
storage/vmware'' -- which resulted in heavy disk activity, and then
hard locked the system after 10 minutes.

I rebooted the machine, but was unable to boot. The machine would hang
on Reading ZFS Configuration: - (the stick wouldn''t even spin.)

I was able to work around that by booting to a live CD, and deleting
the zfs cache on my rpool.


zpool import sees my raidz1 array, but if I try ''zpool import -f
storage'', I get the same behavior of heavy disk activity for
approximately 10 minutes, then a hard lock.


Any clues on this one?

Thanks,
Carl

I have a zpool raidz1 array (called storage) that I created under snv_118.

I then created a zfs filesystem called storage/vmware which I shared
out via iscsi.



I then deleted the vmware filesystem, using ''zpool destroy -f
storage/vmware'' -- which resulted in heavy disk activity, and then
hard locked the system after 10 minutes.

I rebooted the machine, but was unable to boot. The machine would hang
on Reading ZFS Configuration: - (the stick wouldn''t even spin.)

I was able to work around that by booting to a live CD, and deleting
the zfs cache on my rpool.


zpool import sees my raidz1 array, but if I try ''zpool import -f
storage'', I get the same behavior of heavy disk activity for
approximately 10 minutes, then a hard lock.


Any clues on this one?

Thanks,
Carl

Did you happen to set dedup on that zvol that you destroyed?  Your symptoms
sound just like mine.  Check out the threads concerning losing a pool after
destroying a deduped dataset. There''s 3 or  4 of them.  I get heavy
read activity for about 4 minutes and then the systems just hangs and I
can''t do anything.  Others get heavey read activity followed by a
period of really slow writes but eventually get their pool back after a day or
so of importing.
-- 
This message posted from opensolaris.org

On Mon, Jan 4, 2010 at 4:22 PM, tom wagner <mama_jojo at hotmail.com>
wrote:
> Did you happen to set dedup on that zvol that you destroyed? ?Your symptoms
sound just like mine. ?Check out the threads concerning losing a pool after
destroying a deduped dataset. There''s 3 or ?4 of them. ?I get heavy
read activity for about 4 minutes and then the systems just hangs and I
can''t do anything. ?Others get heavey read activity followed by a
period of really slow writes but eventually get their pool back after a day or
so of importing.
> --
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
Hi Tom,

No, I never setup dedup on any of the datasets...

I believe my mistake was using zpool destroy on a zvol, instead of zfs
destroy...

I don''t have any clue on how to start to figure this out. I''d
really
like to not lose all the stuff I have here. Any suggestions are really
great!

Thanks,
Carl

Interesting,

I had assumed the cause of my problem was de-dup because the symptoms are
similar to what others have reported destroying their deduped datasets, but
their system hangs didn''t happen for hours while my system hard locks
in 3 to 4 minutes.  But now you have me thinking because the dataset I was using
was also a zvol shared out through ISCSI (via comstar though) and it was deduped
as well.

Although, my system hung during the destroy which more closely matched the
symptoms from the threads dealing with destroying a deduped zfs dataset and
making the pool non-importable threads.  Did your destroy happen to hangup
mid-way through?

zpool destroy would have just given you some funky error like non-existent
poolname or something. so you probably used zfs destroy correctly.
-- 
This message posted from opensolaris.org

On Jan 4, 2010, at 6:40 AM, Carl Rathman wrote:
> I have a zpool raidz1 array (called storage) that I created under  
> snv_118.
>
> I then created a zfs filesystem called storage/vmware which I shared
> out via iscsi.
>
>
>
> I then deleted the vmware filesystem, using ''zpool destroy -f
> storage/vmware'' -- which resulted in heavy disk activity, and then
> hard locked the system after 10 minutes.
>
> I rebooted the machine, but was unable to boot. The machine would hang
> on Reading ZFS Configuration: - (the stick wouldn''t even spin.)
>
> I was able to work around that by booting to a live CD, and deleting
> the zfs cache on my rpool.
>
>
> zpool import sees my raidz1 array, but if I try ''zpool import -f
> storage'', I get the same behavior of heavy disk activity for
> approximately 10 minutes, then a hard lock.
If you destroyed the pool, h
>
>
> Any clues on this one?
>
> Thanks,
> Carl
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

On Jan 4, 2010, at 6:40 AM, Carl Rathman wrote:
> I have a zpool raidz1 array (called storage) that I created under  
> snv_118.
>
> I then created a zfs filesystem called storage/vmware which I shared
> out via iscsi.
>
>
>
> I then deleted the vmware filesystem, using ''zpool destroy -f
> storage/vmware'' -- which resulted in heavy disk activity, and then
> hard locked the system after 10 minutes.
>
> I rebooted the machine, but was unable to boot. The machine would hang
> on Reading ZFS Configuration: - (the stick wouldn''t even spin.)
>
> I was able to work around that by booting to a live CD, and deleting
> the zfs cache on my rpool.
[clicked the wrong button]
If you destroy the pool, then why try to import?
  -- richard
> zpool import sees my raidz1 array, but if I try ''zpool import -f
> storage'', I get the same behavior of heavy disk activity for
> approximately 10 minutes, then a hard lock.
>
>
> Any clues on this one?
>
> Thanks,
> Carl
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

On Mon, Jan 4, 2010 at 8:59 PM, Richard Elling <richard.elling at
gmail.com> wrote:
>
> On Jan 4, 2010, at 6:40 AM, Carl Rathman wrote:
>
>> I have a zpool raidz1 array (called storage) that I created under
snv_118.
>>
>> I then created a zfs filesystem called storage/vmware which I shared
>> out via iscsi.
>>
>>
>>
>> I then deleted the vmware filesystem, using ''zpool destroy -f
>> storage/vmware'' -- which resulted in heavy disk activity, and
then
>> hard locked the system after 10 minutes.
>>
>> I rebooted the machine, but was unable to boot. The machine would hang
>> on Reading ZFS Configuration: - (the stick wouldn''t even
spin.)
>>
>> I was able to work around that by booting to a live CD, and deleting
>> the zfs cache on my rpool.
>
> [clicked the wrong button]
> If you destroy the pool, then why try to import?
> ?-- richard
>
>> zpool import sees my raidz1 array, but if I try ''zpool import
-f
>> storage'', I get the same behavior of heavy disk activity for
>> approximately 10 minutes, then a hard lock.
>>
>>
>> Any clues on this one?
>>
>> Thanks,
>> Carl
>> _______________________________________________
>> zfs-discuss mailing list
>> zfs-discuss at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
>
I didn''t mean to destroy the pool.  I used zpool destroy on a zvol,
when I should have used zfs destroy.

When I used zpool destroy -f mypool/myvolume the machine hard locked
after about 20 minutes.

I don''t want to destroy the pool, I just wanted to destroy the one
volume. -- Which is why I now want to import the pool itself. Does
that make sense?

Thanks,
Carl

On Jan 5, 2010, at 7:54 AM, Carl Rathman wrote:
> I didn''t mean to destroy the pool.  I used zpool destroy on a
zvol,
> when I should have used zfs destroy.
>
> When I used zpool destroy -f mypool/myvolume the machine hard locked
> after about 20 minutes.
This would be a bug.  "zpool destroy" should only destroy pools.
Volumes are datasets and are destroyed by "zfs destroy."  Using
"zpool destroy -f" will attempt to force unmounts of any mounted
datasets, but volumes are not mounted, per se. Upon reboot, nothing
will be mounted until after the pool is imported.

> I don''t want to destroy the pool, I just wanted to destroy the one
> volume. -- Which is why I now want to import the pool itself. Does
> that make sense?
If the pool was destroyed, then you can try to import using -D.

Are you sure you didn''t "zfs destroy" instead?  Once the pool
is
imported,
"zpool history" will show all of the commands issued against the pool.
  -- richard

On Tue, Jan 5, 2010 at 10:12 AM, Richard Elling
<richard.elling at gmail.com> wrote:
> On Jan 5, 2010, at 7:54 AM, Carl Rathman wrote:
>
>> I didn''t mean to destroy the pool. ?I used zpool destroy on a
zvol,
>> when I should have used zfs destroy.
>>
>> When I used zpool destroy -f mypool/myvolume the machine hard locked
>> after about 20 minutes.
>
> This would be a bug. ?"zpool destroy" should only destroy pools.
> Volumes are datasets and are destroyed by "zfs destroy." ?Using
> "zpool destroy -f" will attempt to force unmounts of any mounted
> datasets, but volumes are not mounted, per se. Upon reboot, nothing
> will be mounted until after the pool is imported.
>
>
>> I don''t want to destroy the pool, I just wanted to destroy the
one
>> volume. -- Which is why I now want to import the pool itself. Does
>> that make sense?
>
> If the pool was destroyed, then you can try to import using -D.
>
> Are you sure you didn''t "zfs destroy" instead? ?Once the
pool is imported,
> "zpool history" will show all of the commands issued against the
pool.
> ?-- richard
>
>
Hi Richard,

If I could import the pool, I''d love to do a history on it.

At this point, if I attempt to import the pool, the machine will have
heavy disk activity on the pool for approximately 10 minutes, then the
machine will hard lock. This will happen when I boot the machine from
its snv_130 rpool, or if I boot the machine from a snv_130 live cd.

Thanks,
Carl

On Tue, Jan 5, 2010 at 10:35 AM, Carl Rathman <crathman at gmail.com>
wrote:
> On Tue, Jan 5, 2010 at 10:12 AM, Richard Elling
> <richard.elling at gmail.com> wrote:
>> On Jan 5, 2010, at 7:54 AM, Carl Rathman wrote:
>>
>>> I didn''t mean to destroy the pool. ?I used zpool destroy
on a zvol,
>>> when I should have used zfs destroy.
>>>
>>> When I used zpool destroy -f mypool/myvolume the machine hard
locked
>>> after about 20 minutes.
>>
>> This would be a bug. ?"zpool destroy" should only destroy
pools.
>> Volumes are datasets and are destroyed by "zfs destroy."
?Using
>> "zpool destroy -f" will attempt to force unmounts of any
mounted
>> datasets, but volumes are not mounted, per se. Upon reboot, nothing
>> will be mounted until after the pool is imported.
>>
>>
>>> I don''t want to destroy the pool, I just wanted to destroy
the one
>>> volume. -- Which is why I now want to import the pool itself. Does
>>> that make sense?
>>
>> If the pool was destroyed, then you can try to import using -D.
>>
>> Are you sure you didn''t "zfs destroy" instead? ?Once
the pool is imported,
>> "zpool history" will show all of the commands issued against
the pool.
>> ?-- richard
>>
>>
>
> Hi Richard,
>
> If I could import the pool, I''d love to do a history on it.
>
> At this point, if I attempt to import the pool, the machine will have
> heavy disk activity on the pool for approximately 10 minutes, then the
> machine will hard lock. This will happen when I boot the machine from
> its snv_130 rpool, or if I boot the machine from a snv_130 live cd.
>
> Thanks,
> Carl
>
Any suggestions on how to begin debugging this, or if data recovery is possible?

Thanks,
Carl

On Wed, Jan 6, 2010 at 12:11 PM, Carl Rathman <crathman at gmail.com>
wrote:
> On Tue, Jan 5, 2010 at 10:35 AM, Carl Rathman <crathman at gmail.com>
wrote:
>> On Tue, Jan 5, 2010 at 10:12 AM, Richard Elling
>> <richard.elling at gmail.com> wrote:
>>> On Jan 5, 2010, at 7:54 AM, Carl Rathman wrote:
>>>
>>>> I didn''t mean to destroy the pool. ?I used zpool
destroy on a zvol,
>>>> when I should have used zfs destroy.
>>>>
>>>> When I used zpool destroy -f mypool/myvolume the machine hard
locked
>>>> after about 20 minutes.
>>>
>>> This would be a bug. ?"zpool destroy" should only destroy
pools.
>>> Volumes are datasets and are destroyed by "zfs destroy."
?Using
>>> "zpool destroy -f" will attempt to force unmounts of any
mounted
>>> datasets, but volumes are not mounted, per se. Upon reboot, nothing
>>> will be mounted until after the pool is imported.
>>>
>>>
>>>> I don''t want to destroy the pool, I just wanted to
destroy the one
>>>> volume. -- Which is why I now want to import the pool itself.
Does
>>>> that make sense?
>>>
>>> If the pool was destroyed, then you can try to import using -D.
>>>
>>> Are you sure you didn''t "zfs destroy" instead?
?Once the pool is imported,
>>> "zpool history" will show all of the commands issued
against the pool.
>>> ?-- richard
>>>
>>>
>>
>> Hi Richard,
>>
>> If I could import the pool, I''d love to do a history on it.
>>
>> At this point, if I attempt to import the pool, the machine will have
>> heavy disk activity on the pool for approximately 10 minutes, then the
>> machine will hard lock. This will happen when I boot the machine from
>> its snv_130 rpool, or if I boot the machine from a snv_130 live cd.
>>
>> Thanks,
>> Carl
>>
>
> Any suggestions on how to begin debugging this, or if data recovery is
possible?
>
> Thanks,
> Carl
>
Just wanted to update everyone on this...

I installed 2009.06 (snv_111b), and gave the import of the pool one
last try. After approximately 20 minutes of grinding, the pool
imported properly! No clue why, but all seems to be working now.

Thanks for the insight from the list, I really appreciate it.

-Carl