thr3ads.net - zfs discuss - [zfs-discuss] ZFS commands hanging in B95 [Aug 2008]

If this information is useful, please help other people find it:
Share via:

Ian Collins

2008-Aug-17 22:35 UTC

[zfs-discuss] ZFS commands hanging in B95

I have a pretty standard ZFS boot AMD64 desktop.  A the moment, most ZFS
related commands are hanging (can''t be killed) .  Running
''truss share''
the last few lines I see are:

ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08044860)    = 0
brk(0x080C1000)                    = 0
ioctl(3, ZFS_IOC_POOL_STATS, 0x080447E0)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x080437E0)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x080437E0)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x080437E0)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08042760)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08043780)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08042760)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08043780)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08042760)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08043780)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08042760)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08043780)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x080437E0)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x080437E0)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08042760)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08043780)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x080437E0)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x080437E0)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08042760)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08043780)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x080437E0)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    = 0
ioctl(3, ZFS_IOC_OBJSET_STATS, 0x080437E0)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08044800)    Err#3 ESRCH
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    = 0
ioctl(3, ZFS_IOC_DATASET_LIST_NEXT, 0x08045880)    Err#3 ESRCH

Where fd 3 came from

open("/dev/zfs", O_RDWR)            = 3

Mark Shellenbaum

2008-Aug-18 20:16 UTC

head link

[zfs-discuss] ZFS commands hanging in B95

Ian Collins wrote:> I have a pretty standard ZFS boot AMD64 desktop.  A the moment, most ZFS
> related commands are hanging (can''t be killed) .  Running
''truss share''
> the last few lines I see are:
> 
Can you provide a kernel thread list report?  You can use mdb -k to get 
that.  Once in mdb type ::threadlist -v


   -Mark

Ian Collins

2008-Aug-19 05:13 UTC

head link

[zfs-discuss] ZFS commands hanging in B95

Mark Shellenbaum wrote:> Ian Collins wrote:
>> I have a pretty standard ZFS boot AMD64 desktop.  A the moment, most
ZFS
>> related commands are hanging (can''t be killed) .  Running
''truss share''
>> the last few lines I see are:
>>
>
> Can you provide a kernel thread list report?  You can use mdb -k to
> get that.  Once in mdb type ::threadlist -v
>Here you go:
> ::threadlist -v            ADDR             PROC              LWP CLS PRI            WCHAN
fffffffffbc29420 fffffffffbc287b0 fffffffffbc2b010   0  96                0
  PC: _resume_from_idle+0xf1    CMD: sched
  stack pointer for thread fffffffffbc29420: fffffffffbc48d60
  [ fffffffffbc48d60 _resume_from_idle+0xf1() ]
    swtch+0x17f()
    sched+0x423()
    main+0x419()
    _locore_start+0x92()

ffffff0007a05c80 fffffffffbc287b0                0   0  -1                0
  PC: _resume_from_idle+0xf1    THREAD: idle()
  stack pointer for thread ffffff0007a05c80: ffffff0007a05c00
  [ ffffff0007a05c00 _resume_from_idle+0xf1() ]
    cpu_idle+0xe9()
    idle+0x10e()
    thread_start+8()

ffffff0007a0bc80 fffffffffbc287b0                0   0  60 fffffffffbcaa8b0
  PC: _resume_from_idle+0xf1    THREAD: thread_reaper()
  stack pointer for thread ffffff0007a0bc80: ffffff0007a0bb90
  [ ffffff0007a0bb90 _resume_from_idle+0xf1() ]
    swtch+0x17f()
    cv_wait+0x61()
    thread_reaper+0xb7()
    thread_start+8()

ffffff0007a11c80 fffffffffbc287b0                0   0  60 ffffff01cce37e90
  PC: _resume_from_idle+0xf1    TASKQ: kmem_move_taskq
  stack pointer for thread ffffff0007a11c80: ffffff0007a11b60
  [ ffffff0007a11b60 _resume_from_idle+0xf1() ]
    swtch+0x17f()
    cv_wait+0x61()
    taskq_thread+0x11e()
    thread_start+8()

ffffff0007a17c80 fffffffffbc287b0                0   0  60 ffffff01cce37db8
  PC: _resume_from_idle+0xf1    TASKQ: kmem_taskq
  stack pointer for thread ffffff0007a17c80: ffffff0007a17b60
  [ ffffff0007a17b60 _resume_from_idle+0xf1() ]
    swtch+0x17f()
    cv_wait+0x61()
    taskq_thread+0x11e()
    thread_start+8()

ffffff0007a1dc80 fffffffffbc287b0                0   0  60 ffffff01cce37ce0
  PC: _resume_from_idle+0xf1    TASKQ: pseudo_nexus_enum_tq
  stack pointer for thread ffffff0007a1dc80: ffffff0007a1db60
  [ ffffff0007a1db60 _resume_from_idle+0xf1() ]
    swtch+0x17f()
    cv_wait+0x61()
    taskq_thread+0x11e()
    thread_start+8()

ffffff0007a23c80 fffffffffbc287b0                0   0  60 ffffff01cce37c08
  PC: _resume_from_idle+0xf1    TASKQ: scsi_vhci_nexus_enum_tq
  stack pointer for thread ffffff0007a23c80: ffffff0007a23b60
  [ ffffff0007a23b60 _resume_from_idle+0xf1() ]
    swtch+0x17f()
    cv_wait+0x61()

Ian

Mark Shellenbaum

2008-Aug-19 12:20 UTC

head link

[zfs-discuss] ZFS commands hanging in B95

Ian Collins wrote:> Mark Shellenbaum wrote:
>> Ian Collins wrote:
>>> I have a pretty standard ZFS boot AMD64 desktop.  A the moment,
most ZFS
>>> related commands are hanging (can''t be killed) .  Running
''truss share''
>>> the last few lines I see are:
>>>
>> Can you provide a kernel thread list report?  You can use mdb -k to
>> get that.  Once in mdb type ::threadlist -v
>>
> Here you go:
> 
That doesn''t appear to be a complete list.  We need to see the ZFS
threads.
>> ::threadlist -v
>             ADDR             PROC              LWP CLS PRI            WCHAN
> fffffffffbc29420 fffffffffbc287b0 fffffffffbc2b010   0  96                0
>   PC: _resume_from_idle+0xf1    CMD: sched
>   stack pointer for thread fffffffffbc29420: fffffffffbc48d60
>   [ fffffffffbc48d60 _resume_from_idle+0xf1() ]
>     swtch+0x17f()
>     sched+0x423()
>     main+0x419()
>     _locore_start+0x92()
> 
> ffffff0007a05c80 fffffffffbc287b0                0   0  -1                0
>   PC: _resume_from_idle+0xf1    THREAD: idle()
>   stack pointer for thread ffffff0007a05c80: ffffff0007a05c00
>   [ ffffff0007a05c00 _resume_from_idle+0xf1() ]
>     cpu_idle+0xe9()
>     idle+0x10e()
>     thread_start+8()
> 
> ffffff0007a0bc80 fffffffffbc287b0                0   0  60 fffffffffbcaa8b0
>   PC: _resume_from_idle+0xf1    THREAD: thread_reaper()
>   stack pointer for thread ffffff0007a0bc80: ffffff0007a0bb90
>   [ ffffff0007a0bb90 _resume_from_idle+0xf1() ]
>     swtch+0x17f()
>     cv_wait+0x61()
>     thread_reaper+0xb7()
>     thread_start+8()
> 
> ffffff0007a11c80 fffffffffbc287b0                0   0  60 ffffff01cce37e90
>   PC: _resume_from_idle+0xf1    TASKQ: kmem_move_taskq
>   stack pointer for thread ffffff0007a11c80: ffffff0007a11b60
>   [ ffffff0007a11b60 _resume_from_idle+0xf1() ]
>     swtch+0x17f()
>     cv_wait+0x61()
>     taskq_thread+0x11e()
>     thread_start+8()
> 
> ffffff0007a17c80 fffffffffbc287b0                0   0  60 ffffff01cce37db8
>   PC: _resume_from_idle+0xf1    TASKQ: kmem_taskq
>   stack pointer for thread ffffff0007a17c80: ffffff0007a17b60
>   [ ffffff0007a17b60 _resume_from_idle+0xf1() ]
>     swtch+0x17f()
>     cv_wait+0x61()
>     taskq_thread+0x11e()
>     thread_start+8()
> 
> ffffff0007a1dc80 fffffffffbc287b0                0   0  60 ffffff01cce37ce0
>   PC: _resume_from_idle+0xf1    TASKQ: pseudo_nexus_enum_tq
>   stack pointer for thread ffffff0007a1dc80: ffffff0007a1db60
>   [ ffffff0007a1db60 _resume_from_idle+0xf1() ]
>     swtch+0x17f()
>     cv_wait+0x61()
>     taskq_thread+0x11e()
>     thread_start+8()
> 
> ffffff0007a23c80 fffffffffbc287b0                0   0  60 ffffff01cce37c08
>   PC: _resume_from_idle+0xf1    TASKQ: scsi_vhci_nexus_enum_tq
>   stack pointer for thread ffffff0007a23c80: ffffff0007a23b60
>   [ ffffff0007a23b60 _resume_from_idle+0xf1() ]
>     swtch+0x17f()
>     cv_wait+0x61()
> 
> Ian
> 
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

zfs discuss - Aug 2008 - ZFS commands hanging in B95

[zfs-discuss] ZFS commands hanging in B95

[zfs-discuss] ZFS commands hanging in B95

[zfs-discuss] ZFS commands hanging in B95

[zfs-discuss] ZFS commands hanging in B95