Andrew Jones
2010-Jun-28 15:59 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Now at 36 hours since zdb process start and: PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP 827 root 4936M 4931M sleep 59 0 0:50:47 0.2% zdb/209 Idling at 0.2% processor for nearly the past 24 hours... feels very stuck. Thoughts on how to determine where and why? -- This message posted from opensolaris.org
Andrew Jones
2010-Jun-28 17:32 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Update: have given up on the zdb write mode repair effort, as least for now.
Hoping for any guidance / direction anyone''s willing to offer...
Re-running ''zpool import -F -f tank'' with some stack trace
debug, as suggested in similar threads elsewhere. Note that this appears hung at
near idle.
ffffff03e278c520 ffffff03e9c60038 ffffff03ef109490 1 60 ffffff0530db4680
PC: _resume_from_idle+0xf1 CMD: zpool import -F -f tank
stack pointer for thread ffffff03e278c520: ffffff00182bbff0
[ ffffff00182bbff0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
zio_wait+0x5d()
dbuf_read+0x1e8()
dnode_next_offset_level+0x129()
dnode_next_offset+0xa2()
get_next_chunk+0xa5()
dmu_free_long_range_impl+0x9e()
dmu_free_object+0xe6()
dsl_dataset_destroy+0x122()
dsl_destroy_inconsistent+0x5f()
findfunc+0x23()
dmu_objset_find_spa+0x38c()
dmu_objset_find_spa+0x153()
dmu_objset_find+0x40()
spa_load_impl+0xb23()
spa_load+0x117()
spa_load_best+0x78()
spa_import+0xee()
zfs_ioc_pool_import+0xc0()
zfsdev_ioctl+0x177()
cdev_ioctl+0x45()
spec_ioctl+0x5a()
fop_ioctl+0x7b()
ioctl+0x18e()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
--
This message posted from opensolaris.org
Roy Sigurd Karlsbakk
2010-Jun-28 18:54 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
----- Original Message -----> Now at 36 hours since zdb process start and: > > > PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP > 827 root 4936M 4931M sleep 59 0 0:50:47 0.2% zdb/209 > > Idling at 0.2% processor for nearly the past 24 hours... feels very > stuck. Thoughts on how to determine where and why?Just a hunch, is this pool using dedup? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.
Malachi de Ælfweald
2010-Jun-28 19:16 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
I had a similar issue on boot after upgrade in the past and it was due to the large number of snapshots I had... don''t know if that could be related or not... Malachi de ?lfweald http://www.google.com/profiles/malachid On Mon, Jun 28, 2010 at 8:59 AM, Andrew Jones <andrewnjones at gmail.com>wrote:> Now at 36 hours since zdb process start and: > > > PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP > 827 root 4936M 4931M sleep 59 0 0:50:47 0.2% zdb/209 > > Idling at 0.2% processor for nearly the past 24 hours... feels very stuck. > Thoughts on how to determine where and why? > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100628/73b2881f/attachment.html>
Andrew Jones
2010-Jun-28 19:25 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Dedup had been turned on in the past for some of the volumes, but I had turned it off altogether before entering production due to performance issues. GZIP compression was turned on for the volume I was trying to delete. -- This message posted from opensolaris.org
Andrew Jones
2010-Jun-28 19:26 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Malachi, Thanks for the reply. There were no snapshots for the CSV1 volume that I recall... very few snapshots on the any volume in the tank. -- This message posted from opensolaris.org
Roy Sigurd Karlsbakk
2010-Jun-28 19:43 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
----- Original Message -----> Dedup had been turned on in the past for some of the volumes, but I > had turned it off altogether before entering production due to > performance issues. GZIP compression was turned on for the volume I > was trying to delete.Was there a lot of deduped data still on disk before it was put into production? Turning off dedup won''t dedup the data, just inhibit deduplication of new data... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.
Andrew Jones
2010-Jun-28 19:50 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Just re-ran ''zdb -e tank'' to confirm the CSV1 volume is still exhibiting error 16: <snip> Could not open tank/CSV1, error 16 <snip> Considering my attempt to delete the CSV1 volume lead to the failure in the first place, I have to think that if I can either 1) complete the deletion of this volume or 2) roll back to a transaction prior to this based on logging or 3) repair whatever corruption has been caused by this partial deletion, that I will then be able to import the pool. What does ''error 16'' mean in the ZDB output, any suggestions? -- This message posted from opensolaris.org
Geoff Shipman
2010-Jun-28 20:29 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Andrew,
Looks like the zpool is telling you the devices are still doing work of
some kind, or that there are locks still held.
From man of section 2 intro page the errors are listed. Number 16
looks to be an EBUSY.
16 EBUSY Device busy
An attempt was made to mount a dev-
ice that was already mounted or an
attempt was made to unmount a device
on which there is an active file
(open file, current directory,
mounted-on file, active text seg-
ment). It will also occur if an
attempt is made to enable accounting
when it is already enabled. The
device or resource is currently una-
vailable. EBUSY is also used by
mutexes, semaphores, condition vari-
ables, and r/w locks, to indicate
that a lock is held, and by the
processor control function
P_ONLINE.
On 06/28/10 01:50 PM, Andrew Jones wrote:> Just re-ran ''zdb -e tank'' to confirm the CSV1 volume is
still exhibiting error 16:
>
> <snip>
> Could not open tank/CSV1, error 16
> <snip>
>
> Considering my attempt to delete the CSV1 volume lead to the failure in the
first place, I have to think that if I can either 1) complete the deletion of
this volume or 2) roll back to a transaction prior to this based on logging or
3) repair whatever corruption has been caused by this partial deletion, that I
will then be able to import the pool.
>
> What does ''error 16'' mean in the ZDB output, any
suggestions?
>
--
Geoff Shipman | Senior Technical Support Engineer
Phone: +13034644710
Oracle Global Customer Services
500 Eldorado Blvd. UBRM-04 | Broomfield, CO 80021
Email: geoff.shipman at sun.com | Hours:9am-5pm MT,Monday-Friday
Victor Latushkin
2010-Jun-28 20:49 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
On Jun 28, 2010, at 9:32 PM, Andrew Jones wrote:> Update: have given up on the zdb write mode repair effort, as least for now. Hoping for any guidance / direction anyone''s willing to offer... > > Re-running ''zpool import -F -f tank'' with some stack trace debug, as suggested in similar threads elsewhere. Note that this appears hung at near idle.It looks like it is processing huge inconsistent data set that was destroyed previously. So you need to wait a bit longer. regards victor> > > ffffff03e278c520 ffffff03e9c60038 ffffff03ef109490 1 60 ffffff0530db4680 > PC: _resume_from_idle+0xf1 CMD: zpool import -F -f tank > stack pointer for thread ffffff03e278c520: ffffff00182bbff0 > [ ffffff00182bbff0 _resume_from_idle+0xf1() ] > swtch+0x145() > cv_wait+0x61() > zio_wait+0x5d() > dbuf_read+0x1e8() > dnode_next_offset_level+0x129() > dnode_next_offset+0xa2() > get_next_chunk+0xa5() > dmu_free_long_range_impl+0x9e() > dmu_free_object+0xe6() > dsl_dataset_destroy+0x122() > dsl_destroy_inconsistent+0x5f() > findfunc+0x23() > dmu_objset_find_spa+0x38c() > dmu_objset_find_spa+0x153() > dmu_objset_find+0x40() > spa_load_impl+0xb23() > spa_load+0x117() > spa_load_best+0x78() > spa_import+0xee() > zfs_ioc_pool_import+0xc0() > zfsdev_ioctl+0x177() > cdev_ioctl+0x45() > spec_ioctl+0x5a() > fop_ioctl+0x7b() > ioctl+0x18e() > dtrace_systrace_syscall32+0x11a() > _sys_sysenter_post_swapgs+0x149() > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Andrew Jones
2010-Jun-28 22:06 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Thanks Victor. I will give it another 24 hrs or so and will let you know how it goes... You are right, a large 2TB volume (CSV1) was not in the process of being deleted, as described above. It is showing error 16 on ''zdb -e'' -- This message posted from opensolaris.org
Andrew Jones
2010-Jun-29 16:30 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Victor, The ''zpool import -f -F tank'' failed at some point last night. The box was completely hung this morning; no core dump, no ability to SSH into the box to diagnose the problem. I had no choice but to reset, as I had no diagnostic ability. I don''t know if there would be anything in the logs? Earlier I ran ''zdb -e -bcsvL tank'' in write mode for 36 hours and gave up to try something different. Now the zpool import has hung the box. Should I try zdb again? Any suggestions? Thanks, Andrew -- This message posted from opensolaris.org
Victor Latushkin
2010-Jun-29 22:31 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
On Jun 29, 2010, at 8:30 PM, Andrew Jones wrote:> Victor, > > The ''zpool import -f -F tank'' failed at some point last night. The box was completely hung this morning; no core dump, no ability to SSH into the box to diagnose the problem. I had no choice but to reset, as I had no diagnostic ability. I don''t know if there would be anything in the logs?It sounds like it might run out of memory. Is it an option for you to add more memory to the box temporarily? Even if it is an option, it is good to prepare for such outcome and have kmdb loaded either at boot time by adding -k to ''kernel$'' line in GRUB menu, or by loading it from console with ''mdb -K'' before attempting import (type '':c'' at mdb prompt to continue). In case it hangs again, you can press ''F1-A'' on the keyboard, drop into kmdb and then use ''$<systemdump'' to force a crashdump. If you hardware has physical or virtual NMI button, you can use that too to drop into kmdb, but you''ll need to set a kernel variable for that to work: http://blogs.sun.com/darren/entry/sending_a_break_to_opensolaris> Earlier I ran ''zdb -e -bcsvL tank'' in write mode for 36 hours and gave up to try something different. Now the zpool import has hung the box.What do you mean be running zdb in write mode? zdb normally is readonly tool. Did you change it in some way?> Should I try zdb again? Any suggestions?It sounds like zdb is not going to be helpful, as inconsistent dataset processing happens only in read-write mode. So you need to try above suggestions with more memory and kmdb/nmi. victor
Andrew Jones
2010-Jun-30 02:36 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
> > On Jun 29, 2010, at 8:30 PM, Andrew Jones wrote: > > > Victor, > > > > The ''zpool import -f -F tank'' failed at some point > last night. The box was completely hung this morning; > no core dump, no ability to SSH into the box to > diagnose the problem. I had no choice but to reset, > as I had no diagnostic ability. I don''t know if there > would be anything in the logs? > > It sounds like it might run out of memory. Is it an > option for you to add more memory to the box > temporarily?I''ll place the order for more memory or transfer some from another machine. Seems quite likely that we did run out of memory.> > Even if it is an option, it is good to prepare for > such outcome and have kmdb loaded either at boot time > by adding -k to ''kernel$'' line in GRUB menu, or by > loading it from console with ''mdb -K'' before > attempting import (type '':c'' at mdb prompt to > continue). In case it hangs again, you can press > ''F1-A'' on the keyboard, drop into kmdb and then use > ''$<systemdump'' to force a crashdump.I''ll prepare the machine this way and repeat the import to reproduce the hang, then break into the kernel and capture the core dump.> > If you hardware has physical or virtual NMI button, > you can use that too to drop into kmdb, but you''ll > need to set a kernel variable for that to work: > > http://blogs.sun.com/darren/entry/sending_a_break_to_o > pensolaris > > > Earlier I ran ''zdb -e -bcsvL tank'' in write mode > for 36 hours and gave up to try something different. > Now the zpool import has hung the box. > > What do you mean be running zdb in write mode? zdb > normally is readonly tool. Did you change it in some > way?I had read elsewhere that set /zfs/:zfs_recover=/1/ and set aok=/1/ placed zdb into some kind of a write/recovery mode. I have set these in /etc/system. Is this a bad idea in this case?> > > Should I try zdb again? Any suggestions? > > It sounds like zdb is not going to be helpful, as > inconsistent dataset processing happens only in > read-write mode. So you need to try above suggestions > with more memory and kmdb/nmi.Will do, thanks!> > victor > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discu > ss >-- This message posted from opensolaris.org
Andrew Jones
2010-Jul-01 06:28 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Victor, I''ve reproduced the crash and have vmdump.0 and dump device files. How do I query the stack on crash for your analysis? What other analysis should I provide? Thanks -- This message posted from opensolaris.org
Andrew Jones
2010-Jul-01 22:20 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Victor, A little more info on the crash, from the messages file is attached here. I have also decompressed the dump with savecore to generate unix.0, vmcore.0, and vmdump.0. Jun 30 19:39:10 HL-SAN unix: [ID 836849 kern.notice] Jun 30 19:39:10 HL-SAN ^Mpanic[cpu3]/thread=ffffff0017909c60: Jun 30 19:39:10 HL-SAN genunix: [ID 335743 kern.notice] BAD TRAP: type=e (#pf Page fault) rp=ffffff0017909790 addr=0 occurred in module "<unknown>" due to a NULL pointer dereference Jun 30 19:39:10 HL-SAN unix: [ID 100000 kern.notice] Jun 30 19:39:10 HL-SAN unix: [ID 839527 kern.notice] sched: Jun 30 19:39:10 HL-SAN unix: [ID 753105 kern.notice] #pf Page fault Jun 30 19:39:10 HL-SAN unix: [ID 532287 kern.notice] Bad kernel fault at addr=0x0 Jun 30 19:39:10 HL-SAN unix: [ID 243837 kern.notice] pid=0, pc=0x0, sp=0xffffff0017909880, eflags=0x10002 Jun 30 19:39:10 HL-SAN unix: [ID 211416 kern.notice] cr0: 8005003b<pg,wp,ne,et,ts,mp,pe> cr4: 6f8<xmme,fxsr,pge,mce,pae,pse,de> Jun 30 19:39:10 HL-SAN unix: [ID 624947 kern.notice] cr2: 0 Jun 30 19:39:10 HL-SAN unix: [ID 625075 kern.notice] cr3: 336a71000 Jun 30 19:39:10 HL-SAN unix: [ID 625715 kern.notice] cr8: c Jun 30 19:39:10 HL-SAN unix: [ID 100000 kern.notice] Jun 30 19:39:10 HL-SAN unix: [ID 592667 kern.notice] rdi: 282 rsi: 15809 rdx: ffffff03edb1e538 Jun 30 19:39:10 HL-SAN unix: [ID 592667 kern.notice] rcx: 5 r8: 0 r9: ffffff03eb2d6a00 Jun 30 19:39:10 HL-SAN unix: [ID 592667 kern.notice] rax: 202 rbx: 0 rbp: ffffff0017909880 Jun 30 19:39:10 HL-SAN unix: [ID 592667 kern.notice] r10: fffffffff80d16d0 r11: 4 r12: 0 Jun 30 19:39:10 HL-SAN unix: [ID 592667 kern.notice] r13: ffffff03e21bca40 r14: ffffff03e1a0d7e8 r15: ffffff03e21bcb58 Jun 30 19:39:10 HL-SAN unix: [ID 592667 kern.notice] fsb: 0 gsb: ffffff03e25fa580 ds: 4b Jun 30 19:39:10 HL-SAN unix: [ID 592667 kern.notice] es: 4b fs: 0 gs: 1c3 Jun 30 19:39:10 HL-SAN unix: [ID 592667 kern.notice] trp: e err: 10 rip: 0 Jun 30 19:39:10 HL-SAN unix: [ID 592667 kern.notice] cs: 30 rfl: 10002 rsp: ffffff0017909880 Jun 30 19:39:10 HL-SAN unix: [ID 266532 kern.notice] ss: 38 Jun 30 19:39:10 HL-SAN unix: [ID 100000 kern.notice] Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909670 unix:die+dd () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909780 unix:trap+177b () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909790 unix:cmntrap+e6 () Jun 30 19:39:10 HL-SAN genunix: [ID 802836 kern.notice] ffffff0017909880 0 () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff00179098a0 unix:debug_enter+38 () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff00179098c0 unix:abort_sequence_enter+35 () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909910 kbtrans:kbtrans_streams_key+102 () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909940 conskbd:conskbdlrput+e7 () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff00179099b0 unix:putnext+21e () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff00179099f0 kbtrans:kbtrans_queueevent+7c () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909a20 kbtrans:kbtrans_queuepress+7c () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909a60 kbtrans:kbtrans_untrans_keypressed_raw+46 () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909a90 kbtrans:kbtrans_processkey+32 () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909ae0 kbtrans:kbtrans_streams_key+175 () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909b10 kb8042:kb8042_process_key+40 () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909b50 kb8042:kb8042_received_byte+109 () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909b80 kb8042:kb8042_intr+6a () Jun 30 19:39:10 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909bb0 i8042:i8042_intr+c5 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909c00 unix:av_dispatch_autovect+7c () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0017909c40 unix:dispatch_hardint+33 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff00183552f0 unix:switch_sp_and_call+13 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355340 unix:do_interrupt+b8 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355350 unix:_interrupt+b8 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff00183554a0 unix:htable_steal+198 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355510 unix:htable_alloc+248 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff00183555c0 unix:htable_create+1dc () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355640 unix:hati_load_common+a3 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff00183556b0 unix:hat_memload+81 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff00183556f0 unix:hat_memload_region+25 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff00183557e0 genunix:segvn_faultpage+937 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355990 genunix:segvn_fault+c13 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355aa0 genunix:as_fault+5ee () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355b20 unix:pagefault+99 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355c30 unix:trap+1624 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355c40 unix:cmntrap+e6 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355d50 unix:bcopy_altentry+1b4 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355df0 genunix:lwp_park+242 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355e30 genunix:syslwp_park+31 () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355ec0 genunix:dtrace_systrace_syscall32+11a () Jun 30 19:39:11 HL-SAN genunix: [ID 655072 kern.notice] ffffff0018355f10 unix:brand_sys_syscall32+192 () Jun 30 19:39:11 HL-SAN unix: [ID 100000 kern.notice] Jun 30 19:39:11 HL-SAN genunix: [ID 672855 kern.notice] syncing file systems... Jun 30 19:39:40 HL-SAN unix: [ID 836849 kern.notice] Jun 30 19:39:40 HL-SAN ^Mpanic[cpu3]/thread=ffffff0017909c60: Jun 30 19:39:40 HL-SAN genunix: [ID 715357 kern.notice] panic sync timeout Jun 30 19:39:40 HL-SAN unix: [ID 100000 kern.notice] Jun 30 19:39:40 HL-SAN genunix: [ID 111219 kern.notice] dumping to /dev/zvol/dsk/syspool/dump, offset 65536, content: kernel + curproc Jun 30 19:39:40 HL-SAN ahci: [ID 405573 kern.info] NOTICE: ahci0: ahci_tran_reset_dport port 2 reset port Jun 30 19:39:40 HL-SAN ahci: [ID 405573 kern.info] NOTICE: ahci0: ahci_tran_reset_dport port 3 reset port Jun 30 20:04:36 HL-SAN genunix: [ID 100000 kern.notice] Jun 30 20:04:36 HL-SAN genunix: [ID 665016 kern.notice] ^M100% done: 3075312 pages dumped, Jun 30 20:04:36 HL-SAN genunix: [ID 851671 kern.notice] dump succeeded Jun 30 22:47:45 HL-SAN genunix: [ID 540533 kern.notice] ^MSunOS Release 5.11 Version NexentaOS_134f 64-bit -- This message posted from opensolaris.org
Victor Latushkin
2010-Jul-02 19:43 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
On Jul 1, 2010, at 10:28 AM, Andrew Jones wrote:> Victor, > > I''ve reproduced the crash and have vmdump.0 and dump device files. How do I query the stack on crash for your analysis? What other analysis should I provide?Output of ''echo "::threadlist -v" | mdb 0'' can be a good start in this case I think. It can be rather big to send to the list, so please find some other way to provide it.
Andrew Jones
2010-Jul-02 20:14 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Victor,
I have attached the entire threadlist.txt for your review.
The most interesting one from what I can see:
ffffff03e30ff900 ffffff03eee70e48 ffffff03e0614bb0 1 59 ffffff03f1411ece
PC: _resume_from_idle+0xf1 CMD: zpool import -f -F tank
stack pointer for thread ffffff03e30ff900: ffffff0017bcd160
[ ffffff0017bcd160 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
txg_wait_open+0x7a()
dmu_tx_wait+0xb3()
dmu_tx_assign+0x4b()
dmu_free_long_range_impl+0x12b()
dmu_free_object+0xe6()
dsl_dataset_destroy+0x122()
dsl_destroy_inconsistent+0x5f()
findfunc+0x23()
dmu_objset_find_spa+0x38c()
dmu_objset_find_spa+0x153()
dmu_objset_find+0x40()
spa_load_impl+0xb23()
spa_load+0x117()
spa_load_best+0x78()
spa_import+0xee()
zfs_ioc_pool_import+0xc0()
zfsdev_ioctl+0x177()
cdev_ioctl+0x45()
spec_ioctl+0x5a()
fop_ioctl+0x7b()
ioctl+0x18e()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
Thanks again for your continued assistance.
/Andrew
On Fri, Jul 2, 2010 at 12:43 PM, Victor Latushkin
<Victor.Latushkin at sun.com>wrote:
>
> On Jul 1, 2010, at 10:28 AM, Andrew Jones wrote:
>
> > Victor,
> >
> > I''ve reproduced the crash and have vmdump.0 and dump device
files. How do
> I query the stack on crash for your analysis? What other analysis should I
> provide?
>
> Output of ''echo "::threadlist -v" | mdb 0'' can
be a good start in this case
> I think. It can be rather big to send to the list, so please find some
other
> way to provide it.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100702/00fc7533/attachment.html>
-------------- next part --------------
ADDR PROC LWP CLS PRI WCHAN
fffffffffbc2e860 fffffffffbc2dbf0 fffffffffbc30320 0 96 ffffff03dd38e24e
PC: _resume_from_idle+0xf1 CMD: sched
stack pointer for thread fffffffffbc2e860: fffffffffbc50430
[ fffffffffbc50430 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
txg_wait_open+0x7a()
dmu_tx_wait+0xb3()
dmu_tx_assign+0x4b()
zvol_strategy+0x1d4()
bdev_strategy+0x75()
spec_startio+0xb4()
spec_pageio+0x39()
fop_pageio+0x80()
swap_putapage+0x443()
swap_putpage+0x1be()
fop_putpage+0x74()
segkp_unlock+0x14b()
segkp_fault+0xf5()
swapout+0x24e()
sched+0x31d()
main+0x46b()
_locore_start+0x92()
ffffff0017005c60 fffffffffbc2dbf0 0 0 -1 0
PC: _resume_from_idle+0xf1 THREAD: idle()
stack pointer for thread ffffff0017005c60: ffffff0017005bf0
[ ffffff0017005bf0 _resume_from_idle+0xf1() ]
swtch+0x145()
idle+0xc4()
thread_start+8()
ffffff001700bc60 fffffffffbc2dbf0 0 0 60 fffffffffbcb48a8
PC: _resume_from_idle+0xf1 THREAD: thread_reaper()
stack pointer for thread ffffff001700bc60: ffffff001700bb80
[ ffffff001700bb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
thread_reaper+0xb1()
thread_start+8()
ffffff0017011c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5e70
PC: _resume_from_idle+0xf1 TASKQ: kmem_move_taskq
stack pointer for thread ffffff0017011c60: ffffff0017011ae0
[ ffffff0017011ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017017c60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 TASKQ: kmem_taskq
stack pointer for thread ffffff0017017c60: ffffff0017017ae0
[ ffffff0017017ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001701dc60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5c40
PC: _resume_from_idle+0xf1 TASKQ: pseudo_nexus_enum_tq
stack pointer for thread ffffff001701dc60: ffffff001701dae0
[ ffffff001701dae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017023c60 fffffffffbc2dbf0 0 0 60 fffffffffbd01f80
PC: _resume_from_idle+0xf1 THREAD: scsi_hba_barrier_daemon()
stack pointer for thread ffffff0017023c60: ffffff0017023b60
[ ffffff0017023b60 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
scsi_hba_barrier_daemon+0x9f()
thread_start+8()
ffffff0017029c60 fffffffffbc2dbf0 0 0 60 fffffffffbd01f82
PC: _resume_from_idle+0xf1 THREAD: scsi_lunchg1_daemon()
stack pointer for thread ffffff0017029c60: ffffff0017029690
[ ffffff0017029690 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
scsi_lunchg1_daemon+0x1f7()
thread_start+8()
ffffff001702fc60 fffffffffbc2dbf0 0 0 60 fffffffffbd01f84
PC: _resume_from_idle+0xf1 THREAD: scsi_lunchg2_daemon()
stack pointer for thread ffffff001702fc60: ffffff001702fb80
[ ffffff001702fb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
scsi_lunchg2_daemon+0x120()
thread_start+8()
ffffff0017035c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5b28
PC: _resume_from_idle+0xf1 TASKQ: scsi_vhci_nexus_enum_tq
stack pointer for thread ffffff0017035c60: ffffff0017035ae0
[ ffffff0017035ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001708fc60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5a10
PC: _resume_from_idle+0xf1 TASKQ: mdi_taskq
stack pointer for thread ffffff001708fc60: ffffff001708fae0
[ ffffff001708fae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001707dc60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5a10
PC: _resume_from_idle+0xf1 TASKQ: mdi_taskq
stack pointer for thread ffffff001707dc60: ffffff001707dae0
[ ffffff001707dae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001706bc60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5a10
PC: _resume_from_idle+0xf1 TASKQ: mdi_taskq
stack pointer for thread ffffff001706bc60: ffffff001706bae0
[ ffffff001706bae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001705fc60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5a10
PC: _resume_from_idle+0xf1 TASKQ: mdi_taskq
stack pointer for thread ffffff001705fc60: ffffff001705fae0
[ ffffff001705fae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017053c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5a10
PC: _resume_from_idle+0xf1 TASKQ: mdi_taskq
stack pointer for thread ffffff0017053c60: ffffff0017053ae0
[ ffffff0017053ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017047c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5a10
PC: _resume_from_idle+0xf1 TASKQ: mdi_taskq
stack pointer for thread ffffff0017047c60: ffffff0017047ae0
[ ffffff0017047ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017041c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5a10
PC: _resume_from_idle+0xf1 TASKQ: mdi_taskq
stack pointer for thread ffffff0017041c60: ffffff0017041ae0
[ ffffff0017041ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001703bc60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5a10
PC: _resume_from_idle+0xf1 TASKQ: mdi_taskq
stack pointer for thread ffffff001703bc60: ffffff001703bae0
[ ffffff001703bae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001704dc60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c58f8
PC: _resume_from_idle+0xf1 TASKQ: vhci_taskq
stack pointer for thread ffffff001704dc60: ffffff001704dae0
[ ffffff001704dae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017161c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c57e0
PC: _resume_from_idle+0xf1 TASKQ: vhci_update_pathstates
stack pointer for thread ffffff0017161c60: ffffff0017161ae0
[ ffffff0017161ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017155c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c57e0
PC: _resume_from_idle+0xf1 TASKQ: vhci_update_pathstates
stack pointer for thread ffffff0017155c60: ffffff0017155ae0
[ ffffff0017155ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017107c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c57e0
PC: _resume_from_idle+0xf1 TASKQ: vhci_update_pathstates
stack pointer for thread ffffff0017107c60: ffffff0017107ae0
[ ffffff0017107ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017095c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c57e0
PC: _resume_from_idle+0xf1 TASKQ: vhci_update_pathstates
stack pointer for thread ffffff0017095c60: ffffff0017095ae0
[ ffffff0017095ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017083c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c57e0
PC: _resume_from_idle+0xf1 TASKQ: vhci_update_pathstates
stack pointer for thread ffffff0017083c60: ffffff0017083ae0
[ ffffff0017083ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017071c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c57e0
PC: _resume_from_idle+0xf1 TASKQ: vhci_update_pathstates
stack pointer for thread ffffff0017071c60: ffffff0017071ae0
[ ffffff0017071ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017065c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c57e0
PC: _resume_from_idle+0xf1 TASKQ: vhci_update_pathstates
stack pointer for thread ffffff0017065c60: ffffff0017065ae0
[ ffffff0017065ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017059c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c57e0
PC: _resume_from_idle+0xf1 TASKQ: vhci_update_pathstates
stack pointer for thread ffffff0017059c60: ffffff0017059ae0
[ ffffff0017059ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017077c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c56c8
PC: _resume_from_idle+0xf1 TASKQ: npe_nexus_enum_tq
stack pointer for thread ffffff0017077c60: ffffff0017077ae0
[ ffffff0017077ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017089c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c55b0
PC: _resume_from_idle+0xf1 TASKQ: isa_nexus_enum_tq
stack pointer for thread ffffff0017089c60: ffffff0017089ae0
[ ffffff0017089ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00170cbc60 fffffffffbc2dbf0 ffffff03e2b84e50 0 169 0
PC: panic_idle+0x20 THREAD: unix`thread_create_intr()
stack pointer for thread ffffff00170cbc60: ffffff00170cb810
0xf0()
do_interrupt+0xf2()
_interrupt+0xb8()
todpc_rtcput+0xb7()
todpc_set+0xd5()
tod_set+0x35()
clock+0x75c()
cyclic_softint+0xdc()
cbe_softclock+0x1a()
av_dispatch_softvect+0x5f()
dispatch_softint+0x34()
switch_sp_and_call+0x13()
dosoftint+0x59()
do_interrupt+0x114()
_interrupt+0xb8()
mutex_enter+0x10()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0xe45()
0xfffffffffb8001d6()
ffffff00170e9c60 fffffffffbc2dbf0 0 0 99 ffffff03dd4c5498
PC: _resume_from_idle+0xf1 TASKQ: timeout_taskq
stack pointer for thread ffffff00170e9c60: ffffff00170e9ae0
[ ffffff00170e9ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00170e3c60 fffffffffbc2dbf0 0 0 99 ffffff03dd4c5498
PC: _resume_from_idle+0xf1 TASKQ: timeout_taskq
stack pointer for thread ffffff00170e3c60: ffffff00170e3ae0
[ ffffff00170e3ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00170ddc60 fffffffffbc2dbf0 0 0 99 ffffff03dd4c5498
PC: _resume_from_idle+0xf1 TASKQ: timeout_taskq
stack pointer for thread ffffff00170ddc60: ffffff00170ddae0
[ ffffff00170ddae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00170d7c60 fffffffffbc2dbf0 0 0 99 ffffff03dd4c5498
PC: _resume_from_idle+0xf1 TASKQ: timeout_taskq
stack pointer for thread ffffff00170d7c60: ffffff00170d7ae0
[ ffffff00170d7ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00170efc60 fffffffffbc2dbf0 0 0 99 fffffffffbc906d0
PC: _resume_from_idle+0xf1 THREAD: timeout_taskq_thread()
stack pointer for thread ffffff00170efc60: ffffff00170efb80
[ ffffff00170efb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
timeout_taskq_thread+0xa7()
thread_start+8()
ffffff00170fbc60 fffffffffbc2dbf0 0 0 99 ffffff03dd4c5380
PC: _resume_from_idle+0xf1 TASKQ: callout_taskq
stack pointer for thread ffffff00170fbc60: ffffff00170fbae0
[ ffffff00170fbae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00170f5c60 fffffffffbc2dbf0 0 0 99 ffffff03dd4c5380
PC: _resume_from_idle+0xf1 TASKQ: callout_taskq
stack pointer for thread ffffff00170f5c60: ffffff00170f5ae0
[ ffffff00170f5ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017ab8c60 fffffffffbc2dbf0 0 0 99 ffffff03e1552a50
PC: _resume_from_idle+0xf1 THREAD: squeue_worker()
stack pointer for thread ffffff0017ab8c60: ffffff0017ab8b80
[ ffffff0017ab8b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_worker+0xc0()
thread_start+8()
ffffff0017abec60 fffffffffbc2dbf0 0 0 99 ffffff03e1552a52
PC: _resume_from_idle+0xf1 THREAD: squeue_polling_thread()
stack pointer for thread ffffff0017abec60: ffffff0017abeb50
[ ffffff0017abeb50 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_polling_thread+0x90()
thread_start+8()
ffffff0017ac4c60 fffffffffbc2dbf0 0 0 99 ffffff03e1552990
PC: _resume_from_idle+0xf1 THREAD: squeue_worker()
stack pointer for thread ffffff0017ac4c60: ffffff0017ac4b80
[ ffffff0017ac4b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_worker+0xc0()
thread_start+8()
ffffff0017acac60 fffffffffbc2dbf0 0 0 99 ffffff03e1552992
PC: _resume_from_idle+0xf1 THREAD: squeue_polling_thread()
stack pointer for thread ffffff0017acac60: ffffff0017acab50
[ ffffff0017acab50 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_polling_thread+0x90()
thread_start+8()
ffffff0017a0fc60 fffffffffbc2dbf0 0 0 99 ffffff03e5093ce8
PC: _resume_from_idle+0xf1 THREAD: mac_srs_worker()
stack pointer for thread ffffff0017a0fc60: ffffff0017a0fb80
[ ffffff0017a0fb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_srs_worker+0x1b4()
thread_start+8()
ffffff0017a15c60 fffffffffbc2dbf0 0 0 99 ffffff03e5094968
PC: _resume_from_idle+0xf1 THREAD: mac_srs_worker()
stack pointer for thread ffffff0017a15c60: ffffff0017a15b80
[ ffffff0017a15b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_srs_worker+0x1b4()
thread_start+8()
ffffff0017a1bc60 fffffffffbc2dbf0 0 0 99 ffffff03e603f6e8
PC: _resume_from_idle+0xf1 THREAD: mac_srs_worker()
stack pointer for thread ffffff0017a1bc60: ffffff0017a1bb80
[ ffffff0017a1bb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_srs_worker+0x1b4()
thread_start+8()
ffffff0017a8ec60 fffffffffbc2dbf0 0 0 99 ffffff03e603f6ea
PC: _resume_from_idle+0xf1 THREAD: mac_rx_srs_poll_ring()
stack pointer for thread ffffff0017a8ec60: ffffff0017a8eb60
[ ffffff0017a8eb60 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_rx_srs_poll_ring+0xa0()
thread_start+8()
ffffff0017a94c60 fffffffffbc2dbf0 0 0 99 ffffff03e507f6e8
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff0017a94c60: ffffff0017a94b80
[ ffffff0017a94b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff0017a9ac60 fffffffffbc2dbf0 0 0 99 ffffff03e507f868
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff0017a9ac60: ffffff0017a9ab80
[ ffffff0017a9ab80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff0017aa0c60 fffffffffbc2dbf0 0 0 99 ffffff03e507f9e8
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff0017aa0c60: ffffff0017aa0b80
[ ffffff0017aa0b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff0017aa6c60 fffffffffbc2dbf0 0 0 99 ffffff03e507fb68
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff0017aa6c60: ffffff0017aa6b80
[ ffffff0017aa6b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff0017aacc60 fffffffffbc2dbf0 0 0 99 ffffff03e507fce8
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff0017aacc60: ffffff0017aacb80
[ ffffff0017aacb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff0017ab2c60 fffffffffbc2dbf0 0 0 99 0
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff0017ab2c60: ffffff0017ab2b80
[ ffffff0017ab2b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff0017b54c60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 TASKQ: system_taskq
stack pointer for thread ffffff0017b54c60: ffffff0017b54a80
[ ffffff0017b54a80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_hires+0xe0()
cv_reltimedwait+0x4f()
taskq_thread_wait+0x74()
taskq_d_thread+0x144()
thread_start+8()
ffffff0017ed3c60 fffffffffbc2dbf0 0 0 60 ffffff05d9edade0
PC: _resume_from_idle+0xf1 TASKQ: system_taskq
stack pointer for thread ffffff0017ed3c60: ffffff0017ed3a80
[ ffffff0017ed3a80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_hires+0xe0()
cv_reltimedwait+0x4f()
taskq_thread_wait+0x74()
taskq_d_thread+0x144()
thread_start+8()
ffffff0019cb6c60 fffffffffbc2dbf0 0 0 60 ffffff05d8827fa8
PC: _resume_from_idle+0xf1 TASKQ: system_taskq
stack pointer for thread ffffff0019cb6c60: ffffff0019cb6a80
[ ffffff0019cb6a80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_hires+0xe0()
cv_reltimedwait+0x4f()
taskq_thread_wait+0x74()
taskq_d_thread+0x144()
thread_start+8()
ffffff0017101c60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 TASKQ: system_taskq
stack pointer for thread ffffff0017101c60: ffffff0017101ae0
[ ffffff0017101ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017a57c60 fffffffffbc2dbf0 0 0 99 ffffff03e60409a8
PC: _resume_from_idle+0xf1 THREAD: mac_srs_worker()
stack pointer for thread ffffff0017a57c60: ffffff0017a57b80
[ ffffff0017a57b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_srs_worker+0x1b4()
thread_start+8()
ffffff0017a5dc60 fffffffffbc2dbf0 0 0 99 ffffff03e6040368
PC: _resume_from_idle+0xf1 THREAD: mac_srs_worker()
stack pointer for thread ffffff0017a5dc60: ffffff0017a5db80
[ ffffff0017a5db80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_srs_worker+0x1b4()
thread_start+8()
ffffff0017a63c60 fffffffffbc2dbf0 0 0 99 ffffff03e603fd28
PC: _resume_from_idle+0xf1 THREAD: mac_srs_worker()
stack pointer for thread ffffff0017a63c60: ffffff0017a63b80
[ ffffff0017a63b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_srs_worker+0x1b4()
thread_start+8()
ffffff0017a69c60 fffffffffbc2dbf0 0 0 99 ffffff03e603fd2a
PC: _resume_from_idle+0xf1 THREAD: mac_rx_srs_poll_ring()
stack pointer for thread ffffff0017a69c60: ffffff0017a69b60
[ ffffff0017a69b60 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_rx_srs_poll_ring+0xa0()
thread_start+8()
ffffff0017a6fc60 fffffffffbc2dbf0 0 0 99 ffffff03e5f44ba8
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff0017a6fc60: ffffff0017a6fb80
[ ffffff0017a6fb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff00179cdc60 fffffffffbc2dbf0 0 0 99 ffffff03e5f44a28
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff00179cdc60: ffffff00179cdb80
[ ffffff00179cdb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff00179d3c60 fffffffffbc2dbf0 0 0 99 ffffff03e5f448a8
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff00179d3c60: ffffff00179d3b80
[ ffffff00179d3b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff00179d9c60 fffffffffbc2dbf0 0 0 99 0
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff00179d9c60: ffffff00179d9b80
[ ffffff00179d9b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff00179dfc60 fffffffffbc2dbf0 0 0 99 ffffff03e5f445a8
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff00179dfc60: ffffff00179dfb80
[ ffffff00179dfb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff00179e5c60 fffffffffbc2dbf0 0 0 99 0
PC: _resume_from_idle+0xf1 THREAD: mac_soft_ring_worker()
stack pointer for thread ffffff00179e5c60: ffffff00179e5b80
[ ffffff00179e5b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mac_soft_ring_worker+0xb0()
thread_start+8()
ffffff00181f4c60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc0c70
PC: _resume_from_idle+0xf1 TASKQ: USB_device_0_pipehndl_tq_2_
stack pointer for thread ffffff00181f4c60: ffffff00181f4ae0
[ ffffff00181f4ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181eec60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc0c70
PC: _resume_from_idle+0xf1 TASKQ: USB_device_0_pipehndl_tq_2_
stack pointer for thread ffffff00181eec60: ffffff00181eeae0
[ ffffff00181eeae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181e8c60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc0c70
PC: _resume_from_idle+0xf1 TASKQ: USB_device_0_pipehndl_tq_2_
stack pointer for thread ffffff00181e8c60: ffffff00181e8ae0
[ ffffff00181e8ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181e2c60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc0c70
PC: _resume_from_idle+0xf1 TASKQ: USB_device_0_pipehndl_tq_2_
stack pointer for thread ffffff00181e2c60: ffffff00181e2ae0
[ ffffff00181e2ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018313c60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc0a40
PC: _resume_from_idle+0xf1 TASKQ: usb_mid_nexus_enum_tq
stack pointer for thread ffffff0018313c60: ffffff0018313ae0
[ ffffff0018313ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001837dc60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 TASKQ: USB_hid_81_pipehndl_tq_0
stack pointer for thread ffffff001837dc60: ffffff001837dae0
[ ffffff001837dae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018377c60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc0928
PC: _resume_from_idle+0xf1 TASKQ: USB_hid_81_pipehndl_tq_0
stack pointer for thread ffffff0018377c60: ffffff0018377ae0
[ ffffff0018377ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018409c60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc0810
PC: _resume_from_idle+0xf1 TASKQ: USB_hid_82_pipehndl_tq_1
stack pointer for thread ffffff0018409c60: ffffff0018409ae0
[ ffffff0018409ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018403c60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc0810
PC: _resume_from_idle+0xf1 TASKQ: USB_hid_82_pipehndl_tq_1
stack pointer for thread ffffff0018403c60: ffffff0018403ae0
[ ffffff0018403ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00179f1c60 fffffffffbc2dbf0 0 0 99 ffffff03e1552bd0
PC: _resume_from_idle+0xf1 THREAD: squeue_worker()
stack pointer for thread ffffff00179f1c60: ffffff00179f1b80
[ ffffff00179f1b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_worker+0xc0()
thread_start+8()
ffffff00179f7c60 fffffffffbc2dbf0 0 0 99 ffffff03e1552bd2
PC: _resume_from_idle+0xf1 THREAD: squeue_polling_thread()
stack pointer for thread ffffff00179f7c60: ffffff00179f7b50
[ ffffff00179f7b50 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_polling_thread+0x90()
thread_start+8()
ffffff00179fdc60 fffffffffbc2dbf0 0 0 99 ffffff03e1552b10
PC: _resume_from_idle+0xf1 THREAD: squeue_worker()
stack pointer for thread ffffff00179fdc60: ffffff00179fdb80
[ ffffff00179fdb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_worker+0xc0()
thread_start+8()
ffffff0017a03c60 fffffffffbc2dbf0 0 0 99 ffffff03e1552b12
PC: _resume_from_idle+0xf1 THREAD: squeue_polling_thread()
stack pointer for thread ffffff0017a03c60: ffffff0017a03b50
[ ffffff0017a03b50 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_polling_thread+0x90()
thread_start+8()
ffffff0017125c60 fffffffffbc2dbf0 0 0 60 fffffffffbce688c
PC: _resume_from_idle+0xf1 THREAD: streams_bufcall_service()
stack pointer for thread ffffff0017125c60: ffffff0017125ba0
[ ffffff0017125ba0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
streams_bufcall_service+0x146()
thread_start+8()
ffffff001712bc60 fffffffffbc2dbf0 0 0 60 fffffffffbcb4158
PC: _resume_from_idle+0xf1 THREAD: streams_qbkgrnd_service()
stack pointer for thread ffffff001712bc60: ffffff001712bb90
[ ffffff001712bb90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
streams_qbkgrnd_service+0xaa()
thread_start+8()
ffffff0017131c60 fffffffffbc2dbf0 0 0 60 fffffffffbcb415a
PC: _resume_from_idle+0xf1 THREAD: streams_sqbkgrnd_service()
stack pointer for thread ffffff0017131c60: ffffff0017131b80
[ ffffff0017131b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
streams_sqbkgrnd_service+0x99()
thread_start+8()
ffffff0017137c60 fffffffffbc2dbf0 0 0 60 fffffffffbc30a70
PC: _resume_from_idle+0xf1 THREAD: page_capture_thread()
stack pointer for thread ffffff0017137c60: ffffff0017137b30
[ ffffff0017137b30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_hires+0xe0()
cv_reltimedwait+0x4f()
page_capture_thread+0xe0()
thread_start+8()
ffffff001713dc60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5d5a
PC: _resume_from_idle+0xf1 THREAD: arc_reclaim_thread()
stack pointer for thread ffffff001713dc60: ffffff001713dae0
[ ffffff001713dae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_wait+0x44()
kmem_cache_reap_now+0x42()
arc_kmem_reap_now+0xd9()
arc_reclaim_thread+0xe5()
thread_start+8()
ffffff0017143c60 fffffffffbc2dbf0 0 0 60 fffffffffbd1da50
PC: _resume_from_idle+0xf1 THREAD: l2arc_feed_thread()
stack pointer for thread ffffff0017143c60: ffffff0017143a70
[ ffffff0017143a70 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
l2arc_dev_get_next+0x17()
l2arc_feed_thread+0x10d()
thread_start+8()
ffffff0017149c60 fffffffffbc2dbf0 0 0 60 fffffffffbcda250
PC: _resume_from_idle+0xf1 THREAD: pm_dep_thread()
stack pointer for thread ffffff0017149c60: ffffff0017149b80
[ ffffff0017149b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
pm_dep_thread+0x95()
thread_start+8()
ffffff001714fc60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5150
PC: _resume_from_idle+0xf1 TASKQ: ppm_nexus_enum_tq
stack pointer for thread ffffff001714fc60: ffffff001714fae0
[ ffffff001714fae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001715bc60 fffffffffbc2dbf0 0 0 60 ffffff03dd4c5038
PC: _resume_from_idle+0xf1 TASKQ: ahci_nexus_enum_tq
stack pointer for thread ffffff001715bc60: ffffff001715bae0
[ ffffff001715bae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001716dc60 fffffffffbc2dbf0 0 0 60 ffffff03dd60ce78
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port0
stack pointer for thread ffffff001716dc60: ffffff001716dae0
[ ffffff001716dae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017167c60 fffffffffbc2dbf0 0 0 60 ffffff03dd60ce78
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port0
stack pointer for thread ffffff0017167c60: ffffff0017167ae0
[ ffffff0017167ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017179c60 fffffffffbc2dbf0 0 0 60 ffffff03dd60cd60
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port1
stack pointer for thread ffffff0017179c60: ffffff0017179ae0
[ ffffff0017179ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017173c60 fffffffffbc2dbf0 0 0 60 ffffff03dd60cd60
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port1
stack pointer for thread ffffff0017173c60: ffffff0017173ae0
[ ffffff0017173ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017185c60 fffffffffbc2dbf0 0 0 60 ffffff03dd60cc48
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port2
stack pointer for thread ffffff0017185c60: ffffff0017185ae0
[ ffffff0017185ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001717fc60 fffffffffbc2dbf0 0 0 60 ffffff03dd60cc48
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port2
stack pointer for thread ffffff001717fc60: ffffff001717fae0
[ ffffff001717fae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017191c60 fffffffffbc2dbf0 0 0 60 ffffff03dd60cb30
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port3
stack pointer for thread ffffff0017191c60: ffffff0017191ae0
[ ffffff0017191ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001718bc60 fffffffffbc2dbf0 0 0 60 ffffff03dd60cb30
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port3
stack pointer for thread ffffff001718bc60: ffffff001718bae0
[ ffffff001718bae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001719dc60 fffffffffbc2dbf0 0 0 60 ffffff03dd60ca18
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port4
stack pointer for thread ffffff001719dc60: ffffff001719dae0
[ ffffff001719dae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017197c60 fffffffffbc2dbf0 0 0 60 ffffff03dd60ca18
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port4
stack pointer for thread ffffff0017197c60: ffffff0017197ae0
[ ffffff0017197ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171a9c60 fffffffffbc2dbf0 0 0 60 ffffff03dd60c900
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port5
stack pointer for thread ffffff00171a9c60: ffffff00171a9ae0
[ ffffff00171a9ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171a3c60 fffffffffbc2dbf0 0 0 60 ffffff03dd60c900
PC: _resume_from_idle+0xf1 TASKQ: ahci_event_handle_taskq_port5
stack pointer for thread ffffff00171a3c60: ffffff00171a3ae0
[ ffffff00171a3ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171afc60 fffffffffbc2dbf0 0 0 60 ffffff03dd60c7e8
PC: _resume_from_idle+0xf1 TASKQ: pci15d9_1_0
stack pointer for thread ffffff00171afc60: ffffff00171afae0
[ ffffff00171afae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171b5c60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 THREAD: sata_event_daemon()
stack pointer for thread ffffff00171b5c60: ffffff00171b5b40
[ ffffff00171b5b40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_hires+0xe0()
cv_reltimedwait+0x4f()
sata_event_daemon+0xfe()
thread_start+8()
ffffff00171e5c60 fffffffffbc2dbf0 0 0 97 ffffff03dd60c6d0
PC: _resume_from_idle+0xf1 TASKQ: sd_drv_taskq
stack pointer for thread ffffff00171e5c60: ffffff00171e5ae0
[ ffffff00171e5ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171dfc60 fffffffffbc2dbf0 0 0 97 ffffff03dd60c6d0
PC: _resume_from_idle+0xf1 TASKQ: sd_drv_taskq
stack pointer for thread ffffff00171dfc60: ffffff00171dfae0
[ ffffff00171dfae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171d9c60 fffffffffbc2dbf0 0 0 97 ffffff03dd60c6d0
PC: _resume_from_idle+0xf1 TASKQ: sd_drv_taskq
stack pointer for thread ffffff00171d9c60: ffffff00171d9ae0
[ ffffff00171d9ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171d3c60 fffffffffbc2dbf0 0 0 97 ffffff03dd60c6d0
PC: _resume_from_idle+0xf1 TASKQ: sd_drv_taskq
stack pointer for thread ffffff00171d3c60: ffffff00171d3ae0
[ ffffff00171d3ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171cdc60 fffffffffbc2dbf0 0 0 97 ffffff03dd60c6d0
PC: _resume_from_idle+0xf1 TASKQ: sd_drv_taskq
stack pointer for thread ffffff00171cdc60: ffffff00171cdae0
[ ffffff00171cdae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171c7c60 fffffffffbc2dbf0 0 0 97 ffffff03dd60c6d0
PC: _resume_from_idle+0xf1 TASKQ: sd_drv_taskq
stack pointer for thread ffffff00171c7c60: ffffff00171c7ae0
[ ffffff00171c7ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171c1c60 fffffffffbc2dbf0 0 0 97 ffffff03dd60c6d0
PC: _resume_from_idle+0xf1 TASKQ: sd_drv_taskq
stack pointer for thread ffffff00171c1c60: ffffff00171c1ae0
[ ffffff00171c1ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171bbc60 fffffffffbc2dbf0 0 0 97 ffffff03dd60c6d0
PC: _resume_from_idle+0xf1 TASKQ: sd_drv_taskq
stack pointer for thread ffffff00171bbc60: ffffff00171bbae0
[ ffffff00171bbae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171ebc60 fffffffffbc2dbf0 0 0 97 ffffff03dd60c5b8
PC: _resume_from_idle+0xf1 TASKQ: sd_rmw_taskq
stack pointer for thread ffffff00171ebc60: ffffff00171ebae0
[ ffffff00171ebae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171f7c60 fffffffffbc2dbf0 0 0 97 ffffff03dd60c4a0
PC: _resume_from_idle+0xf1 TASKQ: xbuf_taskq
stack pointer for thread ffffff00171f7c60: ffffff00171f7ae0
[ ffffff00171f7ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017232c60 ffffff03e05e5348 ffffff03da3c6be0 2 99 ffffff03e06084a8
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017232c60: ffffff00172329f0
[ ffffff00172329f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017238c60 ffffff03e05e5348 ffffff03da3c6650 2 99 ffffff03e06085c0
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017238c60: ffffff00172389f0
[ ffffff00172389f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174f0c60 ffffff03e05e5348 ffffff03da3c3660 2 99 ffffff03e06086d8
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00174f0c60: ffffff00174f09f0
[ ffffff00174f09f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174eac60 ffffff03e05e5348 ffffff03da3c3bf0 2 99 ffffff03e06086d8
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00174eac60: ffffff00174ea9f0
[ ffffff00174ea9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174e4c60 ffffff03e05e5348 ffffff03da3c4180 2 99 ffffff03e06086d8
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00174e4c60: ffffff00174e49f0
[ ffffff00174e49f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017256c60 ffffff03e05e5348 ffffff03da3c4710 2 99 ffffff03e06086d8
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017256c60: ffffff00172569f0
[ ffffff00172569f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017250c60 ffffff03e05e5348 ffffff03da3c5010 2 0 ffffff03e06086d8
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017250c60: ffffff00172509f0
[ ffffff00172509f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001724ac60 ffffff03e05e5348 ffffff03da3c55a0 2 0 ffffff03e06086d8
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001724ac60: ffffff001724a9f0
[ ffffff001724a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017244c60 ffffff03e05e5348 ffffff03da3c5b30 2 0 ffffff03e06086d8
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017244c60: ffffff00172449f0
[ ffffff00172449f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001723ec60 ffffff03e05e5348 ffffff03da3c60c0 2 0 ffffff03e06086d8
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001723ec60: ffffff001723e9f0
[ ffffff001723e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001789bc60 ffffff03e05e5348 ffffff03e0627c00 2 0 ffffff03e06087f0
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001789bc60: ffffff001789b9f0
[ ffffff001789b9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017921c60 ffffff03e05e5348 ffffff03e0624c10 2 0 ffffff03e06087f0
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017921c60: ffffff00179219f0
[ ffffff00179219f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00172f2c60 ffffff03e05e5348 ffffff03e0628190 2 99 ffffff03e06087f0
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00172f2c60: ffffff00172f29f0
[ ffffff00172f29f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017209c60 ffffff03e05e5348 ffffff03da3c30d0 2 0 ffffff03e06087f0
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017209c60: ffffff00172099f0
[ ffffff00172099f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017895c60 ffffff03e05e5348 ffffff03e0625730 2 99 ffffff03e0608908
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017895c60: ffffff00178959f0
[ ffffff00178959f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00178d3c60 ffffff03e05e5348 ffffff03e06270e0 2 99 ffffff03e0608908
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00178d3c60: ffffff00178d39f0
[ ffffff00178d39f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017847c60 ffffff03e05e5348 ffffff03e0627670 2 99 ffffff03e0608908
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017847c60: ffffff00178479f0
[ ffffff00178479f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001720fc60 ffffff03e05e5348 ffffff03e0653810 2 99 fffffffffbc2cf02
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001720fc60: ffffff001720f370
[ ffffff001720f370 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
kmem_zalloc+0x6a()
vdev_mirror_map_alloc+0x158()
vdev_mirror_io_start+0x1f()
zio_vdev_io_start+0x1ba()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff0017556c60 ffffff03e05e5348 ffffff03e0651c40 2 0 ffffff03e0608a20
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017556c60: ffffff00175569f0
[ ffffff00175569f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017550c60 ffffff03e05e5348 ffffff03e06521d0 2 0 ffffff03e0608a20
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017550c60: ffffff00175509f0
[ ffffff00175509f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017221c60 ffffff03e05e5348 ffffff03e0652760 2 0 ffffff03e0608a20
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017221c60: ffffff00172219f0
[ ffffff00172219f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001721bc60 ffffff03e05e5348 ffffff03e0652cf0 2 0 ffffff03e0608a20
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001721bc60: ffffff001721b9f0
[ ffffff001721b9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017215c60 ffffff03e05e5348 ffffff03e0653280 2 0 ffffff03e0608a20
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017215c60: ffffff00172159f0
[ ffffff00172159f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017586c60 ffffff03e05e5348 ffffff03e064ec50 2 99 ffffff03e0608b38
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017586c60: ffffff00175869f0
[ ffffff00175869f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017580c60 ffffff03e05e5348 ffffff03e064f1e0 2 99 ffffff03e0608b38
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017580c60: ffffff00175809f0
[ ffffff00175809f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001757ac60 ffffff03e05e5348 ffffff03e064f770 2 99 ffffff03e0608b38
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001757ac60: ffffff001757a9f0
[ ffffff001757a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017574c60 ffffff03e05e5348 ffffff03e064fd00 2 99 ffffff03e0608b38
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017574c60: ffffff00175749f0
[ ffffff00175749f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001756ec60 ffffff03e05e5348 ffffff03e0650290 2 99 ffffff03e0608b38
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001756ec60: ffffff001756e9f0
[ ffffff001756e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017568c60 ffffff03e05e5348 ffffff03e0650820 2 99 ffffff03e0608b38
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017568c60: ffffff00175689f0
[ ffffff00175689f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017562c60 ffffff03e05e5348 ffffff03e0651120 2 99 ffffff03e0608b38
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017562c60: ffffff00175629f0
[ ffffff00175629f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001755cc60 ffffff03e05e5348 ffffff03e06516b0 2 99 ffffff03e0608b38
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001755cc60: ffffff001755c9f0
[ ffffff001755c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175dac60 ffffff03e05e5348 ffffff03e0b05790 2 0 ffffff03e0608c50
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175dac60: ffffff00175da9f0
[ ffffff00175da9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175d4c60 ffffff03e05e5348 ffffff03e0b05d20 2 0 ffffff03e0608c50
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175d4c60: ffffff00175d49f0
[ ffffff00175d49f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017598c60 ffffff03e05e5348 ffffff03e064d830 2 0 ffffff03e0608c50
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017598c60: ffffff00175989f0
[ ffffff00175989f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017592c60 ffffff03e05e5348 ffffff03e064e130 2 99 ffffff03e0608c50
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017592c60: ffffff00175929f0
[ ffffff00175929f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001758cc60 ffffff03e05e5348 ffffff03e064e6c0 2 99 ffffff03e0608c50
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001758cc60: ffffff001758c9f0
[ ffffff001758c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001782cc60 ffffff03e05e5348 ffffff03e0b3f850 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001782cc60: ffffff001782c9f0
[ ffffff001782c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017826c60 ffffff03e05e5348 ffffff03e0b3fde0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017826c60: ffffff00178269f0
[ ffffff00178269f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017820c60 ffffff03e05e5348 ffffff03e0b40370 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017820c60: ffffff00178209f0
[ ffffff00178209f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001781ac60 ffffff03e05e5348 ffffff03e0b40900 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001781ac60: ffffff001781a9f0
[ ffffff001781a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017814c60 ffffff03e05e5348 ffffff03e0b41200 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017814c60: ffffff00178149f0
[ ffffff00178149f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001780ec60 ffffff03e05e5348 ffffff03e0b41790 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001780ec60: ffffff001780e9f0
[ ffffff001780e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017808c60 ffffff03e05e5348 ffffff03e0b41d20 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017808c60: ffffff00178089f0
[ ffffff00178089f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177f0c60 ffffff03e05e5348 ffffff03e0b422b0 2 0 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177f0c60: ffffff00177f09f0
[ ffffff00177f09f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177eac60 ffffff03e05e5348 ffffff03e0b42840 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177eac60: ffffff00177ea9f0
[ ffffff00177ea9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177e4c60 ffffff03e05e5348 ffffff03e0b42dd0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177e4c60: ffffff00177e49f0
[ ffffff00177e49f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177dec60 ffffff03e05e5348 ffffff03e0b43360 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177dec60: ffffff00177de9f0
[ ffffff00177de9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177d8c60 ffffff03e05e5348 ffffff03e0b438f0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177d8c60: ffffff00177d89f0
[ ffffff00177d89f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177d2c60 ffffff03e05e5348 ffffff03e0b441f0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177d2c60: ffffff00177d29f0
[ ffffff00177d29f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177ccc60 ffffff03e05e5348 ffffff03e0b44780 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177ccc60: ffffff00177cc9f0
[ ffffff00177cc9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177c6c60 ffffff03e05e5348 ffffff03e0b44d10 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177c6c60: ffffff00177c69f0
[ ffffff00177c69f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177c0c60 ffffff03e05e5348 ffffff03e0b452a0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177c0c60: ffffff00177c09f0
[ ffffff00177c09f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177bac60 ffffff03e05e5348 ffffff03e0b45830 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177bac60: ffffff00177ba9f0
[ ffffff00177ba9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177b4c60 ffffff03e05e5348 ffffff03e0b45dc0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177b4c60: ffffff00177b49f0
[ ffffff00177b49f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177aec60 ffffff03e05e5348 ffffff03e0b46350 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177aec60: ffffff00177ae9f0
[ ffffff00177ae9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177a8c60 ffffff03e05e5348 ffffff03e0b468e0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177a8c60: ffffff00177a89f0
[ ffffff00177a89f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177a2c60 ffffff03e05e5348 ffffff03e0ae91e0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00177a2c60: ffffff00177a29f0
[ ffffff00177a29f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001779cc60 ffffff03e05e5348 ffffff03e0ae9770 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001779cc60: ffffff001779c9f0
[ ffffff001779c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017790c60 ffffff03e05e5348 ffffff03e0aea290 2 0 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017790c60: ffffff00177909f0
[ ffffff00177909f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001778ac60 ffffff03e05e5348 ffffff03e0aea820 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001778ac60: ffffff001778a9f0
[ ffffff001778a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017784c60 ffffff03e05e5348 ffffff03e0aeadb0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017784c60: ffffff00177849f0
[ ffffff00177849f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001777ec60 ffffff03e05e5348 ffffff03e0aeb340 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001777ec60: ffffff001777e9f0
[ ffffff001777e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017778c60 ffffff03e05e5348 ffffff03e0aeb8d0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017778c60: ffffff00177789f0
[ ffffff00177789f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017772c60 ffffff03e05e5348 ffffff03e0aec1d0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017772c60: ffffff00177729f0
[ ffffff00177729f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001776cc60 ffffff03e05e5348 ffffff03e0aec760 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001776cc60: ffffff001776c9f0
[ ffffff001776c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017766c60 ffffff03e05e5348 ffffff03e0aeccf0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017766c60: ffffff00177669f0
[ ffffff00177669f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017760c60 ffffff03e05e5348 ffffff03e0aed280 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017760c60: ffffff00177609f0
[ ffffff00177609f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001775ac60 ffffff03e05e5348 ffffff03e0aed810 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001775ac60: ffffff001775a9f0
[ ffffff001775a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017754c60 ffffff03e05e5348 ffffff03e0aedda0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017754c60: ffffff00177549f0
[ ffffff00177549f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001774ec60 ffffff03e05e5348 ffffff03e0aee330 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001774ec60: ffffff001774e9f0
[ ffffff001774e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017748c60 ffffff03e05e5348 ffffff03e0aee8c0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017748c60: ffffff00177489f0
[ ffffff00177489f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017742c60 ffffff03e05e5348 ffffff03e0aef1c0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017742c60: ffffff00177429f0
[ ffffff00177429f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001773cc60 ffffff03e05e5348 ffffff03e0aef750 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001773cc60: ffffff001773c9f0
[ ffffff001773c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017736c60 ffffff03e05e5348 ffffff03e0aefce0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017736c60: ffffff00177369f0
[ ffffff00177369f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017730c60 ffffff03e05e5348 ffffff03e0af0270 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017730c60: ffffff00177309f0
[ ffffff00177309f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001772ac60 ffffff03e05e5348 ffffff03e0af0800 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001772ac60: ffffff001772a9f0
[ ffffff001772a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017724c60 ffffff03e05e5348 ffffff03e0af0d90 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017724c60: ffffff00177249f0
[ ffffff00177249f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001771ec60 ffffff03e05e5348 ffffff03e0af1320 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001771ec60: ffffff001771e9f0
[ ffffff001771e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017718c60 ffffff03e05e5348 ffffff03e0af18b0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017718c60: ffffff00177189f0
[ ffffff00177189f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017712c60 ffffff03e05e5348 ffffff03e0af21b0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017712c60: ffffff00177129f0
[ ffffff00177129f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001770cc60 ffffff03e05e5348 ffffff03e0af2740 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001770cc60: ffffff001770c9f0
[ ffffff001770c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017706c60 ffffff03e05e5348 ffffff03e0af2cd0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017706c60: ffffff00177069f0
[ ffffff00177069f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176fac60 ffffff03e05e5348 ffffff03e0af37f0 2 0 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176fac60: ffffff00176fa9f0
[ ffffff00176fa9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176f4c60 ffffff03e05e5348 ffffff03e0af3d80 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176f4c60: ffffff00176f49f0
[ ffffff00176f49f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176eec60 ffffff03e05e5348 ffffff03e0af4310 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176eec60: ffffff00176ee9f0
[ ffffff00176ee9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176e8c60 ffffff03e05e5348 ffffff03e0af48a0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176e8c60: ffffff00176e89f0
[ ffffff00176e89f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176e2c60 ffffff03e05e5348 ffffff03e0af51a0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176e2c60: ffffff00176e29f0
[ ffffff00176e29f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176dcc60 ffffff03e05e5348 ffffff03e0af5730 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176dcc60: ffffff00176dc9f0
[ ffffff00176dc9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176d6c60 ffffff03e05e5348 ffffff03e0af5cc0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176d6c60: ffffff00176d69f0
[ ffffff00176d69f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176d0c60 ffffff03e05e5348 ffffff03e0af6250 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176d0c60: ffffff00176d09f0
[ ffffff00176d09f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176c4c60 ffffff03e05e5348 ffffff03e0af6d70 2 0 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176c4c60: ffffff00176c49f0
[ ffffff00176c49f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176bec60 ffffff03e05e5348 ffffff03e0af7300 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176bec60: ffffff00176be9f0
[ ffffff00176be9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176b8c60 ffffff03e05e5348 ffffff03e0af7890 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176b8c60: ffffff00176b89f0
[ ffffff00176b89f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176b2c60 ffffff03e05e5348 ffffff03e0af8190 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176b2c60: ffffff00176b29f0
[ ffffff00176b29f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176acc60 ffffff03e05e5348 ffffff03e0af8720 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176acc60: ffffff00176ac9f0
[ ffffff00176ac9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176a6c60 ffffff03e05e5348 ffffff03e0af8cb0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176a6c60: ffffff00176a69f0
[ ffffff00176a69f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176a0c60 ffffff03e05e5348 ffffff03e0af9240 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176a0c60: ffffff00176a09f0
[ ffffff00176a09f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001769ac60 ffffff03e05e5348 ffffff03e0af97d0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001769ac60: ffffff001769a9f0
[ ffffff001769a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017694c60 ffffff03e05e5348 ffffff03e0af9d60 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017694c60: ffffff00176949f0
[ ffffff00176949f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001768ec60 ffffff03e05e5348 ffffff03e0afa2f0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001768ec60: ffffff001768e9f0
[ ffffff001768e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017688c60 ffffff03e05e5348 ffffff03e0afa880 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017688c60: ffffff00176889f0
[ ffffff00176889f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017682c60 ffffff03e05e5348 ffffff03e0afb180 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017682c60: ffffff00176829f0
[ ffffff00176829f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001767cc60 ffffff03e05e5348 ffffff03e0afb710 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001767cc60: ffffff001767c9f0
[ ffffff001767c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017676c60 ffffff03e05e5348 ffffff03e0afbca0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017676c60: ffffff00176769f0
[ ffffff00176769f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017670c60 ffffff03e05e5348 ffffff03e0afc230 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017670c60: ffffff00176709f0
[ ffffff00176709f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017664c60 ffffff03e05e5348 ffffff03e0afcd50 2 0 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017664c60: ffffff00176649f0
[ ffffff00176649f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001765ec60 ffffff03e05e5348 ffffff03e0afd2e0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001765ec60: ffffff001765e9f0
[ ffffff001765e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017658c60 ffffff03e05e5348 ffffff03e0afd870 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017658c60: ffffff00176589f0
[ ffffff00176589f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017652c60 ffffff03e05e5348 ffffff03e0afe170 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017652c60: ffffff00176529f0
[ ffffff00176529f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001764cc60 ffffff03e05e5348 ffffff03e0afe700 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001764cc60: ffffff001764c9f0
[ ffffff001764c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017646c60 ffffff03e05e5348 ffffff03e0afec90 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017646c60: ffffff00176469f0
[ ffffff00176469f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017640c60 ffffff03e05e5348 ffffff03e0aff220 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017640c60: ffffff00176409f0
[ ffffff00176409f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001763ac60 ffffff03e05e5348 ffffff03e0aff7b0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001763ac60: ffffff001763a9f0
[ ffffff001763a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017634c60 ffffff03e05e5348 ffffff03e0affd40 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017634c60: ffffff00176349f0
[ ffffff00176349f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001762ec60 ffffff03e05e5348 ffffff03e0b002d0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001762ec60: ffffff001762e9f0
[ ffffff001762e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017628c60 ffffff03e05e5348 ffffff03e0b00860 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017628c60: ffffff00176289f0
[ ffffff00176289f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017622c60 ffffff03e05e5348 ffffff03e0b01160 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017622c60: ffffff00176229f0
[ ffffff00176229f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001761cc60 ffffff03e05e5348 ffffff03e0b016f0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001761cc60: ffffff001761c9f0
[ ffffff001761c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017616c60 ffffff03e05e5348 ffffff03e0b01c80 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017616c60: ffffff00176169f0
[ ffffff00176169f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017610c60 ffffff03e05e5348 ffffff03e0b02210 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017610c60: ffffff00176109f0
[ ffffff00176109f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001760ac60 ffffff03e05e5348 ffffff03e0b027a0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001760ac60: ffffff001760a9f0
[ ffffff001760a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017604c60 ffffff03e05e5348 ffffff03e0b02d30 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017604c60: ffffff00176049f0
[ ffffff00176049f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175fec60 ffffff03e05e5348 ffffff03e0b032c0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175fec60: ffffff00175fe9f0
[ ffffff00175fe9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175f8c60 ffffff03e05e5348 ffffff03e0b03850 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175f8c60: ffffff00175f89f0
[ ffffff00175f89f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175f2c60 ffffff03e05e5348 ffffff03e0b04150 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175f2c60: ffffff00175f29f0
[ ffffff00175f29f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175ecc60 ffffff03e05e5348 ffffff03e0b046e0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175ecc60: ffffff00175ec9f0
[ ffffff00175ec9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175e6c60 ffffff03e05e5348 ffffff03e0b04c70 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175e6c60: ffffff00175e69f0
[ ffffff00175e69f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175cec60 ffffff03e05e5348 ffffff03e0b062b0 2 0 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175cec60: ffffff00175ce9f0
[ ffffff00175ce9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175c8c60 ffffff03e05e5348 ffffff03e0b06840 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175c8c60: ffffff00175c89f0
[ ffffff00175c89f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175c2c60 ffffff03e05e5348 ffffff03e064b140 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175c2c60: ffffff00175c29f0
[ ffffff00175c29f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175bcc60 ffffff03e05e5348 ffffff03e064b6d0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175bcc60: ffffff00175bc9f0
[ ffffff00175bc9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175b6c60 ffffff03e05e5348 ffffff03e064bc60 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175b6c60: ffffff00175b69f0
[ ffffff00175b69f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175b0c60 ffffff03e05e5348 ffffff03e064c1f0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175b0c60: ffffff00175b09f0
[ ffffff00175b09f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175aac60 ffffff03e05e5348 ffffff03e064c780 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175aac60: ffffff00175aa9f0
[ ffffff00175aa9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175a4c60 ffffff03e05e5348 ffffff03e064cd10 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175a4c60: ffffff00175a49f0
[ ffffff00175a49f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001759ec60 ffffff03e05e5348 ffffff03e064d2a0 2 99 ffffff03e0608d68
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001759ec60: ffffff001759e9f0
[ ffffff001759e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00175e0c60 ffffff03e05e5348 ffffff03e0b05200 2 99 ffffff03e0608e80
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00175e0c60: ffffff00175e09f0
[ ffffff00175e09f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001766ac60 ffffff03e05e5348 ffffff03e0afc7c0 2 99 ffffff03dd60c040
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001766ac60: ffffff001766a9f0
[ ffffff001766a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00176cac60 ffffff03e05e5348 ffffff03e0af67e0 2 99 ffffff03dd60c158
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff00176cac60: ffffff00176ca9f0
[ ffffff00176ca9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017700c60 ffffff03e05e5348 ffffff03e0af3260 2 99 ffffff03dd60c270
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017700c60: ffffff00177009f0
[ ffffff00177009f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017796c60 ffffff03e05e5348 ffffff03e0ae9d00 2 99 ffffff03dd60c388
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff0017796c60: ffffff00177969f0
[ ffffff00177969f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001722cc60 ffffff03e05e5348 ffffff03da3c7170 2 99 ffffff03de005278
PC: _resume_from_idle+0xf1 CMD: zpool-syspool
stack pointer for thread ffffff001722cc60: ffffff001722ca90
[ ffffff001722ca90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
spa_thread+0x20b()
thread_start+8()
ffffff0017832c60 fffffffffbc2dbf0 0 0 60 ffffff03e0608390
PC: _resume_from_idle+0xf1 TASKQ: zfs_vn_rele_taskq
stack pointer for thread ffffff0017832c60: ffffff0017832ae0
[ ffffff0017832ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017838c60 fffffffffbc2dbf0 0 0 60 ffffff03dd38e24c
PC: _resume_from_idle+0xf1 THREAD: txg_quiesce_thread()
stack pointer for thread ffffff0017838c60: ffffff0017838b10
[ ffffff0017838b10 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
txg_thread_wait+0x5f()
txg_quiesce_thread+0x94()
thread_start+8()
ffffff0017841c60 fffffffffbc2dbf0 0 0 60 ffffff04718e9990
PC: _resume_from_idle+0xf1 THREAD: txg_sync_thread()
stack pointer for thread ffffff0017841c60: ffffff00178419d0
[ ffffff00178419d0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
zio_wait+0x5d()
dsl_pool_sync+0x2bc()
spa_sync+0x37b()
txg_sync_thread+0x247()
thread_start+8()
ffffff00173b8c60 fffffffffbc2dbf0 0 0 60 ffffff03e0608278
PC: _resume_from_idle+0xf1 TASKQ: zil_clean
stack pointer for thread ffffff00173b8c60: ffffff00173b8ae0
[ ffffff00173b8ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001744ec60 fffffffffbc2dbf0 0 0 60 ffffff03e0608160
PC: _resume_from_idle+0xf1 TASKQ: acpinex_nexus_enum_tq
stack pointer for thread ffffff001744ec60: ffffff001744eae0
[ ffffff001744eae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017460c60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 TASKQ: cpudrv_cpudrv_monitor
stack pointer for thread ffffff0017460c60: ffffff0017460ae0
[ ffffff0017460ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017203c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556e88
PC: _resume_from_idle+0xf1 TASKQ: IP_INJECT_QUEUE_OUT
stack pointer for thread ffffff0017203c60: ffffff0017203ae0
[ ffffff0017203ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017802c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556d70
PC: _resume_from_idle+0xf1 TASKQ: IP_INJECT_QUEUE_IN
stack pointer for thread ffffff0017802c60: ffffff0017802ae0
[ ffffff0017802ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177fcc60 fffffffffbc2dbf0 0 0 60 ffffff03e1556c58
PC: _resume_from_idle+0xf1 TASKQ: IP_NIC_EVENT_QUEUE
stack pointer for thread ffffff00177fcc60: ffffff00177fcae0
[ ffffff00177fcae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00177f6c60 fffffffffbc2dbf0 0 0 99 ffffff03de003808
PC: _resume_from_idle+0xf1 THREAD: ipsec_loader()
stack pointer for thread ffffff00177f6c60: ffffff00177f6b80
[ ffffff00177f6b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
ipsec_loader+0x9d()
thread_start+8()
ffffff00173bec60 fffffffffbc2dbf0 0 0 99 ffffff03e1552ed0
PC: _resume_from_idle+0xf1 THREAD: squeue_worker()
stack pointer for thread ffffff00173bec60: ffffff00173beb80
[ ffffff00173beb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_worker+0xc0()
thread_start+8()
ffffff00173c4c60 fffffffffbc2dbf0 0 0 99 ffffff03e1552ed2
PC: _resume_from_idle+0xf1 THREAD: squeue_polling_thread()
stack pointer for thread ffffff00173c4c60: ffffff00173c4b50
[ ffffff00173c4b50 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_polling_thread+0x90()
thread_start+8()
ffffff0017508c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556b40
PC: _resume_from_idle+0xf1 TASKQ: sctp_def_recvq_taskq
stack pointer for thread ffffff0017508c60: ffffff0017508ae0
[ ffffff0017508ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174f6c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556b40
PC: _resume_from_idle+0xf1 TASKQ: sctp_def_recvq_taskq
stack pointer for thread ffffff00174f6c60: ffffff00174f6ae0
[ ffffff00174f6ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001745ac60 fffffffffbc2dbf0 0 0 60 ffffff03e1556b40
PC: _resume_from_idle+0xf1 TASKQ: sctp_def_recvq_taskq
stack pointer for thread ffffff001745ac60: ffffff001745aae0
[ ffffff001745aae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017454c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556b40
PC: _resume_from_idle+0xf1 TASKQ: sctp_def_recvq_taskq
stack pointer for thread ffffff0017454c60: ffffff0017454ae0
[ ffffff0017454ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174fcc60 fffffffffbc2dbf0 0 0 60 ffffff03de000f70
PC: _resume_from_idle+0xf1 THREAD: ill_taskq_dispatch()
stack pointer for thread ffffff00174fcc60: ffffff00174fcb40
[ ffffff00174fcb40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
ill_taskq_dispatch+0x128()
thread_start+8()
ffffff0017502c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556a28
PC: _resume_from_idle+0xf1 TASKQ: ilb_rule_taskq_ffffff03da0bce0
stack pointer for thread ffffff0017502c60: ffffff0017502ae0
[ ffffff0017502ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017526c60 fffffffffbc2dbf0 0 0 60 fffffffffbd346b0
PC: _resume_from_idle+0xf1 THREAD: dld_taskq_dispatch()
stack pointer for thread ffffff0017526c60: ffffff0017526b80
[ ffffff0017526b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
dld_taskq_dispatch+0xfc()
thread_start+8()
ffffff001752cc60 fffffffffbc2dbf0 0 0 60 ffffff03e1556910
PC: _resume_from_idle+0xf1 TASKQ: i8042_nexus_enum_tq
stack pointer for thread ffffff001752cc60: ffffff001752cae0
[ ffffff001752cae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017532c60 fffffffffbc2dbf0 0 0 60 ffffff03e15567f8
PC: _resume_from_idle+0xf1 TASKQ: pcieb_nexus_enum_tq
stack pointer for thread ffffff0017532c60: ffffff0017532ae0
[ ffffff0017532ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017538c60 fffffffffbc2dbf0 0 0 60 ffffff03e15566e0
PC: _resume_from_idle+0xf1 TASKQ: pcieb_nexus_enum_tq
stack pointer for thread ffffff0017538c60: ffffff0017538ae0
[ ffffff0017538ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001753ec60 fffffffffbc2dbf0 0 0 60 ffffff03e15565c8
PC: _resume_from_idle+0xf1 TASKQ: pci_pci_nexus_enum_tq
stack pointer for thread ffffff001753ec60: ffffff001753eae0
[ ffffff001753eae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001754ac60 fffffffffbc2dbf0 0 0 60 ffffff03e15564b0
PC: _resume_from_idle+0xf1 TASKQ: ehci_nexus_enum_tq
stack pointer for thread ffffff001754ac60: ffffff001754aae0
[ ffffff001754aae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00173e2c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556398
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_0_pipehndl_tq_0
stack pointer for thread ffffff00173e2c60: ffffff00173e2ae0
[ ffffff00173e2ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00173dcc60 fffffffffbc2dbf0 0 0 60 ffffff03e1556398
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_0_pipehndl_tq_0
stack pointer for thread ffffff00173dcc60: ffffff00173dcae0
[ ffffff00173dcae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00173d6c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556398
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_0_pipehndl_tq_0
stack pointer for thread ffffff00173d6c60: ffffff00173d6ae0
[ ffffff00173d6ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00173d0c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556398
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_0_pipehndl_tq_0
stack pointer for thread ffffff00173d0c60: ffffff00173d0ae0
[ ffffff00173d0ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00173eec60 fffffffffbc2dbf0 0 0 60 ffffff03e1556280
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_81_pipehndl_tq_0
stack pointer for thread ffffff00173eec60: ffffff00173eeae0
[ ffffff00173eeae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00173e8c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556280
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_81_pipehndl_tq_0
stack pointer for thread ffffff00173e8c60: ffffff00173e8ae0
[ ffffff00173e8ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00173f4c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556168
PC: _resume_from_idle+0xf1 TASKQ: ehci_nexus_enum_tq
stack pointer for thread ffffff00173f4c60: ffffff00173f4ae0
[ ffffff00173f4ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001740cc60 fffffffffbc2dbf0 0 0 60 ffffff03e1556050
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_0_pipehndl_tq_1
stack pointer for thread ffffff001740cc60: ffffff001740cae0
[ ffffff001740cae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017406c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556050
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_0_pipehndl_tq_1
stack pointer for thread ffffff0017406c60: ffffff0017406ae0
[ ffffff0017406ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017400c60 fffffffffbc2dbf0 0 0 60 ffffff03e1556050
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_0_pipehndl_tq_1
stack pointer for thread ffffff0017400c60: ffffff0017400ae0
[ ffffff0017400ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00173fac60 fffffffffbc2dbf0 0 0 60 ffffff03e1556050
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_0_pipehndl_tq_1
stack pointer for thread ffffff00173fac60: ffffff00173faae0
[ ffffff00173faae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017418c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478e90
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_81_pipehndl_tq_1
stack pointer for thread ffffff0017418c60: ffffff0017418ae0
[ ffffff0017418ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017412c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478e90
PC: _resume_from_idle+0xf1 TASKQ: USB_ehci_81_pipehndl_tq_1
stack pointer for thread ffffff0017412c60: ffffff0017412ae0
[ ffffff0017412ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001741ec60 fffffffffbc2dbf0 0 0 60 ffffff03e2478d78
PC: _resume_from_idle+0xf1 TASKQ: uhci_nexus_enum_tq
stack pointer for thread ffffff001741ec60: ffffff001741eae0
[ ffffff001741eae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017436c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478c60
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_0
stack pointer for thread ffffff0017436c60: ffffff0017436ae0
[ ffffff0017436ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017430c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478c60
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_0
stack pointer for thread ffffff0017430c60: ffffff0017430ae0
[ ffffff0017430ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001742ac60 fffffffffbc2dbf0 0 0 60 ffffff03e2478c60
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_0
stack pointer for thread ffffff001742ac60: ffffff001742aae0
[ ffffff001742aae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017424c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478c60
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_0
stack pointer for thread ffffff0017424c60: ffffff0017424ae0
[ ffffff0017424ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017442c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478b48
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_0
stack pointer for thread ffffff0017442c60: ffffff0017442ae0
[ ffffff0017442ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001743cc60 fffffffffbc2dbf0 0 0 60 ffffff03e2478b48
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_0
stack pointer for thread ffffff001743cc60: ffffff001743cae0
[ ffffff001743cae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017448c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478a30
PC: _resume_from_idle+0xf1 TASKQ: uhci_nexus_enum_tq
stack pointer for thread ffffff0017448c60: ffffff0017448ae0
[ ffffff0017448ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017478c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478918
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_1
stack pointer for thread ffffff0017478c60: ffffff0017478ae0
[ ffffff0017478ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017472c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478918
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_1
stack pointer for thread ffffff0017472c60: ffffff0017472ae0
[ ffffff0017472ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001746cc60 fffffffffbc2dbf0 0 0 60 ffffff03e2478918
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_1
stack pointer for thread ffffff001746cc60: ffffff001746cae0
[ ffffff001746cae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017466c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478918
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_1
stack pointer for thread ffffff0017466c60: ffffff0017466ae0
[ ffffff0017466ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017484c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478800
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_1
stack pointer for thread ffffff0017484c60: ffffff0017484ae0
[ ffffff0017484ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001747ec60 fffffffffbc2dbf0 0 0 60 ffffff03e2478800
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_1
stack pointer for thread ffffff001747ec60: ffffff001747eae0
[ ffffff001747eae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001748ac60 fffffffffbc2dbf0 0 0 60 ffffff03e24786e8
PC: _resume_from_idle+0xf1 TASKQ: uhci_nexus_enum_tq
stack pointer for thread ffffff001748ac60: ffffff001748aae0
[ ffffff001748aae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174a2c60 fffffffffbc2dbf0 0 0 60 ffffff03e24785d0
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_2
stack pointer for thread ffffff00174a2c60: ffffff00174a2ae0
[ ffffff00174a2ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001749cc60 fffffffffbc2dbf0 0 0 60 ffffff03e24785d0
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_2
stack pointer for thread ffffff001749cc60: ffffff001749cae0
[ ffffff001749cae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017496c60 fffffffffbc2dbf0 0 0 60 ffffff03e24785d0
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_2
stack pointer for thread ffffff0017496c60: ffffff0017496ae0
[ ffffff0017496ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017490c60 fffffffffbc2dbf0 0 0 60 ffffff03e24785d0
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_2
stack pointer for thread ffffff0017490c60: ffffff0017490ae0
[ ffffff0017490ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174aec60 fffffffffbc2dbf0 0 0 60 ffffff03e24784b8
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_2
stack pointer for thread ffffff00174aec60: ffffff00174aeae0
[ ffffff00174aeae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174a8c60 fffffffffbc2dbf0 0 0 60 ffffff03e24784b8
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_2
stack pointer for thread ffffff00174a8c60: ffffff00174a8ae0
[ ffffff00174a8ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174b4c60 fffffffffbc2dbf0 0 0 60 ffffff03e24783a0
PC: _resume_from_idle+0xf1 TASKQ: uhci_nexus_enum_tq
stack pointer for thread ffffff00174b4c60: ffffff00174b4ae0
[ ffffff00174b4ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174ccc60 fffffffffbc2dbf0 0 0 60 ffffff03e2478288
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_3
stack pointer for thread ffffff00174ccc60: ffffff00174ccae0
[ ffffff00174ccae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174c6c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478288
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_3
stack pointer for thread ffffff00174c6c60: ffffff00174c6ae0
[ ffffff00174c6ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174c0c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478288
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_3
stack pointer for thread ffffff00174c0c60: ffffff00174c0ae0
[ ffffff00174c0ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174bac60 fffffffffbc2dbf0 0 0 60 ffffff03e2478288
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_3
stack pointer for thread ffffff00174bac60: ffffff00174baae0
[ ffffff00174baae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174d8c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478170
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_3
stack pointer for thread ffffff00174d8c60: ffffff00174d8ae0
[ ffffff00174d8ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174d2c60 fffffffffbc2dbf0 0 0 60 ffffff03e2478170
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_3
stack pointer for thread ffffff00174d2c60: ffffff00174d2ae0
[ ffffff00174d2ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00174dec60 fffffffffbc2dbf0 0 0 60 ffffff03e2478058
PC: _resume_from_idle+0xf1 TASKQ: uhci_nexus_enum_tq
stack pointer for thread ffffff00174dec60: ffffff00174deae0
[ ffffff00174deae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001726ec60 fffffffffbc2dbf0 0 0 60 ffffff03e246be98
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_4
stack pointer for thread ffffff001726ec60: ffffff001726eae0
[ ffffff001726eae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017268c60 fffffffffbc2dbf0 0 0 60 ffffff03e246be98
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_4
stack pointer for thread ffffff0017268c60: ffffff0017268ae0
[ ffffff0017268ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017262c60 fffffffffbc2dbf0 0 0 60 ffffff03e246be98
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_4
stack pointer for thread ffffff0017262c60: ffffff0017262ae0
[ ffffff0017262ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001725cc60 fffffffffbc2dbf0 0 0 60 ffffff03e246be98
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_4
stack pointer for thread ffffff001725cc60: ffffff001725cae0
[ ffffff001725cae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001727ac60 fffffffffbc2dbf0 0 0 60 ffffff03e246bd80
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_4
stack pointer for thread ffffff001727ac60: ffffff001727aae0
[ ffffff001727aae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017274c60 fffffffffbc2dbf0 0 0 60 ffffff03e246bd80
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_4
stack pointer for thread ffffff0017274c60: ffffff0017274ae0
[ ffffff0017274ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017280c60 fffffffffbc2dbf0 0 0 60 ffffff03e246bc68
PC: _resume_from_idle+0xf1 TASKQ: uhci_nexus_enum_tq
stack pointer for thread ffffff0017280c60: ffffff0017280ae0
[ ffffff0017280ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017298c60 fffffffffbc2dbf0 0 0 60 ffffff03e246bb50
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_5
stack pointer for thread ffffff0017298c60: ffffff0017298ae0
[ ffffff0017298ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017292c60 fffffffffbc2dbf0 0 0 60 ffffff03e246bb50
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_5
stack pointer for thread ffffff0017292c60: ffffff0017292ae0
[ ffffff0017292ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001728cc60 fffffffffbc2dbf0 0 0 60 ffffff03e246bb50
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_5
stack pointer for thread ffffff001728cc60: ffffff001728cae0
[ ffffff001728cae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017286c60 fffffffffbc2dbf0 0 0 60 ffffff03e246bb50
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_0_pipehndl_tq_5
stack pointer for thread ffffff0017286c60: ffffff0017286ae0
[ ffffff0017286ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00172a4c60 fffffffffbc2dbf0 0 0 60 ffffff03e246ba38
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_5
stack pointer for thread ffffff00172a4c60: ffffff00172a4ae0
[ ffffff00172a4ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001729ec60 fffffffffbc2dbf0 0 0 60 ffffff03e246ba38
PC: _resume_from_idle+0xf1 TASKQ: USB_uhci_81_pipehndl_tq_5
stack pointer for thread ffffff001729ec60: ffffff001729eae0
[ ffffff001729eae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00180b5c60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 TASKQ: STMF_SVC_TASKQ
stack pointer for thread ffffff00180b5c60: ffffff00180b5a50
[ ffffff00180b5a50 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_hires+0xe0()
cv_reltimedwait+0x4f()
stmf_svc+0x423()
taskq_thread+0x248()
thread_start+8()
ffffff00172e0c60 fffffffffbc2dbf0 0 0 60 ffffff03da0b0490
PC: _resume_from_idle+0xf1 THREAD: evch_delivery_thr()
stack pointer for thread ffffff00172e0c60: ffffff00172e0ae0
[ ffffff00172e0ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
evch_delivery_hold+0x85()
evch_delivery_thr+0x23e()
thread_start+8()
ffffff00172e6c60 fffffffffbc2dbf0 0 0 60 ffffff03dd4d6c78
PC: _resume_from_idle+0xf1 THREAD: evch_delivery_thr()
stack pointer for thread ffffff00172e6c60: ffffff00172e6ae0
[ ffffff00172e6ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
evch_delivery_hold+0x85()
evch_delivery_thr+0x23e()
thread_start+8()
ffffff00172ecc60 fffffffffbc2dbf0 0 0 109 0
PC: _resume_from_idle+0xf1 THREAD: cpu_pause()
stack pointer for thread ffffff00172ecc60: ffffff00172ecbd0
[ ffffff00172ecbd0 _resume_from_idle+0xf1() ]
swtch+0x145()
cpu_pause+0x8c()
thread_start+8()
ffffff00173a3c60 fffffffffbc2dbf0 0 0 99 ffffff03e1552e10
PC: _resume_from_idle+0xf1 THREAD: squeue_worker()
stack pointer for thread ffffff00173a3c60: ffffff00173a3b80
[ ffffff00173a3b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_worker+0xc0()
thread_start+8()
ffffff00173a9c60 fffffffffbc2dbf0 0 0 99 ffffff03e1552e12
PC: _resume_from_idle+0xf1 THREAD: squeue_polling_thread()
stack pointer for thread ffffff00173a9c60: ffffff00173a9b50
[ ffffff00173a9b50 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_polling_thread+0x90()
thread_start+8()
ffffff00173afc60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 TASKQ: cpudrv_cpudrv_monitor
stack pointer for thread ffffff00173afc60: ffffff00173afae0
[ ffffff00173afae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00172f8c60 fffffffffbc2dbf0 0 0 -1 0
PC: _resume_from_idle+0xf1 THREAD: idle()
stack pointer for thread ffffff00172f8c60: ffffff00172f8bf0
[ ffffff00172f8bf0 _resume_from_idle+0xf1() ]
swtch+0x145()
idle+0xc4()
thread_start+8()
ffffff001733ac60 fffffffffbc2dbf0 0 0 109 0
PC: _resume_from_idle+0xf1 THREAD: cpu_pause()
stack pointer for thread ffffff001733ac60: ffffff001733abd0
[ ffffff001733abd0 _resume_from_idle+0xf1() ]
swtch+0x145()
cpu_pause+0x8c()
thread_start+8()
ffffff00178c1c60 fffffffffbc2dbf0 0 0 99 ffffff03e1552d50
PC: _resume_from_idle+0xf1 THREAD: squeue_worker()
stack pointer for thread ffffff00178c1c60: ffffff00178c1b80
[ ffffff00178c1b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_worker+0xc0()
thread_start+8()
ffffff00178c7c60 fffffffffbc2dbf0 0 0 99 ffffff03e1552d52
PC: _resume_from_idle+0xf1 THREAD: squeue_polling_thread()
stack pointer for thread ffffff00178c7c60: ffffff00178c7b50
[ ffffff00178c7b50 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_polling_thread+0x90()
thread_start+8()
ffffff00178cdc60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 TASKQ: cpudrv_cpudrv_monitor
stack pointer for thread ffffff00178cdc60: ffffff00178cdae0
[ ffffff00178cdae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001784dc60 fffffffffbc2dbf0 0 0 -1 0
PC: _resume_from_idle+0xf1 THREAD: idle()
stack pointer for thread ffffff001784dc60: ffffff001784dbf0
[ ffffff001784dbf0 _resume_from_idle+0xf1() ]
swtch+0x145()
idle+0xc4()
thread_start+8()
ffffff001788fc60 fffffffffbc2dbf0 0 0 109 0
PC: _resume_from_idle+0xf1 THREAD: cpu_pause()
stack pointer for thread ffffff001788fc60: ffffff001788fbd0
[ ffffff001788fbd0 _resume_from_idle+0xf1() ]
swtch+0x145()
cpu_pause+0x8c()
thread_start+8()
ffffff001792cc60 fffffffffbc2dbf0 0 0 99 ffffff03e1552c90
PC: _resume_from_idle+0xf1 THREAD: squeue_worker()
stack pointer for thread ffffff001792cc60: ffffff001792cb80
[ ffffff001792cb80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_worker+0xc0()
thread_start+8()
ffffff0017932c60 fffffffffbc2dbf0 0 0 99 ffffff03e1552c92
PC: _resume_from_idle+0xf1 THREAD: squeue_polling_thread()
stack pointer for thread ffffff0017932c60: ffffff0017932b50
[ ffffff0017932b50 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
squeue_polling_thread+0x90()
thread_start+8()
ffffff0017938c60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 TASKQ: cpudrv_cpudrv_monitor
stack pointer for thread ffffff0017938c60: ffffff0017938ae0
[ ffffff0017938ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00178d9c60 fffffffffbc2dbf0 0 0 -1 0
PC: _resume_from_idle+0xf1 THREAD: idle()
stack pointer for thread ffffff00178d9c60: ffffff00178d9bf0
[ ffffff00178d9bf0 _resume_from_idle+0xf1() ]
swtch+0x145()
idle+0xc4()
thread_start+8()
ffffff0017909c60 fffffffffbc2dbf0 ffffff03e060d220 0 164 0
PC: panicsys+0x9b THREAD: unix`thread_create_intr()
stack pointer for thread ffffff0017909c60: ffffff00179094f0
panic+0x94()
die+0xdd()
trap+0x177b()
0xfffffffffb8001d6()
debug_enter+0x38()
abort_sequence_enter+0x35()
kbtrans_streams_key+0x102()
conskbdlrput+0xe7()
putnext+0x21e()
kbtrans_queueevent+0x7c()
kbtrans_queuepress+0x7c()
kbtrans_untrans_keypressed_raw+0x46()
kbtrans_processkey+0x32()
kbtrans_streams_key+0x175()
kb8042_process_key+0x40()
kb8042_received_byte+0x109()
kb8042_intr+0x6a()
i8042_intr+0xc5()
av_dispatch_autovect+0x7c()
dispatch_hardint+0x33()
switch_sp_and_call+0x13()
do_interrupt+0xb8()
_interrupt+0xb8()
htable_steal+0x198()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
lwp_park+0x242()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff001791bc60 fffffffffbc2dbf0 0 0 109 0
PC: _resume_from_idle+0xf1 THREAD: cpu_pause()
stack pointer for thread ffffff001791bc60: ffffff001791bbd0
[ ffffff001791bbd0 _resume_from_idle+0xf1() ]
swtch+0x145()
cpu_pause+0x8c()
thread_start+8()
ffffff001798bc60 fffffffffbc2dbf0 0 0 99 0
PC: _resume_from_idle+0xf1 TASKQ: callout_taskq
stack pointer for thread ffffff001798bc60: ffffff001798bae0
[ ffffff001798bae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00178acc60 fffffffffbc2dbf0 0 0 99 0
PC: panic_idle+0x20 TASKQ: callout_taskq
stack pointer for thread ffffff00178acc60: ffffff00178ac820
dld_wput+0x14d()
putnext+0x411()
0xf0()
do_interrupt+0xf2()
_interrupt+0xb8()
ddi_get32+0x14()
ehci_handle_root_hub_status_change+0xe5()
callout_list_expire+0x77()
callout_expire+0x31()
callout_execute+0x1e()
taskq_thread+0x248()
thread_start+8()
ffffff0017997c60 fffffffffbc2dbf0 0 0 99 0
PC: panic_idle+0x20 TASKQ: callout_taskq
stack pointer for thread ffffff0017997c60: ffffff00179978e0
0xf0()
do_interrupt+0xf2()
_interrupt+0xb8()
kmem_cache_update+0x11()
kmem_cache_applyall+0x67()
kmem_update+0x25()
callout_list_expire+0x77()
callout_expire+0x31()
callout_execute+0x1e()
taskq_thread+0x248()
thread_start+8()
ffffff0017991c60 fffffffffbc2dbf0 0 0 99 ffffff03e246b290
PC: _resume_from_idle+0xf1 TASKQ: callout_taskq
stack pointer for thread ffffff0017991c60: ffffff0017991ae0
[ ffffff0017991ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00179a3c60 fffffffffbc2dbf0 0 0 99 ffffff03e246b178
PC: _resume_from_idle+0xf1 TASKQ: callout_taskq
stack pointer for thread ffffff00179a3c60: ffffff00179a3ae0
[ ffffff00179a3ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001799dc60 fffffffffbc2dbf0 0 0 99 0
PC: _resume_from_idle+0xf1 TASKQ: callout_taskq
stack pointer for thread ffffff001799dc60: ffffff001799dae0
[ ffffff001799dae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00182d6c60 fffffffffbc2dbf0 0 0 98 ffffff03e2bc0b58
PC: _resume_from_idle+0xf1 TASKQ: console_taskq
stack pointer for thread ffffff00182d6c60: ffffff00182d6ae0
[ ffffff00182d6ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff03e279f020 ffffff03e05e08d8 ffffff03e0614620 1 59 ffffff03e05e1358
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e279f020: ffffff00178a1d80
[ ffffff00178a1d80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig+0x14d()
door_unref+0x89()
doorfs32+0x82()
sys_syscall32+0xff()
ffffff03e2f0b040 ffffff03e05e08d8 ffffff03e061bc40 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e2f0b040: ffffff001841bc80
[ ffffff001841bc80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff03e30e7180 ffffff03e05e08d8 ffffff03e06630e0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e30e7180: ffffff00173762f0
[ ffffff00173762f0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
lwp_park+0x242()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff03e31003e0 ffffff03e05e08d8 ffffff03e0620050 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e31003e0: ffffff0018458d10
[ ffffff0018458d10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e32ee900 ffffff03e05e08d8 ffffff03e2b957b0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e32ee900: ffffff0017370d10
[ ffffff0017370d10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e37eeb00 ffffff03e05e08d8 ffffff03e0615140 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e37eeb00: ffffff0019d3ad10
[ ffffff0019d3ad10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e2f0c8c0 ffffff03e05e08d8 ffffff03e0626030 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e2f0c8c0: ffffff0018446d10
[ ffffff0018446d10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e2f0b3c0 ffffff03e05e08d8 ffffff03e0626b50 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e2f0b3c0: ffffff0018415c80
[ ffffff0018415c80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff03e32ff3a0 ffffff03e05e08d8 ffffff03e065b750 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e32ff3a0: ffffff00180c1d10
[ ffffff00180c1d10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e27a08a0 ffffff03e05e08d8 ffffff03e06240f0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e27a08a0: ffffff0018452d10
[ ffffff0018452d10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e30ff580 ffffff03e05e08d8 ffffff03e061f750 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e30ff580: ffffff001847cd10
[ ffffff001847cd10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e32ff020 ffffff03e05e08d8 ffffff03e0660680 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e32ff020: ffffff00180c7c80
[ ffffff00180c7c80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff03e32f5e60 ffffff03e05e08d8 ffffff03e2b92230 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e32f5e60: ffffff001836bd10
[ ffffff001836bd10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e30fe080 ffffff03e05e08d8 ffffff03e2b86d90 1 59 ffffff03e30fe25e
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e30fe080: ffffff0019d28c30
[ ffffff0019d28c30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff03e2f0cc40 ffffff03e05e08d8 ffffff03e0613200 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e2f0cc40: ffffff00178b2d10
[ ffffff00178b2d10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e279f3a0 ffffff03e05e08d8 ffffff03e061c1d0 1 59 ffffff03e279f57e
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.configd
stack pointer for thread ffffff03e279f3a0: ffffff0018483cb0
[ ffffff0018483cb0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
sigtimedwait+0x1ca()
sys_syscall32+0xff()
ffffff03e279faa0 ffffff03e05e1540 ffffff03e0628720 1 59 ffffff03e2e012fc
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e279faa0: ffffff0017b1ea90
[ ffffff0017b1ea90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig+0x14d()
cte_get_event+0xd7()
ctfs_endpoint_ioctl+0xa7()
ctfs_bu_ioctl+0x51()
fop_ioctl+0x7b()
ioctl+0x18e()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff03e2f0c540 ffffff03e05e1540 ffffff03e060b0c0 1 59 ffffff03e2f0c71e
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e2f0c540: ffffff00178b8c30
[ ffffff00178b8c30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff0018898c60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc06f8
PC: _resume_from_idle+0xf1 TASKQ: zil_clean
stack pointer for thread ffffff0018898c60: ffffff0018898ae0
[ ffffff0018898ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00173cac60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc05e0
PC: _resume_from_idle+0xf1 TASKQ: zil_clean
stack pointer for thread ffffff00173cac60: ffffff00173caae0
[ ffffff00173caae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff03e363b540 ffffff03e20656e8 ffffff03e2b818d0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /sbin/dlmgmtd
stack pointer for thread ffffff03e363b540: ffffff0019d9ad10
[ ffffff0019d9ad10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e32e5e20 ffffff03e20656e8 ffffff03e3a4f990 1 59 0
PC: _resume_from_idle+0xf1 CMD: /sbin/dlmgmtd
stack pointer for thread ffffff03e32e5e20: ffffff0018383d10
[ ffffff0018383d10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e32ed400 ffffff03e20656e8 ffffff03e3a4f400 1 59 0
PC: _resume_from_idle+0xf1 CMD: /sbin/dlmgmtd
stack pointer for thread ffffff03e32ed400: ffffff0018389d30
[ ffffff0018389d30 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e37f8760 ffffff03e20656e8 ffffff03e2b848c0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /sbin/dlmgmtd
stack pointer for thread ffffff03e37f8760: ffffff0019d6ad10
[ ffffff0019d6ad10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e32f5ae0 ffffff03e20656e8 ffffff03e0b3ed30 1 59 0
PC: _resume_from_idle+0xf1 CMD: /sbin/dlmgmtd
stack pointer for thread ffffff03e32f5ae0: ffffff00183d5c80
[ ffffff00183d5c80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff0018168c60 fffffffffbc2dbf0 0 0 60 ffffffffc010bc28
PC: _resume_from_idle+0xf1 THREAD: softmac_taskq_dispatch()
stack pointer for thread ffffff0018168c60: ffffff0018168b80
[ ffffff0018168b80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
softmac_taskq_dispatch+0x104()
thread_start+8()
ffffff03e32f93c0 ffffff03e20656e8 ffffff03e0662250 1 59 ffffff03e32f959e
PC: _resume_from_idle+0xf1 CMD: /sbin/dlmgmtd
stack pointer for thread ffffff03e32f93c0: ffffff00183abdf0
[ ffffff00183abdf0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
pause+0x48()
sys_syscall32+0xff()
ffffff0018319c60 fffffffffbc2dbf0 0 0 60 ffffff03e3a678c8
PC: _resume_from_idle+0xf1 THREAD: i_mac_notify_thread()
stack pointer for thread ffffff0018319c60: ffffff0018319b60
[ ffffff0018319b60 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
i_mac_notify_thread+0xc7()
thread_start+8()
ffffff0019bcfc60 fffffffffbc2dbf0 0 0 60 ffffff03e3a67100
PC: _resume_from_idle+0xf1 THREAD: i_mac_notify_thread()
stack pointer for thread ffffff0019bcfc60: ffffff0019bcfb60
[ ffffff0019bcfb60 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
i_mac_notify_thread+0xc7()
thread_start+8()
ffffff001816ec60 fffffffffbc2dbf0 0 0 60 ffffff03dd427528
PC: _resume_from_idle+0xf1 THREAD: kcf_failover_thread()
stack pointer for thread ffffff001816ec60: ffffff001816eb70
[ ffffff001816eb70 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
kcf_failover_thread+0x185()
thread_start+8()
ffffff0018162c60 fffffffffbc2dbf0 0 0 60 fffffffffbd10cb8
PC: _resume_from_idle+0xf1 THREAD: crypto_bufcall_service()
stack pointer for thread ffffff0018162c60: ffffff0018162ba0
[ ffffff0018162ba0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
crypto_bufcall_service+0x152()
thread_start+8()
ffffff03e32ee200 ffffff03e2064a80 ffffff03eeefa8a0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/crypto/kcfd
stack pointer for thread ffffff03e32ee200: ffffff00183f4ca0
[ ffffff00183f4ca0 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32e9e00 ffffff03e2064a80 ffffff03e2b80290 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/crypto/kcfd
stack pointer for thread ffffff03e32e9e00: ffffff0018174c80
[ ffffff0018174c80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37ee080 ffffff03e2064a80 ffffff03e2b82980 1 59 ffffff03e37ee25e
PC: _resume_from_idle+0xf1 CMD: /lib/crypto/kcfd
stack pointer for thread ffffff03e37ee080: ffffff0019d88df0
[ ffffff0019d88df0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
pause+0x48()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32ffe20 ffffff03e205e6f0 ffffff03e2b883d0 1 59 ffffff03e32ffffe
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e32ffe20: ffffff0017364c30
[ ffffff0017364c30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32fa540 ffffff03e205e6f0 ffffff03e0618130 1 59 ffffff03e32fa71e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e32fa540: ffffff001735ec30
[ ffffff001735ec30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32fa8c0 ffffff03e205e6f0 ffffff03e06186c0 1 59 ffffff03e32faa9e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e32fa8c0: ffffff001845ecb0
[ ffffff001845ecb0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
sigtimedwait+0x1e3()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30ffc80 ffffff03e205e6f0 ffffff03e0667180 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e30ffc80: ffffff0019bf9c80
[ ffffff0019bf9c80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37faac0 ffffff03e205e6f0 ffffff03e0612150 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e37faac0: ffffff00183dbd30
[ ffffff00183dbd30 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37f98e0 ffffff03e205e6f0 ffffff03e060bbe0 1 59 ffffff03e37f9abe
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e37f98e0: ffffff001739dcc0
[ ffffff001739dcc0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3628700 ffffff03e205e6f0 ffffff03e2b853e0 1 59 ffffff03e36288de
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e3628700: ffffff0019cecc30
[ ffffff0019cecc30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30f18c0 ffffff03e205e6f0 ffffff03e0616780 1 59 ffffff03e30f1a9e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e30f18c0: ffffff001842dc30
[ ffffff001842dc30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30f1c40 ffffff03e205e6f0 ffffff03e060e640 1 59 ffffff03e30f1e1e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e30f1c40: ffffff00182dcc30
[ ffffff00182dcc30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37fae40 ffffff03e205da88 ffffff03e060d7b0 1 59 ffffff03e37fb01e
PC: _resume_from_idle+0xf1 CMD: devfsadmd
stack pointer for thread ffffff03e37fae40: ffffff00182a9cc0
[ ffffff00182a9cc0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32f5060 ffffff03e205da88 ffffff03e065c270 1 60 fffffffffbc61d30
PC: _resume_from_idle+0xf1 CMD: devfsadmd
stack pointer for thread ffffff03e32f5060: ffffff00184952f0
[ ffffff00184952f0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
lwp_park+0x242()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32f68e0 ffffff03e205da88 ffffff03e061b6b0 1 59 0
PC: _resume_from_idle+0xf1 CMD: devfsadmd
stack pointer for thread ffffff03e32f68e0: ffffff00184eed30
[ ffffff00184eed30 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37f8060 ffffff03e205da88 ffffff03e065c800 1 59 0
PC: _resume_from_idle+0xf1 CMD: devfsadmd
stack pointer for thread ffffff03e37f8060: ffffff001849bc80
[ ffffff001849bc80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37ee400 ffffff03e205da88 ffffff03e065d100 1 59 ffffff03e37ee5de
PC: _resume_from_idle+0xf1 CMD: devfsadmd
stack pointer for thread ffffff03e37ee400: ffffff00184a1c30
[ ffffff00184a1c30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37f8e60 ffffff03e205da88 ffffff03e065e1b0 1 59 ffffff03e37f903e
PC: _resume_from_idle+0xf1 CMD: devfsadmd
stack pointer for thread ffffff03e37f8e60: ffffff001848fdf0
[ ffffff001848fdf0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
pause+0x75()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3101560 ffffff03e205e6f0 ffffff03e060f6f0 1 59 ffffff03e310173e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e3101560: ffffff0017a75c30
[ ffffff0017a75c30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e27a0520 ffffff03e205e6f0 ffffff03e0610210 1 59 ffffff03e27a06fe
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e27a0520: ffffff001734bc30
[ ffffff001734bc30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3634ae0 ffffff03e205e6f0 ffffff03e06126e0 1 59 ffffff03e3634cbe
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e3634ae0: ffffff0019ca4c30
[ ffffff0019ca4c30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37ea380 ffffff03e205e6f0 ffffff03e061ab90 1 59 ffffff03e37ea55e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e37ea380: ffffff0019d52c30
[ ffffff0019d52c30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32ea880 ffffff03e205e6f0 ffffff03e0621c20 1 59 ffffff03e32eaa5e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e32ea880: ffffff0019cfec30
[ ffffff0019cfec30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e279f720 ffffff03e205e6f0 ffffff03e2b83280 1 52 ffffff03e279f8fe
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e279f720: ffffff0019ce6cc0
[ ffffff0019ce6cc0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32e9a80 ffffff03e205e6f0 ffffff03e060c170 1 59 ffffff03e32e9c5e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e32e9a80: ffffff0019bedc30
[ ffffff0019bedc30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff0019bf3c60 fffffffffbc2dbf0 0 0 60 fffffffffbc937ac
PC: _resume_from_idle+0xf1 THREAD: log_event_deliver()
stack pointer for thread ffffff0019bf3c60: ffffff0019bf3b90
[ ffffff0019bf3b90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
log_event_deliver+0x20e()
thread_start+8()
ffffff03e30e6e00 ffffff03e205e6f0 ffffff03e0617080 1 59 ffffff03e30e6fde
PC: _resume_from_idle+0xf1 CMD: /usr/lib/sysevent/syseventd
stack pointer for thread ffffff03e30e6e00: ffffff001846adf0
[ ffffff001846adf0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
pause+0x75()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3622720 ffffff03e2060c78 ffffff03e061a070 1 59 ffffff03e2e9a7ca
PC: _resume_from_idle+0xf1 CMD: /usr/lib/power/powerd
stack pointer for thread ffffff03e3622720: ffffff0019d4cc40
[ ffffff0019d4cc40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37fbc40 ffffff03e2060c78 ffffff03e2b83810 1 59 ffffff03e37fbe1e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/power/powerd
stack pointer for thread ffffff03e37fbc40: ffffff0019dbcdc0
[ ffffff0019dbcdc0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
sigsuspend+0xeb()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37fb8c0 ffffff03e2060c78 ffffff03e0665240 1 59 ffffff03e360818a
PC: _resume_from_idle+0xf1 CMD: /usr/lib/power/powerd
stack pointer for thread ffffff03e37fb8c0: ffffff00183b1c40
[ ffffff00183b1c40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3623520 ffffff03e2060c78 ffffff03e0617ba0 1 59 ffffff03e36236fe
PC: _resume_from_idle+0xf1 CMD: /usr/lib/power/powerd
stack pointer for thread ffffff03e3623520: ffffff0019d64dc0
[ ffffff0019d64dc0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
sigsuspend+0xeb()
_sys_sysenter_post_swapgs+0x149()
ffffff03e362c080 ffffff03e05e21a8 ffffff03e2b84330 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e362c080: ffffff0018476330
[ ffffff0018476330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30e6700 ffffff03e05e21a8 ffffff03e06657d0 1 59 ffffff03e05e2c28
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e30e6700: ffffff001825ad80
[ ffffff001825ad80 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig+0x14d()
door_unref+0x89()
doorfs32+0x82()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30f0040 ffffff03e05e21a8 ffffff03e2b87e40 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e30f0040: ffffff0019d40330
[ ffffff0019d40330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32e6520 ffffff03e05e21a8 ffffff03e2b81340 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e32e6520: ffffff0019da0330
[ ffffff0019da0330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37ea000 ffffff03e05e21a8 ffffff03e2b86270 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e37ea000: ffffff0019d94330
[ ffffff0019d94330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30fe780 ffffff03e05e21a8 ffffff03e2b86800 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e30fe780: ffffff0018146330
[ ffffff0018146330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30f9880 ffffff03e05e21a8 ffffff03e0621690 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e30f9880: ffffff0019d1c330
[ ffffff0019d1c330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea77b180 ffffff03e05e21a8 ffffff03e0615c60 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03ea77b180: ffffff0018152330
[ ffffff0018152330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e363aac0 ffffff03e05e21a8 ffffff03e0b3f2c0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e363aac0: ffffff0018341330
[ ffffff0018341330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32fa1c0 ffffff03e05e21a8 ffffff03e06110a0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e32fa1c0: ffffff00182af5c0
[ ffffff00182af5c0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0xe45()
0xfffffffffb8001d6()
ffffff03e30ec060 ffffff03e05e21a8 ffffff03e3a4d830 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e30ec060: ffffff00183c9330
[ ffffff00183c9330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37eb500 ffffff03e05e21a8 ffffff03e060ebd0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e37eb500: ffffff0019be7330
[ ffffff0019be7330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea699e80 ffffff03e05e21a8 ffffff03eeefb3c0 1 60 fffffffffbc61d30
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03ea699e80: ffffff0018670330
[ ffffff0018670330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37ea700 ffffff03e05e21a8 ffffff03e0664cb0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e37ea700: ffffff001838fc80
[ ffffff001838fc80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea49cc40 ffffff03e05e21a8 ffffff03e060f160 1 59 ffffff03ea49ce1e
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03ea49cc40: ffffff0017a4dc60
[ ffffff0017a4dc60 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_sig_hires+0x1e9()
cv_waituntil_sig+0xba()
nanosleep+0x120()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea49c8c0 ffffff03e05e21a8 ffffff03e061c760 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03ea49c8c0: ffffff00184b9330
[ ffffff00184b9330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea575e00 ffffff03e05e21a8 ffffff03e06627e0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03ea575e00: ffffff001859d330
[ ffffff001859d330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea575a80 ffffff03e05e21a8 ffffff03e0667710 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03ea575a80: ffffff00185a3330
[ ffffff00185a3330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32e53a0 ffffff03e05e21a8 ffffff03e065dc20 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e32e53a0: ffffff0019d762f0
[ ffffff0019d762f0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
lwp_park+0x242()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30edc60 ffffff03e05e21a8 ffffff03e2b80db0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e30edc60: ffffff00184cb330
[ ffffff00184cb330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30f51a0 ffffff03e05e21a8 ffffff03e2b80820 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e30f51a0: ffffff0019be1330
[ ffffff0019be1330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37f9c60 ffffff03e05e21a8 ffffff03e06161f0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e37f9c60: ffffff0019d46c80
[ ffffff0019d46c80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37fa740 ffffff03e05e21a8 ffffff03e0668230 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e37fa740: ffffff00182545c0
[ ffffff00182545c0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0xe45()
0xfffffffffb8001d6()
ffffff03e32e5aa0 ffffff03e05e21a8 ffffff03e0664190 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e32e5aa0: ffffff00182bb330
[ ffffff00182bb330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e36223a0 ffffff03e05e21a8 ffffff03e0661cc0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e36223a0: ffffff00182c1330
[ ffffff00182c1330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3622020 ffffff03e05e21a8 ffffff03e06611a0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e3622020: ffffff00182c7330
[ ffffff00182c7330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37fa040 ffffff03e05e21a8 ffffff03e065bce0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e37fa040: ffffff00182cd330
[ ffffff00182cd330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37f83e0 ffffff03e05e21a8 ffffff03e0620b70 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e37f83e0: ffffff0018489330
[ ffffff0018489330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3629c00 ffffff03e05e21a8 ffffff03e0666bf0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e3629c00: ffffff0018266d10
[ ffffff0018266d10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
_sys_sysenter_post_swapgs+0x149()
ffffff03e363ae40 ffffff03e05e21a8 ffffff03e0663670 1 60 ffffff03e363f608
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e363ae40: ffffff001826c950
[ ffffff001826c950 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig+0x14d()
so_dequeue_msg+0x2a2()
so_recvmsg+0x185()
socket_recvmsg+0x3d()
socket_vop_read+0x52()
fop_read+0x6b()
read+0x2b8()
read32+0x22()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32e61a0 ffffff03e05e21a8 ffffff03e06660d0 1 59 ffffff03e32e637e
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/nscd
stack pointer for thread ffffff03e32e61a0: ffffff0018260df0
[ ffffff0018260df0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
pause+0x48()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea5901e0 ffffff03e20576f8 ffffff03e0b3e210 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/picl/picld
stack pointer for thread ffffff03ea5901e0: ffffff00184d1c80
[ ffffff00184d1c80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea576c00 ffffff03e20576f8 ffffff03e065ecd0 1 59 ffffff03ea576dde
PC: _resume_from_idle+0xf1 CMD: /usr/lib/picl/picld
stack pointer for thread ffffff03ea576c00: ffffff001852ec30
[ ffffff001852ec30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea576880 ffffff03e20576f8 ffffff03e065f260 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/picl/picld
stack pointer for thread ffffff03ea576880: ffffff0018534d30
[ ffffff0018534d30 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
_sys_sysenter_post_swapgs+0x149()
ffffff03e6435aa0 ffffff03e20576f8 ffffff03e065a110 1 59 ffffff03e6435c7e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/picl/picld
stack pointer for thread ffffff03e6435aa0: ffffff00184c5df0
[ ffffff00184c5df0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
pause+0x48()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea6b5720 ffffff03e2058360 ffffff03e0b3e7a0 3 100 ffffff03dd38e24e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/inet/xntpd
stack pointer for thread ffffff03ea6b5720: ffffff00185f1670
[ ffffff00185f1670 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
txg_wait_open+0x7a()
dmu_tx_wait+0xb3()
zfs_create+0x465()
fop_create+0xfc()
vn_createat+0x5e1()
vn_openat+0x1fb()
copen+0x435()
open32+0x2f()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea58d400 ffffff03e205f358 ffffff03e06221b0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/method/iscsi-initiator
stack pointer for thread ffffff03ea58d400: ffffff001850cd30
[ ffffff001850cd30 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
_sys_sysenter_post_swapgs+0x149()
ffffff00185bbc60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc04c8
PC: _resume_from_idle+0xf1 TASKQ: idm_global_taskq
stack pointer for thread ffffff00185bbc60: ffffff00185bbae0
[ ffffff00185bbae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00185c1c60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 THREAD: idm_wd_thread()
stack pointer for thread ffffff00185c1c60: ffffff00185c1b30
[ ffffff00185c1b30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_hires+0xe0()
cv_reltimedwait+0x4f()
idm_wd_thread+0x1d7()
thread_start+8()
ffffff00185c7c60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc03b0
PC: _resume_from_idle+0xf1 TASKQ: iscsi_nexus_enum_tq
stack pointer for thread ffffff00185c7c60: ffffff00185c7ae0
[ ffffff00185c7ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00185cdc60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc0298
PC: _resume_from_idle+0xf1 TASKQ: isns_reg_query_taskq
stack pointer for thread ffffff00185cdc60: ffffff00185cdae0
[ ffffff00185cdae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00185d3c60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc0180
PC: _resume_from_idle+0xf1 TASKQ: isns_scn_taskq
stack pointer for thread ffffff00185d3c60: ffffff00185d3ae0
[ ffffff00185d3ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00185d9c60 fffffffffbc2dbf0 0 0 60 ffffff03e2bc0068
PC: _resume_from_idle+0xf1 TASKQ: iscsi_Static
stack pointer for thread ffffff00185d9c60: ffffff00185d9ae0
[ ffffff00185d9ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00185dfc60 fffffffffbc2dbf0 0 0 60 ffffff03ea6deea8
PC: _resume_from_idle+0xf1 TASKQ: iscsi_SendTarget
stack pointer for thread ffffff00185dfc60: ffffff00185dfae0
[ ffffff00185dfae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00185e5c60 fffffffffbc2dbf0 0 0 60 ffffff03ea6ded90
PC: _resume_from_idle+0xf1 TASKQ: iscsi_SLP
stack pointer for thread ffffff00185e5c60: ffffff00185e5ae0
[ ffffff00185e5ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00185ebc60 fffffffffbc2dbf0 0 0 60 ffffff03ea6dec78
PC: _resume_from_idle+0xf1 TASKQ: iscsi_iSNS
stack pointer for thread ffffff00185ebc60: ffffff00185ebae0
[ ffffff00185ebae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff03ea58d780 ffffff03e205f358 ffffff03e065ac30 1 59 ffffff03ea58d95e
PC: _resume_from_idle+0xf1 CMD: /lib/svc/method/iscsi-initiator
stack pointer for thread ffffff03ea58d780: ffffff0018506cb0
[ ffffff0018506cb0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
sigtimedwait+0x1ca()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37f91e0 ffffff03ea8fba98 ffffff03e2b94390 1 59 ffffff03e366a76a
PC: _resume_from_idle+0xf1 CMD: /usr/bin/dbus-daemon --system
stack pointer for thread ffffff03e37f91e0: ffffff0019dacc40
[ ffffff0019dacc40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff03e363a040 ffffff03e2066350 ffffff03e2b93e00 1 59 ffffff03e05ad4b2
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/winbindd
stack pointer for thread ffffff03e363a040: ffffff0018371bb0
[ ffffff0018371bb0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32e6c20 ffffff03e20618e0 ffffff03e2b89d80 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/winbindd
stack pointer for thread ffffff03e32e6c20: ffffff00180e4270
[ ffffff00180e4270 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1a4()
poll_common+0x614()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e33001a0 ffffff03eee6d040 ffffff03f1fd70a0 1 60 fffffffffbd1da50
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/cron
stack pointer for thread ffffff03e33001a0: ffffff00173511a0
[ ffffff00173511a0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
spa_open_common+0x69()
spa_open+0x23()
dsl_dir_open_spa+0x95()
dsl_dataset_hold+0x41()
dsl_prop_get+0x4c()
zfs_is_wormed+0x73()
zfs_create+0x69e()
fop_create+0xfc()
vn_createat+0x5e1()
vn_openat+0x1fb()
copen+0x435()
open64+0x34()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30e8080 ffffff04f4f46388 ffffff050d79bc80 1 60 ffffff04d629c7d0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/cron
stack pointer for thread ffffff03e30e8080: ffffff0019d7c630
[ ffffff0019d7c630 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
zfs_dirent_lock+0x226()
zfs_create+0x214()
fop_create+0xfc()
vn_createat+0x5e1()
vn_openat+0x1fb()
copen+0x435()
open64+0x34()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea6b53a0 ffffff03eee72718 ffffff050d79a0b0 1 60 ffffff04d629c7d0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/cron
stack pointer for thread ffffff03ea6b53a0: ffffff0018682630
[ ffffff0018682630 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
zfs_dirent_lock+0x226()
zfs_create+0x214()
fop_create+0xfc()
vn_createat+0x5e1()
vn_openat+0x1fb()
copen+0x435()
open64+0x34()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30f8380 ffffff03e20631b0 ffffff03e2b8d890 1 59 ffffff03e2bcf4b2
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/cron
stack pointer for thread ffffff03e30f8380: ffffff00183c3ba0
[ ffffff00183c3ba0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_sig_hires+0x1e9()
cv_waituntil_sig+0xba()
poll_common+0x462()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32e9000 ffffff03e2052c88 ffffff03e2b8d300 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/hotplugd
stack pointer for thread ffffff03e32e9000: ffffff00183cfd30
[ ffffff00183cfd30 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3629180 ffffff03e2052c88 ffffff03e2b93870 1 59 ffffff03e362935e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/hotplugd
stack pointer for thread ffffff03e3629180: ffffff0019db2cc0
[ ffffff0019db2cc0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
_sys_sysenter_post_swapgs+0x149()
ffffff001822ac60 fffffffffbc2dbf0 0 0 60 ffffff03ea6deb60
PC: _resume_from_idle+0xf1 TASKQ: iscsit_dispatch
stack pointer for thread ffffff001822ac60: ffffff001822aae0
[ ffffff001822aae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff03e30ec3e0 ffffff03ea8f88f8 ffffff03e065a6a0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/inet/in.ndpd
stack pointer for thread ffffff03e30ec3e0: ffffff00184b3100
[ ffffff00184b3100 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x320()
savecontext32+0x8c()
sendsig32+0x48b()
psig+0x3e2()
post_syscall+0x49e()
syscall_exit+0x59()
0xfffffffffb800ed1()
ffffff03e30f9180 ffffff03e205a8e8 ffffff03e2b823f0 1 59 ffffff03e3608c2a
PC: _resume_from_idle+0xf1 CMD: /usr/lib/hal/hald-addon-network-discovery
stack pointer for thread ffffff03e30f9180: ffffff0019d8ec40
[ ffffff0019d8ec40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3634e60 ffffff03e2054558 ffffff03e2b878b0 1 59 ffffff03e36088ba
PC: _resume_from_idle+0xf1 CMD: /usr/lib/hal/hald-addon-cpufreq
stack pointer for thread ffffff03e3634e60: ffffff00185b5c40
[ ffffff00185b5c40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32ea180 ffffff03e20538f0 ffffff03e2b84e50 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/hal/hald-addon-acpi
stack pointer for thread ffffff03e32ea180: ffffff0018335610
[ ffffff0018335610 _resume_from_idle+0xf1() ]
mutex_enter+0x10()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0xe45()
0xfffffffffb8001d6()
ffffff03e3628000 ffffff03e2060010 ffffff03e2b8f240 1 59 ffffff03e36183aa
PC: _resume_from_idle+0xf1 CMD: hald-runner
stack pointer for thread ffffff03e3628000: ffffff0019bd5c40
[ ffffff0019bd5c40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37fa3c0 ffffff03ea8fd368 ffffff03e2b89260 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/hal/hald --daemon=yes
stack pointer for thread ffffff03e37fa3c0: ffffff0019c27ca0
[ ffffff0019c27ca0 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e362c780 ffffff03ea8fd368 ffffff03e2b92d50 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/hal/hald --daemon=yes
stack pointer for thread ffffff03e362c780: ffffff0018512c80
[ ffffff0018512c80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3629500 ffffff03ea8fd368 ffffff03e2b927c0 1 59 ffffff03e36296de
PC: _resume_from_idle+0xf1 CMD: /usr/lib/hal/hald --daemon=yes
stack pointer for thread ffffff03e3629500: ffffff00183eec30
[ ffffff00183eec30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30f4720 ffffff03ea8fd368 ffffff03e2b8c250 1 59 ffffff03e360ee52
PC: _resume_from_idle+0xf1 CMD: /usr/lib/hal/hald --daemon=yes
stack pointer for thread ffffff03e30f4720: ffffff00180eabb0
[ ffffff00180eabb0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30e9900 ffffff03e05dfc70 ffffff03e2b90880 1 59 ffffff03e366d7d2
PC: _resume_from_idle+0xf1 CMD:
dbus-daemon --fork --config-file=/var/lib/nza/nms.conf --print-address
stack pointer for thread ffffff03e30e9900: ffffff0019d82bb0
[ ffffff0019d82bb0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30fe400 ffffff03ea8f9560 ffffff03e2b8a8a0 1 59 ffffff03e29f095a
PC: _resume_from_idle+0xf1 CMD:
dbus-daemon --fork --config-file=/var/lib/nza/nmcd.conf --print-address
stack pointer for thread ffffff03e30fe400: ffffff00180d8bb0
[ ffffff00180d8bb0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30f5520 ffffff03e205b550 ffffff03e060c700 1 59 ffffff03e3697ea2
PC: _resume_from_idle+0xf1 CMD:
dbus-daemon --fork --config-file=/var/lib/nza/nmdtrace.conf --print-address
stack pointer for thread ffffff03e30f5520: ffffff00180f0c40
[ ffffff00180f0c40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea699080 ffffff03e2052020 ffffff03e060fc80 1 59 ffffff03e366a1ca
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/rpcbind
stack pointer for thread ffffff03ea699080: ffffff00184bfbb0
[ ffffff00184bfbb0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30e8e80 ffffff03e2059c80 ffffff03e0619770 1 59 ffffff03e2f590a2
PC: _resume_from_idle+0xf1 CMD: /usr/lib/rmvolmgr -s
stack pointer for thread ffffff03e30e8e80: ffffff001814cc40
[ ffffff001814cc40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37eb880 ffffff03e2055e28 ffffff03e2b85970 1 59 ffffff03e360813a
PC: _resume_from_idle+0xf1 CMD: /usr/lib/nfs/statd
stack pointer for thread ffffff03e37eb880: ffffff0017358c40
[ ffffff0017358c40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff0018230c60 fffffffffbc2dbf0 0 0 60 ffffff03e36643ca
PC: _resume_from_idle+0xf1 THREAD: svc_thread_creator()
stack pointer for thread ffffff0018230c60: ffffff0018230b70
[ ffffff0018230b70 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
svc_thread_creator+0xbf()
thread_start+8()
ffffff03e30f9c00 ffffff03ea8f7028 ffffff03e06156d0 4 60 ffffff03e3664422
PC: _resume_from_idle+0xf1 CMD: /usr/lib/nfs/nfs4cbd
stack pointer for thread ffffff03e30f9c00: ffffff0019d34b30
[ ffffff0019d34b30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig+0x14d()
svc_wait+0xb9()
nfssys+0x792()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37f9560 ffffff03ea8f7028 ffffff03e065e740 4 60 ffffff03e29f0a9a
PC: _resume_from_idle+0xf1 CMD: /usr/lib/nfs/nfs4cbd
stack pointer for thread ffffff03e37f9560: ffffff00184adc40
[ ffffff00184adc40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff0018236c60 fffffffffbc2dbf0 0 0 99 ffffff03ea6dea48
PC: _resume_from_idle+0xf1 TASKQ: dtrace_taskq
stack pointer for thread ffffff0018236c60: ffffff0018236ae0
[ ffffff0018236ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff03e3628a80 ffffff03e20551c0 ffffff03e2b8b950 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/nfs/nfsmapid
stack pointer for thread ffffff03e3628a80: ffffff00182f35c0
[ ffffff00182f35c0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0xe45()
0xfffffffffb8001d6()
ffffff03e30f58a0 ffffff03e20551c0 ffffff03e2b81e60 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/nfs/nfsmapid
stack pointer for thread ffffff03e30f58a0: ffffff0019d0a330
[ ffffff0019d0a330 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
nanosleep+0x1bc()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3635560 ffffff03e20551c0 ffffff03e2b87320 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/nfs/nfsmapid
stack pointer for thread ffffff03e3635560: ffffff0019d22d30
[ ffffff0019d22d30 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32f5760 ffffff03e20551c0 ffffff03e0613790 1 59 ffffff03e32f593e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/nfs/nfsmapid
stack pointer for thread ffffff03e32f5760: ffffff0019cf2df0
[ ffffff0019cf2df0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
pause+0x48()
_sys_sysenter_post_swapgs+0x149()
ffffff001823cc60 fffffffffbc2dbf0 0 0 60 ffffff03e083b4d2
PC: _resume_from_idle+0xf1 THREAD: svc_thread_creator()
stack pointer for thread ffffff001823cc60: ffffff001823cb70
[ ffffff001823cb70 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
svc_thread_creator+0xbf()
thread_start+8()
ffffff03ea58e200 ffffff03e205ce20 ffffff03e06600f0 4 60 ffffff03e083b52a
PC: _resume_from_idle+0xf1 CMD: /usr/lib/nfs/lockd
stack pointer for thread ffffff03ea58e200: ffffff0019da6b30
[ ffffff0019da6b30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig+0x14d()
svc_wait+0xb9()
nfssys+0x792()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30ff200 ffffff03e205ce20 ffffff03e2b97910 4 60 ffffff03e2f59a02
PC: _resume_from_idle+0xf1 CMD: /usr/lib/nfs/lockd
stack pointer for thread ffffff03e30ff200: ffffff0018464c40
[ ffffff0018464c40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff03e363a3c0 ffffff03ea8fc700 ffffff03e0617610 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/inet/inetd start
stack pointer for thread ffffff03e363a3c0: ffffff00184f4d30
[ ffffff00184f4d30 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea590c60 ffffff03ea8fc700 ffffff03e06107a0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/inet/inetd start
stack pointer for thread ffffff03ea590c60: ffffff00182b5d10
[ ffffff00182b5d10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
_sys_sysenter_post_swapgs+0x149()
ffffff0018242c60 fffffffffbc2dbf0 0 0 60 ffffff03e3454448
PC: _resume_from_idle+0xf1 THREAD: evch_delivery_thr()
stack pointer for thread ffffff0018242c60: ffffff0018242ae0
[ ffffff0018242ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
evch_delivery_hold+0x85()
evch_delivery_thr+0x23e()
thread_start+8()
ffffff03e37eb180 ffffff03ea8fc700 ffffff03e06687c0 1 59 ffffff03e3731b3a
PC: _resume_from_idle+0xf1 CMD: /usr/lib/inet/inetd start
stack pointer for thread ffffff03e37eb180: ffffff0019d16c40
[ ffffff0019d16c40 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
poll_common+0x462()
pollsys+0xe4()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3100e60 ffffff03ea8f31d0 ffffff03e2b8ae30 1 60 fffffffffbd1da50
PC: _resume_from_idle+0xf1 CMD: /usr/lib/utmpd
stack pointer for thread ffffff03e3100e60: ffffff00184701a0
[ ffffff00184701a0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
spa_open_common+0x69()
spa_open+0x23()
dsl_dir_open_spa+0x95()
dsl_dataset_hold+0x41()
dsl_prop_get+0x4c()
zfs_is_wormed+0x73()
zfs_create+0x69e()
fop_create+0xfc()
vn_createat+0x5e1()
vn_openat+0x1fb()
copen+0x435()
open32+0x2f()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30f1540 ffffff03ea8f2568 ffffff03e2b83da0 1 59 ffffff03e30f171e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/saf/ttymon
stack pointer for thread ffffff03e30f1540: ffffff001832fd60
[ ffffff001832fd60 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
pause+0x48()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e37eae00 ffffff03ea8f4aa0 ffffff03e0618c50 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/saf/sac -t 300
stack pointer for thread ffffff03e37eae00: ffffff001844c100
[ ffffff001844c100 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x320()
savecontext32+0x8c()
sendsig32+0x48b()
psig+0x3e2()
post_syscall+0x49e()
syscall_exit+0x59()
0xfffffffffb800ed1()
ffffff03e37efc80 ffffff03ea8f5708 ffffff03e2b913a0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/autofs/automountd
stack pointer for thread ffffff03e37efc80: ffffff0018216d30
[ ffffff0018216d30 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
_sys_sysenter_post_swapgs+0x149()
ffffff03e36351e0 ffffff03ea8f6370 ffffff03e0611bc0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/autofs/automountd
stack pointer for thread ffffff03e36351e0: ffffff0019c052f0
[ ffffff0019c052f0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
lwp_park+0x242()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32ee580 ffffff03ea8f6370 ffffff03eb1c5860 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/autofs/automountd
stack pointer for thread ffffff03e32ee580: ffffff00172aa0f0
[ ffffff00172aa0f0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x324()
copyout_nowatch+0x48()
door_stack_copyout+0x9c()
door_server_dispatch+0x2e3()
door_return+0x287()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3300c20 ffffff03ea8f6370 ffffff03e061db80 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/autofs/automountd
stack pointer for thread ffffff03e3300c20: ffffff0019c0b0f0
[ ffffff0019c0b0f0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x324()
copyout_nowatch+0x48()
door_stack_copyout+0x9c()
door_server_dispatch+0x2e3()
door_return+0x287()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff00187fec60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 THREAD: unmount_zone_tree()
stack pointer for thread ffffff00187fec60: ffffff00187fe7e0
[ ffffff00187fe7e0 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_upcall+0x243()
door_ki_upcall_limited+0x69()
stubs_common_code+0x51()
auto_calldaemon+0x2f2()
auto_null_request+0x2a()
unmount_tree+0x79()
unmount_zone_tree+0x1e()
thread_start+8()
ffffff00187d3c60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 THREAD: unmount_zone_tree()
stack pointer for thread ffffff00187d3c60: ffffff00187d37e0
[ ffffff00187d37e0 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_upcall+0x243()
door_ki_upcall_limited+0x69()
stubs_common_code+0x51()
auto_calldaemon+0x2f2()
auto_null_request+0x2a()
unmount_tree+0x79()
unmount_zone_tree+0x1e()
thread_start+8()
ffffff0017ef1c60 fffffffffbc2dbf0 0 0 60 ffffff03ea8f6dd8
PC: _resume_from_idle+0xf1 THREAD: unmount_zone_tree()
stack pointer for thread ffffff0017ef1c60: ffffff0017ef1770
[ ffffff0017ef1770 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
cv_wait_sig_swap_core+0x2aa()
door_get_server+0xae()
door_upcall+0x155()
door_ki_upcall_limited+0x69()
stubs_common_code+0x51()
auto_calldaemon+0x2f2()
auto_null_request+0x2a()
unmount_tree+0x79()
unmount_zone_tree+0x1e()
thread_start+8()
ffffff0017f2ec60 fffffffffbc2dbf0 0 0 60 ffffff03ea8f6dd8
PC: _resume_from_idle+0xf1 THREAD: unmount_zone_tree()
stack pointer for thread ffffff0017f2ec60: ffffff0017f2e770
[ ffffff0017f2e770 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
cv_wait_sig_swap_core+0x2aa()
door_get_server+0xae()
door_upcall+0x155()
door_ki_upcall_limited+0x69()
stubs_common_code+0x51()
auto_calldaemon+0x2f2()
auto_null_request+0x2a()
unmount_tree+0x79()
unmount_zone_tree+0x1e()
thread_start+8()
ffffff0018615c60 fffffffffbc2dbf0 0 0 60 ffffff03ea8f6dd8
PC: _resume_from_idle+0xf1 THREAD: unmount_zone_tree()
stack pointer for thread ffffff0018615c60: ffffff0018615770
[ ffffff0018615770 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
cv_wait_sig_swap_core+0x2aa()
door_get_server+0xae()
door_upcall+0x155()
door_ki_upcall_limited+0x69()
stubs_common_code+0x51()
auto_calldaemon+0x2f2()
auto_null_request+0x2a()
unmount_tree+0x79()
unmount_zone_tree+0x1e()
thread_start+8()
ffffff0018248c60 fffffffffbc2dbf0 0 0 60 fffffffffbcd97c8
PC: _resume_from_idle+0xf1 THREAD: auto_do_unmount()
stack pointer for thread ffffff0018248c60: ffffff0018248ab0
[ ffffff0018248ab0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_hires+0xe0()
cv_timedwait+0x5a()
zone_status_timedwait+0x53()
auto_do_unmount+0xba()
thread_start+8()
ffffff03e30f0e40 ffffff03ea8f6370 ffffff03e0664720 1 59 ffffff03e30f101e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/autofs/automountd
stack pointer for thread ffffff03e30f0e40: ffffff0019bffdf0
[ ffffff0019bffdf0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
pause+0x48()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3629880 ffffff03ea8f5708 ffffff03e060b650 1 59 ffffff03ea8f57c8
PC: _resume_from_idle+0xf1 CMD: /usr/lib/autofs/automountd
stack pointer for thread ffffff03e3629880: ffffff0018222c70
[ ffffff0018222c70 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
waitid+0x2a0()
waitsys32+0x30()
_sys_sysenter_post_swapgs+0x149()
ffffff001879dc60 fffffffffbc2dbf0 0 0 60 ffffff03eb9c5c80
PC: _resume_from_idle+0xf1 TASKQ: ah_taskq
stack pointer for thread ffffff001879dc60: ffffff001879dae0
[ ffffff001879dae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187dac60 fffffffffbc2dbf0 0 0 60 ffffff03eb9c5a50
PC: _resume_from_idle+0xf1 TASKQ: esp_taskq
stack pointer for thread ffffff00187dac60: ffffff00187daae0
[ ffffff00187daae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017d71c60 fffffffffbc2dbf0 0 0 60 ffffffffc017d838
PC: _resume_from_idle+0xf1 THREAD: smb_thread_entry_point()
stack pointer for thread ffffff0017d71c60: ffffff0017d71ae0
[ ffffff0017d71ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
smb_thread_continue_timedwait_locked+0x60()
smb_thread_continue+0x2a()
smb_notify_change_daemon+0x3f()
smb_thread_entry_point+0x69()
thread_start+8()
ffffff0017ec1c60 fffffffffbc2dbf0 0 0 60 ffffff03eb9c5938
PC: _resume_from_idle+0xf1 TASKQ: simnet_simnet
stack pointer for thread ffffff0017ec1c60: ffffff0017ec1ae0
[ ffffff0017ec1ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017efec60 fffffffffbc2dbf0 0 0 60 ffffff03eb9c5820
PC: _resume_from_idle+0xf1 TASKQ: bridge_bridge
stack pointer for thread ffffff0017efec60: ffffff0017efeae0
[ ffffff0017efeae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017f3bc60 fffffffffbc2dbf0 0 0 60 ffffff03eb9c5d98
PC: _resume_from_idle+0xf1 TASKQ: iptun_taskq
stack pointer for thread ffffff0017f3bc60: ffffff0017f3bae0
[ ffffff0017f3bae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff03e3622aa0 ffffff03ea8fae30 ffffff03e3a4ee70 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/bin/perl /lib/svc/method/nmdtrace -d
stack pointer for thread ffffff03e3622aa0: ffffff001833b270
[ ffffff001833b270 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
poll_common+0x614()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea58f060 ffffff03ea8f3e38 ffffff03eb1cc360 1 59 ffffff03ea58f23e
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/syslogd
ffffff03ea5908e0 ffffff03ea8f3e38 ffffff03eb1cbdd0 1 59 ffffff03ea590abe
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/syslogd
ffffff03ea58d080 ffffff03ea8f3e38 ffffff03eb1cb840 1 59 ffffff03ea58d25e
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/syslogd
ffffff03ea576500 ffffff03ea8f3e38 ffffff03eb1cb2b0 1 59 ffffff03ea5766de
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/syslogd
ffffff03ea576180 ffffff03ea8f3e38 ffffff03eb1ca9b0 1 59 ffffff03ea57635e
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/syslogd
ffffff03ea6b6c20 ffffff03ea8f3e38 ffffff03eb1ca420 1 59 ffffff03ea6b6dfe
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/syslogd
ffffff03ea6b68a0 ffffff03ea8f3e38 ffffff03eb1c9e90 1 59 ffffff03e3608b3a
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/syslogd
ffffff03ea575000 ffffff03ea8f3e38 ffffff03e3a4ddc0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/syslogd
stack pointer for thread ffffff03ea575000: ffffff0019cd8c80
[ ffffff0019cd8c80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea575380 ffffff03ea8f3e38 ffffff03eb1c9900 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/syslogd
stack pointer for thread ffffff03ea575380: ffffff0017956d10
[ ffffff0017956d10 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
_sys_sysenter_post_swapgs+0x149()
ffffff03e6435720 ffffff03ea8f3e38 ffffff03eb1cce80 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/sbin/syslogd
ffffff03ea699400 ffffff03eee6dca8 ffffff03f1fd7bc0 1 60 0
PC: _resume_from_idle+0xf1 CMD: prstat
stack pointer for thread ffffff03ea699400: ffffff001796e2b0
[ ffffff001796e2b0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
poll_common+0x614()
pollsys+0xe4()
dtrace_systrace_syscall+0x11a()
sys_syscall+0x17a()
ffffff03e32f9ac0 ffffff03ea8f0c98 ffffff03eeef5ef0 1 59 ffffff03ea8f0d58
PC: _resume_from_idle+0xf1 CMD: bash
ffffff03e32f53e0 ffffff03e205c1b8 ffffff03e061d060 1 59 ffffff03e205c278
PC: _resume_from_idle+0xf1 CMD: -bash
ffffff03e362d580 ffffff03ea8e9038 ffffff03eeef6a10 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/ssh/sshd
stack pointer for thread ffffff03e362d580: ffffff0017a21270
[ ffffff0017a21270 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x19c()
poll_common+0x614()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e64353a0 ffffff03ea8e9ca0 ffffff03e0611630 1 59 ffffff03ed72404a
PC: _resume_from_idle+0xf1 CMD: /usr/lib/ssh/sshd
ffffff03e32e68a0 ffffff03ea8f7c90 ffffff03eb1c82c0 1 59 ffffff03e3618f3a
PC: _resume_from_idle+0xf1 CMD: /usr/lib/ssh/sshd
ffffff001824ec60 fffffffffbc2dbf0 0 0 60 ffffff03ea6de930
PC: _resume_from_idle+0xf1 TASKQ: fcoe_nexus_enum_tq
stack pointer for thread ffffff001824ec60: ffffff001824eae0
[ ffffff001824eae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001853ac60 fffffffffbc2dbf0 0 0 60 ffffff03ea6de818
PC: _resume_from_idle+0xf1 TASKQ: fcoe_mac
stack pointer for thread ffffff001853ac60: ffffff001853aae0
[ ffffff001853aae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001834bc60 fffffffffbc2dbf0 0 0 60 ffffff03eb264f48
PC: _resume_from_idle+0xf1 TASKQ: fcoe_mac
stack pointer for thread ffffff001834bc60: ffffff001834bb00
[ ffffff001834bb00 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
fcoe_watchdog+0xc8()
taskq_thread+0x248()
thread_start+8()
ffffff00181fac60 fffffffffbc2dbf0 0 0 60 ffffff03eb2e5570
PC: _resume_from_idle+0xf1 TASKQ: FCOE_WORKER_TASKQ
stack pointer for thread ffffff00181fac60: ffffff00181fab00
[ ffffff00181fab00 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
fcoe_worker_frame+0xd0()
taskq_thread+0x248()
thread_start+8()
ffffff0018296c60 fffffffffbc2dbf0 0 0 60 ffffff03eb2e5538
PC: _resume_from_idle+0xf1 TASKQ: FCOE_WORKER_TASKQ
stack pointer for thread ffffff0018296c60: ffffff0018296b00
[ ffffff0018296b00 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
fcoe_worker_frame+0xd0()
taskq_thread+0x248()
thread_start+8()
ffffff00182e2c60 fffffffffbc2dbf0 0 0 60 ffffff03eb2e5500
PC: _resume_from_idle+0xf1 TASKQ: FCOE_WORKER_TASKQ
stack pointer for thread ffffff00182e2c60: ffffff00182e2b00
[ ffffff00182e2b00 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
fcoe_worker_frame+0xd0()
taskq_thread+0x248()
thread_start+8()
ffffff00180cdc60 fffffffffbc2dbf0 0 0 60 ffffff03eb2e54c8
PC: _resume_from_idle+0xf1 TASKQ: FCOE_WORKER_TASKQ
stack pointer for thread ffffff00180cdc60: ffffff00180cdb00
[ ffffff00180cdb00 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
fcoe_worker_frame+0xd0()
taskq_thread+0x248()
thread_start+8()
ffffff001864fc60 fffffffffbc2dbf0 0 0 60 ffffff03ea6de3b8
PC: _resume_from_idle+0xf1 TASKQ: mpt_nexus_enum_tq
stack pointer for thread ffffff001864fc60: ffffff001864fae0
[ ffffff001864fae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018660c60 fffffffffbc2dbf0 0 0 60 ffffff03ea6de2a0
PC: _resume_from_idle+0xf1 TASKQ: mpt_mpt_event_taskq
stack pointer for thread ffffff0018660c60: ffffff0018660ae0
[ ffffff0018660ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018666c60 fffffffffbc2dbf0 0 0 60 ffffff03ea6de188
PC: _resume_from_idle+0xf1 TASKQ: mpt_mpt_dr_taskq
stack pointer for thread ffffff0018666c60: ffffff0018666ae0
[ ffffff0018666ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b24c60 fffffffffbc2dbf0 0 0 60 ffffff03eb2a3618
PC: _resume_from_idle+0xf1 THREAD: mpt_doneq_thread()
stack pointer for thread ffffff0017b24c60: ffffff0017b24b90
[ ffffff0017b24b90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mpt_doneq_thread+0x62()
thread_start+8()
ffffff0017b2ac60 fffffffffbc2dbf0 0 0 60 ffffff03eb2a3658
PC: _resume_from_idle+0xf1 THREAD: mpt_doneq_thread()
stack pointer for thread ffffff0017b2ac60: ffffff0017b2ab90
[ ffffff0017b2ab90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mpt_doneq_thread+0x62()
thread_start+8()
ffffff0017b30c60 fffffffffbc2dbf0 0 0 60 ffffff03eb2a3698
PC: _resume_from_idle+0xf1 THREAD: mpt_doneq_thread()
stack pointer for thread ffffff0017b30c60: ffffff0017b30b90
[ ffffff0017b30b90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mpt_doneq_thread+0x62()
thread_start+8()
ffffff0017b36c60 fffffffffbc2dbf0 0 0 60 ffffff03eb2a36d8
PC: _resume_from_idle+0xf1 THREAD: mpt_doneq_thread()
stack pointer for thread ffffff0017b36c60: ffffff0017b36b90
[ ffffff0017b36b90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mpt_doneq_thread+0x62()
thread_start+8()
ffffff0017b3cc60 fffffffffbc2dbf0 0 0 60 ffffff03eb2a3718
PC: _resume_from_idle+0xf1 THREAD: mpt_doneq_thread()
stack pointer for thread ffffff0017b3cc60: ffffff0017b3cb90
[ ffffff0017b3cb90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mpt_doneq_thread+0x62()
thread_start+8()
ffffff0017b42c60 fffffffffbc2dbf0 0 0 60 ffffff03eb2a3758
PC: _resume_from_idle+0xf1 THREAD: mpt_doneq_thread()
stack pointer for thread ffffff0017b42c60: ffffff0017b42b90
[ ffffff0017b42b90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mpt_doneq_thread+0x62()
thread_start+8()
ffffff0017b48c60 fffffffffbc2dbf0 0 0 60 ffffff03eb2a3798
PC: _resume_from_idle+0xf1 THREAD: mpt_doneq_thread()
stack pointer for thread ffffff0017b48c60: ffffff0017b48b90
[ ffffff0017b48b90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mpt_doneq_thread+0x62()
thread_start+8()
ffffff0017b4ec60 fffffffffbc2dbf0 0 0 60 ffffff03eb2a37d8
PC: _resume_from_idle+0xf1 THREAD: mpt_doneq_thread()
stack pointer for thread ffffff0017b4ec60: ffffff0017b4eb90
[ ffffff0017b4eb90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mpt_doneq_thread+0x62()
thread_start+8()
ffffff001797fc60 fffffffffbc2dbf0 0 0 60 ffffff03ea6de5e8
PC: _resume_from_idle+0xf1 TASKQ: acpinex_nexus_enum_tq
stack pointer for thread ffffff001797fc60: ffffff001797fae0
[ ffffff001797fae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017985c60 fffffffffbc2dbf0 0 0 60 ffffff03ea6de4d0
PC: _resume_from_idle+0xf1 TASKQ: pseudo_nexus_enum_tq
stack pointer for thread ffffff0017985c60: ffffff0017985ae0
[ ffffff0017985ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017afdc60 fffffffffbc2dbf0 0 0 60 ffffff03ea6de070
PC: _resume_from_idle+0xf1 TASKQ: ipnet
stack pointer for thread ffffff0017afdc60: ffffff0017afdae0
[ ffffff0017afdae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b0fc60 fffffffffbc2dbf0 0 0 60 ffffff03eb9c5eb0
PC: _resume_from_idle+0xf1 TASKQ: ipnet_nic_event_queue
stack pointer for thread ffffff0017b0fc60: ffffff0017b0fae0
[ ffffff0017b0fae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b09c60 fffffffffbc2dbf0 0 0 60 ffffffffc01988a6
PC: _resume_from_idle+0xf1 THREAD: ufs_thread_idle()
stack pointer for thread ffffff0017b09c60: ffffff0017b09b90
[ ffffff0017b09b90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
ufs_thread_idle+0x152()
thread_start+8()
ffffff0018596c60 fffffffffbc2dbf0 0 0 60 ffffffffc0198e56
PC: _resume_from_idle+0xf1 THREAD: ufs_thread_hlock()
stack pointer for thread ffffff0018596c60: ffffff0018596b20
[ ffffff0018596b20 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
ufs_thread_run+0x12d()
ufs_thread_hlock+0x6f()
thread_start+8()
ffffff03e363bc40 ffffff03ea8ee710 ffffff03eb1c9370 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
stack pointer for thread ffffff03e363bc40: ffffff001839f5c0
[ ffffff001839f5c0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0xe45()
0xfffffffffb8001d6()
ffffff03e37ef580 ffffff03ea8ee710 ffffff03e3a4d2a0 1 59 ffffff03e37ef75e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03e3634060 ffffff03ea8ee710 ffffff03e2b897f0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
stack pointer for thread ffffff03e3634060: ffffff0019cf8ca0
[ ffffff0019cf8ca0 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e363b8c0 ffffff03ea8ee710 ffffff03e2b902f0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
stack pointer for thread ffffff03e363b8c0: ffffff001828ac80
[ ffffff001828ac80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30f9500 ffffff03ea8ee710 ffffff03e061f1c0 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
stack pointer for thread ffffff03e30f9500: ffffff0019cccc80
[ ffffff0019cccc80 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_return+0x21a()
doorfs32+0x134()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff0017ba3c60 fffffffffbc2dbf0 0 0 60 ffffff03edfca7e0
PC: _resume_from_idle+0xf1 THREAD: evch_delivery_thr()
stack pointer for thread ffffff0017ba3c60: ffffff0017ba3ae0
[ ffffff0017ba3ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
evch_delivery_hold+0x85()
evch_delivery_thr+0x23e()
thread_start+8()
ffffff03e30f8700 ffffff03ea8ee710 ffffff03e2b8e940 1 59 ffffff03e30f88de
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03ea699780 ffffff03ea8ee710 ffffff03e2b96860 1 59 ffffff03ea69995e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03ea49c540 ffffff03ea8ee710 ffffff03eb1c79c0 1 59 ffffff03ee67fa52
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03e3635c60 ffffff03ea8ee710 ffffff03eb1cd410 1 59 ffffff03e3635e3e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03ea699b00 ffffff03ea8ee710 ffffff03e0612c70 1 59 ffffff03ea699cde
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff001868ec60 fffffffffbc2dbf0 0 0 60 ffffff03ee258d28
PC: _resume_from_idle+0xf1 THREAD: evch_delivery_thr()
stack pointer for thread ffffff001868ec60: ffffff001868eae0
[ ffffff001868eae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
evch_delivery_hold+0x85()
evch_delivery_thr+0x23e()
thread_start+8()
ffffff03e30e8b00 ffffff03ea8ee710 ffffff03e3a4e8e0 1 59 ffffff03e30e8cde
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03e30f8a80 ffffff03ea8ee710 ffffff03e2b8f7d0 1 59 ffffff03e30f8c5e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03e30f8e00 ffffff03ea8ee710 ffffff03e2b97380 1 59 ffffff03e30f8fde
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
stack pointer for thread ffffff03e30f8e00: ffffff0018433c30
[ ffffff0018433c30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e27a01a0 ffffff03ea8ee710 ffffff03e2b8fd60 1 59 ffffff03e27a037e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03e6436520 ffffff03ea8ee710 ffffff03e060e0b0 1 59 ffffff03e64366fe
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03e6436c20 ffffff03ea8ee710 ffffff03e0663c00 1 59 ffffff03e6436dfe
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03e30e6a80 ffffff03ea8ee710 ffffff03e2b8cd70 1 59 ffffff03e30e6c5e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03ea69c1e0 ffffff03ea8ee710 ffffff03f1fd86e0 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
stack pointer for thread ffffff03ea69c1e0: ffffff00185a92f0
[ ffffff00185a92f0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
lwp_park+0x242()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea590560 ffffff03ea8ee710 ffffff03e0621100 1 59 fffffffffbd1da50
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
stack pointer for thread ffffff03ea590560: ffffff00183e8930
[ ffffff00183e8930 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
spa_open_common+0x69()
spa_get_stats+0x50()
zfs_ioc_pool_stats+0x32()
zfsdev_ioctl+0x177()
cdev_ioctl+0x45()
spec_ioctl+0x5a()
fop_ioctl+0x7b()
ioctl+0x18e()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e32f61e0 ffffff03ea8ee710 ffffff03eb1c7430 1 59 ffffff03e32f63be
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
stack pointer for thread ffffff03e32f61e0: ffffff0019d2ec30
[ ffffff0019d2ec30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea6b61a0 ffffff03ea8ee710 ffffff03eb1c8de0 1 59 ffffff03ea6b637e
PC: _resume_from_idle+0xf1 CMD: /usr/lib/fm/fmd/fmd
ffffff03ea49b040 ffffff03ea8ec1d8 ffffff03e2b95220 1 59 0
PC: _resume_from_idle+0xf1 CMD: /usr/perl5/bin/perl /usr/lib/intrd
stack pointer for thread ffffff03ea49b040: ffffff001736ad60
[ ffffff001736ad60 _resume_from_idle+0xf1() ]
swtch+0x145()
swapout_lwp+0xa3()
ts_trapret+0x21b()
post_syscall+0x622()
syscall_exit+0x59()
0xfffffffffb800ed1()
ffffff00180fcc60 fffffffffbc2dbf0 0 0 60 ffffffffc0144d40
PC: _resume_from_idle+0xf1 THREAD: kmdbmod`kctl_wr_thread()
stack pointer for thread ffffff00180fcc60: ffffff00180fcb70
[ ffffff00180fcb70 _resume_from_idle+0xf1() ]
swtch+0x145()
sema_p+0x1d9()
kmdbmod`kctl_wr_thread+0x9b()
thread_start+8()
ffffff0017da1c60 ffffff03ea8fa1c8 ffffff03e06205e0 2 0 ffffff03e246b808
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017da1c60: ffffff0017da19f0
[ ffffff0017da19f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017dddc60 ffffff03ea8fa1c8 ffffff03eef05440 2 99 ffffff03e246b920
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017dddc60: ffffff0017ddd9f0
[ ffffff0017ddd9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00172b0c60 ffffff03ea8fa1c8 ffffff03ef8553e0 2 99 ffffff03f1451a58
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00172b0c60: ffffff00172b09f0
[ ffffff00172b09f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c11c60 ffffff03ea8fa1c8 ffffff03ef854e50 2 99 ffffff03f1451a58
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c11c60: ffffff0019c119f0
[ ffffff0019c119f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019cb0c60 ffffff03ea8fa1c8 ffffff03e06265c0 2 99 ffffff03f1451a58
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019cb0c60: ffffff0019cb09f0
[ ffffff0019cb09f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019cd2c60 ffffff03ea8fa1c8 ffffff03eeef48b0 2 99 ffffff03f1451a58
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019cd2c60: ffffff0019cd29f0
[ ffffff0019cd29f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017d83c60 ffffff03ea8fa1c8 ffffff03ef855970 2 99 ffffff03f1451a58
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017d83c60: ffffff0017d839f0
[ ffffff0017d839f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017db9c60 ffffff03ea8fa1c8 ffffff03e2b90e10 2 99 ffffff03f1451a58
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017db9c60: ffffff0017db99f0
[ ffffff0017db99f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017d95c60 ffffff03ea8fa1c8 ffffff03e0614090 2 99 ffffff03f1451a58
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017d95c60: ffffff0017d959f0
[ ffffff0017d959f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017e07c60 ffffff03ea8fa1c8 ffffff03eeef4e40 2 99 ffffff03f1451a58
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017e07c60: ffffff0017e079f0
[ ffffff0017e079f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018688c60 ffffff03ea8fa1c8 ffffff03ef8548c0 2 0 ffffff03eb9c5190
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018688c60: ffffff00186889f0
[ ffffff00186889f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00171f1c60 ffffff03ea8fa1c8 ffffff03eb1cc8f0 2 0 ffffff03eb9c5190
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00171f1c60: ffffff00171f19f0
[ ffffff00171f19f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017a09c60 ffffff03ea8fa1c8 ffffff03e061ec30 2 0 ffffff03eb9c5190
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017a09c60: ffffff0017a099f0
[ ffffff0017a099f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017dfbc60 ffffff03ea8fa1c8 ffffff03e061d5f0 2 99 ffffff03eb9c5190
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017dfbc60: ffffff0017dfb9f0
[ ffffff0017dfb9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001857cc60 ffffff03ea8fa1c8 ffffff03eeef4320 2 99 fffffffffbc2cf02
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001857cc60: ffffff001857c1b0
[ ffffff001857c1b0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
zio_create+0x53()
zio_vdev_child_io+0xba()
vdev_raidz_io_start+0x1d4()
zio_vdev_io_start+0x17d()
zio_execute+0x8d()
zio_nowait+0x42()
vdev_mirror_io_start+0x148()
zio_vdev_io_start+0x1ba()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff0018570c60 ffffff03ea8fa1c8 ffffff03ef854330 2 99 fffffffffbc2cf02
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018570c60: ffffff00185701b0
[ ffffff00185701b0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
zio_create+0x53()
zio_vdev_child_io+0xba()
vdev_raidz_io_start+0x1d4()
zio_vdev_io_start+0x17d()
zio_execute+0x8d()
zio_nowait+0x42()
vdev_mirror_io_start+0x148()
zio_vdev_io_start+0x1ba()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff00179ebc60 ffffff03ea8fa1c8 ffffff03eef059d0 2 99 fffffffffbc2cf02
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00179ebc60: ffffff00179eb1b0
[ ffffff00179eb1b0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
zio_create+0x53()
zio_vdev_child_io+0xba()
vdev_raidz_io_start+0x1d4()
zio_vdev_io_start+0x17d()
zio_execute+0x8d()
zio_nowait+0x42()
vdev_mirror_io_start+0x148()
zio_vdev_io_start+0x1ba()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff00171fdc60 ffffff03ea8fa1c8 ffffff03e065d690 2 99 fffffffffbc2cf02
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00171fdc60: ffffff00171fd1b0
[ ffffff00171fd1b0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
zio_create+0x53()
zio_vdev_child_io+0xba()
vdev_raidz_io_start+0x1d4()
zio_vdev_io_start+0x17d()
zio_execute+0x8d()
zio_nowait+0x42()
vdev_mirror_io_start+0x148()
zio_vdev_io_start+0x1ba()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff0018609c60 ffffff03ea8fa1c8 ffffff03efcd98d0 2 99 ffffff03f1451710
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018609c60: ffffff00186099f0
[ ffffff00186099f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018102c60 ffffff03ea8fa1c8 ffffff03efcda3f0 2 99 ffffff03f1451710
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018102c60: ffffff00181029f0
[ ffffff00181029f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017dadc60 ffffff03ea8fa1c8 ffffff03efcdaf10 2 99 ffffff03f1451710
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017dadc60: ffffff0017dad9f0
[ ffffff0017dad9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001862dc60 ffffff03ea8fa1c8 ffffff03efcdb4a0 2 99 ffffff03f1451710
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001862dc60: ffffff001862d9f0
[ ffffff001862d9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018540c60 ffffff03ea8fa1c8 ffffff03efcdba30 2 99 ffffff03f1451710
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018540c60: ffffff00185409f0
[ ffffff00185409f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017119c60 ffffff03ea8fa1c8 ffffff03f1fc4990 2 99 0
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017119c60: ffffff0017118f10
[ ffffff0017118f10 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
kmem_alloc+0xdd()
bplist_append+0x2e()
zio_free+0x33()
dsl_free+0x20()
dsl_dataset_block_kill+0x275()
dbuf_write_done+0x5a()
arc_write_done+0x1d0()
zio_done+0x383()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff0018645c60 ffffff03ea8fa1c8 ffffff03f1fc0360 2 99 fffffffffbc2cf02
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018645c60: ffffff0018644f10
[ ffffff0018644f10 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
kmem_alloc+0xdd()
bplist_append+0x2e()
zio_free+0x33()
dsl_free+0x20()
dsl_dataset_block_kill+0x275()
dbuf_write_done+0x5a()
arc_write_done+0x1d0()
zio_done+0x383()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff001858ec60 ffffff03ea8fa1c8 ffffff03f1fc1410 2 99 fffffffffbc2cf02
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001858ec60: ffffff001858df10
[ ffffff001858df10 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
kmem_alloc+0xdd()
bplist_append+0x2e()
zio_free+0x33()
dsl_free+0x20()
dsl_dataset_block_kill+0x275()
dbuf_write_done+0x5a()
arc_write_done+0x1d0()
zio_done+0x383()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff00185fdc60 ffffff03ea8fa1c8 ffffff03f1fc1f30 2 99 0
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00185fdc60: ffffff00185fcf10
[ ffffff00185fcf10 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
kmem_alloc+0xdd()
bplist_append+0x2e()
zio_free+0x33()
dsl_free+0x20()
dsl_dataset_block_kill+0x275()
dbuf_write_done+0x5a()
arc_write_done+0x1d0()
zio_done+0x383()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff0018621c60 ffffff03ea8fa1c8 ffffff03f1fc2a50 2 99 fffffffffbc2cf02
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018621c60: ffffff0018620f10
[ ffffff0018620f10 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
kmem_alloc+0xdd()
bplist_append+0x2e()
zio_free+0x33()
dsl_free+0x20()
dsl_dataset_block_kill+0x275()
dbuf_write_done+0x5a()
arc_write_done+0x1d0()
zio_done+0x383()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff001860fc60 ffffff03ea8fa1c8 ffffff03efcd9340 2 99 fffffffffbc2cf02
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001860fc60: ffffff001860ef10
[ ffffff001860ef10 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
kmem_alloc+0xdd()
bplist_append+0x2e()
zio_free+0x33()
dsl_free+0x20()
dsl_dataset_block_kill+0x275()
dbuf_write_done+0x5a()
arc_write_done+0x1d0()
zio_done+0x383()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff00183fac60 ffffff03ea8fa1c8 ffffff03efcd9e60 2 99 0
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00183fac60: ffffff00183f9f10
[ ffffff00183f9f10 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
kmem_alloc+0xdd()
bplist_append+0x2e()
zio_free+0x33()
dsl_free+0x20()
dsl_dataset_block_kill+0x275()
dbuf_write_done+0x5a()
arc_write_done+0x1d0()
zio_done+0x383()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff0017a45c60 ffffff03ea8fa1c8 ffffff03efcda980 2 99 0
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017a45c60: ffffff0017a44f10
[ ffffff0017a44f10 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
page_create_throttle+0x154()
page_create_va+0xd7()
segkmem_page_create+0x8d()
segkmem_xalloc+0xc0()
segkmem_alloc_vn+0xcd()
segkmem_alloc+0x24()
vmem_xalloc+0x546()
vmem_alloc+0x161()
kmem_slab_create+0x81()
kmem_slab_alloc+0x5b()
kmem_cache_alloc+0x1fa()
kmem_alloc+0xdd()
bplist_append+0x2e()
zio_free+0x33()
dsl_free+0x20()
dsl_dataset_block_kill+0x275()
dbuf_write_done+0x5a()
arc_write_done+0x1d0()
zio_done+0x383()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
zio_notify_parent+0xa6()
zio_done+0x3e2()
zio_execute+0x8d()
taskq_thread+0x248()
thread_start+8()
ffffff0017113c60 ffffff03ea8fa1c8 ffffff03f1fc4f20 2 99 ffffff03f1451b70
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017113c60: ffffff00171139f0
[ ffffff00171139f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001863fc60 ffffff03ea8fa1c8 ffffff03f1fc08f0 2 99 ffffff03f1451b70
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001863fc60: ffffff001863f9f0
[ ffffff001863f9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018588c60 ffffff03ea8fa1c8 ffffff03f1fc19a0 2 99 ffffff03f1451b70
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018588c60: ffffff00185889f0
[ ffffff00185889f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00185f7c60 ffffff03ea8fa1c8 ffffff03f1fc24c0 2 99 ffffff03f1451b70
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00185f7c60: ffffff00185f79f0
[ ffffff00185f79f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001861bc60 ffffff03ea8fa1c8 ffffff03f1fc5a40 2 99 ffffff03f1451b70
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001861bc60: ffffff001861b9f0
[ ffffff001861b9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187f2c60 ffffff03ea8fa1c8 ffffff03f1fd8c70 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187f2c60: ffffff00187f29f0
[ ffffff00187f29f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187ecc60 ffffff03ea8fa1c8 ffffff03f1fd9200 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187ecc60: ffffff00187ec9f0
[ ffffff00187ec9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187e6c60 ffffff03ea8fa1c8 ffffff03f1fd9790 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187e6c60: ffffff00187e69f0
[ ffffff00187e69f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187e0c60 ffffff03ea8fa1c8 ffffff03f1fda090 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187e0c60: ffffff00187e09f0
[ ffffff00187e09f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00180afc60 ffffff03ea8fa1c8 ffffff03f1fda620 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00180afc60: ffffff00180af9f0
[ ffffff00180af9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00180a9c60 ffffff03ea8fa1c8 ffffff03f1fdabb0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00180a9c60: ffffff00180a99f0
[ ffffff00180a99f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00180a3c60 ffffff03ea8fa1c8 ffffff03f1fdb140 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00180a3c60: ffffff00180a39f0
[ ffffff00180a39f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001809dc60 ffffff03ea8fa1c8 ffffff03f1fdb6d0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001809dc60: ffffff001809d9f0
[ ffffff001809d9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018097c60 ffffff03ea8fa1c8 ffffff03f1fdbc60 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018097c60: ffffff00180979f0
[ ffffff00180979f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018091c60 ffffff03ea8fa1c8 ffffff03f1fdc1f0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018091c60: ffffff00180919f0
[ ffffff00180919f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001808bc60 ffffff03ea8fa1c8 ffffff03f1fdc780 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001808bc60: ffffff001808b9f0
[ ffffff001808b9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018085c60 ffffff03ea8fa1c8 ffffff03f1fdd080 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018085c60: ffffff00180859f0
[ ffffff00180859f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001807fc60 ffffff03ea8fa1c8 ffffff03f1fdd610 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001807fc60: ffffff001807f9f0
[ ffffff001807f9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018079c60 ffffff03ea8fa1c8 ffffff03f1fddba0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018079c60: ffffff00180799f0
[ ffffff00180799f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018073c60 ffffff03ea8fa1c8 ffffff03f1fde130 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018073c60: ffffff00180739f0
[ ffffff00180739f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001806dc60 ffffff03ea8fa1c8 ffffff03f1fde6c0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001806dc60: ffffff001806d9f0
[ ffffff001806d9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018067c60 ffffff03ea8fa1c8 ffffff03f1fdec50 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018067c60: ffffff00180679f0
[ ffffff00180679f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018061c60 ffffff03ea8fa1c8 ffffff03f1fdf1e0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018061c60: ffffff00180619f0
[ ffffff00180619f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001805bc60 ffffff03ea8fa1c8 ffffff03f1fdf770 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001805bc60: ffffff001805b9f0
[ ffffff001805b9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018055c60 ffffff03ea8fa1c8 ffffff03f1fe0070 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018055c60: ffffff00180559f0
[ ffffff00180559f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001804fc60 ffffff03ea8fa1c8 ffffff03f1fe0600 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001804fc60: ffffff001804f9f0
[ ffffff001804f9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018049c60 ffffff03ea8fa1c8 ffffff03f1fe0b90 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018049c60: ffffff00180499f0
[ ffffff00180499f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018043c60 ffffff03ea8fa1c8 ffffff03f1fe1120 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018043c60: ffffff00180439f0
[ ffffff00180439f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001803dc60 ffffff03ea8fa1c8 ffffff03f1fe16b0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001803dc60: ffffff001803d9f0
[ ffffff001803d9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018037c60 ffffff03ea8fa1c8 ffffff03f1fe1c40 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018037c60: ffffff00180379f0
[ ffffff00180379f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018031c60 ffffff03ea8fa1c8 ffffff03f1fe21d0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018031c60: ffffff00180319f0
[ ffffff00180319f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c99c60 ffffff03ea8fa1c8 ffffff03f1fe2760 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c99c60: ffffff0019c999f0
[ ffffff0019c999f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c93c60 ffffff03ea8fa1c8 ffffff03f1fe3060 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c93c60: ffffff0019c939f0
[ ffffff0019c939f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c8dc60 ffffff03ea8fa1c8 ffffff03f1fe35f0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c8dc60: ffffff0019c8d9f0
[ ffffff0019c8d9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c87c60 ffffff03ea8fa1c8 ffffff03f1fe3b80 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c87c60: ffffff0019c879f0
[ ffffff0019c879f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c81c60 ffffff03ea8fa1c8 ffffff03f1fe4110 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c81c60: ffffff0019c819f0
[ ffffff0019c819f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c7bc60 ffffff03ea8fa1c8 ffffff03f1fe46a0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c7bc60: ffffff0019c7b9f0
[ ffffff0019c7b9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c75c60 ffffff03ea8fa1c8 ffffff03f1fe4c30 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c75c60: ffffff0019c759f0
[ ffffff0019c759f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c6fc60 ffffff03ea8fa1c8 ffffff03f1fe51c0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c6fc60: ffffff0019c6f9f0
[ ffffff0019c6f9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c69c60 ffffff03ea8fa1c8 ffffff03f1fe5750 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c69c60: ffffff0019c699f0
[ ffffff0019c699f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c63c60 ffffff03ea8fa1c8 ffffff03f1fa8050 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c63c60: ffffff0019c639f0
[ ffffff0019c639f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c5dc60 ffffff03ea8fa1c8 ffffff03f1fa85e0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c5dc60: ffffff0019c5d9f0
[ ffffff0019c5d9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c57c60 ffffff03ea8fa1c8 ffffff03f1fa8b70 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c57c60: ffffff0019c579f0
[ ffffff0019c579f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c51c60 ffffff03ea8fa1c8 ffffff03f1fa9100 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c51c60: ffffff0019c519f0
[ ffffff0019c519f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c4bc60 ffffff03ea8fa1c8 ffffff03f1fa9690 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c4bc60: ffffff0019c4b9f0
[ ffffff0019c4b9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c45c60 ffffff03ea8fa1c8 ffffff03f1fa9c20 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c45c60: ffffff0019c459f0
[ ffffff0019c459f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c3fc60 ffffff03ea8fa1c8 ffffff03f1faa1b0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c3fc60: ffffff0019c3f9f0
[ ffffff0019c3f9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c39c60 ffffff03ea8fa1c8 ffffff03f1faa740 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c39c60: ffffff0019c399f0
[ ffffff0019c399f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c33c60 ffffff03ea8fa1c8 ffffff03f1fab040 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c33c60: ffffff0019c339f0
[ ffffff0019c339f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0019c2dc60 ffffff03ea8fa1c8 ffffff03f1fab5d0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0019c2dc60: ffffff0019c2d9f0
[ ffffff0019c2d9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181dac60 ffffff03ea8fa1c8 ffffff03f1fabb60 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00181dac60: ffffff00181da9f0
[ ffffff00181da9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181d4c60 ffffff03ea8fa1c8 ffffff03f1fac0f0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00181d4c60: ffffff00181d49f0
[ ffffff00181d49f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181cec60 ffffff03ea8fa1c8 ffffff03f1fac680 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00181cec60: ffffff00181ce9f0
[ ffffff00181ce9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181c8c60 ffffff03ea8fa1c8 ffffff03f1facc10 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00181c8c60: ffffff00181c89f0
[ ffffff00181c89f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181c2c60 ffffff03ea8fa1c8 ffffff03f1fad1a0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00181c2c60: ffffff00181c29f0
[ ffffff00181c29f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181bcc60 ffffff03ea8fa1c8 ffffff03f1fad730 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00181bcc60: ffffff00181bc9f0
[ ffffff00181bc9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181b6c60 ffffff03ea8fa1c8 ffffff03f1fae030 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00181b6c60: ffffff00181b69f0
[ ffffff00181b69f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181b0c60 ffffff03ea8fa1c8 ffffff03f1fae5c0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00181b0c60: ffffff00181b09f0
[ ffffff00181b09f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181aac60 ffffff03ea8fa1c8 ffffff03f1faeb50 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00181aac60: ffffff00181aa9f0
[ ffffff00181aa9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00181a4c60 ffffff03ea8fa1c8 ffffff03f1faf0e0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00181a4c60: ffffff00181a49f0
[ ffffff00181a49f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001819ec60 ffffff03ea8fa1c8 ffffff03f1faf670 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001819ec60: ffffff001819e9f0
[ ffffff001819e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018198c60 ffffff03ea8fa1c8 ffffff03f1fafc00 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018198c60: ffffff00181989f0
[ ffffff00181989f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018192c60 ffffff03ea8fa1c8 ffffff03f1fb0190 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018192c60: ffffff00181929f0
[ ffffff00181929f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001818cc60 ffffff03ea8fa1c8 ffffff03f1fb0720 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001818cc60: ffffff001818c9f0
[ ffffff001818c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018186c60 ffffff03ea8fa1c8 ffffff03f1fb1020 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018186c60: ffffff00181869f0
[ ffffff00181869f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018180c60 ffffff03ea8fa1c8 ffffff03f1fb15b0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018180c60: ffffff00181809f0
[ ffffff00181809f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001817ac60 ffffff03ea8fa1c8 ffffff03f1fb1b40 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001817ac60: ffffff001817a9f0
[ ffffff001817a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b9cc60 ffffff03ea8fa1c8 ffffff03f1fb20d0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b9cc60: ffffff0017b9c9f0
[ ffffff0017b9c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b96c60 ffffff03ea8fa1c8 ffffff03f1fb2660 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b96c60: ffffff0017b969f0
[ ffffff0017b969f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b90c60 ffffff03ea8fa1c8 ffffff03f1fb2bf0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b90c60: ffffff0017b909f0
[ ffffff0017b909f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b8ac60 ffffff03ea8fa1c8 ffffff03f1fb3180 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b8ac60: ffffff0017b8a9f0
[ ffffff0017b8a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b84c60 ffffff03ea8fa1c8 ffffff03f1fb3710 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b84c60: ffffff0017b849f0
[ ffffff0017b849f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b7ec60 ffffff03ea8fa1c8 ffffff03f1fb4010 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b7ec60: ffffff0017b7e9f0
[ ffffff0017b7e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b78c60 ffffff03ea8fa1c8 ffffff03f1fb45a0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b78c60: ffffff0017b789f0
[ ffffff0017b789f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b72c60 ffffff03ea8fa1c8 ffffff03f1fb4b30 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b72c60: ffffff0017b729f0
[ ffffff0017b729f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b6cc60 ffffff03ea8fa1c8 ffffff03f1fb50c0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b6cc60: ffffff0017b6c9f0
[ ffffff0017b6c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b66c60 ffffff03ea8fa1c8 ffffff03f1fb5650 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b66c60: ffffff0017b669f0
[ ffffff0017b669f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b60c60 ffffff03ea8fa1c8 ffffff03f1fb5be0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b60c60: ffffff0017b609f0
[ ffffff0017b609f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017b5ac60 ffffff03ea8fa1c8 ffffff03f1fb6170 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017b5ac60: ffffff0017b5a9f0
[ ffffff0017b5a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017f34c60 ffffff03ea8fa1c8 ffffff03f1fb6700 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017f34c60: ffffff0017f349f0
[ ffffff0017f349f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017f28c60 ffffff03ea8fa1c8 ffffff03f1fba380 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017f28c60: ffffff0017f289f0
[ ffffff0017f289f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017f22c60 ffffff03ea8fa1c8 ffffff03f1fba910 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017f22c60: ffffff0017f229f0
[ ffffff0017f229f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017f16c60 ffffff03ea8fa1c8 ffffff03f1fbaea0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017f16c60: ffffff0017f169f0
[ ffffff0017f169f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017f0ac60 ffffff03ea8fa1c8 ffffff03f1fbb430 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017f0ac60: ffffff0017f0a9f0
[ ffffff0017f0a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017f04c60 ffffff03ea8fa1c8 ffffff03f1fbb9c0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017f04c60: ffffff0017f049f0
[ ffffff0017f049f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017ef7c60 ffffff03ea8fa1c8 ffffff03f1fbbf50 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017ef7c60: ffffff0017ef79f0
[ ffffff0017ef79f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017eebc60 ffffff03ea8fa1c8 ffffff03f1fbc4e0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017eebc60: ffffff0017eeb9f0
[ ffffff0017eeb9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017edfc60 ffffff03ea8fa1c8 ffffff03f1fb7000 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017edfc60: ffffff0017edf9f0
[ ffffff0017edf9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017ecdc60 ffffff03ea8fa1c8 ffffff03f1fb7590 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017ecdc60: ffffff0017ecd9f0
[ ffffff0017ecd9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017ec7c60 ffffff03ea8fa1c8 ffffff03f1fb7b20 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017ec7c60: ffffff0017ec79f0
[ ffffff0017ec79f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187cdc60 ffffff03ea8fa1c8 ffffff03f1fb80b0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187cdc60: ffffff00187cd9f0
[ ffffff00187cd9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187c7c60 ffffff03ea8fa1c8 ffffff03f1fb8640 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187c7c60: ffffff00187c79f0
[ ffffff00187c79f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187c1c60 ffffff03ea8fa1c8 ffffff03f1fb8bd0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187c1c60: ffffff00187c19f0
[ ffffff00187c19f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187bbc60 ffffff03ea8fa1c8 ffffff03f1fb9160 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187bbc60: ffffff00187bb9f0
[ ffffff00187bb9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187afc60 ffffff03ea8fa1c8 ffffff03f1fbca70 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187afc60: ffffff00187af9f0
[ ffffff00187af9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187a9c60 ffffff03ea8fa1c8 ffffff03f1fbd370 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187a9c60: ffffff00187a99f0
[ ffffff00187a99f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018564c60 ffffff03ea8fa1c8 ffffff03f1fbde90 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018564c60: ffffff00185649f0
[ ffffff00185649f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001855ec60 ffffff03ea8fa1c8 ffffff03f1fbe420 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001855ec60: ffffff001855e9f0
[ ffffff001855e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018552c60 ffffff03ea8fa1c8 ffffff03f1fbef40 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018552c60: ffffff00185529f0
[ ffffff00185529f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001854cc60 ffffff03ea8fa1c8 ffffff03f1fbf4d0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001854cc60: ffffff001854c9f0
[ ffffff001854c9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001751ac60 ffffff03ea8fa1c8 ffffff03f1fc3350 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001751ac60: ffffff001751a9f0
[ ffffff001751a9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017514c60 ffffff03ea8fa1c8 ffffff03f1fc38e0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017514c60: ffffff00175149f0
[ ffffff00175149f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001711fc60 ffffff03ea8fa1c8 ffffff03f1fc4400 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001711fc60: ffffff001711f9f0
[ ffffff001711f9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001710dc60 ffffff03ea8fa1c8 ffffff03f1fc54b0 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001710dc60: ffffff001710d9f0
[ ffffff001710d9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018639c60 ffffff03ea8fa1c8 ffffff03f1fc0e80 2 0 ffffff03f14515f8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018639c60: ffffff00186399f0
[ ffffff00186399f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff001750ec60 ffffff03ea8fa1c8 ffffff03f1fc3e70 2 99 ffffff03f1451c88
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff001750ec60: ffffff001750e9f0
[ ffffff001750e9f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017520c60 ffffff03ea8fa1c8 ffffff03f1fbfa60 2 99 ffffff03e2bc0d88
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017520c60: ffffff00175209f0
[ ffffff00175209f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0018558c60 ffffff03ea8fa1c8 ffffff03f1fbe9b0 2 99 ffffff03eb9c53c0
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0018558c60: ffffff00185589f0
[ ffffff00185589f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187a3c60 ffffff03ea8fa1c8 ffffff03f1fbd900 2 99 ffffff03eb9c52a8
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187a3c60: ffffff00187a39f0
[ ffffff00187a39f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff00187b5c60 ffffff03ea8fa1c8 ffffff03f1fb96f0 2 99 ffffff03eb9c5708
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff00187b5c60: ffffff00187b59f0
[ ffffff00187b59f0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017e19c60 ffffff03ea8fa1c8 ffffff03e2b88960 2 99 ffffff03ef6f4d38
PC: _resume_from_idle+0xf1 CMD: zpool-tank
stack pointer for thread ffffff0017e19c60: ffffff0017e19a90
[ ffffff0017e19a90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
spa_thread+0x20b()
thread_start+8()
ffffff00187f8c60 fffffffffbc2dbf0 0 0 60 ffffff03f1451eb8
PC: _resume_from_idle+0xf1 TASKQ: zfs_vn_rele_taskq
stack pointer for thread ffffff00187f8c60: ffffff00187f8ae0
[ ffffff00187f8ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
taskq_thread_wait+0x84()
taskq_thread+0x2d1()
thread_start+8()
ffffff0017ed9c60 fffffffffbc2dbf0 0 0 60 ffffff03f1411ecc
PC: _resume_from_idle+0xf1 THREAD: txg_quiesce_thread()
stack pointer for thread ffffff0017ed9c60: ffffff0017ed9b10
[ ffffff0017ed9b10 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
txg_thread_wait+0x5f()
txg_quiesce_thread+0x94()
thread_start+8()
ffffff0017d8cc60 fffffffffbc2dbf0 0 0 60 ffffff049030e640
PC: _resume_from_idle+0xf1 THREAD: txg_sync_thread()
stack pointer for thread ffffff0017d8cc60: ffffff0017d8c9d0
[ ffffff0017d8c9d0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
zio_wait+0x5d()
dsl_pool_sync+0x2bc()
spa_sync+0x37b()
txg_sync_thread+0x247()
thread_start+8()
ffffff03e30ff900 ffffff03eee70e48 ffffff03e0614bb0 1 59 ffffff03f1411ece
PC: _resume_from_idle+0xf1 CMD: zpool import -f -F tank
stack pointer for thread ffffff03e30ff900: ffffff0017bcd160
[ ffffff0017bcd160 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
txg_wait_open+0x7a()
dmu_tx_wait+0xb3()
dmu_tx_assign+0x4b()
dmu_free_long_range_impl+0x12b()
dmu_free_object+0xe6()
dsl_dataset_destroy+0x122()
dsl_destroy_inconsistent+0x5f()
findfunc+0x23()
dmu_objset_find_spa+0x38c()
dmu_objset_find_spa+0x153()
dmu_objset_find+0x40()
spa_load_impl+0xb23()
spa_load+0x117()
spa_load_best+0x78()
spa_import+0xee()
zfs_ioc_pool_import+0xc0()
zfsdev_ioctl+0x177()
cdev_ioctl+0x45()
spec_ioctl+0x5a()
fop_ioctl+0x7b()
ioctl+0x18e()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30fee80 ffffff03e2059018 ffffff03eeefae30 1 59 ffffff03e20590d8
PC: _resume_from_idle+0xf1 CMD: bash
ffffff03ea58fae0 ffffff03ea8eb570 ffffff03e3a4e350 1 59 ffffff03ea8eb630
PC: _resume_from_idle+0xf1 CMD: -bash
ffffff03e32f6c60 ffffff03ea8edaa8 ffffff03eeefa310 1 60 fffffffffbd1da50
PC: _resume_from_idle+0xf1 CMD:
/usr/bin/perl /var/lib/nza/triggers/hosts-check
stack pointer for thread ffffff03e32f6c60: ffffff00179681a0
[ ffffff00179681a0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
spa_open_common+0x69()
spa_open+0x23()
dsl_dir_open_spa+0x95()
dsl_dataset_hold+0x41()
dsl_prop_get+0x4c()
zfs_is_wormed+0x73()
zfs_create+0x69e()
fop_create+0xfc()
vn_createat+0x5e1()
vn_openat+0x1fb()
copen+0x435()
open64+0x34()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03ea77b500 ffffff03eee73380 ffffff03f1fd8150 1 60 fffffffffbd1da50
PC: _resume_from_idle+0xf1 CMD: touch /var/lib/nza/.uptime
stack pointer for thread ffffff03ea77b500: ffffff001867c1a0
[ ffffff001867c1a0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
spa_open_common+0x69()
spa_open+0x23()
dsl_dir_open_spa+0x95()
dsl_dataset_hold+0x41()
dsl_prop_get+0x4c()
zfs_is_wormed+0x73()
zfs_create+0x69e()
fop_create+0xfc()
vn_createat+0x5e1()
vn_openat+0x1fb()
copen+0x435()
open64+0x34()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e362ce80 ffffff03ea8f1900 ffffff03eeef6480 1 59 ffffff03ea8f19c0
PC: _resume_from_idle+0xf1 CMD: /usr/bin/perl /lib/svc/method/nms -d
ffffff03e30f5c20 ffffff03eee71ab0 ffffff03eeefb950 1 59 ffffff03e30f5dfe
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e37ef900 ffffff03eee71ab0 ffffff03e2b94920 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
stack pointer for thread ffffff03e37ef900: ffffff001811c5c0
[ ffffff001811c5c0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0xe45()
0xfffffffffb8001d6()
ffffff03e32e5720 ffffff03eee71ab0 ffffff03e0661730 1 59 ffffff03eda5785a
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
stack pointer for thread ffffff03e32e5720: ffffff00172d4ba0
[ ffffff00172d4ba0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_sig_hires+0x1e9()
cv_waituntil_sig+0xba()
poll_common+0x462()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e3100060 ffffff03eee71ab0 ffffff03ef855f00 1 59 ffffff03e310023e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30ec760 ffffff03eee71ab0 ffffff03ef856490 1 59 ffffff03e30ec93e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e32e5020 ffffff03eee71ab0 ffffff03eb1c52d0 1 59 ffffff03e32e51fe
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30f03c0 ffffff03eee71ab0 ffffff03e2b91930 1 59 ffffff03e30f059e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e33008a0 ffffff03eee71ab0 ffffff03e0660c10 1 59 ffffff03e3300a7e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea49c1c0 ffffff03eee71ab0 ffffff03e2b8de20 1 59 ffffff03ea49c39e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e3623c20 ffffff03eee71ab0 ffffff03eb1c6380 1 59 ffffff03e3623dfe
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e37fb540 ffffff03eee71ab0 ffffff03e061e110 1 59 ffffff03e37fb71e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e3634760 ffffff03eee71ab0 ffffff03e2b8a310 1 59 ffffff03e363493e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea6a1e40 ffffff03eee71ab0 ffffff03e065b1c0 1 59 ffffff03ea6a201e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e31011e0 ffffff03eee71ab0 ffffff03eb1c6ea0 1 59 ffffff03e31013be
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e362dc80 ffffff03eee71ab0 ffffff03eef04eb0 1 59 ffffff03e362de5e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e32ffaa0 ffffff03eee71ab0 ffffff03eef04920 1 59 ffffff03e32ffc7e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e32ff720 ffffff03eee71ab0 ffffff03eef04390 1 59 ffffff03e32ff8fe
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea6b6520 ffffff03eee71ab0 ffffff03e061e6a0 1 59 ffffff03ea6b66fe
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30ece60 ffffff03eee71ab0 ffffff03eb1cd9a0 1 59 ffffff03e30ed03e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e37ef200 ffffff03eee71ab0 ffffff03eef03870 1 59 ffffff03e37ef3de
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30e7500 ffffff03eee71ab0 ffffff03e2b96df0 1 59 ffffff03e30e76de
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e64368a0 ffffff03eee71ab0 ffffff03eef03e00 1 59 ffffff03e6436a7e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea69b760 ffffff03eee71ab0 ffffff03e2b962d0 1 59 ffffff03ea69b93e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea77bc00 ffffff03eee71ab0 ffffff03e2b8e3b0 1 59 ffffff03ea77bdde
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e32ede80 ffffff03eee71ab0 ffffff03eef032e0 1 59 ffffff03e32ee05e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea6a28c0 ffffff03eee71ab0 ffffff03eef029e0 1 59 ffffff03ea6a2a9e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea6a2540 ffffff03eee71ab0 ffffff03eeefde20 1 59 ffffff03ea6a271e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea49b740 ffffff03eee71ab0 ffffff03e061b120 1 59 ffffff03ea49b91e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea49bac0 ffffff03eee71ab0 ffffff03eb1c5df0 1 59 ffffff03ea49bc9e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e32eec80 ffffff03eee71ab0 ffffff03e2b8c7e0 1 59 ffffff03e32eee5e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e32f9e40 ffffff03eee71ab0 ffffff03e06191e0 1 59 ffffff03e32fa01e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea6a21c0 ffffff03eee71ab0 ffffff03da3c7700 1 59 ffffff03ea6a239e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30f0ac0 ffffff03eee71ab0 ffffff03eeeff460 1 59 ffffff03e30f0c9e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e362cb00 ffffff03eee71ab0 ffffff03eeefe940 1 59 ffffff03e362ccde
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea58f3e0 ffffff03eee71ab0 ffffff03eef00880 1 59 ffffff03ea58f5be
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e3622e20 ffffff03eee71ab0 ffffff03eef01ec0 1 59 ffffff03e3622ffe
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30e6380 ffffff03eee71ab0 ffffff03eeefd300 1 59 ffffff03e30e655e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e32edb00 ffffff03eee71ab0 ffffff03eeefd890 1 59 ffffff03e32edcde
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea779c20 ffffff03eee71ab0 ffffff03eef013a0 1 59 ffffff03ea779dfe
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea58de80 ffffff03eee71ab0 ffffff03eef00e10 1 59 ffffff03ea58e05e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30e8780 ffffff03eee71ab0 ffffff03eeefbee0 1 59 ffffff03e30e895e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e363b1c0 ffffff03eee71ab0 ffffff03e2b8b3c0 1 59 ffffff03e363b39e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea6a2c40 ffffff03eee71ab0 ffffff03eeefeed0 1 59 ffffff03ea6a2e1e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30e9200 ffffff03eee71ab0 ffffff03eef01930 1 59 ffffff03e30e93de
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30ecae0 ffffff03eee71ab0 ffffff03eeefc470 1 59 ffffff03e30eccbe
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30e6000 ffffff03eee71ab0 ffffff03e0666660 1 59 ffffff03e30e61de
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30f8000 ffffff03eee71ab0 ffffff03eeefe3b0 1 59 ffffff03e30f81de
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e3628e00 ffffff03eee71ab0 ffffff03eb1c6910 1 59 ffffff03e3628fde
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30f11c0 ffffff03eee71ab0 ffffff03eef02450 1 59 ffffff03e30f139e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea49be40 ffffff03eee71ab0 ffffff03eef002f0 1 59 ffffff03ea49c01e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e363a740 ffffff03eee71ab0 ffffff03ef856a20 1 59 ffffff03e363a91e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea58e900 ffffff03eee71ab0 ffffff03eeefca00 1 59 ffffff03ea58eade
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e37ee780 ffffff03eee71ab0 ffffff03eb1c8850 1 59 ffffff03e37ee95e
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03ea77aa80 ffffff03eee71ab0 ffffff03e2b95d40 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
stack pointer for thread ffffff03ea77aa80: ffffff00172ce270
[ ffffff00172ce270 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
poll_common+0x614()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
_sys_sysenter_post_swapgs+0x149()
ffffff03e30e7880 ffffff03eee71ab0 ffffff03eeef5960 1 60 0
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
stack pointer for thread ffffff03e30e7880: ffffff001738e5c0
[ ffffff001738e5c0 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0xe45()
0xfffffffffb8001d6()
ffffff03ea58fe60 ffffff03ea8ef378 ffffff03e2b932e0 1 59 ffffff03ea8ef438
PC: _resume_from_idle+0xf1 CMD: /usr/bin/python /usr/bin/nmv.py
ffffff03e30e9580 ffffff03e2056a90 ffffff03eeeff9f0 1 59 ffffff03ed724d6a
PC: _resume_from_idle+0xf1 CMD: /usr/bin/perl /usr/bin/nmc -s
ffffff03e2f0c1c0 ffffff03e05e1540 ffffff03e0667ca0 1 59 ffffff03e2f0c39e
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e2f0c1c0: ffffff00182fdc30
[ ffffff00182fdc30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff03e2f0be40 ffffff03e05e1540 ffffff03e065f7f0 1 59 ffffff03e3091444
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e2f0be40: ffffff0018303a90
[ ffffff0018303a90 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig+0x14d()
cte_get_event+0xd7()
ctfs_endpoint_ioctl+0xa7()
ctfs_bu_ioctl+0x51()
fop_ioctl+0x7b()
ioctl+0x18e()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff03e2f0bac0 ffffff03e05e1540 ffffff03e0623b60 1 59 ffffff03e2f70a14
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e2f0bac0: ffffff0018309b00
[ ffffff0018309b00 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
port_getn+0x23b()
portfs+0x282()
portfs32+0x57()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff03e30ed560 ffffff03e05e1540 ffffff03e060d220 1 60 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e30ed560: ffffff0018355350
[ ffffff0018355350 _resume_from_idle+0xf1() ]
htable_steal+0x198()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
lwp_park+0x242()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff0017bd4c60 fffffffffbc2dbf0 0 0 60 ffffff03e312d5f8
PC: _resume_from_idle+0xf1 THREAD: evch_delivery_thr()
stack pointer for thread ffffff0017bd4c60: ffffff0017bd4ae0
[ ffffff0017bd4ae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
evch_delivery_hold+0x85()
evch_delivery_thr+0x23e()
thread_start+8()
ffffff00180bbc60 fffffffffbc2dbf0 0 0 60 ffffff03e312d588
PC: _resume_from_idle+0xf1 THREAD: evch_delivery_thr()
stack pointer for thread ffffff00180bbc60: ffffff00180bbae0
[ ffffff00180bbae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
evch_delivery_hold+0x85()
evch_delivery_thr+0x23e()
thread_start+8()
ffffff03e37fb1c0 ffffff03e05e1540 ffffff03e061a600 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e37fb1c0: ffffff0018158d30
[ ffffff0018158d30 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff03e30e9c80 ffffff03e05e1540 ffffff03e060cc90 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e30e9c80: ffffff00183e2d30
[ ffffff00183e2d30 _resume_from_idle+0xf1() ]
swtch+0x145()
shuttle_swtch+0x256()
door_return+0x242()
doorfs32+0x134()
sys_syscall32+0xff()
ffffff00172dac60 fffffffffbc2dbf0 0 0 60 ffffff03e312d4a8
PC: _resume_from_idle+0xf1 THREAD: evch_delivery_thr()
stack pointer for thread ffffff00172dac60: ffffff00172daae0
[ ffffff00172daae0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
evch_delivery_hold+0x85()
evch_delivery_thr+0x23e()
thread_start+8()
ffffff03e2f0b740 ffffff03e05e1540 ffffff03e06235d0 1 59 ffffff03e2f0b91e
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e2f0b740: ffffff001840fcc0
[ ffffff001840fcc0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
sys_syscall32+0xff()
ffffff03e3101c60 ffffff03e05e1540 ffffff03e0623040 1 59 0
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e3101c60: ffffff0018421c30
[ ffffff0018421c30 _resume_from_idle+0xf1() ]
swtch_to+0xe6()
shuttle_resume+0x325()
door_call+0x2b1()
doorfs32+0x141()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff03e31018e0 ffffff03e05e1540 ffffff03e0622740 1 59 ffffff03e3101abe
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e31018e0: ffffff0018427c30
[ ffffff0018427c30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
cv_waituntil_sig+0x13c()
lwp_park+0x157()
syslwp_park+0x31()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff03e279fe20 ffffff03e05e1540 ffffff03e0624680 1 59 ffffff03e279fffe
PC: _resume_from_idle+0xf1 CMD: /lib/svc/bin/svc.startd
stack pointer for thread ffffff03e279fe20: ffffff0018399dc0
[ ffffff0018399dc0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait_sig_swap_core+0x174()
cv_wait_sig_swap+0x18()
sigsuspend+0x109()
sys_syscall32+0xff()
ffffff03e27a0c20 ffffff03e05e46e0 ffffff03da3c2b40 1 60 0
PC: _resume_from_idle+0xf1 CMD: /sbin/init
stack pointer for thread ffffff03e27a0c20: ffffff00179a9270
[ ffffff00179a9270 _resume_from_idle+0xf1() ]
swtch+0x145()
turnstile_block+0x760()
mutex_vector_enter+0x261()
htable_steal+0x19d()
htable_alloc+0x248()
htable_create+0x1dc()
hati_load_common+0xa3()
hat_memload+0x81()
hat_memload_region+0x25()
segvn_faultpage+0x937()
segvn_fault+0xc13()
as_fault+0x5ee()
pagefault+0x99()
trap+0x1624()
0xfffffffffb8001d6()
bcopy+0x1b4()
poll_common+0x614()
pollsys+0xe4()
dtrace_systrace_syscall32+0x11a()
sys_syscall32+0xff()
ffffff00179b5c60 ffffff03e05e3a78 ffffff03e06251a0 0 97 0
PC: _resume_from_idle+0xf1 CMD: pageout
stack pointer for thread ffffff00179b5c60: ffffff00179b5a30
[ ffffff00179b5a30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
pageout_scanner+0x132()
thread_start+8()
ffffff00179afc60 ffffff03e05e3a78 ffffff03da3c25b0 0 98 ffffff03dd38e24e
PC: _resume_from_idle+0xf1 CMD: pageout
stack pointer for thread ffffff00179afc60: ffffff00179af5d0
[ ffffff00179af5d0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
txg_wait_open+0x7a()
dmu_tx_wait+0xb3()
dmu_tx_assign+0x4b()
zvol_strategy+0x1d4()
bdev_strategy+0x75()
spec_startio+0xb4()
spec_pageio+0x39()
fop_pageio+0x80()
swap_putapage+0x443()
swap_putpage+0x1be()
fop_putpage+0x74()
pageout+0x2e4()
thread_start+8()
ffffff00179bbc60 ffffff03e05e2e10 ffffff03da3c2020 0 60 ffffff03dd38e24e
PC: _resume_from_idle+0xf1 CMD: fsflush
stack pointer for thread ffffff00179bbc60: ffffff00179bb620
[ ffffff00179bb620 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
txg_wait_open+0x7a()
dmu_tx_wait+0xb3()
zfs_putapage+0x1fb()
pvn_vplist_dirty+0x362()
zfs_putpage+0x289()
fop_putpage+0x74()
fsflush_do_pages+0x2c5()
fsflush+0x39a()
thread_start+8()
ffffff00179c1c60 fffffffffbc2dbf0 0 0 60 fffffffffbce7178
PC: _resume_from_idle+0xf1 THREAD: mod_uninstall_daemon()
stack pointer for thread ffffff00179c1c60: ffffff00179c1ba0
[ ffffff00179c1ba0 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_wait+0x61()
mod_uninstall_daemon+0x99()
thread_start+8()
ffffff00179c7c60 fffffffffbc2dbf0 0 0 60 0
PC: _resume_from_idle+0xf1 THREAD: seg_pasync_thread()
stack pointer for thread ffffff00179c7c60: ffffff00179c7b30
[ ffffff00179c7b30 _resume_from_idle+0xf1() ]
swtch+0x145()
cv_timedwait_hires+0xe0()
cv_reltimedwait+0x4f()
seg_pasync_thread+0xcb()
thread_start+8()
Andrew Jones
2010-Jul-02 20:49 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
> Andrew, > > Looks like the zpool is telling you the devices are > still doing work of > some kind, or that there are locks still held. >Agreed; it appears the CSV1 volume is in a fundamentally inconsistent state following the aborted zfs destroy attempt. See later in this thread where Victor has identified this to be the case. I am awaiting his analysis of the latest crash.> From man of section 2 intro page the errors are > listed. Number 16 > ooks to be an EBUSY. > > > 16 EBUSY Device busy > An attempt was made to mount > a dev- > ice that was already > mounted or an > attempt was made to > unmount a device > on which there is > an active file > (open file, current > directory, > mounted-on file, active > text seg- > ment). It will also > occur if an > attempt is made to > enable accounting > when it is already > enabled. The > device or resource is > currently una- > vailable. EBUSY is > also used by > mutexes, semaphores, > condition vari- > ables, and r/w locks, > to indicate > that a lock is held, > and by the > processor control > function > P_ONLINE. > ndrew Jones wrote: > > Just re-ran ''zdb -e tank'' to confirm the CSV1 > volume is still exhibiting error 16: > > > > <snip> > > Could not open tank/CSV1, error 16 > > <snip> > > > > Considering my attempt to delete the CSV1 volume > lead to the failure in the first place, I have to > think that if I can either 1) complete the deletion > of this volume or 2) roll back to a transaction prior > to this based on logging or 3) repair whatever > corruption has been caused by this partial deletion, > that I will then be able to import the pool. > > > > What does ''error 16'' mean in the ZDB output, any > suggestions? > > > > -- > Geoff Shipman | Senior Technical Support Engineer > Phone: +13034644710 > Oracle Global Customer Services > 500 Eldorado Blvd. UBRM-04 | Broomfield, CO 80021 > Email: geoff.shipman at sun.com | Hours:9am-5pm > MT,Monday-Friday > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discu > ss >-- This message posted from opensolaris.org
Andrew Jones
2010-Jul-04 00:58 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
Victor,
The zpool import succeeded on the next attempt following the crash that I
reported to you by private e-mail!
For completeness, this is the final status of the pool:
pool: tank
state: ONLINE
scan: resilvered 1.50K in 165h28m with 0 errors on Sat Jul 3 08:02:30 2010
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
c0t0d0 ONLINE 0 0 0
c0t1d0 ONLINE 0 0 0
c0t2d0 ONLINE 0 0 0
c0t3d0 ONLINE 0 0 0
c0t4d0 ONLINE 0 0 0
c0t5d0 ONLINE 0 0 0
c0t6d0 ONLINE 0 0 0
c0t7d0 ONLINE 0 0 0
cache
c2t0d0 ONLINE 0 0 0
errors: No known data errors
Thank you very much for your help. We did not need to add additional RAM to
solve this, in the end. Instead, we needed to persist with the import through
several panics to finally work our way through the large inconsistent dataset;
it is unclear whether the resilvering caused additional processing delay.
Unfortunately, the delay made much of the data quite stale, now that
it''s been recovered.
It does seem that zfs would benefit tremendously from a better (quicker and more
intuitive?) set of recovery tools, that are available to a wider range of users.
It''s really a shame, because the features and functionality in zfs are
otherwise absolutely second to none.
/Andrew[i][/i][i][/i][i][/i][i][/i][i][/i]
--
This message posted from opensolaris.org
Roy Sigurd Karlsbakk
2010-Jul-04 11:41 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
----- Original Message -----> Victor, > > The zpool import succeeded on the next attempt following the crash > that I reported to you by private e-mail! > > For completeness, this is the final status of the pool: > > > pool: tank > state: ONLINE > scan: resilvered 1.50K in 165h28m with 0 errors on Sat Jul 3 08:02:30Out of curiosity, what sort of drives are you using here? Resilvering in 165h28m is close to a week, which is rather bad imho. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.
Andrew Jones
2010-Jul-04 16:18 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
> > ----- Original Message ----- > > Victor, > > > > The zpool import succeeded on the next attempt > following the crash > > that I reported to you by private e-mail! > > > > For completeness, this is the final status of the > pool: > > > > > > pool: tank > > state: ONLINE > > scan: resilvered 1.50K in 165h28m with 0 errors on > Sat Jul 3 08:02:30 > > Out of curiosity, what sort of drives are you using > here? Resilvering in 165h28m is close to a week, > which is rather bad imho.I think the resilvering statistic is quite misleading, in this case. We''re using very average 1TB retail Hitachi disks, which perform just fine when the pool is healthy. What happened here is that the zpool-tank process was performing a resilvering task in parallel with the processing of a very large inconsistent dataset, which took the overwhelming majority of the time to complete. Why it actually took over a week to process the 2TB volume in an inconsistent state is my primary concern with the performance of ZFS, in this case.> > Vennlige hilsener / Best regards > > roy > -- > Roy Sigurd Karlsbakk > (+47) 97542685 > roy at karlsbakk.net > http://blogg.karlsbakk.net/ > -- > I all pedagogikk er det essensielt at pensum > presenteres intelligibelt. Det er et element?rt > imperativ for alle pedagoger ? unng? eksessiv > anvendelse av idiomer med fremmed opprinnelse. I de > fleste tilfeller eksisterer adekvate og relevante > synonymer p? norsk. > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discu > ss-- This message posted from opensolaris.org
Victor Latushkin
2010-Jul-06 14:41 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
On Jul 4, 2010, at 4:58 AM, Andrew Jones wrote:> Victor, > > The zpool import succeeded on the next attempt following the crash that I reported to you by private e-mail!>From the threadlist it looked like system was pretty low on memory with stacks of userland stuff swapped out, hence system was not responsive, but it was able to complete inconsistent dataset processing in the end.> > For completeness, this is the final status of the pool: > > > pool: tank > state: ONLINE > scan: resilvered 1.50K in 165h28m with 0 errors on Sat Jul 3 08:02:30 2010 > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > raidz2-0 ONLINE 0 0 0 > c0t0d0 ONLINE 0 0 0 > c0t1d0 ONLINE 0 0 0 > c0t2d0 ONLINE 0 0 0 > c0t3d0 ONLINE 0 0 0 > c0t4d0 ONLINE 0 0 0 > c0t5d0 ONLINE 0 0 0 > c0t6d0 ONLINE 0 0 0 > c0t7d0 ONLINE 0 0 0 > cache > c2t0d0 ONLINE 0 0 0 > > errors: No known data errors >Good. Run ''zpool scrub'' to make sure there are no other errors. regards victor> Thank you very much for your help. We did not need to add additional RAM to solve this, in the end. Instead, we needed to persist with the import through several panics to finally work our way through the large inconsistent dataset; it is unclear whether the resilvering caused additional processing delay. Unfortunately, the delay made much of the data quite stale, now that it''s been recovered. > > It does seem that zfs would benefit tremendously from a better (quicker and more intuitive?) set of recovery tools, that are available to a wider range of users. It''s really a shame, because the features and functionality in zfs are otherwise absolutely second to none. > > /Andrew[i][/i][i][/i][i][/i][i][/i][i][/i] > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Andrew Jones
2010-Jul-06 17:32 UTC
[zfs-discuss] zpool import hangs indefinitely (retry post in parts; too long?)
> > Good. Run ''zpool scrub'' to make sure there are no > other errors. > > regards > victor >Yes, scrubbed successfully with no errors. Thanks again for all of your generous assistance. /AJ -- This message posted from opensolaris.org