Hi, After upgrading RELENG_6 from Jul 11 to Sep 30 on an i386 box, everytime I run tar to backup my system to a mounted nfs volume. After one hour of operation, it panics with sleeping thread. Upgrading to RELENG_6_2 does not help. Also, the console is complete hang, I can not break into DDB at all. The only thing is do power cycling. Also, the only harddisk on that host is the ips(4), so I can not obtain a kernel dump. I'm not sure if this is a hardware failure, at least, no led on the panel is shown red... OK, the only information on console is attached below. Any suggesstion are welcome. Thanks, Rong-En Fan = ips0: WARNING: command timeout. Adapter is in toaster mode, resetting to known s tate ips0: resetting adapter, this may take up to 5 minutes ips0: syncing config Sleeping thread (tid 100002, pid 14) owns a non-sleepable lock sched_switch(c5feec00,0,1,8577f833,d14d2103,...) at sched_switch+0x158 mi_switch(1,0) at mi_switch+0x1d5 sleepq_switch(c60c2604,e9f77b8c,c051acd3,4,1,...) at sleepq_switch+0x93 sleepq_wait(c60c2604,c60c25e0,c06e6957,1,1,...) at sleepq_wait+0x75 cv_wait(c60c2604,c60c25e0,a,e9f77c04,5,...) at cv_wait+0x151 _sema_wait(c60c25e0,0,0,c60c2400,c60c2400,...) at _sema_wait+0x64 ips_send_config_sync_cmd(c60f5000,e9f77c08,1,c60f5000,7,...) at ips_send_config_sync_cmd+0x78 ips_clear_adapter(c60c2400,c60b6e00,0,4,c60f5000,...) at ips_clear_adapter+0x60 ips_morpheus_reinit(c60c2400,1,c053abf7,c0740100,c5feec00,...) at ips_morpheus_reinit+0x2ac ips_timeout(c60c2400,c053a7f5,c5feec00,c5feea80,d69f60ed,...) at ips_timeout+0xf8 softclock(0,e9f77cd4,15dbe,c43ec589,c5feec00,...) at softclock+0x35d ithread_execute_handlers(c5fed430,c6042000,0,0,0,...) at ithread_execute_handlers+0x162 ithread_loop(c5fbc880,e9f77d38,0,0,0,...) at ithread_loop+0x64 fork_exit(c050b22a,c5fbc880,e9f77d38) at fork_exit+0x7b fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe9f77d6c, ebp = 0 --- panic: sleeping thread cpuid = 2 KDB: stack backtrace: kdb_backtrace(c0702303,2,c06ef01b,e9f98bf0,0,...) at kdb_backtrace+0x2f panic(c06ef01b,ffffffff,e,c051ac54,1,...) at panic+0x129 propagate_priority(c5fefd80,c5fefd80,0,0,0,...) at propagate_priority+0x69 turnstile_wait(c60c25a8,c5feec00,c610c000,c7d064a4,4,...) at turnstile_wait+0x32f _mtx_lock_sleep(c60c25a8,c5fefd80,0,0,0,...) at _mtx_lock_sleep+0xfd ipsd_strategy(c7d064a4,43,200,0,c04e31a1,...) at ipsd_strategy+0x70 g_disk_start(c7d1e4a4,c073bac8,24c,c06e8406,64,...) at g_disk_start+0x1b1 g_io_schedule_down(c5fefd80,4c,c5fefd80,c04e39c1,e9f98d04,...) at g_io_schedule_down+0x15f g_down_procbody(0,e9f98d38,0,0,0,...) at g_down_procbody+0xb3 fork_exit(c04e39c1,0,e9f98d38) at fork_exit+0x7b fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe9f98d6c, ebp = 0 ---
I'll look at this. Scott Rong-en Fan wrote:> Hi, > > After upgrading RELENG_6 from Jul 11 to Sep 30 on an i386 box, > everytime I run tar to backup my system to a mounted nfs volume. > After one hour of operation, it panics with sleeping thread. Upgrading > to RELENG_6_2 does not help. Also, the console is complete > hang, I can not break into DDB at all. The only thing is do power > cycling. > > Also, the only harddisk on that host is the ips(4), so I can not obtain > a kernel dump. I'm not sure if this is a hardware failure, at least, no > led on the panel is shown red... > > OK, the only information on console is attached below. Any suggesstion > are welcome. > > Thanks, > Rong-En Fan > > => > ips0: WARNING: command timeout. Adapter is in toaster mode, resetting to > known s > tate > ips0: resetting adapter, this may take up to 5 minutes > ips0: syncing config > Sleeping thread (tid 100002, pid 14) owns a non-sleepable lock > sched_switch(c5feec00,0,1,8577f833,d14d2103,...) at sched_switch+0x158 > mi_switch(1,0) at mi_switch+0x1d5 > sleepq_switch(c60c2604,e9f77b8c,c051acd3,4,1,...) at sleepq_switch+0x93 > sleepq_wait(c60c2604,c60c25e0,c06e6957,1,1,...) at sleepq_wait+0x75 > cv_wait(c60c2604,c60c25e0,a,e9f77c04,5,...) at cv_wait+0x151 > _sema_wait(c60c25e0,0,0,c60c2400,c60c2400,...) at _sema_wait+0x64 > ips_send_config_sync_cmd(c60f5000,e9f77c08,1,c60f5000,7,...) at > ips_send_config_sync_cmd+0x78 > ips_clear_adapter(c60c2400,c60b6e00,0,4,c60f5000,...) at > ips_clear_adapter+0x60 > ips_morpheus_reinit(c60c2400,1,c053abf7,c0740100,c5feec00,...) at > ips_morpheus_reinit+0x2ac > ips_timeout(c60c2400,c053a7f5,c5feec00,c5feea80,d69f60ed,...) at > ips_timeout+0xf8 > softclock(0,e9f77cd4,15dbe,c43ec589,c5feec00,...) at softclock+0x35d > ithread_execute_handlers(c5fed430,c6042000,0,0,0,...) at > ithread_execute_handlers+0x162 > ithread_loop(c5fbc880,e9f77d38,0,0,0,...) at ithread_loop+0x64 > fork_exit(c050b22a,c5fbc880,e9f77d38) at fork_exit+0x7b > fork_trampoline() at fork_trampoline+0x8 > --- trap 0x1, eip = 0, esp = 0xe9f77d6c, ebp = 0 --- > panic: sleeping thread > cpuid = 2 > KDB: stack backtrace: > kdb_backtrace(c0702303,2,c06ef01b,e9f98bf0,0,...) at kdb_backtrace+0x2f > panic(c06ef01b,ffffffff,e,c051ac54,1,...) at panic+0x129 > propagate_priority(c5fefd80,c5fefd80,0,0,0,...) at propagate_priority+0x69 > turnstile_wait(c60c25a8,c5feec00,c610c000,c7d064a4,4,...) at > turnstile_wait+0x32f > _mtx_lock_sleep(c60c25a8,c5fefd80,0,0,0,...) at _mtx_lock_sleep+0xfd > ipsd_strategy(c7d064a4,43,200,0,c04e31a1,...) at ipsd_strategy+0x70 > g_disk_start(c7d1e4a4,c073bac8,24c,c06e8406,64,...) at g_disk_start+0x1b1 > g_io_schedule_down(c5fefd80,4c,c5fefd80,c04e39c1,e9f98d04,...) at > g_io_schedule_down+0x15f > g_down_procbody(0,e9f98d38,0,0,0,...) at g_down_procbody+0xb3 > fork_exit(c04e39c1,0,e9f98d38) at fork_exit+0x7b > fork_trampoline() at fork_trampoline+0x8 > --- trap 0x1, eip = 0, esp = 0xe9f98d6c, ebp = 0 --- > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
It seems that after upgrading ips firmware to the latest version available on ibm.com solves this problem. One changelog caught me eye: "increase timeout when tape driver is attached to the adapter". Indeed, we have a tape on ahd(4), which I think ips(4) is sharing this adapter. however, the mystery is why ips stops work suddenly. Perhaps, I tweaked some hardware settings and I forgot it. On 11/18/06, Scott Long <scottl@samsco.org> wrote:> I'll look at this. > > Scott > > > Rong-en Fan wrote: > > Hi, > > > > After upgrading RELENG_6 from Jul 11 to Sep 30 on an i386 box, > > everytime I run tar to backup my system to a mounted nfs volume. > > After one hour of operation, it panics with sleeping thread. Upgrading > > to RELENG_6_2 does not help. Also, the console is complete > > hang, I can not break into DDB at all. The only thing is do power > > cycling. > > > > Also, the only harddisk on that host is the ips(4), so I can not obtain > > a kernel dump. I'm not sure if this is a hardware failure, at least, no > > led on the panel is shown red... > > > > OK, the only information on console is attached below. Any suggesstion > > are welcome. > > > > Thanks, > > Rong-En Fan > > > > => > > > ips0: WARNING: command timeout. Adapter is in toaster mode, resetting to > > known s > > tate > > ips0: resetting adapter, this may take up to 5 minutes > > ips0: syncing config > > Sleeping thread (tid 100002, pid 14) owns a non-sleepable lock > > sched_switch(c5feec00,0,1,8577f833,d14d2103,...) at sched_switch+0x158 > > mi_switch(1,0) at mi_switch+0x1d5 > > sleepq_switch(c60c2604,e9f77b8c,c051acd3,4,1,...) at sleepq_switch+0x93 > > sleepq_wait(c60c2604,c60c25e0,c06e6957,1,1,...) at sleepq_wait+0x75 > > cv_wait(c60c2604,c60c25e0,a,e9f77c04,5,...) at cv_wait+0x151 > > _sema_wait(c60c25e0,0,0,c60c2400,c60c2400,...) at _sema_wait+0x64 > > ips_send_config_sync_cmd(c60f5000,e9f77c08,1,c60f5000,7,...) at > > ips_send_config_sync_cmd+0x78 > > ips_clear_adapter(c60c2400,c60b6e00,0,4,c60f5000,...) at > > ips_clear_adapter+0x60 > > ips_morpheus_reinit(c60c2400,1,c053abf7,c0740100,c5feec00,...) at > > ips_morpheus_reinit+0x2ac > > ips_timeout(c60c2400,c053a7f5,c5feec00,c5feea80,d69f60ed,...) at > > ips_timeout+0xf8 > > softclock(0,e9f77cd4,15dbe,c43ec589,c5feec00,...) at softclock+0x35d > > ithread_execute_handlers(c5fed430,c6042000,0,0,0,...) at > > ithread_execute_handlers+0x162 > > ithread_loop(c5fbc880,e9f77d38,0,0,0,...) at ithread_loop+0x64 > > fork_exit(c050b22a,c5fbc880,e9f77d38) at fork_exit+0x7b > > fork_trampoline() at fork_trampoline+0x8 > > --- trap 0x1, eip = 0, esp = 0xe9f77d6c, ebp = 0 --- > > panic: sleeping thread > > cpuid = 2 > > KDB: stack backtrace: > > kdb_backtrace(c0702303,2,c06ef01b,e9f98bf0,0,...) at kdb_backtrace+0x2f > > panic(c06ef01b,ffffffff,e,c051ac54,1,...) at panic+0x129 > > propagate_priority(c5fefd80,c5fefd80,0,0,0,...) at propagate_priority+0x69 > > turnstile_wait(c60c25a8,c5feec00,c610c000,c7d064a4,4,...) at > > turnstile_wait+0x32f > > _mtx_lock_sleep(c60c25a8,c5fefd80,0,0,0,...) at _mtx_lock_sleep+0xfd > > ipsd_strategy(c7d064a4,43,200,0,c04e31a1,...) at ipsd_strategy+0x70 > > g_disk_start(c7d1e4a4,c073bac8,24c,c06e8406,64,...) at g_disk_start+0x1b1 > > g_io_schedule_down(c5fefd80,4c,c5fefd80,c04e39c1,e9f98d04,...) at > > g_io_schedule_down+0x15f > > g_down_procbody(0,e9f98d38,0,0,0,...) at g_down_procbody+0xb3 > > fork_exit(c04e39c1,0,e9f98d38) at fork_exit+0x7b > > fork_trampoline() at fork_trampoline+0x8 > > --- trap 0x1, eip = 0, esp = 0xe9f98d6c, ebp = 0 --- > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > >