thr3ads.net - crossbow discuss - [crossbow-discuss] Panic in mac:mac_tx_srs_wakeup on freed flow control entry [May 2009]

If this information is useful, please help other people find it:
Share via:

Saurabh Misra

2009-May-21 18:23 UTC

[crossbow-discuss] Panic in mac:mac_tx_srs_wakeup on freed flow control entry

Hi --

I''m seeing a panic while running NICDRV. It''s happening when
running
test09 (Nemo_Trunk) of the testsuite. I didn''t see this before though.

Just want to check if it''s my problem or not. It does seem like a race 
between TX interrupt calling mac_tx_update() and thread removing TX src 
(mac src client).

 > $C
ffffff0003ec59f0 vpanic()
ffffff0003ec5a20 mutex_panic+0x73(fffffffffb92d478, ffffffcc53dc82c0)
ffffff0003ec5a80 mutex_vector_enter+0x446(ffffffcc53dc82c0)
ffffff0003ec5ad0 mac_tx_srs_wakeup+0x2b(ffffffcc53dc82c0, 0)
ffffff0003ec5b10 i_mac_tx_srs_notify+0x79(ffffff0153f3a738, 0)
ffffff0003ec5b30 mac_tx_update+0x18(ffffff0153f3a738)
ffffff0003ec5b60 bfe_tx_done+0x34(ffffff0153b0dd00, 1000000)
ffffff0003ec5bb0 bfe_interrupt+0xbb(ffffff0153b0dd00, 0)
ffffff0003ec5c00 av_dispatch_autovect+0x7c(15)
ffffff0003ec5c40 dispatch_hardint+0x33(15, 0)
ffffff00045888d0 switch_sp_and_call+0x13()
ffffff0004588920 do_interrupt+0x9e(ffffff0004588930, ffffffcc53dc88b8)
ffffff0004588930 _interrupt+0xba()
ffffff0004588a30 mac_srs_free+0x5a(ffffffcc53dc82c0)
ffffff0004588a90 mac_srs_group_teardown+0xf5(ffffff6972044130, 
ffffff9a92f95428
, 1)
ffffff0004588af0 mac_datapath_teardown+0x8e(ffffff6972044130, 
ffffff9a92f95428,
1)
ffffff0004588b40 mac_unicast_remove+0xfa(ffffff6972044130, ffffff84bda00c40)
ffffff0004588b70 aggr_port_delete+0xc2(ffffff84bd9ea790)
ffffff0004588be0 aggr_grp_delete+0x1aa(11c)
ffffff0004588c20 aggr_ioc_delete+0x27(ffffff84c246dd60, 8046f74, 100003,
ffffff7302534830, ffffff0004588de4)
ffffff0004588cc0 drv_ioctl+0x137(b200000000, a660002, 8046f74, 100003,
ffffff7302534830, ffffff0004588de4)
ffffff0004588d00 cdev_ioctl+0x45(b200000000, a660002, 8046f74, 100003,
ffffff7302534830, ffffff0004588de4)
ffffff0004588d40 spec_ioctl+0x83(ffffff01513c7200, a660002, 8046f74, 
100003,
ffffff7302534830, ffffff0004588de4, 0)
ffffff0004588dc0 fop_ioctl+0x7b(ffffff01513c7200, a660002, 8046f74, 100003,
ffffff7302534830, ffffff0004588de4, 0)
ffffff0004588ec0 ioctl+0x18e(3, a660002, 8046f74)
ffffff0004588f10 _sys_sysenter_post_swapgs+0x14b()


Here''s the cclient list

 > ffffff0153f3a738::print mac_impl_t mi_clients_list | ::list 
mac_client_impl_t mci_client_next | ::print mac_client_impl_t mci_flent 
| ::print flow_entry_t fe_tx_srs
fe_tx_srs = 0xffffffcc53dc82c0
fe_tx_srs = 0

And the mac src (TX) is freed already by the pinned thread perhaps.

 > 0xffffffcc53dc82c0::print mac_soft_ring_set_t srs_lock
{
     srs_lock._opaque = [ 0xdeadbeefdeadbeef ]
}
 >

I don''t see any detach in progress so my driver didn''t call 
mac_unregister while interrupt was still active.

 From test09 logs :

stdout| Configuring nfs . DONE
stdout| Distributing the test binary . . DONE
stdout| Configuring mount points . . DONE
stdout| Starting Corrupt . . DONE
stdout| Corrupt Started.  Running for 600 seconds
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed
stdout| trunk_switch: aggr1 deleted
stdout| trunk_switch: aggr1 plumbed


I''m running snv_113.

Thanks,
--
http://saurabhslr.blogspot.com/

rajagopal kunhappan

2009-May-21 19:02 UTC

head link

[crossbow-discuss] Panic in mac:mac_tx_srs_wakeup on freed flow control entry

Saurabh,

If you have time, take a look at other NICs and see why it doesn''t 
happen there. Otherwise feel free to file a bug and one of us will get 
back to you. At this time, everyone is chasing one hot issue or other.

Thanks,
-krgopi
> Hi --
> 
> I''m seeing a panic while running NICDRV. It''s happening
when running
> test09 (Nemo_Trunk) of the testsuite. I didn''t see this before
though.
> 
> Just want to check if it''s my problem or not. It does seem like a
race
> between TX interrupt calling mac_tx_update() and thread removing TX src 
> (mac src client).
> 
>  > $C
> ffffff0003ec59f0 vpanic()
> ffffff0003ec5a20 mutex_panic+0x73(fffffffffb92d478, ffffffcc53dc82c0)
> ffffff0003ec5a80 mutex_vector_enter+0x446(ffffffcc53dc82c0)
> ffffff0003ec5ad0 mac_tx_srs_wakeup+0x2b(ffffffcc53dc82c0, 0)
> ffffff0003ec5b10 i_mac_tx_srs_notify+0x79(ffffff0153f3a738, 0)
> ffffff0003ec5b30 mac_tx_update+0x18(ffffff0153f3a738)
> ffffff0003ec5b60 bfe_tx_done+0x34(ffffff0153b0dd00, 1000000)
> ffffff0003ec5bb0 bfe_interrupt+0xbb(ffffff0153b0dd00, 0)
> ffffff0003ec5c00 av_dispatch_autovect+0x7c(15)
> ffffff0003ec5c40 dispatch_hardint+0x33(15, 0)
> ffffff00045888d0 switch_sp_and_call+0x13()
> ffffff0004588920 do_interrupt+0x9e(ffffff0004588930, ffffffcc53dc88b8)
> ffffff0004588930 _interrupt+0xba()
> ffffff0004588a30 mac_srs_free+0x5a(ffffffcc53dc82c0)
> ffffff0004588a90 mac_srs_group_teardown+0xf5(ffffff6972044130, 
> ffffff9a92f95428
> , 1)
> ffffff0004588af0 mac_datapath_teardown+0x8e(ffffff6972044130, 
> ffffff9a92f95428,
> 1)
> ffffff0004588b40 mac_unicast_remove+0xfa(ffffff6972044130, 
> ffffff84bda00c40)
> ffffff0004588b70 aggr_port_delete+0xc2(ffffff84bd9ea790)
> ffffff0004588be0 aggr_grp_delete+0x1aa(11c)
> ffffff0004588c20 aggr_ioc_delete+0x27(ffffff84c246dd60, 8046f74, 100003,
> ffffff7302534830, ffffff0004588de4)
> ffffff0004588cc0 drv_ioctl+0x137(b200000000, a660002, 8046f74, 100003,
> ffffff7302534830, ffffff0004588de4)
> ffffff0004588d00 cdev_ioctl+0x45(b200000000, a660002, 8046f74, 100003,
> ffffff7302534830, ffffff0004588de4)
> ffffff0004588d40 spec_ioctl+0x83(ffffff01513c7200, a660002, 8046f74, 
> 100003,
> ffffff7302534830, ffffff0004588de4, 0)
> ffffff0004588dc0 fop_ioctl+0x7b(ffffff01513c7200, a660002, 8046f74, 100003,
> ffffff7302534830, ffffff0004588de4, 0)
> ffffff0004588ec0 ioctl+0x18e(3, a660002, 8046f74)
> ffffff0004588f10 _sys_sysenter_post_swapgs+0x14b()
> 
> 
> Here''s the cclient list
> 
>  > ffffff0153f3a738::print mac_impl_t mi_clients_list | ::list 
> mac_client_impl_t mci_client_next | ::print mac_client_impl_t mci_flent 
> | ::print flow_entry_t fe_tx_srs
> fe_tx_srs = 0xffffffcc53dc82c0
> fe_tx_srs = 0
> 
> And the mac src (TX) is freed already by the pinned thread perhaps.
> 
>  > 0xffffffcc53dc82c0::print mac_soft_ring_set_t srs_lock
> {
>     srs_lock._opaque = [ 0xdeadbeefdeadbeef ]
> }
>  >
> 
> I don''t see any detach in progress so my driver didn''t
call
> mac_unregister while interrupt was still active.
> 
>  From test09 logs :
> 
> stdout| Configuring nfs . DONE
> stdout| Distributing the test binary . . DONE
> stdout| Configuring mount points . . DONE
> stdout| Starting Corrupt . . DONE
> stdout| Corrupt Started.  Running for 600 seconds
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> stdout| trunk_switch: aggr1 deleted
> stdout| trunk_switch: aggr1 plumbed
> 
> 
> I''m running snv_113.
> 
> Thanks,
> -- 
> http://saurabhslr.blogspot.com/
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

--

crossbow discuss - May 2009 - Panic in mac:mac_tx_srs_wakeup on freed flow control entry

[crossbow-discuss] Panic in mac:mac_tx_srs_wakeup on freed flow control entry

[crossbow-discuss] Panic in mac:mac_tx_srs_wakeup on freed flow control entry