thr3ads.net - Xen devel - [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out [Nov 2010]

If this information is useful, please help other people find it:
Share via:

Qiangmin Lin

2010-Nov-04 03:23 UTC

[Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

Step 1.

CPU:Intel(R) Xeon(R) CPU E5620  @ 2.40GHz * 2
Nic:Broadcom 5715S * 6

Kernel:2.6.32.12-0.7-xen
Xen:3.4.2

Step 2.

A timeout error appears and dumps the following stack, after about a week.

we change Nic Intel 82576 and 82571, the timeout error as follow too.


[ 8385.279738] ------------[ cut here ]------------

[ 8385.279749] WARNING: at
/usr/src/linux-2.6.32.12-0.7/net/sched/sch_generic.c:261
dev_watchdog+0x2a5/0x2c0()

[ 8385.279751] Hardware name:          

[ 8385.279753] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

[ 8385.279755] Modules linked in: tun(N) iptable_mangle(N) xt_physdev(N)
xt_pkttype(N) ipt_MASQUERADE(N) iptable_nat(N) nf_nat(N) xt_tcpudp(N) bridge(N)
gntdev(N) netbk(N) blkbk(N) blkback_pagemap(N) blktap(N) domctl(N) xenbus_be(N)
ipmi_devintf(N) ipmi_si(N) ipmi_msghandler(N) cryptomgr(N) aead(N) pcompress(N)
crypto_blkcipher(N) crc32c(N) crypto_hash(N) crypto_algapi(N) iscsi_tcp(N)
libiscsi_tcp(N) libiscsi(N) scsi_transport_iscsi(N) 8021q(N) garp(N) stp(N)
llc(N) bonding(N) microcode(N) kbox_netdev(N) dl_kbox(N) binfmt_misc(N)
ip6t_REJECT(N) nf_conntrack_ipv6(N) ip6table_raw(N) xt_NOTRACK(N) ipt_REJECT(N)
xt_state(N) iptable_raw(N) iptable_filter(N) ip6table_mangle(N)
nf_conntrack_netbios_ns(N) nf_conntrack_ipv4(N) nf_conntrack(N)
nf_defrag_ipv4(N) ip_tables(N) ip6table_filter(N) ip6_tables(N) x_tables(N)
ipv6(N) usbhid(N) hid(N) loop(N) dm_mod(N) tpm_tis(N) 8250_pnp(N) tg3(N) tpm(N)
iTCO_wdt(N) tpm_bios(N) pcspkr(N) shpchp(N) pci_hotplug(N)
iTCO_vendor_support(N) serio_raw(N) mptctl(N) i2c_i801(N) 8250(N) i2c_core(N)
serial_core(N) button(N) uhci_hcd(N) ehci_hcd(N) usbcore(N) xenblk(N) cdrom(N)
xennet(N) upVhba edd(N) fan(N) thermal(N) processor(N) thermal_sys(N) hwmon(N)
ata_piix(N) libata(N) mptsas(N) mptscsih(N) mptbase(N) scsi_transport_sas(N)
upUpper sg(N) sd_mod(N) crc_t10dif(N) scsi_mod(N)

[ 8385.279817] Supported: Yes

[ 8385.279820] Pid: 0, comm: swapper Tainted: G          N  2.6.32.12-0.7-xen #1

[ 8385.279822] Call Trace:

[ 8385.279840]  [<ffffffff80009a75>] dump_trace+0x65/0x180

[ 8385.279849]  [<ffffffff8036d496>] dump_stack+0x69/0x73

[ 8385.279858]  [<ffffffff8003d2e4>] warn_slowpath_common+0x74/0xd0

[ 8385.279863]  [<ffffffff8003d390>] warn_slowpath_fmt+0x40/0x50

[ 8385.279867]  [<ffffffff802e5bb5>] dev_watchdog+0x2a5/0x2c0

[ 8385.279875]  [<ffffffff800494e2>] run_timer_softirq+0x1b2/0x2e0

[ 8385.279881]  [<ffffffff80043a3e>] __do_softirq+0xde/0x1a0

[ 8385.279886]  [<ffffffff80007f3c>] call_softirq+0x1c/0x30

[ 8385.279892]  [<ffffffff80009535>] do_softirq+0xa5/0xe0

[ 8385.279899]  [<ffffffff802746e2>] evtchn_do_upcall+0x212/0x2a0

[ 8385.279905]  [<ffffffff8000799e>] do_hypervisor_callback+0x1e/0x30

[ 8385.279913]  [<ffffffff800033aa>] 0xffffffff800033aa

[ 8385.279923]  [<ffffffff8000a826>] xen_safe_halt+0xb6/0x130

[ 8385.279928]  [<ffffffff8000de1d>] xen_idle+0x5d/0x70

[ 8385.279933]  [<ffffffff800065cf>] cpu_idle+0x5f/0xa0

[ 8385.279941]  [<ffffffff8063bbd5>] start_kernel+0x374/0x443

[ 8385.279945] ---[ end trace 38582b64ba009d90 ]---

[ 8385.279949] pci 0000:04:00.0: eth2: transmit timed out, resetting

[ 8385.279953] pci 0000:04:00.0: eth2: DEBUG: MAC_TX_STATUS[00000008]
MAC_RX_STATUS[00000000]

[ 8385.279957] pci 0000:04:00.0: eth2: DEBUG: RDMAC_STATUS[00000000]
WDMAC_STATUS[00000000]

[ 8385.279962] pci 0000:04:00.0: eth3: transmit timed out, resetting

[ 8385.279966] pci 0000:04:00.0: eth3: DEBUG: MAC_TX_STATUS[00000008]
MAC_RX_STATUS[00000000]

[ 8385.279970] pci 0000:04:00.0: eth3: DEBUG: RDMAC_STATUS[00000000]
WDMAC_STATUS[00000000]

[ 8385.296378] pci 0000:04:00.0: eth2: Link is down

[ 8385.334989] pci 0000:04:00.0: eth3: Link is down

Please let me know if you need further information.
So perhaps you can help.

Many thanks in advance, 
best regards,

qiangmin lin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

wei song

2010-Nov-04 03:37 UTC

head link

Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

It''s seems that a timeout take place at tx action. What''s the
nic driver
version you used for Broadcom?  In addition, what''s the parameters
added
when insmod nic driver.

-James (Song Wei)

2010/11/4 Qiangmin Lin <linqiangmin@huawei.com>
>  Step 1.
>
> CPU:Intel(R) Xeon(R) CPU E5620  @ 2.40GHz * 2
> Nic:Broadcom 5715S * 6
>
> Kernel:2.6.32.12-0.7-xen
> Xen:3.4.2
>
> Step 2.
>
> A timeout error appears and dumps the following stack, after about a week.
>
> we change Nic Intel 82576 and 82571, the timeout error as follow too.
>
>
> [ 8385.279738] ------------[ cut here ]------------
>
> [ 8385.279749] WARNING: at
> /usr/src/linux-2.6.32.12-0.7/net/sched/sch_generic.c:261
> dev_watchdog+0x2a5/0x2c0()
>
> [ 8385.279751] Hardware name:
>
> [ 8385.279753] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out
>
> [ 8385.279755] Modules linked in: tun(N) iptable_mangle(N) xt_physdev(N)
> xt_pkttype(N) ipt_MASQUERADE(N) iptable_nat(N) nf_nat(N) xt_tcpudp(N)
> bridge(N) gntdev(N) netbk(N) blkbk(N) blkback_pagemap(N) blktap(N)
domctl(N)
> xenbus_be(N) ipmi_devintf(N) ipmi_si(N) ipmi_msghandler(N) cryptomgr(N)
> aead(N) pcompress(N) crypto_blkcipher(N) crc32c(N) crypto_hash(N)
> crypto_algapi(N) iscsi_tcp(N) libiscsi_tcp(N) libiscsi(N)
> scsi_transport_iscsi(N) 8021q(N) garp(N) stp(N) llc(N) bonding(N)
> microcode(N) kbox_netdev(N) dl_kbox(N) binfmt_misc(N) ip6t_REJECT(N)
> nf_conntrack_ipv6(N) ip6table_raw(N) xt_NOTRACK(N) ipt_REJECT(N)
xt_state(N)
> iptable_raw(N) iptable_filter(N) ip6table_mangle(N)
> nf_conntrack_netbios_ns(N) nf_conntrack_ipv4(N) nf_conntrack(N)
> nf_defrag_ipv4(N) ip_tables(N) ip6table_filter(N) ip6_tables(N) x_tables(N)
> ipv6(N) usbhid(N) hid(N) loop(N) dm_mod(N) tpm_tis(N) 8250_pnp(N) tg3(N)
> tpm(N) iTCO_wdt(N) tpm_bios(N) pcspkr(N) shpchp(N) pci_hotplug(N)
> iTCO_vendor_support(N) serio_raw(N) mptctl(N) i2c_i801(N) 8250(N)
> i2c_core(N) serial_core(N) button(N) uhci_hcd(N) ehci_hcd(N) usbcore(N)
> xenblk(N) cdrom(N) xennet(N) upVhba edd(N) fan(N) thermal(N) processor(N)
> thermal_sys(N) hwmon(N) ata_piix(N) libata(N) mptsas(N) mptscsih(N)
> mptbase(N) scsi_transport_sas(N) upUpper sg(N) sd_mod(N) crc_t10dif(N)
> scsi_mod(N)
>
> [ 8385.279817] Supported: Yes
>
> [ 8385.279820] Pid: 0, comm: swapper Tainted: G          N
> 2.6.32.12-0.7-xen #1
>
> [ 8385.279822] Call Trace:
>
> [ 8385.279840]  [<ffffffff80009a75>] dump_trace+0x65/0x180
>
> [ 8385.279849]  [<ffffffff8036d496>] dump_stack+0x69/0x73
>
> [ 8385.279858]  [<ffffffff8003d2e4>] warn_slowpath_common+0x74/0xd0
>
> [ 8385.279863]  [<ffffffff8003d390>] warn_slowpath_fmt+0x40/0x50
>
> [ 8385.279867]  [<ffffffff802e5bb5>] dev_watchdog+0x2a5/0x2c0
>
> [ 8385.279875]  [<ffffffff800494e2>] run_timer_softirq+0x1b2/0x2e0
>
> [ 8385.279881]  [<ffffffff80043a3e>] __do_softirq+0xde/0x1a0
>
> [ 8385.279886]  [<ffffffff80007f3c>] call_softirq+0x1c/0x30
>
> [ 8385.279892]  [<ffffffff80009535>] do_softirq+0xa5/0xe0
>
> [ 8385.279899]  [<ffffffff802746e2>] evtchn_do_upcall+0x212/0x2a0
>
> [ 8385.279905]  [<ffffffff8000799e>] do_hypervisor_callback+0x1e/0x30
>
> [ 8385.279913]  [<ffffffff800033aa>] 0xffffffff800033aa
>
> [ 8385.279923]  [<ffffffff8000a826>] xen_safe_halt+0xb6/0x130
>
> [ 8385.279928]  [<ffffffff8000de1d>] xen_idle+0x5d/0x70
>
> [ 8385.279933]  [<ffffffff800065cf>] cpu_idle+0x5f/0xa0
>
> [ 8385.279941]  [<ffffffff8063bbd5>] start_kernel+0x374/0x443
>
> [ 8385.279945] ---[ end trace 38582b64ba009d90 ]---
>
> [ 8385.279949] pci 0000:04:00.0: eth2: transmit timed out, resetting
>
> [ 8385.279953] pci 0000:04:00.0: eth2: DEBUG: MAC_TX_STATUS[00000008]
> MAC_RX_STATUS[00000000]
>
> [ 8385.279957] pci 0000:04:00.0: eth2: DEBUG: RDMAC_STATUS[00000000]
> WDMAC_STATUS[00000000]
>
> [ 8385.279962] pci 0000:04:00.0: eth3: transmit timed out, resetting
>
> [ 8385.279966] pci 0000:04:00.0: eth3: DEBUG: MAC_TX_STATUS[00000008]
> MAC_RX_STATUS[00000000]
>
> [ 8385.279970] pci 0000:04:00.0: eth3: DEBUG: RDMAC_STATUS[00000000]
> WDMAC_STATUS[00000000]
>
> [ 8385.296378] pci 0000:04:00.0: eth2: Link is down
>
> [ 8385.334989] pci 0000:04:00.0: eth3: Link is down
>
> Please let me know if you need further information.
> So perhaps you can help.
>
> Many thanks in advance,
> best regards,
>
> qiangmin lin
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Qiangmin Lin

2010-Nov-04 04:03 UTC

head link

Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

The nic driver version for Broadcom is tg3 3.106 and 3.110

we set no parameters when insmod nic driver.

tks
QiangMin Lin


  ----- Original Message ----- 
  From: wei song 
  To: Qiangmin Lin 
  Cc: xen-devel@lists.xensource.com 
  Sent: Thursday, November 04, 2010 11:37 AM
  Subject: Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed
out


  It's seems that a timeout take place at tx action. What's the nic
driver version you used for Broadcom?  In addition, what's the parameters
added when insmod nic driver.

  -James (Song Wei)


  2010/11/4 Qiangmin Lin <linqiangmin@huawei.com>

    Step 1.

    CPU:Intel(R) Xeon(R) CPU E5620  @ 2.40GHz * 2
    Nic:Broadcom 5715S * 6

    Kernel:2.6.32.12-0.7-xen
    Xen:3.4.2

    Step 2.

    A timeout error appears and dumps the following stack, after about a week.

    we change Nic Intel 82576 and 82571, the timeout error as follow too.


    [ 8385.279738] ------------[ cut here ]------------

    [ 8385.279749] WARNING: at
/usr/src/linux-2.6.32.12-0.7/net/sched/sch_generic.c:261
dev_watchdog+0x2a5/0x2c0()

    [ 8385.279751] Hardware name:          

    [ 8385.279753] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

    [ 8385.279755] Modules linked in: tun(N) iptable_mangle(N) xt_physdev(N)
xt_pkttype(N) ipt_MASQUERADE(N) iptable_nat(N) nf_nat(N) xt_tcpudp(N) bridge(N)
gntdev(N) netbk(N) blkbk(N) blkback_pagemap(N) blktap(N) domctl(N) xenbus_be(N)
ipmi_devintf(N) ipmi_si(N) ipmi_msghandler(N) cryptomgr(N) aead(N) pcompress(N)
crypto_blkcipher(N) crc32c(N) crypto_hash(N) crypto_algapi(N) iscsi_tcp(N)
libiscsi_tcp(N) libiscsi(N) scsi_transport_iscsi(N) 8021q(N) garp(N) stp(N)
llc(N) bonding(N) microcode(N) kbox_netdev(N) dl_kbox(N) binfmt_misc(N)
ip6t_REJECT(N) nf_conntrack_ipv6(N) ip6table_raw(N) xt_NOTRACK(N) ipt_REJECT(N)
xt_state(N) iptable_raw(N) iptable_filter(N) ip6table_mangle(N)
nf_conntrack_netbios_ns(N) nf_conntrack_ipv4(N) nf_conntrack(N)
nf_defrag_ipv4(N) ip_tables(N) ip6table_filter(N) ip6_tables(N) x_tables(N)
ipv6(N) usbhid(N) hid(N) loop(N) dm_mod(N) tpm_tis(N) 8250_pnp(N) tg3(N) tpm(N)
iTCO_wdt(N) tpm_bios(N) pcspkr(N) shpchp(N) pci_hotplug(N)
iTCO_vendor_support(N) serio_raw(N) mptctl(N) i2c_i801(N) 8250(N) i2c_core(N)
serial_core(N) button(N) uhci_hcd(N) ehci_hcd(N) usbcore(N) xenblk(N) cdrom(N)
xennet(N) upVhba edd(N) fan(N) thermal(N) processor(N) thermal_sys(N) hwmon(N)
ata_piix(N) libata(N) mptsas(N) mptscsih(N) mptbase(N) scsi_transport_sas(N)
upUpper sg(N) sd_mod(N) crc_t10dif(N) scsi_mod(N)

    [ 8385.279817] Supported: Yes

    [ 8385.279820] Pid: 0, comm: swapper Tainted: G          N 
2.6.32.12-0.7-xen #1

    [ 8385.279822] Call Trace:

    [ 8385.279840]  [<ffffffff80009a75>] dump_trace+0x65/0x180

    [ 8385.279849]  [<ffffffff8036d496>] dump_stack+0x69/0x73

    [ 8385.279858]  [<ffffffff8003d2e4>] warn_slowpath_common+0x74/0xd0

    [ 8385.279863]  [<ffffffff8003d390>] warn_slowpath_fmt+0x40/0x50

    [ 8385.279867]  [<ffffffff802e5bb5>] dev_watchdog+0x2a5/0x2c0

    [ 8385.279875]  [<ffffffff800494e2>] run_timer_softirq+0x1b2/0x2e0

    [ 8385.279881]  [<ffffffff80043a3e>] __do_softirq+0xde/0x1a0

    [ 8385.279886]  [<ffffffff80007f3c>] call_softirq+0x1c/0x30

    [ 8385.279892]  [<ffffffff80009535>] do_softirq+0xa5/0xe0

    [ 8385.279899]  [<ffffffff802746e2>] evtchn_do_upcall+0x212/0x2a0

    [ 8385.279905]  [<ffffffff8000799e>] do_hypervisor_callback+0x1e/0x30

    [ 8385.279913]  [<ffffffff800033aa>] 0xffffffff800033aa

    [ 8385.279923]  [<ffffffff8000a826>] xen_safe_halt+0xb6/0x130

    [ 8385.279928]  [<ffffffff8000de1d>] xen_idle+0x5d/0x70

    [ 8385.279933]  [<ffffffff800065cf>] cpu_idle+0x5f/0xa0

    [ 8385.279941]  [<ffffffff8063bbd5>] start_kernel+0x374/0x443

    [ 8385.279945] ---[ end trace 38582b64ba009d90 ]---

    [ 8385.279949] pci 0000:04:00.0: eth2: transmit timed out, resetting

    [ 8385.279953] pci 0000:04:00.0: eth2: DEBUG: MAC_TX_STATUS[00000008]
MAC_RX_STATUS[00000000]

    [ 8385.279957] pci 0000:04:00.0: eth2: DEBUG: RDMAC_STATUS[00000000]
WDMAC_STATUS[00000000]

    [ 8385.279962] pci 0000:04:00.0: eth3: transmit timed out, resetting

    [ 8385.279966] pci 0000:04:00.0: eth3: DEBUG: MAC_TX_STATUS[00000008]
MAC_RX_STATUS[00000000]

    [ 8385.279970] pci 0000:04:00.0: eth3: DEBUG: RDMAC_STATUS[00000000]
WDMAC_STATUS[00000000]

    [ 8385.296378] pci 0000:04:00.0: eth2: Link is down

    [ 8385.334989] pci 0000:04:00.0: eth3: Link is down

    Please let me know if you need further information.
    So perhaps you can help.

    Many thanks in advance, 
    best regards,

    qiangmin lin

    _______________________________________________
    Xen-devel mailing list
    Xen-devel@lists.xensource.com
    http://lists.xensource.com/xen-devel





_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

wei song

2010-Nov-04 05:58 UTC

head link

Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

try with disable_msi = 1 and increase the timeout val in tg3 driver.

-James (Songwei)

2010/11/4 Qiangmin Lin <linqiangmin@huawei.com>
>  The nic driver version for Broadcom is tg3 3.106 and 3.110
>
> we set no parameters when insmod nic driver.
>
> tks
> QiangMin Lin
>
>  ----- Original Message -----
> *From:* wei song <james.songwei@gmail.com>
> *To:* Qiangmin Lin <linqiangmin@huawei.com>
> *Cc:* xen-devel@lists.xensource.com
> *Sent:* Thursday, November 04, 2010 11:37 AM
> *Subject:* Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0
> timed out
>
> It''s seems that a timeout take place at tx action. What''s
the nic driver
> version you used for Broadcom?  In addition, what''s the parameters
added
> when insmod nic driver.
>
> -James (Song Wei)
>
> 2010/11/4 Qiangmin Lin <linqiangmin@huawei.com>
>
>>  Step 1.
>>
>> CPU:Intel(R) Xeon(R) CPU E5620  @ 2.40GHz * 2
>> Nic:Broadcom 5715S * 6
>>
>> Kernel:2.6.32.12-0.7-xen
>> Xen:3.4.2
>>
>> Step 2.
>>
>> A timeout error appears and dumps the following stack, after about a
week.
>>
>> we change Nic Intel 82576 and 82571, the timeout error as follow too.
>>
>>
>> [ 8385.279738] ------------[ cut here ]------------
>>
>> [ 8385.279749] WARNING: at
>> /usr/src/linux-2.6.32.12-0.7/net/sched/sch_generic.c:261
>> dev_watchdog+0x2a5/0x2c0()
>>
>> [ 8385.279751] Hardware name:
>>
>> [ 8385.279753] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out
>>
>> [ 8385.279755] Modules linked in: tun(N) iptable_mangle(N)
xt_physdev(N)
>> xt_pkttype(N) ipt_MASQUERADE(N) iptable_nat(N) nf_nat(N) xt_tcpudp(N)
>> bridge(N) gntdev(N) netbk(N) blkbk(N) blkback_pagemap(N) blktap(N)
domctl(N)
>> xenbus_be(N) ipmi_devintf(N) ipmi_si(N) ipmi_msghandler(N) cryptomgr(N)
>> aead(N) pcompress(N) crypto_blkcipher(N) crc32c(N) crypto_hash(N)
>> crypto_algapi(N) iscsi_tcp(N) libiscsi_tcp(N) libiscsi(N)
>> scsi_transport_iscsi(N) 8021q(N) garp(N) stp(N) llc(N) bonding(N)
>> microcode(N) kbox_netdev(N) dl_kbox(N) binfmt_misc(N) ip6t_REJECT(N)
>> nf_conntrack_ipv6(N) ip6table_raw(N) xt_NOTRACK(N) ipt_REJECT(N)
xt_state(N)
>> iptable_raw(N) iptable_filter(N) ip6table_mangle(N)
>> nf_conntrack_netbios_ns(N) nf_conntrack_ipv4(N) nf_conntrack(N)
>> nf_defrag_ipv4(N) ip_tables(N) ip6table_filter(N) ip6_tables(N)
x_tables(N)
>> ipv6(N) usbhid(N) hid(N) loop(N) dm_mod(N) tpm_tis(N) 8250_pnp(N)
tg3(N)
>> tpm(N) iTCO_wdt(N) tpm_bios(N) pcspkr(N) shpchp(N) pci_hotplug(N)
>> iTCO_vendor_support(N) serio_raw(N) mptctl(N) i2c_i801(N) 8250(N)
>> i2c_core(N) serial_core(N) button(N) uhci_hcd(N) ehci_hcd(N) usbcore(N)
>> xenblk(N) cdrom(N) xennet(N) upVhba edd(N) fan(N) thermal(N)
processor(N)
>> thermal_sys(N) hwmon(N) ata_piix(N) libata(N) mptsas(N) mptscsih(N)
>> mptbase(N) scsi_transport_sas(N) upUpper sg(N) sd_mod(N) crc_t10dif(N)
>> scsi_mod(N)
>>
>> [ 8385.279817] Supported: Yes
>>
>> [ 8385.279820] Pid: 0, comm: swapper Tainted: G          N
>> 2.6.32.12-0.7-xen #1
>>
>> [ 8385.279822] Call Trace:
>>
>> [ 8385.279840]  [<ffffffff80009a75>] dump_trace+0x65/0x180
>>
>> [ 8385.279849]  [<ffffffff8036d496>] dump_stack+0x69/0x73
>>
>> [ 8385.279858]  [<ffffffff8003d2e4>]
warn_slowpath_common+0x74/0xd0
>>
>> [ 8385.279863]  [<ffffffff8003d390>] warn_slowpath_fmt+0x40/0x50
>>
>> [ 8385.279867]  [<ffffffff802e5bb5>] dev_watchdog+0x2a5/0x2c0
>>
>> [ 8385.279875]  [<ffffffff800494e2>]
run_timer_softirq+0x1b2/0x2e0
>>
>> [ 8385.279881]  [<ffffffff80043a3e>] __do_softirq+0xde/0x1a0
>>
>> [ 8385.279886]  [<ffffffff80007f3c>] call_softirq+0x1c/0x30
>>
>> [ 8385.279892]  [<ffffffff80009535>] do_softirq+0xa5/0xe0
>>
>> [ 8385.279899]  [<ffffffff802746e2>] evtchn_do_upcall+0x212/0x2a0
>>
>> [ 8385.279905]  [<ffffffff8000799e>]
do_hypervisor_callback+0x1e/0x30
>>
>> [ 8385.279913]  [<ffffffff800033aa>] 0xffffffff800033aa
>>
>> [ 8385.279923]  [<ffffffff8000a826>] xen_safe_halt+0xb6/0x130
>>
>> [ 8385.279928]  [<ffffffff8000de1d>] xen_idle+0x5d/0x70
>>
>> [ 8385.279933]  [<ffffffff800065cf>] cpu_idle+0x5f/0xa0
>>
>> [ 8385.279941]  [<ffffffff8063bbd5>] start_kernel+0x374/0x443
>>
>> [ 8385.279945] ---[ end trace 38582b64ba009d90 ]---
>>
>> [ 8385.279949] pci 0000:04:00.0: eth2: transmit timed out, resetting
>>
>> [ 8385.279953] pci 0000:04:00.0: eth2: DEBUG: MAC_TX_STATUS[00000008]
>> MAC_RX_STATUS[00000000]
>>
>> [ 8385.279957] pci 0000:04:00.0: eth2: DEBUG: RDMAC_STATUS[00000000]
>> WDMAC_STATUS[00000000]
>>
>> [ 8385.279962] pci 0000:04:00.0: eth3: transmit timed out, resetting
>>
>> [ 8385.279966] pci 0000:04:00.0: eth3: DEBUG: MAC_TX_STATUS[00000008]
>> MAC_RX_STATUS[00000000]
>>
>> [ 8385.279970] pci 0000:04:00.0: eth3: DEBUG: RDMAC_STATUS[00000000]
>> WDMAC_STATUS[00000000]
>>
>> [ 8385.296378] pci 0000:04:00.0: eth2: Link is down
>>
>> [ 8385.334989] pci 0000:04:00.0: eth3: Link is down
>>
>> Please let me know if you need further information.
>> So perhaps you can help.
>>
>> Many thanks in advance,
>> best regards,
>>
>> qiangmin lin
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Qiangmin Lin

2010-Nov-04 06:32 UTC

head link

Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

I tried it , but there is no effect

QiangMin Lin
  ----- Original Message ----- 
  From: wei song 
  To: Qiangmin Lin 
  Cc: xen-devel@lists.xensource.com 
  Sent: Thursday, November 04, 2010 1:58 PM
  Subject: Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed
out


  try with disable_msi = 1 and increase the timeout val in tg3 driver. 

  -James (Songwei)


  2010/11/4 Qiangmin Lin <linqiangmin@huawei.com>

    The nic driver versionÂ for Broadcom is tg3 3.106 and 3.110
    Â 
    we set noÂ parameters when insmod nic driver.
    Â 
    tks
    QiangMin Lin


      ----- Original Message ----- 
      From: wei song 
      To: Qiangmin Lin 
      Cc: xen-devel@lists.xensource.com 
      Sent: Thursday, November 04, 2010 11:37 AM
      Subject: Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0
timed out


      It's seems that a timeout take place at tx action. What's the nic
driver version you used for Broadcom?Â  In addition, what's the parameters
added when insmod nic driver.

      -James (Song Wei)


      2010/11/4 Qiangmin Lin <linqiangmin@huawei.com>

        Step 1.
        Â 
        CPU:Intel(R) Xeon(R) CPU E5620Â  @ 2.40GHz * 2
        Nic:Broadcom 5715S * 6
        Â 
        Kernel:2.6.32.12-0.7-xen
        Xen:3.4.2
        Â 
        Step 2.
        Â 
        A timeout error appears and dumps the following stack, after about a
week.
        Â 
        we change Nic Intel 82576 and 82571, the timeout error as follow too.
        Â 

        [ 8385.279738] ------------[ cut here ]------------
        Â 
        [ 8385.279749] WARNING: at
/usr/src/linux-2.6.32.12-0.7/net/sched/sch_generic.c:261
dev_watchdog+0x2a5/0x2c0()
        Â 
        [ 8385.279751] Hardware name:Â Â Â Â Â Â Â Â Â  
        Â 
        [ 8385.279753] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out
        Â 
        [ 8385.279755] Modules linked in: tun(N) iptable_mangle(N) xt_physdev(N)
xt_pkttype(N) ipt_MASQUERADE(N) iptable_nat(N) nf_nat(N) xt_tcpudp(N) bridge(N)
gntdev(N) netbk(N) blkbk(N) blkback_pagemap(N) blktap(N) domctl(N) xenbus_be(N)
ipmi_devintf(N) ipmi_si(N) ipmi_msghandler(N) cryptomgr(N) aead(N) pcompress(N)
crypto_blkcipher(N) crc32c(N) crypto_hash(N) crypto_algapi(N) iscsi_tcp(N)
libiscsi_tcp(N) libiscsi(N) scsi_transport_iscsi(N) 8021q(N) garp(N) stp(N)
llc(N) bonding(N) microcode(N) kbox_netdev(N) dl_kbox(N) binfmt_misc(N)
ip6t_REJECT(N) nf_conntrack_ipv6(N) ip6table_raw(N) xt_NOTRACK(N) ipt_REJECT(N)
xt_state(N) iptable_raw(N) iptable_filter(N) ip6table_mangle(N)
nf_conntrack_netbios_ns(N) nf_conntrack_ipv4(N) nf_conntrack(N)
nf_defrag_ipv4(N) ip_tables(N) ip6table_filter(N) ip6_tables(N) x_tables(N)
ipv6(N) usbhid(N) hid(N) loop(N) dm_mod(N) tpm_tis(N) 8250_pnp(N) tg3(N) tpm(N)
iTCO_wdt(N) tpm_bios(N) pcspkr(N) shpchp(N) pci_hotplug(N)
iTCO_vendor_support(N) serio_raw(N) mptctl(N) i2c_i801(N) 8250(N) i2c_core(N)
serial_core(N) button(N) uhci_hcd(N) ehci_hcd(N) usbcore(N) xenblk(N) cdrom(N)
xennet(N) upVhba edd(N) fan(N) thermal(N) processor(N) thermal_sys(N) hwmon(N)
ata_piix(N) libata(N) mptsas(N) mptscsih(N) mptbase(N) scsi_transport_sas(N)
upUpper sg(N) sd_mod(N) crc_t10dif(N) scsi_mod(N)
        Â 
        [ 8385.279817] Supported: Yes
        Â 
        [ 8385.279820] Pid: 0, comm: swapper Tainted: GÂ Â Â Â Â Â Â Â Â  NÂ 
2.6.32.12-0.7-xen #1
        Â 
        [ 8385.279822] Call Trace:
        Â 
        [ 8385.279840]Â  [<ffffffff80009a75>] dump_trace+0x65/0x180
        Â 
        [ 8385.279849]Â  [<ffffffff8036d496>] dump_stack+0x69/0x73
        Â 
        [ 8385.279858]Â  [<ffffffff8003d2e4>]
warn_slowpath_common+0x74/0xd0
        Â 
        [ 8385.279863]Â  [<ffffffff8003d390>] warn_slowpath_fmt+0x40/0x50
        Â 
        [ 8385.279867]Â  [<ffffffff802e5bb5>] dev_watchdog+0x2a5/0x2c0
        Â 
        [ 8385.279875]Â  [<ffffffff800494e2>]
run_timer_softirq+0x1b2/0x2e0
        Â 
        [ 8385.279881]Â  [<ffffffff80043a3e>] __do_softirq+0xde/0x1a0
        Â 
        [ 8385.279886]Â  [<ffffffff80007f3c>] call_softirq+0x1c/0x30
        Â 
        [ 8385.279892]Â  [<ffffffff80009535>] do_softirq+0xa5/0xe0
        Â 
        [ 8385.279899]Â  [<ffffffff802746e2>] evtchn_do_upcall+0x212/0x2a0
        Â 
        [ 8385.279905]Â  [<ffffffff8000799e>]
do_hypervisor_callback+0x1e/0x30
        Â 
        [ 8385.279913]Â  [<ffffffff800033aa>] 0xffffffff800033aa
        Â 
        [ 8385.279923]Â  [<ffffffff8000a826>] xen_safe_halt+0xb6/0x130
        Â 
        [ 8385.279928]Â  [<ffffffff8000de1d>] xen_idle+0x5d/0x70
        Â 
        [ 8385.279933]Â  [<ffffffff800065cf>] cpu_idle+0x5f/0xa0
        Â 
        [ 8385.279941]Â  [<ffffffff8063bbd5>] start_kernel+0x374/0x443
        Â 
        [ 8385.279945] ---[ end trace 38582b64ba009d90 ]---
        Â 
        [ 8385.279949] pci 0000:04:00.0: eth2: transmit timed out, resetting
        Â 
        [ 8385.279953] pci 0000:04:00.0: eth2: DEBUG: MAC_TX_STATUS[00000008]
MAC_RX_STATUS[00000000]
        Â 
        [ 8385.279957] pci 0000:04:00.0: eth2: DEBUG: RDMAC_STATUS[00000000]
WDMAC_STATUS[00000000]
        Â 
        [ 8385.279962] pci 0000:04:00.0: eth3: transmit timed out, resetting
        Â 
        [ 8385.279966] pci 0000:04:00.0: eth3: DEBUG: MAC_TX_STATUS[00000008]
MAC_RX_STATUS[00000000]
        Â 
        [ 8385.279970] pci 0000:04:00.0: eth3: DEBUG: RDMAC_STATUS[00000000]
WDMAC_STATUS[00000000]
        Â 
        [ 8385.296378] pci 0000:04:00.0: eth2: Link is down
        Â 
        [ 8385.334989] pci 0000:04:00.0: eth3: Link is down
        Â 
        Please let me know if you need further information.
        So perhaps you can help.
        Â 
        Many thanks in advance, 
        best regards,

        qiangmin lin

        _______________________________________________
        Xen-devel mailing list
        Xen-devel@lists.xensource.com
        http://lists.xensource.com/xen-devel







_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

wei song

2010-Nov-04 08:22 UTC

head link

Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

disable irqbalance and set the affinity of irq manually

-James (Song Wei)

2010/11/4 Qiangmin Lin <linqiangmin@huawei.com>
>  I tried it , but there is no effect
>
> QiangMin Lin
>
> ----- Original Message -----
> *From:* wei song <james.songwei@gmail.com>
> *To:* Qiangmin Lin <linqiangmin@huawei.com>
> *Cc:* xen-devel@lists.xensource.com
> *Sent:* Thursday, November 04, 2010 1:58 PM
> *Subject:* Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0
> timed out
>
> try with disable_msi = 1 and increase the timeout val in tg3 driver.
>
> -James (Songwei)
>
> 2010/11/4 Qiangmin Lin <linqiangmin@huawei.com>
>
>>  The nic driver versionÂ for Broadcom is tg3 3.106 and 3.110
>> Â
>> we set noÂ parameters when insmod nic driver.
>> Â
>> tks
>> QiangMin Lin
>>
>>  ----- Original Message -----
>> *From:* wei song <james.songwei@gmail.com>
>> *To:* Qiangmin Lin <linqiangmin@huawei.com>
>> *Cc:* xen-devel@lists.xensource.com
>> *Sent:* Thursday, November 04, 2010 11:37 AM
>> *Subject:* Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue
0
>> timed out
>>
>> It''s seems that a timeout take place at tx action.
What''s the nic driver
>> version you used for Broadcom?Â  In addition, what''s the
parameters added
>> when insmod nic driver.
>>
>> -James (Song Wei)
>>
>> 2010/11/4 Qiangmin Lin <linqiangmin@huawei.com>
>>
>>>  Step 1.
>>> Â
>>> CPU:Intel(R) Xeon(R) CPU E5620Â  @ 2.40GHz * 2
>>> Nic:Broadcom 5715S * 6
>>> Â
>>> Kernel:2.6.32.12-0.7-xen
>>> Xen:3.4.2
>>> Â
>>> Step 2.
>>> Â
>>> A timeout error appears and dumps the following stack, after about
a
>>> week.
>>> Â
>>> we change Nic Intel 82576 and 82571, the timeout error as follow
too.
>>> Â
>>>
>>> [ 8385.279738] ------------[ cut here ]------------
>>> Â
>>> [ 8385.279749] WARNING: at
>>> /usr/src/linux-2.6.32.12-0.7/net/sched/sch_generic.c:261
>>> dev_watchdog+0x2a5/0x2c0()
>>> Â
>>> [ 8385.279751] Hardware name:Â Â Â Â Â Â Â Â Â
>>> Â
>>> [ 8385.279753] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed
out
>>> Â
>>> [ 8385.279755] Modules linked in: tun(N) iptable_mangle(N)
xt_physdev(N)
>>> xt_pkttype(N) ipt_MASQUERADE(N) iptable_nat(N) nf_nat(N)
xt_tcpudp(N)
>>> bridge(N) gntdev(N) netbk(N) blkbk(N) blkback_pagemap(N) blktap(N)
domctl(N)
>>> xenbus_be(N) ipmi_devintf(N) ipmi_si(N) ipmi_msghandler(N)
cryptomgr(N)
>>> aead(N) pcompress(N) crypto_blkcipher(N) crc32c(N) crypto_hash(N)
>>> crypto_algapi(N) iscsi_tcp(N) libiscsi_tcp(N) libiscsi(N)
>>> scsi_transport_iscsi(N) 8021q(N) garp(N) stp(N) llc(N) bonding(N)
>>> microcode(N) kbox_netdev(N) dl_kbox(N) binfmt_misc(N)
ip6t_REJECT(N)
>>> nf_conntrack_ipv6(N) ip6table_raw(N) xt_NOTRACK(N) ipt_REJECT(N)
xt_state(N)
>>> iptable_raw(N) iptable_filter(N) ip6table_mangle(N)
>>> nf_conntrack_netbios_ns(N) nf_conntrack_ipv4(N) nf_conntrack(N)
>>> nf_defrag_ipv4(N) ip_tables(N) ip6table_filter(N) ip6_tables(N)
x_tables(N)
>>> ipv6(N) usbhid(N) hid(N) loop(N) dm_mod(N) tpm_tis(N) 8250_pnp(N)
tg3(N)
>>> tpm(N) iTCO_wdt(N) tpm_bios(N) pcspkr(N) shpchp(N) pci_hotplug(N)
>>> iTCO_vendor_support(N) serio_raw(N) mptctl(N) i2c_i801(N) 8250(N)
>>> i2c_core(N) serial_core(N) button(N) uhci_hcd(N) ehci_hcd(N)
usbcore(N)
>>> xenblk(N) cdrom(N) xennet(N) upVhba edd(N) fan(N) thermal(N)
processor(N)
>>> thermal_sys(N) hwmon(N) ata_piix(N) libata(N) mptsas(N) mptscsih(N)
>>> mptbase(N) scsi_transport_sas(N) upUpper sg(N) sd_mod(N)
crc_t10dif(N)
>>> scsi_mod(N)
>>> Â
>>> [ 8385.279817] Supported: Yes
>>> Â
>>> [ 8385.279820] Pid: 0, comm: swapper Tainted: GÂ Â Â Â Â Â Â Â Â 
NÂ
>>> 2.6.32.12-0.7-xen #1
>>> Â
>>> [ 8385.279822] Call Trace:
>>> Â
>>> [ 8385.279840]Â  [<ffffffff80009a75>] dump_trace+0x65/0x180
>>> Â
>>> [ 8385.279849]Â  [<ffffffff8036d496>] dump_stack+0x69/0x73
>>> Â
>>> [ 8385.279858]Â  [<ffffffff8003d2e4>]
warn_slowpath_common+0x74/0xd0
>>> Â
>>> [ 8385.279863]Â  [<ffffffff8003d390>]
warn_slowpath_fmt+0x40/0x50
>>> Â
>>> [ 8385.279867]Â  [<ffffffff802e5bb5>]
dev_watchdog+0x2a5/0x2c0
>>> Â
>>> [ 8385.279875]Â  [<ffffffff800494e2>]
run_timer_softirq+0x1b2/0x2e0
>>> Â
>>> [ 8385.279881]Â  [<ffffffff80043a3e>] __do_softirq+0xde/0x1a0
>>> Â
>>> [ 8385.279886]Â  [<ffffffff80007f3c>] call_softirq+0x1c/0x30
>>> Â
>>> [ 8385.279892]Â  [<ffffffff80009535>] do_softirq+0xa5/0xe0
>>> Â
>>> [ 8385.279899]Â  [<ffffffff802746e2>]
evtchn_do_upcall+0x212/0x2a0
>>> Â
>>> [ 8385.279905]Â  [<ffffffff8000799e>]
do_hypervisor_callback+0x1e/0x30
>>> Â
>>> [ 8385.279913]Â  [<ffffffff800033aa>] 0xffffffff800033aa
>>> Â
>>> [ 8385.279923]Â  [<ffffffff8000a826>]
xen_safe_halt+0xb6/0x130
>>> Â
>>> [ 8385.279928]Â  [<ffffffff8000de1d>] xen_idle+0x5d/0x70
>>> Â
>>> [ 8385.279933]Â  [<ffffffff800065cf>] cpu_idle+0x5f/0xa0
>>> Â
>>> [ 8385.279941]Â  [<ffffffff8063bbd5>]
start_kernel+0x374/0x443
>>> Â
>>> [ 8385.279945] ---[ end trace 38582b64ba009d90 ]---
>>> Â
>>> [ 8385.279949] pci 0000:04:00.0: eth2: transmit timed out,
resetting
>>> Â
>>> [ 8385.279953] pci 0000:04:00.0: eth2: DEBUG:
MAC_TX_STATUS[00000008]
>>> MAC_RX_STATUS[00000000]
>>> Â
>>> [ 8385.279957] pci 0000:04:00.0: eth2: DEBUG:
RDMAC_STATUS[00000000]
>>> WDMAC_STATUS[00000000]
>>> Â
>>> [ 8385.279962] pci 0000:04:00.0: eth3: transmit timed out,
resetting
>>> Â
>>> [ 8385.279966] pci 0000:04:00.0: eth3: DEBUG:
MAC_TX_STATUS[00000008]
>>> MAC_RX_STATUS[00000000]
>>> Â
>>> [ 8385.279970] pci 0000:04:00.0: eth3: DEBUG:
RDMAC_STATUS[00000000]
>>> WDMAC_STATUS[00000000]
>>> Â
>>> [ 8385.296378] pci 0000:04:00.0: eth2: Link is down
>>> Â
>>> [ 8385.334989] pci 0000:04:00.0: eth3: Link is down
>>> Â
>>> Please let me know if you need further information.
>>> So perhaps you can help.
>>> Â
>>> Many thanks in advance,
>>> best regards,
>>>
>>> qiangmin lin
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>>>
>>>
>>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Nov 2010 - NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

[Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out

Re: [Xen-devel] NETDEV WATCHDOG: eth2 (tg3): transmit queue 0 timed out