Rémi BERNIER
2011-Feb-02 10:55 UTC
[Xen-users] suddenly no network, peth0: transmit timed out
On a stable server with an uptime to 3-4 months, i had suddenly no network but I had access to console, i have restart network, ifup eth0 but nothing to do, link peth0 is down !! I have restart server for it works again. On my syslog: Feb 2 09:07:58 xen kernel: NETDEV WATCHDOG: peth0: transmit timed out Feb 2 09:07:58 xen kernel: tg3: peth0: transmit timed out, resetting Feb 2 09:07:58 xen kernel: tg3: DEBUG: MAC_TX_STATUS[00000008] MAC_RX_STATUS[00000000] Feb 2 09:07:58 xen kernel: tg3: DEBUG: RDMAC_STATUS[00000000] WDMAC_STATUS[00000000] Feb 2 09:07:58 xen kernel: tg3: tg3_stop_block timed out, ofs=1400 enable_bit=2 Feb 2 09:07:58 xen kernel: tg3: peth0: Link is down. Feb 2 09:07:58 xen kernel: eth0: port 1(peth0) entering disabled state I have search on google, user-list but I have not found any solution Maybe disabled acpi ?? My server is on debian lenny, xen 3.4.2 with linux-2.6.18-xen. I have activate iommu for a tape drive on my domU win2k3 If you have met this problem I listen you !! Thanks in advance for your answer !! Regards, Rémi _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Rémi BERNIER
2011-Apr-12 08:37 UTC
Re: [Xen-users] suddenly no network, peth0: transmit timed out
I had the same problem today. My server doesn''t respond on ssh, ping ... but i have access on console. This is the logs of my kernel: Apr 12 09:13:10 svxen kernel: eth0: port 2(vif2.0) entering disabled state Apr 12 09:13:10 svxen kernel: eth0: port 1(peth0) entering disabled state Apr 12 09:13:50 svxen kernel: eth0: port 2(vif2.0) entering learning state Apr 12 09:13:50 svxen kernel: eth0: port 1(peth0) entering learning state Apr 12 09:13:50 svxen kernel: eth0: topology change detected, propagating Apr 12 09:13:50 svxen kernel: eth0: port 2(vif2.0) entering forwarding state Apr 12 09:13:50 svxen kernel: eth0: topology change detected, propagating Apr 12 09:13:50 svxen kernel: eth0: port 1(peth0) entering forwarding state Apr 12 09:14:01 svxen kernel: eth0: no IPv6 routers present Apr 12 09:14:15 svxen kernel: NETDEV WATCHDOG: peth0: transmit timed out Apr 12 09:14:15 svxen kernel: tg3: peth0: transmit timed out, resetting Apr 12 09:14:15 svxen kernel: tg3: DEBUG: MAC_TX_STATUS[00000008] MAC_RX_STATUS[00000000] Apr 12 09:14:15 svxen kernel: tg3: DEBUG: RDMAC_STATUS[00000000] WDMAC_STATUS[00000000] Apr 12 09:14:15 svxen kernel: tg3: peth0: Link is down. Anyone have met this problem ? My network card is: 05:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5722 Gigabit Ethernet PCI Express modinfo tg3 filename: /lib/modules/2.6.18.8/kernel/drivers/net/tg3.ko version: 3.99k license: GPL description: Broadcom Tigon3 ethernet driver author: David S. Miller (davem@redhat.com) and Jeff Garzik (jgarzik@pobox.com) srcversion: 1F42BAC68F1166C6BAD2BF4 alias: pci:v0000106Bd00001645sv*sd*bc*sc*i* depends: vermagic: 2.6.18.8 SMP mod_unload Xen gcc-4.3 parm: tg3_debug:Tigon3 bitmapped debugging message enable value (int) Thanks in advance for your answer. Regards, Rémi On a stable server with an uptime to 3-4 months, i had suddenly no network but I had access to console, i have restart network, ifup eth0 but nothing to do, link peth0 is down !! I have restart server for it works again. On my syslog: Feb 2 09:07:58 xen kernel: NETDEV WATCHDOG: peth0: transmit timed out Feb 2 09:07:58 xen kernel: tg3: peth0: transmit timed out, resetting Feb 2 09:07:58 xen kernel: tg3: DEBUG: MAC_TX_STATUS[00000008] MAC_RX_STATUS[00000000] Feb 2 09:07:58 xen kernel: tg3: DEBUG: RDMAC_STATUS[00000000] WDMAC_STATUS[00000000] Feb 2 09:07:58 xen kernel: tg3: tg3_stop_block timed out, ofs=1400 enable_bit=2 Feb 2 09:07:58 xen kernel: tg3: peth0: Link is down. Feb 2 09:07:58 xen kernel: eth0: port 1(peth0) entering disabled state I have search on google, user-list but I have not found any solution Maybe disabled acpi ?? My server is on debian lenny, xen 3.4.2 with linux-2.6.18-xen. I have activate iommu for a tape drive on my domU win2k3 If you have met this problem I listen you !! Thanks in advance for your answer !! Regards, Rémi_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Todd Deshane
2011-Apr-13 05:20 UTC
Re: [Xen-users] suddenly no network, peth0: transmit timed out
On Tue, Apr 12, 2011 at 8:37 AM, Rémi BERNIER <R.BERNIER@grelier.com> wrote:> > I had the same problem today. My server doesn''t respond on ssh, ping ... but > i have access on console. > This is the logs of my kernel: > > Apr 12 09:13:10 svxen kernel: eth0: port 2(vif2.0) entering disabled state > Apr 12 09:13:10 svxen kernel: eth0: port 1(peth0) entering disabled state > Apr 12 09:13:50 svxen kernel: eth0: port 2(vif2.0) entering learning state > Apr 12 09:13:50 svxen kernel: eth0: port 1(peth0) entering learning state > Apr 12 09:13:50 svxen kernel: eth0: topology change detected, propagating > Apr 12 09:13:50 svxen kernel: eth0: port 2(vif2.0) entering forwarding state > Apr 12 09:13:50 svxen kernel: eth0: topology change detected, propagating > Apr 12 09:13:50 svxen kernel: eth0: port 1(peth0) entering forwarding state > Apr 12 09:14:01 svxen kernel: eth0: no IPv6 routers present > Apr 12 09:14:15 svxen kernel: NETDEV WATCHDOG: peth0: transmit timed out > Apr 12 09:14:15 svxen kernel: tg3: peth0: transmit timed out, resetting > Apr 12 09:14:15 svxen kernel: tg3: DEBUG: MAC_TX_STATUS[00000008] > MAC_RX_STATUS[00000000] > Apr 12 09:14:15 svxen kernel: tg3: DEBUG: RDMAC_STATUS[00000000] > WDMAC_STATUS[00000000] > Apr 12 09:14:15 svxen kernel: tg3: peth0: Link is down. > > Anyone have met this problem ? > > My network card is: > 05:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5722 Gigabit > Ethernet PCI Express > > modinfo tg3 > filename: /lib/modules/2.6.18.8/kernel/drivers/net/tg3.ko > version: 3.99k > license: GPL > description: Broadcom Tigon3 ethernet driver > author: David S. Miller (davem@redhat.com) and Jeff Garzik > (jgarzik@pobox.com) > srcversion: 1F42BAC68F1166C6BAD2BF4 > alias: pci:v0000106Bd00001645sv*sd*bc*sc*i* > depends: > vermagic: 2.6.18.8 SMP mod_unload Xen gcc-4.3 > parm: tg3_debug:Tigon3 bitmapped debugging message enable value > (int) >Have you tried disabling checksumming? see: http://xen.markmail.org/search/?q="ethtool+-K"#query:%22ethtool%20-K%22+page:1+mid:ozu24u7azscw2zwf+state:results> > Thanks in advance for your answer. > Regards, > Rémi > > > > > > > On a stable server with an uptime to 3-4 months, i had suddenly no network > but I had access to console, i have restart network, ifup eth0 but nothing > to do, link peth0 is down !! I have restart server for it works again. > > On my syslog: > Feb 2 09:07:58 xen kernel: NETDEV WATCHDOG: peth0: transmit timed out > Feb 2 09:07:58 xen kernel: tg3: peth0: transmit timed out, resetting > Feb 2 09:07:58 xen kernel: tg3: DEBUG: MAC_TX_STATUS[00000008] > MAC_RX_STATUS[00000000] > Feb 2 09:07:58 xen kernel: tg3: DEBUG: RDMAC_STATUS[00000000] > WDMAC_STATUS[00000000] > Feb 2 09:07:58 xen kernel: tg3: tg3_stop_block timed out, ofs=1400 > enable_bit=2 > Feb 2 09:07:58 xen kernel: tg3: peth0: Link is down. > Feb 2 09:07:58 xen kernel: eth0: port 1(peth0) entering disabled state > > I have search on google, user-list but I have not found any solution > Maybe disabled acpi ?? My server is on debian lenny, xen 3.4.2 with > linux-2.6.18-xen. I have activate iommu for a tape drive on my domU win2k3 > > If you have met this problem I listen you !! > Thanks in advance for your answer !! > Regards, > > Rémi_______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >-- Todd Deshane http://www.linkedin.com/in/deshantm http://www.xen.org/products/cloudxen.html _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Rémi BERNIER
2011-Apr-15 08:16 UTC
Re: [Xen-users] suddenly no network, peth0: transmit timed out
Thanks Todd for your answer ! I have seen your link, it''s very interesting !! For the next time, I will try the command line: ethtool -K eth0 tx off And if it''s works fine, i will add this line in the file: /etc/rc.local Regards, Rémi On Tue, Apr 12, 2011 at 8:37 AM, Rémi BERNIER <R.BERNIER@grelier.com> wrote:> > I had the same problem today. My server doesn''t respond on ssh, ping ...but> i have access on console. > This is the logs of my kernel: > > Apr 12 09:13:10 svxen kernel: eth0: port 2(vif2.0) entering disabledstate> Apr 12 09:13:10 svxen kernel: eth0: port 1(peth0) entering disabledstate> Apr 12 09:13:50 svxen kernel: eth0: port 2(vif2.0) entering learningstate> Apr 12 09:13:50 svxen kernel: eth0: port 1(peth0) entering learningstate> Apr 12 09:13:50 svxen kernel: eth0: topology change detected,propagating> Apr 12 09:13:50 svxen kernel: eth0: port 2(vif2.0) entering forwardingstate> Apr 12 09:13:50 svxen kernel: eth0: topology change detected,propagating> Apr 12 09:13:50 svxen kernel: eth0: port 1(peth0) entering forwardingstate> Apr 12 09:14:01 svxen kernel: eth0: no IPv6 routers present > Apr 12 09:14:15 svxen kernel: NETDEV WATCHDOG: peth0: transmit timed out > Apr 12 09:14:15 svxen kernel: tg3: peth0: transmit timed out, resetting > Apr 12 09:14:15 svxen kernel: tg3: DEBUG: MAC_TX_STATUS[00000008] > MAC_RX_STATUS[00000000] > Apr 12 09:14:15 svxen kernel: tg3: DEBUG: RDMAC_STATUS[00000000] > WDMAC_STATUS[00000000] > Apr 12 09:14:15 svxen kernel: tg3: peth0: Link is down. > > Anyone have met this problem ? > > My network card is: > 05:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5722Gigabit> Ethernet PCI Express > > modinfo tg3 > filename: /lib/modules/2.6.18.8/kernel/drivers/net/tg3.ko > version: 3.99k > license: GPL > description: Broadcom Tigon3 ethernet driver > author: David S. Miller (davem@redhat.com) and Jeff Garzik > (jgarzik@pobox.com) > srcversion: 1F42BAC68F1166C6BAD2BF4 > alias: pci:v0000106Bd00001645sv*sd*bc*sc*i* > depends: > vermagic: 2.6.18.8 SMP mod_unload Xen gcc-4.3 > parm: tg3_debug:Tigon3 bitmapped debugging message enablevalue> (int) >Have you tried disabling checksumming? see: http://xen.markmail.org/search/?q"ethtool+-K"#query:%22ethtool%20-K%22+page:1+mid:ozu24u7azscw2zwf+state:results> > Thanks in advance for your answer. > Regards, > Rémi > > > > > > > On a stable server with an uptime to 3-4 months, i had suddenly nonetwork> but I had access to console, i have restart network, ifup eth0 butnothing> to do, link peth0 is down !! I have restart server for it works again. > > On my syslog: > Feb 2 09:07:58 xen kernel: NETDEV WATCHDOG: peth0: transmit timed out > Feb 2 09:07:58 xen kernel: tg3: peth0: transmit timed out, resetting > Feb 2 09:07:58 xen kernel: tg3: DEBUG: MAC_TX_STATUS[00000008] > MAC_RX_STATUS[00000000] > Feb 2 09:07:58 xen kernel: tg3: DEBUG: RDMAC_STATUS[00000000] > WDMAC_STATUS[00000000] > Feb 2 09:07:58 xen kernel: tg3: tg3_stop_block timed out, ofs=1400 > enable_bit=2 > Feb 2 09:07:58 xen kernel: tg3: peth0: Link is down. > Feb 2 09:07:58 xen kernel: eth0: port 1(peth0) entering disabled state > > I have search on google, user-list but I have not found any solution > Maybe disabled acpi ?? My server is on debian lenny, xen 3.4.2 with > linux-2.6.18-xen. I have activate iommu for a tape drive on my domUwin2k3> > If you have met this problem I listen you !! > Thanks in advance for your answer !! > Regards, > > Rémi_______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >-- Todd Deshane http://www.linkedin.com/in/deshantm http://www.xen.org/products/cloudxen.html _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Rémi BERNIER
2011-Aug-10 06:32 UTC
[Xen-users] eth0: port 1(peth0) entering disabled state
I have always the same problem every two-three weeks. I am now with the kernel 2.6.26-2-xen-amd64 and xen 3.4.2: # lsb_release -a Distributor ID: Debian Description: Debian GNU/Linux 5.0.8 (lenny) Release: 5.0.8 Codename: lenny This is the log on console, after this, i have access on the console server but no network on all VMs and the physical server: Aug 10 06:43:00 xxxx kernel: [1121060.140412] NETDEV WATCHDOG: peth0: transmit timed out Aug 10 06:43:00 xxxx kernel: [1121060.140461] tg3: peth0: transmit timed out, resetting Aug 10 06:43:00 xxxx kernel: [1121060.140502] tg3: DEBUG: MAC_TX_STATUS[00000008] MAC_RX_STATUS[00000000] Aug 10 06:43:00 xxxx kernel: [1121060.140541] tg3: DEBUG: RDMAC_STATUS[00000000] WDMAC_STATUS[00000000] Aug 10 06:43:00 xxxx kernel: [1121060.140585] ------------[ cut here ]------------ Aug 10 06:43:00 xxxx kernel: [1121060.140619] WARNING: at net/sched/sch_generic.c:222 dev_watchdog+0xa6/0x119() Aug 10 06:43:00 xxxx kernel: [1121060.140672] Modules linked in: mptctl nls_utf8 cifs nls_base tun nf_conntrack_ipv4 xt_state nf_conntrack xt_physdev iptable_filter ip_tables x_tables bridge ipmi_devintf ipmi_si ipmi_msghandler ipv6 fuse xenscsi xen_scsibk loop shpchp pcspkr pci_hotplug button dcdbas evdev ext3 jbd mbcache dm_mirror dm_log dm_snapshot dm_mod sd_mod usbhid hid ff_memless ahci tg3 libata dock mptsas mptscsih mptbase scsi_transport_sas scsi_mod ehci_hcd thermal processor fan thermal_sys [last unloaded: scsi_wait_scan] Aug 10 06:43:00 xxxx kernel: [1121060.141170] Pid: 0, comm: swapper Not tainted 2.6.26-2-xen-amd64 #1 Aug 10 06:43:00 xxxx kernel: [1121060.141206] Aug 10 06:43:00 xxxx kernel: [1121060.141207] Call Trace: Aug 10 06:43:00 xxxx kernel: [1121060.141258] <IRQ> [<ffffffff8022cb3e>] warn_on_slowpath+0x51/0x78 Aug 10 06:43:00 xxxx kernel: [1121060.141312] [<ffffffff8023f716>] autoremove_wake_function+0x9/0x2e Aug 10 06:43:00 xxxx kernel: [1121060.141351] [<ffffffff80221555>] __wake_up_common+0x41/0x74 Aug 10 06:43:00 xxxx kernel: [1121060.141391] [<ffffffff80223845>] __wake_up+0x38/0x4f Aug 10 06:43:00 xxxx kernel: [1121060.141432] [<ffffffff8023c8e5>] __queue_work+0x23/0x33 Aug 10 06:43:00 xxxx kernel: [1121060.141468] [<ffffffff803d8915>] dev_watchdog+0x0/0x119 Aug 10 06:43:00 xxxx kernel: [1121060.141504] [<ffffffff803d8915>] dev_watchdog+0x0/0x119 Aug 10 06:43:00 xxxx kernel: [1121060.141540] [<ffffffff803d89bb>] dev_watchdog+0xa6/0x119 Aug 10 06:43:00 xxxx kernel: [1121060.141577] [<ffffffff802356d3>] run_timer_softirq+0x190/0x237 Aug 10 06:43:00 xxxx kernel: [1121060.141618] [<ffffffff80231cc4>] __do_softirq+0x77/0x103 Aug 10 06:43:00 xxxx kernel: [1121060.141657] [<ffffffff8020c13c>] call_softirq+0x1c/0x28 Aug 10 06:43:00 xxxx kernel: [1121060.141695] [<ffffffff8020e092>] do_softirq+0x55/0xbb Aug 10 06:43:00 xxxx kernel: [1121060.141731] [<ffffffff8020e175>] do_IRQ+0x7d/0x9a Aug 10 06:43:00 xxxx kernel: [1121060.141768] [<ffffffff8037e218>] evtchn_do_upcall+0x13c/0x1fc Aug 10 06:43:00 xxxx kernel: [1121060.141809] [<ffffffff8020bbde>] do_hypervisor_callback+0x1e/0x30 Aug 10 06:43:00 xxxx kernel: [1121060.141845] <EOI> [<ffffffff8020e79d>] xen_safe_halt+0x90/0xa6 Aug 10 06:43:00 xxxx kernel: [1121060.141900] [<ffffffff8020a0ce>] xen_idle+0x2e/0x66 Aug 10 06:43:00 xxxx kernel: [1121060.141935] [<ffffffff80209d49>] cpu_idle+0x97/0xb9 Aug 10 06:43:00 xxxx kernel: [1121060.141976] Aug 10 06:43:00 xxxx kernel: [1121060.142002] ---[ end trace 3ce4835172a40db7 ]--- Aug 10 06:43:00 xxxx kernel: [1121060.171016] tg3: peth0: Link is down. Aug 10 06:43:00 xxxx kernel: [1121060.248834] eth0: port 1(peth0) entering disabled state Thanks in advance for your answer !! Regards, Rémi _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Diego Augusto Molina
2011-Aug-10 06:57 UTC
Re: [Xen-users] eth0: port 1(peth0) entering disabled state
Have you tried testing the hardware? Look at the general load of your systems: can you find any pattern between one or more processes and the link breaking? If so, the hardware or device driver may not respond correctly to certain loads (though the tg3 is a longly tested module). Can you check with another NIC? (e.g. add a NIC and change udev rules so that it becomes the new eth0/peth0). Is it possible for you to switch to a newer kernel and test? Also check the patchcords and the phisical network. Have you performed any of these or other tests? This is just to cover basic stuff (don''t get angry at me). -- Diego Augusto Molina diegoaugustomolina@gmail.com ES: Por favor, evite adjuntar documentos de Microsoft Office. Serán desestimados. EN: Please, avoid attaching Microsoft Office documents. They shall be discarded. LINK: http://www.gnu.org/philosophy/no-word-attachments.html _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Rémi BERNIER
2011-Aug-10 07:14 UTC
Re: [Xen-users] eth0: port 1(peth0) entering disabled state
Thanks for your answer Diego, I haven''t testing another NIC but my server is a Dell PowerEdge T110 who just have 1 year and 6 months.. Before, i was on the kernel 2.6.18-xen but i had the same problem. I will try to check the patchcords and the phisical network with another NIC. Thanks for your advice, (I''m not angry with you) Rémi Have you tried testing the hardware? Look at the general load of your systems: can you find any pattern between one or more processes and the link breaking? If so, the hardware or device driver may not respond correctly to certain loads (though the tg3 is a longly tested module). Can you check with another NIC? (e.g. add a NIC and change udev rules so that it becomes the new eth0/peth0). Is it possible for you to switch to a newer kernel and test? Also check the patchcords and the phisical network. Have you performed any of these or other tests? This is just to cover basic stuff (don''t get angry at me). -- Diego Augusto Molina diegoaugustomolina@gmail.com ES: Por favor, evite adjuntar documentos de Microsoft Office. Serán desestimados. EN: Please, avoid attaching Microsoft Office documents. They shall be discarded. LINK: http://www.gnu.org/philosophy/no-word-attachments.html _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Diego Augusto Molina
2011-Aug-10 08:06 UTC
Re: [Xen-users] eth0: port 1(peth0) entering disabled state
2011/8/10, Rémi BERNIER <R.BERNIER@grelier.com>:> Thanks for your answer Diego, > I haven''t testing another NIC but my server is a Dell PowerEdge T110 who > just have 1 year and 6 months.. > Before, i was on the kernel 2.6.18-xen but i had the same problem. > I will try to check the patchcords and the phisical network with another > NIC. > Thanks for your advice, (I''m not angry with you) > > RémiReally? I''ve got assigned the administration of a server exactly like that. I installed it about 6-8 months when it came to the office (bright new). I use XEN 3.4.2 with xenified Linux kernel 2.6.34-r4 (the release part may be particular to my distro) for Gentoo/Linux distro up to date and never had that problem before. Note that on this distro you compile from source _everything_, including the Linux kernel, the XEN hypervisor and the XEN admin tools (so we probably don''t have much in common), but I may help you compiling a kernel of that version (I don''t have it right now but I will in a couple of hours when I got to job, it''s 5 in the morning here and I''m still studying). Of course, the first thing to do would be to check the cables because a longer downtime for kernel upgrade may be unnecessary. -- Diego Augusto Molina diegoaugustomolina@gmail.com ES: Por favor, evite adjuntar documentos de Microsoft Office. Serán desestimados. EN: Please, avoid attaching Microsoft Office documents. They shall be discarded. LINK: http://www.gnu.org/philosophy/no-word-attachments.html _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users