Am I going about this wrong? It''s quite trivial to deadlock the Fedora kernel. # modprobe netloop ; rmmod netloop unregister_netdevice: waiting for veth3 to become free. Usage count = 1 unregister_netdevice: waiting for veth3 to become free. Usage count = 1 unregister_netdevice: waiting for veth3 to become free. Usage count = 1 ... This is without doing anything else with xend or the other modules. So why/who/what grabbed a hold of that veth? I just did this today on 2.6.20-1.2933 and have done it before with one or more of the 2.6.18 builds. ../C
On Fri, Apr 13, 2007 at 01:52:46PM -0700, Curtis Doty wrote:> Am I going about this wrong? It''s quite trivial to deadlock the Fedora > kernel. > > # modprobe netloop ; rmmod netloop > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > ... > > This is without doing anything else with xend or the other modules. So > why/who/what grabbed a hold of that veth?This is basically a limitation of the netloop module - its not written to allow its removal once loaded. Fortunately we don''t need to worry about fixing this, because in the very near future we will be blasting netloop into oblivion :-) This will make networking in Dom0 much simpler because you won''t have this crazy set of extra devices for Dom0 - just the real eth*, the bridge device & any VIFs for guest VMs. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Sat, Apr 14, 2007 at 12:18:26AM +0100, Daniel P. Berrange wrote:> On Fri, Apr 13, 2007 at 01:52:46PM -0700, Curtis Doty wrote: > > Am I going about this wrong? It''s quite trivial to deadlock the Fedora > > kernel. > > > > # modprobe netloop ; rmmod netloop > > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > > ... > > > > This is without doing anything else with xend or the other modules. So > > why/who/what grabbed a hold of that veth? > > This is basically a limitation of the netloop module - its not written to > allow its removal once loaded. Fortunately we don''t need to worry about > fixing this, because in the very near future we will be blasting netloop > into oblivion :-) This will make networking in Dom0 much simpler because > you won''t have this crazy set of extra devices for Dom0 - just the real > eth*, the bridge device & any VIFs for guest VMs. >Can you tell more about this? Redhat is planning not to use the xen network-bridge script anymore and allow more flexible configuration with the tools provided by the distro? What does "in the very near future" mean.. ? for fc7? fc8? I''m interested in this because I''ve made my own scripts for use in dom0 instead of xen network-bridge.. kind of "add-on" scripts for standard fedora network scripts. Thanks! -- Pasi
On Sat, Apr 14, 2007 at 08:26:16PM +0300, Pasi K?rkk?inen wrote:> On Sat, Apr 14, 2007 at 12:18:26AM +0100, Daniel P. Berrange wrote: > > On Fri, Apr 13, 2007 at 01:52:46PM -0700, Curtis Doty wrote: > > > Am I going about this wrong? It''s quite trivial to deadlock the Fedora > > > kernel. > > > > > > # modprobe netloop ; rmmod netloop > > > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > > > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > > > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > > > ... > > > > > > This is without doing anything else with xend or the other modules. So > > > why/who/what grabbed a hold of that veth? > > > > This is basically a limitation of the netloop module - its not written to > > allow its removal once loaded. Fortunately we don''t need to worry about > > fixing this, because in the very near future we will be blasting netloop > > into oblivion :-) This will make networking in Dom0 much simpler because > > you won''t have this crazy set of extra devices for Dom0 - just the real > > eth*, the bridge device & any VIFs for guest VMs. > > > > Can you tell more about this? Redhat is planning not to use the xen > network-bridge script anymore and allow more flexible configuration with > the tools provided by the distro?For compatability with FC5/6 we need to keep using network-bridge in the default RPM setup. What we''re trying to do is alter the network-bridge script so that it doesn''t need to use the netloop module. Part of this work involves kernel changes to the way the Xen network module transfer data to avoid some potential denial-of-service conditions that arise if you''re not using netloop. So we''ll have a simplified network-bridge, but will also have the option of just setting the config file option ''(network-script /bin/true)'' and then using normal distro networking scripts for bridging/bonding/whatever.> What does "in the very near future" mean.. ? for fc7? fc8?Fedora 7 is the goal.> I''m interested in this because I''ve made my own scripts for use in dom0 > instead of xen network-bridge.. kind of "add-on" scripts for standard fedora > network scripts.You''ll be able to keep using these - netloop will still exist for compatability with any user custom scripts, we simply won''t use it in the default scripts Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Sat, Apr 14, 2007 at 07:12:16PM +0100, Daniel P. Berrange wrote:> On Sat, Apr 14, 2007 at 08:26:16PM +0300, Pasi K?rkk?inen wrote: > > On Sat, Apr 14, 2007 at 12:18:26AM +0100, Daniel P. Berrange wrote: > > > On Fri, Apr 13, 2007 at 01:52:46PM -0700, Curtis Doty wrote: > > > > Am I going about this wrong? It''s quite trivial to deadlock the Fedora > > > > kernel. > > > > > > > > # modprobe netloop ; rmmod netloop > > > > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > > > > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > > > > unregister_netdevice: waiting for veth3 to become free. Usage count = 1 > > > > ... > > > > > > > > This is without doing anything else with xend or the other modules. So > > > > why/who/what grabbed a hold of that veth? > > > > > > This is basically a limitation of the netloop module - its not written to > > > allow its removal once loaded. Fortunately we don''t need to worry about > > > fixing this, because in the very near future we will be blasting netloop > > > into oblivion :-) This will make networking in Dom0 much simpler because > > > you won''t have this crazy set of extra devices for Dom0 - just the real > > > eth*, the bridge device & any VIFs for guest VMs. > > > > > > > Can you tell more about this? Redhat is planning not to use the xen > > network-bridge script anymore and allow more flexible configuration with > > the tools provided by the distro? > > For compatability with FC5/6 we need to keep using network-bridge in the > default RPM setup. What we''re trying to do is alter the network-bridge > script so that it doesn''t need to use the netloop module. Part of this > work involves kernel changes to the way the Xen network module transfer > data to avoid some potential denial-of-service conditions that arise if > you''re not using netloop. > > So we''ll have a simplified network-bridge, but will also have the option > of just setting the config file option ''(network-script /bin/true)'' and > then using normal distro networking scripts for bridging/bonding/whatever. >OK. That''s a good thing. The biggest problem with default xen network-bridge for me is the lack of VLAN support..> > What does "in the very near future" mean.. ? for fc7? fc8? > > Fedora 7 is the goal. > > > I''m interested in this because I''ve made my own scripts for use in dom0 > > instead of xen network-bridge.. kind of "add-on" scripts for standard fedora > > network scripts. > > You''ll be able to keep using these - netloop will still exist for compatability > with any user custom scripts, we simply won''t use it in the default scripts > >OK. My scripts are meant for using VLAN interfaces with xen bridges, and also with bonding. Example: With the normal distro (fc6) network scripts I do: - Reset MAC addresses of ethX devices - Set up bond0 (from ethX+ethY) - Create VLAN interfaces (bond0.xyz) - Configure vethX for dom0 mgmt (including MAC address from physical dev) - Create bridges for VLAN interfaces Then with my custom scripts: - Set NOARP for interfaces - Do the bridge "addif" to connect the VLAN interfaces to corresponding bridges All of that basicly should be doable with the normal fc6 network scripts, but it seems bridging with MAC changes doesn''t seem to work, and there were some problems with NOARP too.. -- Pasi
On Sat, 2007-04-14 at 19:12 +0100, Daniel P. Berrange wrote:> For compatability with FC5/6 we need to keep using network-bridge in the > default RPM setup. What we''re trying to do is alter the network-bridge > script so that it doesn''t need to use the netloop module. Part of this > work involves kernel changes to the way the Xen network module transfer > data to avoid some potential denial-of-service conditions that arise if > you''re not using netloop.Just to elaborate ... The only reason netloop exists is to prevent the netback driver from "seizing up" if packets from DomU, destined for Dom0, are queued up indefinitely in Dom0 socket buffers causing hypervisor resources (grant table entries?) to be exhausted. The netloop solution is to copy packets from DomU just before they are delivered to Dom0''s networking stack. The new solution is for netback to keep track of packets and to periodically copy any packets which have been queued up for too long. That way you don''t need the netloop device at all. Cheers, Mark.
Will this decrease or increase the chance of a domU loosing its network and the dom0 complaining about memory squeezes in the netback driver? On Mon, 16 Apr 2007, Mark McLoughlin wrote:> On Sat, 2007-04-14 at 19:12 +0100, Daniel P. Berrange wrote: > >> For compatability with FC5/6 we need to keep using network-bridge in the >> default RPM setup. What we''re trying to do is alter the network-bridge >> script so that it doesn''t need to use the netloop module. Part of this >> work involves kernel changes to the way the Xen network module transfer >> data to avoid some potential denial-of-service conditions that arise if >> you''re not using netloop. > > Just to elaborate ... > > The only reason netloop exists is to prevent the netback driver from > "seizing up" if packets from DomU, destined for Dom0, are queued up > indefinitely in Dom0 socket buffers causing hypervisor resources (grant > table entries?) to be exhausted. The netloop solution is to copy packets > from DomU just before they are delivered to Dom0''s networking stack. > > The new solution is for netback to keep track of packets and to > periodically copy any packets which have been queued up for too long. > That way you don''t need the netloop device at all. > > Cheers, > Mark. > > -- > Fedora-xen mailing list > Fedora-xen@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-xen >