Fischer Udo Attila
2009-Oct-26 11:40 UTC
[Xen-users] xen bridge jumbo frame problems starting new DomU
Hi all, My problem is using jumbo frames in vm-s using networking via bridge: Facts I discovered: - bridge will set automatically his mtu to the lowest interface MTU - MTU can only set to max 1500 in the initialization phase of an DomU (via vif-bridge or vif common) - MTU can only set higher some seconds after the DomU started So the problem I have to wait some seconds after I started a DomO and set manually It sets the bridge back to MTU 1500 when the initialization script is running, affecting all DomUs using this bridge!!! It is not acceptable I think! So I have a workaround: I have created an additional bridge set it in the DomU config and a vifname for the interface for easier scripting start the DomU sleep 5 brctl delif br2 xxx #where br2 is the additional bridge ifconfig xxx mtu 9000 brctl addif br1 xxx #where br1 is the right one So the problem is here not having network in the DomU for 5 seconds... (The smalles time was 2 seconds, but 5 for safety.. Ok I can write a script tries till its succeed.) It is a very dirty solution.... Is there any other solution? Is there any patch witch removes the 1500 MTU limit in the driver for the initialization phase, or a type=xxxx that allowes setting it higher? Is there any solution planned to remove this behavior in the future? I think using Jumbo Frames is now everyday requirement to lower CPU IO utilization by Gigabit networks. So the main point is, the bridge should not change the MTU downto 1500 by starting a new DomU affecting existing DomUs... Other question: Is there any patch that change the bridge behavior to the following way: try to set the interface to the bridge MTU if it fails it goes down in MTU till it works, and than set all other interfaces and the bridge down to this MTU... It not solves the first problem (init phase may 1500 MTU), but would be remove the problem by adding a new interface without increased MTU) Best regards, Udo Attila Fischer _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Peter Viskup
2009-Oct-26 14:29 UTC
Re: [Xen-users] xen bridge jumbo frame problems starting new DomU
Hello Udo, I sent email to Xen-devel mailing group [1] regarding MTU configuration option for VIF interfaces just some days before. MTU is hard-coded now and there is no way to change it within Xen configuration files. At this time you have to change MTU settings manually. I think that my very simple patch will not work - anyway you can test it and probably somebody more experienced in Xen development could work on this. It will be very helpful to have this option and it is good to see that I am not the only one looking for that. [1] - http://lists.xensource.com/archives/html/xen-devel/2009-10/msg00893.html Best regards, Peter Viskup 2009/10/26 Fischer Udo Attila <udo@udo.hu>> Hi all, > > My problem is using jumbo frames in vm-s using networking via bridge: > > Facts I discovered: > - bridge will set automatically his mtu to the lowest interface MTU > - MTU can only set to max 1500 in the initialization phase of an DomU (via > vif-bridge or vif common) > - MTU can only set higher some seconds after the DomU started > > So the problem > I have to wait some seconds after I started a DomO and set manually > It sets the bridge back to MTU 1500 when the initialization script is > running, affecting all DomUs using this bridge!!! It is not acceptable I > think! > > So I have a workaround: > I have created an additional bridge > set it in the DomU config and a vifname for the interface for easier > scripting > start the DomU > sleep 5 > brctl delif br2 xxx #where br2 is the additional bridge > ifconfig xxx mtu 9000 > brctl addif br1 xxx #where br1 is the right one > So the problem is here not having network in the DomU for 5 seconds... (The > smalles time was 2 seconds, but 5 for safety.. Ok I can write a script tries > till its succeed.) > > It is a very dirty solution.... > > Is there any other solution? > Is there any patch witch removes the 1500 MTU limit in the driver for the > initialization phase, or a type=xxxx that allowes setting it higher? > > > Is there any solution planned to remove this behavior in the future? I > think using Jumbo Frames is now everyday requirement to lower CPU IO > utilization by Gigabit networks. > > So the main point is, the bridge should not change the MTU downto 1500 by > starting a new DomU affecting existing DomUs... > > > Other question: > Is there any patch that change the bridge behavior to the following way: > try to set the interface to the bridge MTU > if it fails it goes down in MTU till it works, and than set all other > interfaces and the bridge down to this MTU... > It not solves the first problem (init phase may 1500 MTU), but would be > remove the problem by adding a new interface without increased MTU) > > Best regards, > Udo Attila Fischer > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fischer Udo Attila
2009-Oct-26 15:41 UTC
Re: [Xen-users] xen bridge jumbo frame problems starting new DomU
Hi Peter,
I have already programmed it only a little different:
and it works if you want to set it below 1500, but fails if you want to
go over 1500..
the domain start fails if it is set over 1500 (RTNETLINK answers:
Invalid argument)
Best Regards,
Udo Attila Fischer
PS: Maybe it the following code should not posted here but maybe it is
usefull if somebody want to change settings below 1500.
in the /etc/xen/scripts
--- vif-common.sh.orig 2009-10-24 01:15:59.000000000 +0200
+++ vif-common.sh 2009-10-26 16:15:54.000000000 +0100
@@ -63,6 +63,11 @@
vif="$vifname"
fi
+mtu=$(xenstore_read_default "$XENBUS_PATH/mtu" "")
+if [ "$mtu" ] && [ "$command" == "online"
]
+then
+ ip link set $vif mtu "$mtu"
+fi
frob_iptable()
{
in the xen phyton scripts directory:
--- xm/create.py.orig 2009-10-24 02:19:04.000000000 +0200
+++ xm/create.py 2009-10-24 02:22:07.000000000 +0200
@@ -354,7 +354,7 @@
keymap overrides the XendD configured default layout
file.""")
gopts.var(''vif'',
val="type=TYPE,mac=MAC,bridge=BRIDGE,ip=IPADDR,script=SCRIPT," + \
-
"backend=DOM,vifname=NAME,rate=RATE,model=MODEL,accel=ACCEL",
+
"backend=DOM,vifname=NAME,rate=RATE,model=MODEL,accel=ACCEL,mtu=MTU",
fn=append_value, default=[],
use="""Add a network interface with the given MAC
address and
bridge.
The vif is configured by calling the given configuration script.
@@ -850,7 +850,7 @@
def f(k):
if k not in [''backend'',
''bridge'', ''ip'', ''mac'',
''script'',
''type'',
''vifname'', ''rate'',
''model'', ''accel'',
- ''policy'',
''label'']:
+ ''policy'', ''label'',
''mtu'']:
err(''Invalid vif option: '' + k)
config_vif.append([k, d[k]])
--- xend/server/netif.py.orig 2009-01-05 21:27:55.000000000 +0100
+++ xend/server/netif.py 2009-10-24 03:46:17.000000000 +0200
@@ -115,6 +115,7 @@
model = config.get(''model'')
accel = config.get(''accel'')
sec_lab = config.get(''security_label'')
+ mtu = config.get(''mtu'')
if not mac:
raise VmError("MAC address not specified or generated.")
@@ -141,6 +142,8 @@
back[''accel''] = accel
if sec_lab:
back[''security_label''] = sec_lab
+ if mtu:
+ back[''mtu''] = mtu
back[''handle''] = "%i" % devid
back[''script''] =
os.path.join(xoptions.network_script_dir, script)
@@ -187,7 +190,7 @@
for x in ( ''script'', ''ip'',
''bridge'', ''mac'',
''type'', ''vifname'',
''rate'', ''uuid'', ''model'',
''accel'',
- ''security_label''):
+ ''security_label'',
''mtu''):
if transaction is None:
y = self.readBackend(devid, x)
else:
--- xm/main.py.orig 2009-10-24 02:55:52.000000000 +0200
+++ xm/main.py 2009-10-24 03:02:08.000000000 +0200
@@ -188,7 +188,7 @@
''network-attach'': (''<Domain>
[type=<type>] [mac=<mac>]
[bridge=<bridge>] ''
''[ip=<ip>] [script=<script>]
[backend=<BackDomain>] ''
''[vifname=<name>] [rate=<rate>]
[model=<model>]''
- ''[accel=<accel>]'',
+ ''[accel=<accel>]''
''[mtu=<mtu>]'',
''Create a new virtual network
device.''),
''network-detach'': (''<Domain>
<DevId> [-f|--force]'',
''Destroy a domain\''s virtual network
device.''),
@@ -2416,7 +2416,7 @@
dom = args[0]
vif = [''vif'']
vif_params = [''type'', ''mac'',
''bridge'', ''ip'', ''script'',
\
- ''backend'', ''vifname'',
''rate'', ''model'', ''accel'']
+ ''backend'', ''vifname'',
''rate'', ''model'', ''accel'',
''mtu'']
if serverType == SERVER_XEN_API:
vif_record = {
@@ -2465,7 +2465,9 @@
''model'':
lambda x: None,
''accel'':
- lambda x: set([''other_config'',
''accel''], x)
+ lambda x: set([''other_config'',
''accel''], x),
+ ''mtu'':
+ lambda x: set([''other_config'',
''mtu''], x)
}
for a in args[1:]:
--- xend/XendDomainInfo.py.orig 2009-10-24 05:05:37.000000000 +0200
+++ xend/XendDomainInfo.py 2009-10-24 05:25:20.000000000 +0200
@@ -3213,7 +3213,8 @@
# some point we''re going to have to figure out how
to
# handle that properly.
- config[''MTU''] = 1500 # TODO
+ #config[''MTU''] = 1500 # TODO
+ config[''MTU''] =
config.get(''mtu'', 1500) # TODO
if self._stateGet() not in (XEN_API_VM_POWER_STATE_HALTED,):
xennode = XendNode.instance()
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Fischer Udo Attila
2009-Nov-03 19:24 UTC
Re: [Xen-users] xen bridge jumbo frame problems starting new DomU
Hi all, So no solution for Jumbo Frame support at xen DomU startup, only with the temporary bridge and wait in the background and set new MTU when possible and move to the right bridge workaround? Or any other solution not to break the other DomUs attached to the bridge? So than I will open a bug ticket at xensource.org, because I think direct Jumbo Frame support is a every day need today... Best Regards, Udo Attila Fischer _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2010-Jan-18 12:36 UTC
Re: [Xen-users] xen bridge jumbo frame problems starting new DomU
On Tue, Nov 03, 2009 at 08:24:10PM +0100, Fischer Udo Attila wrote:> Hi all, > > So no solution for Jumbo Frame support at xen DomU startup, only with > the temporary bridge and wait in the background and set new MTU when > possible and move to the right bridge workaround? Or any other solution > not to break the other DomUs attached to the bridge? > > So than I will open a bug ticket at xensource.org, because I think > direct Jumbo Frame support is a every day need today... >Hello, Was this ever resolved? -- Pasi _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
nope, I just installed xen3.4.4 on centos5.8 with gitco repos two days ago(2012.10.02) the bug is still there. -- View this message in context: http://xen.1045712.n5.nabble.com/xen-bridge-jumbo-frame-problems-starting-new-DomU-tp2608719p5711706.html Sent from the Xen - User mailing list archive at Nabble.com.
I found that redhat have been solve this problem: https://bugzilla.redhat.com/show_bug.cgi?id=697021 but the bug is still there on xen 3.4.4 On Thu, Oct 4, 2012 at 11:23 AM, alfiechan <darkblue2000@gmail.com> wrote:> nope, I just installed xen3.4.4 on centos5.8 with gitco repos two days > ago(2012.10.02) > the bug is still there. > > > > -- > View this message in context: > http://xen.1045712.n5.nabble.com/xen-bridge-jumbo-frame-problems-starting-new-DomU-tp2608719p5711706.html > Sent from the Xen - User mailing list archive at Nabble.com. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Pasi Kärkkäinen
2012-Nov-30 10:03 UTC
Re: xen bridge jumbo frame problems starting new DomU
On Thu, Oct 04, 2012 at 11:36:41AM +0800, darkblue wrote:> I found that redhat have been solve this problem: > [1]https://bugzilla.redhat.com/show_bug.cgi?id=697021 > > but the bug is still there on xen 3.4.4 >Yes, the vif-bridge script needs to be patched, the instructions are on the bugzilla. (and merged to upstream Xen in later versions, iirc). Also the kernel drivers need to support jumbos aswell, especially the xen-netback driver in dom0 kernel. -- Pasi> On Thu, Oct 4, 2012 at 11:23 AM, alfiechan <[2]darkblue2000@gmail.com> > wrote: > > nope, I just installed xen3.4.4 on centos5.8 with gitco repos two days > ago(2012.10.02) > the bug is still there. > > -- > View this message in context: > [3]http://xen.1045712.n5.nabble.com/xen-bridge-jumbo-frame-problems-starting-new-DomU-tp2608719p5711706.html > Sent from the Xen - User mailing list archive at Nabble.com. > > _______________________________________________ > Xen-users mailing list > [4]Xen-users@lists.xen.org > [5]http://lists.xen.org/xen-users > > References > > Visible links > 1. https://bugzilla.redhat.com/show_bug.cgi?id=697021 > 2. mailto:darkblue2000@gmail.com > 3. http://xen.1045712.n5.nabble.com/xen-bridge-jumbo-frame-problems-starting-new-DomU-tp2608719p5711706.html > 4. mailto:Xen-users@lists.xen.org > 5. http://lists.xen.org/xen-users> _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users