Xen devel - Oct 2007 - Re: [Xen-changelog] [xen-3.1-testing] xend: fix server/netif.py so that it respects type=None.

On Thu, Oct 04, 2007 at 08:40:16AM -0700, Xen patchbot-3.1-testing wrote:
> xend: fix server/netif.py so that it respects type=None.
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> xen-unstable changeset:   15972:3a799196ff69d3a3d5e4a891c13434aa61ce60a9
> xen-unstable date:        Thu Sep 27 17:44:03 2007 +0100
>  
> -        if not typ:
> -            typ = xoptions.netback_type
> -
Grumble, this makes xoptions.netback_type now clearly unused. I''m not
sure whether this breaks us yet, but certainly there''s either cleanup
needed or it needs fixing.

What does type=None do?

regards
john

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 01:21:24AM +0100, John Levon
wrote:
> On Thu, Oct 04, 2007 at 08:40:16AM -0700, Xen patchbot-3.1-testing wrote:
> 
> > xend: fix server/netif.py so that it respects type=None.
> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> > xen-unstable changeset:  
15972:3a799196ff69d3a3d5e4a891c13434aa61ce60a9
> > xen-unstable date:        Thu Sep 27 17:44:03 2007 +0100
> >  
> > -        if not typ:
> > -            typ = xoptions.netback_type
> > -
> 
> Grumble, this makes xoptions.netback_type now clearly unused. I''m
not
> sure whether this breaks us yet, but certainly there''s either
cleanup
> needed or it needs fixing.
It shouldn''t break anything unless it was already broken.

Basically with type=None, then the in-memory VM would be booted with type
of None, but at the same time the VM config would be written out to disk
with type of ''netfront''. You could stop & start the HVM
guest many times
and XenD would always use a type of None when configuring qemu/netback.
The moment you restarted XenD (or rebooted the machine), then it would read
the config in from disk & now see type=netfront. So your config which had
been working fine, suddenly changes when XenD is restarted & your qemu 
NICs disappear. So if this breaks anything for Solaris, then it must have
been broken all along, because this only changes the config written to disk,
not the in-memory one XenD is actuall using to start off with.
> What does type=None do?
The behaviour is

  - type == netfront  -> setup netback only
  - type == ioemu     -> setup qemu-dm only
  - type is None      -> setup netback & qemu-dm

The latter is the preferred behavour because it means the same VM config works
whether the guest is using the RTL8139 emulated NIC, or PV drivers.

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 01:38:40AM +0100, Daniel P. Berrange
wrote:
> On Fri, Oct 05, 2007 at 01:21:24AM +0100, John Levon wrote:
> > On Thu, Oct 04, 2007 at 08:40:16AM -0700, Xen patchbot-3.1-testing
wrote:
> > 
> > > xend: fix server/netif.py so that it respects type=None.
> > > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> > > xen-unstable changeset:  
15972:3a799196ff69d3a3d5e4a891c13434aa61ce60a9
> > > xen-unstable date:        Thu Sep 27 17:44:03 2007 +0100
> > >  
> > > -        if not typ:
> > > -            typ = xoptions.netback_type
> > > -
> > 
> > Grumble, this makes xoptions.netback_type now clearly unused.
I''m not
> > sure whether this breaks us yet, but certainly there''s either
cleanup
> > needed or it needs fixing.
> 
> It shouldn''t break anything unless it was already broken.
> 
> Basically with type=None, then the in-memory VM would be booted with type
> of None, but at the same time the VM config would be written out to disk
> with type of ''netfront''. You could stop & start the
HVM guest many times
> and XenD would always use a type of None when configuring qemu/netback.
> The moment you restarted XenD (or rebooted the machine), then it would read
> the config in from disk & now see type=netfront. So your config which
had
> been working fine, suddenly changes when XenD is restarted & your qemu 
> NICs disappear. So if this breaks anything for Solaris, then it must have
> been broken all along, because this only changes the config written to
disk,
> not the in-memory one XenD is actuall using to start off with.
Actually if your Dom0/DomU drivers are working differently to the Linux ones
it could conceivably cause an issue. To explain what goes on in XenD when
starting a VM:

 1 The netif.getDeviceDetails() method is called to get the dict for
   front and backend devices. If type == ''ioemu'' then,
frontend dict is an
   empty stub (ie no handle/mac fields present).

 2 DevController.createDevice() writes the front & back dicts into xenstore

       - frontend VIF info is written into /local/[DOMID]/device/vif/[DEVID]
       - backend VIF info is written info /local/0/backends/device/vif/[DEVID]

 3 netback sees the info in /local/0/backends/device/vif/[DEVID] and sets up
   the backend devices.

 4 if the frontend dict was a stub (ie no handle/mac fields present) then it
   doesn''t do anything, otherwise it connects to backend.

 5 in image.py if type is None or ''ioemu'' then qemu is given a
-nic arg to
   add an rtl8139 nic.

 6 VM config is saved, by calling DevController.configurations(), which
   calls configuration() on netif, which reads info out of xenstore in the
   backend/frontend paths. 


So, when in Step 1, the getDeviceDetails() was converting a type of
''None''
into ''netfront'', when we get to step 6 it reads the type of
''netfront'' out of
xenstore and writes it to disk. So next time XenD restarts, the type is fixed
to be ''netfront'', and not None. This means that in step
''5'', qemu nevers sets
up a -nic arg.

So the one place where Solaris could in theory find trouble is if the kernel
modules for back/front actually look at the ''type'' field in
xenstore. In the
Linux impl, neither the back/front drivers ever look at the
''type'' field - it
is only ever interpreted by XenD.
> > What does type=None do?
> 
> The behaviour is
> 
>   - type == netfront  -> setup netback only
>   - type == ioemu     -> setup qemu-dm only
>   - type is None      -> setup netback & qemu-dm
> 
> The latter is the preferred behavour because it means the same VM config
works
> whether the guest is using the RTL8139 emulated NIC, or PV drivers.

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 02:22:33AM +0100, Daniel P. Berrange
wrote:
> So the one place where Solaris could in theory find trouble is if
> the kernel modules for back/front actually look at the
''type'' field
> in xenstore. In the Linux impl, neither the back/front drivers ever
> look at the ''type'' field - it is only ever interpreted by
XenD.
We do look at the ''type'' field, to determine how to connect up
the
backend to a physical device.

"type=ioemu" and "type=netfront" both behave similarly to
the Linux
code ("netfront" creates a pseudo-ethernet device that could be used
by IP or a bridge, "ioemu" prompts qemu-dm to do the right thing
(which is different on Solaris, but anyway)).

"type=SUNW_mac", the default on Solaris (well, previously the
default), results in different approach where we expect to open a
physical or pseudo-physical device directly and relay
packets. Typically it''s a pseudo-physical device created by the
hotplug scripts using Crossbow[1] VNICs.

I''ll look at the changes more closely and figure out what to do.

[1] http://opensolaris.org/os/project/crossbow/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 5/10/07 14:41, "David Edmondson" <dme@sun.com> wrote:
> "type=SUNW_mac", the default on Solaris (well, previously the
> default), results in different approach where we expect to open a
> physical or pseudo-physical device directly and relay
> packets. Typically it''s a pseudo-physical device created by the
> hotplug scripts using Crossbow[1] VNICs.
> 
> I''ll look at the changes more closely and figure out what to do.
Perhaps slightly gross, but could we perhaps:
    if not typ and xoptions.netback_type != "netfront":
        typ = xoptions.netback_type

Bear in mind that you need a fix quite quickly for 3.1.1...

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 03:00:31PM +0100, Keir Fraser
wrote:
> Perhaps slightly gross, but could we perhaps:
Yes, "blech".

It seems that at the place where ''None'' is interpreted to mean
''netfront'' we need to use xoptions.netback_type rather than
''netfront''.
> Bear in mind that you need a fix quite quickly for 3.1.1...
What is "quite quickly"?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 03:33:01PM +0100, David Edmondson
wrote:
> On Fri, Oct 05, 2007 at 03:00:31PM +0100, Keir Fraser wrote:
> > Perhaps slightly gross, but could we perhaps:
> 
> Yes, "blech".
> 
> It seems that at the place where ''None'' is interpreted to
mean
> ''netfront'' we need to use xoptions.netback_type rather
than
> ''netfront''.
That would be the kernel...   Can''t your code which looks at xenstore
for the type field just treat the absence of it as ''SUNW_mac''
or
whatever default value you want to use ?

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 04:29:06PM +0100, Daniel P. Berrange
wrote:
> On Fri, Oct 05, 2007 at 03:33:01PM +0100, David Edmondson wrote:
> > On Fri, Oct 05, 2007 at 03:00:31PM +0100, Keir Fraser wrote:
> > > Perhaps slightly gross, but could we perhaps:
> > 
> > Yes, "blech".
> > 
> > It seems that at the place where ''None'' is
interpreted to mean
> > ''netfront'' we need to use xoptions.netback_type
rather than
> > ''netfront''.
> 
> That would be the kernel...  Can''t your code which looks at
xenstore
> for the type field just treat the absence of it as
''SUNW_mac'' or
> whatever default value you want to use ?
Yes, that''s what we''ll probably do.

Will you remove the netback_type default stuff?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 5/10/07 15:33, "David Edmondson" <dme@sun.com> wrote:
>> Bear in mind that you need a fix quite quickly for 3.1.1...
> 
> What is "quite quickly"?
We plan to release within a week. Really this needs to be fixed by end of
Monday. Seems to me we can knock up something suitable in that timeframe
though.

Daniel: Could we simply make xoptions.netback_type = None by default (for
all but Solaris) instead of "netfront"? Then we could put:
 if not typ:
    typ = xoptions.netback_type

Back in as is and it''d do the right thing?

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 04:56:28PM +0100, Keir Fraser
wrote:
> On 5/10/07 15:33, "David Edmondson" <dme@sun.com> wrote:
> 
> >> Bear in mind that you need a fix quite quickly for 3.1.1...
> > 
> > What is "quite quickly"?
> 
> We plan to release within a week. Really this needs to be fixed by end of
> Monday. Seems to me we can knock up something suitable in that timeframe
> though.
> 
> Daniel: Could we simply make xoptions.netback_type = None by default (for
> all but Solaris) instead of "netfront"? Then we could put:
>  if not typ:
>     typ = xoptions.netback_type
> 
> Back in as is and it''d do the right thing?
That would certainly work for Linux, though it would not enable the HVM
guests on a Solaris host to be configured with both PV drivers & QEMU at
the same time. Upto solaris guys if that''s a good enough solution...

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 5/10/07 17:06, "Daniel P. Berrange" <berrange@redhat.com>
wrote:
>> Daniel: Could we simply make xoptions.netback_type = None by default
(for
>> all but Solaris) instead of "netfront"? Then we could put:
>>  if not typ:
>>     typ = xoptions.netback_type
>> 
>> Back in as is and it''d do the right thing?
> 
> That would certainly work for Linux, though it would not enable the HVM
> guests on a Solaris host to be configured with both PV drivers & QEMU
at
> the same time. Upto solaris guys if that''s a good enough
solution...
Not clear there''s a perfect solution that doesn''t involve
modifying the
Solaris PV netback to interpret None as SunW_mac (or whatever it was).
That''s probably the nicest solution of all, and get rid of netback_type
default altogether. But that may be unacceptable due to backward compat
issues with existing Solaris PV netback?

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 05:06:56PM +0100, Daniel P. Berrange
wrote:
> That would certainly work for Linux, though it would not enable the
> HVM guests on a Solaris host to be configured with both PV drivers &
> QEMU at the same time. Upto solaris guys if that''s a good enough
> solution...
We''ll make ''None'' mean ''our preferred
default'' in our implementation.

The whole "PV and IOEMU" at the same time makes me uncomfortable,
though it looks to be more problematic with disks than network
devices.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 05:09:55PM +0100, Keir Fraser
wrote:
> Not clear there''s a perfect solution that doesn''t involve
modifying the
> Solaris PV netback to interpret None as SunW_mac (or whatever it was).
> That''s probably the nicest solution of all, and get rid of
netback_type
> default altogether. But that may be unacceptable due to backward compat
> issues with existing Solaris PV netback?
I don''t believe that there are any backwards compatibility issues
here. The people using the current OpenSolaris bits are versed in
dealing with change.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 05:16:35PM +0100, David Edmondson
wrote:
> On Fri, Oct 05, 2007 at 05:06:56PM +0100, Daniel P. Berrange wrote:
> > That would certainly work for Linux, though it would not enable the
> > HVM guests on a Solaris host to be configured with both PV drivers
&
> > QEMU at the same time. Upto solaris guys if that''s a good
enough
> > solution...
> 
> We''ll make ''None'' mean ''our preferred
default'' in our implementation.
> 
> The whole "PV and IOEMU" at the same time makes me uncomfortable,
> though it looks to be more problematic with disks than network
> devices.
The key is that the Dom0 configuration of the guest should allow the guest
kernel to choose the drivers. Obviously you don''t want both drivers
active
at once, so one idea is for the PV drivers to ''grab'' the PCI
resources
associated with the emulated NIC. So once you load the PV driver,
there''s
no way for the RTL8139 driver to get access to the NIC, thus ensuring only
one is ever active at once. The same principle applies to disk adapters.

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 06:30:48PM +0100, Daniel P. Berrange
wrote:
> > The whole "PV and IOEMU" at the same time makes me
uncomfortable,
> > though it looks to be more problematic with disks than network
> > devices.
> 
> The key is that the Dom0 configuration of the guest should allow the
> guest kernel to choose the drivers.
Understood.
> Obviously you don''t want both drivers active at once, so one idea
is
> for the PV drivers to ''grab'' the PCI resources associated
with the
> emulated NIC.
That seems so wrong.

In Solaris drivers are loaded as a result of a mapping from the PCI ID
of the device to a driver name (I''ve no idea if it''s the same
in
Linux). What happens if the RTL8139 driver gets in first?
> So once you load the PV driver, there''s no way for the RTL8139
> driver to get access to the NIC, thus ensuring only one is ever
> active at once. The same principle applies to disk adapters.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Oct 05, 2007 at 06:36:22PM +0100, David Edmondson
wrote:
> On Fri, Oct 05, 2007 at 06:30:48PM +0100, Daniel P. Berrange wrote:
> > > The whole "PV and IOEMU" at the same time makes me
uncomfortable,
> > > though it looks to be more problematic with disks than network
> > > devices.
> > 
> > The key is that the Dom0 configuration of the guest should allow the
> > guest kernel to choose the drivers.
> 
> Understood.
> 
> > Obviously you don''t want both drivers active at once, so one
idea is
> > for the PV drivers to ''grab'' the PCI resources
associated with the
> > emulated NIC.
> 
> That seems so wrong.
> 
> In Solaris drivers are loaded as a result of a mapping from the PCI ID
> of the device to a driver name (I''ve no idea if it''s the
same in
> Linux). What happens if the RTL8139 driver gets in first?
If the PV driver is unable to grab the PCI resources of the RTL NIC
because RTL8139 is already loaded, then it wouldn''t try to initialize 
the PV nic. So this basically provides the safety net against both
being active (accidentally or by delibrate admuin actions). Then you 
need to figure out a way to prioritization the driver you really want.

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 5/10/07 17:17, "David Edmondson" <dme@sun.com> wrote:
> On Fri, Oct 05, 2007 at 05:09:55PM +0100, Keir Fraser wrote:
>> Not clear there''s a perfect solution that doesn''t
involve modifying the
>> Solaris PV netback to interpret None as SunW_mac (or whatever it was).
>> That''s probably the nicest solution of all, and get rid of
netback_type
>> default altogether. But that may be unacceptable due to backward compat
>> issues with existing Solaris PV netback?
> 
> I don''t believe that there are any backwards compatibility issues
> here. The people using the current OpenSolaris bits are versed in
> dealing with change.
Okay, so what''s the conclusion? It sounds like:
 1. What is currently in 3.1-testing and unstable is okay, and you will
modify your netback driver; but
 2. We should clean up by removing xoption.netback_type in a further patch,
because it''s not used any longer.

Right?

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Sat, Oct 06, 2007 at 07:41:44AM +0100, Keir Fraser
wrote:
> Okay, so what''s the conclusion? It sounds like:
>  1. What is currently in 3.1-testing and unstable is okay, and you will
> modify your netback driver; but
>  2. We should clean up by removing xoption.netback_type in a further patch,
> because it''s not used any longer.
> 
> Right?
Yes, I believe that is correct.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel