Xen devel - Mar 2008 - incompatibility of netfront driver with bonding module

Hi all,
 
I am trying a live migration of pass-through enabled domU. The idea is
configuring a bonding interface and enslave the pass-through NIC to it.
During migration, enslave a vif and detach the NIC. But this is not
working because of the incompatibility of netfront driver with bonding
module.
 
1. Normally, bonding will inherit the mac from the first slave and
assign it to all the other slaves added later. So the mac of vif will be
updated. But during migration, the mac of vif is getting updated from
xenstore(/vm/). So how about having a set_mac_address entry in netfront
that update the xenstore?
2. arp link monitoring method relies on the slave driver''s rx/tx
timestamp. But in netfront, dev->last_rx and dev->trans_start are not
being used, and fails to detect the link status.
3. In mii link monitoring, it relies on the netif_carrier_ok status. But
as netfront is using itsown netfront_carrier_* functions, the status is
not getting updated to bonding module.
 
So I need help, especially on #1. Is it ok to update the xenstore /vm/
keys from domU? how to do this?
 
thanks,
Masroor


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi,

Masroor Vettuparambil wrote:
> 1. Normally, bonding will inherit the mac from the first slave and 
> assign it to all the other slaves added later. So the mac of vif will be 
> updated. But during migration, the mac of vif is getting updated from 
> xenstore(/vm/). So how about having a set_mac_address entry in netfront 
> that update the xenstore?
> So I need help, especially on #1. Is it ok to update the xenstore 
> /vm/ keys from domU? how to do this?
I tried a bit to find out the way to update /vm/ keys from frontend,
but didn''t succeed. Either way, I don''t think it is a good
idea to
update the xenstore key of the vif mac address, because the original
mac will be lost and we won''t be able to reset it e.g. after the guest
reboot.

So the attached patch adds set mac_address() support to netfront.
It doesn''t touch xenstore at all, but just preserves the modified mac
address in netfront_info structure and keeps using it after migration.

Thanks,
-----------------------
Yosuke Iwamatsu
         NEC Corporation


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Thanks for your work.
I could manage the live migration using a simple check in netfront while
resuming on destination.

if (!(netdev->flags & IFF_SLAVE))
	memcpy(netdev->dev_addr, info->mac, ETH_ALEN);
But this will work only if the interface is enslaved.

In your patch, should we let to change the MAC while interface is up? 
I need to include the support for arp link monitoring to netfront.

Regards
Masroor,
 

-----Original Message-----
From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
Sent: Thursday, April 03, 2008 12:00 PM
To: Masroor Vettuparambil
Cc: xen-devel@lists.xensource.com
Subject: incompatibility of netfront driver with bonding module

Hi,

Masroor Vettuparambil wrote:
> 1. Normally, bonding will inherit the mac from the first slave and 
> assign it to all the other slaves added later. So the mac of vif will 
> be updated. But during migration, the mac of vif is getting updated 
> from xenstore(/vm/). So how about having a set_mac_address entry in 
> netfront that update the xenstore?
> So I need help, especially on #1. Is it ok to update the xenstore /vm/
> keys from domU? how to do this?
I tried a bit to find out the way to update /vm/ keys from frontend, but
didn''t succeed. Either way, I don''t think it is a good idea to
update
the xenstore key of the vif mac address, because the original mac will
be lost and we won''t be able to reset it e.g. after the guest reboot.

So the attached patch adds set mac_address() support to netfront.
It doesn''t touch xenstore at all, but just preserves the modified mac
address in netfront_info structure and keeps using it after migration.

Thanks,
-----------------------
Yosuke Iwamatsu
         NEC Corporation

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Masroor Vettuparambil wrote:
> Thanks for your work.
> I could manage the live migration using a simple check in netfront while
> resuming on destination.
> 
> if (!(netdev->flags & IFF_SLAVE))
> 	memcpy(netdev->dev_addr, info->mac, ETH_ALEN);
> But this will work only if the interface is enslaved.
> 
> In your patch, should we let to change the MAC while interface is up?
It might be desirable, but I don''t know for now if we can achieve it
easily.
> I need to include the support for arp link monitoring to netfront.
Is link monitoring really necessary?
If you mean you use link monitoring to change the active slave of the 
bond0, I think we can do that using ifenslave command inside domU.
  #ifenslave bond0 eth1     -> enslave new netif(eth1) to bonding device
  #ifenslave -d bond0 eth0  -> release old netif(eth0) from bonding

Regards,
   Yosuke
> 
> Regards
> Masroor,
>  
> 
> -----Original Message-----
> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
> Sent: Thursday, April 03, 2008 12:00 PM
> To: Masroor Vettuparambil
> Cc: xen-devel@lists.xensource.com
> Subject: incompatibility of netfront driver with bonding module
> 
> Hi,
> 
> Masroor Vettuparambil wrote:
>> 1. Normally, bonding will inherit the mac from the first slave and 
>> assign it to all the other slaves added later. So the mac of vif will 
>> be updated. But during migration, the mac of vif is getting updated 
>> from xenstore(/vm/). So how about having a set_mac_address entry in 
>> netfront that update the xenstore?
> 
>> So I need help, especially on #1. Is it ok to update the xenstore /vm/
> 
>> keys from domU? how to do this?
> 
> I tried a bit to find out the way to update /vm/ keys from frontend, but
> didn''t succeed. Either way, I don''t think it is a good
idea to update
> the xenstore key of the vif mac address, because the original mac will
> be lost and we won''t be able to reset it e.g. after the guest
reboot.
> 
> So the attached patch adds set mac_address() support to netfront.
> It doesn''t touch xenstore at all, but just preserves the modified
mac
> address in netfront_info structure and keeps using it after migration.
> 
> Thanks,
> -----------------------
> Yosuke Iwamatsu
>          NEC Corporation

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

The attached patch provides the arp link monitoring support for
netfront.
Also it prevents setting the mac while the interface is up.

Regards
Masroor 

-----Original Message-----
From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
Sent: Monday, April 07, 2008 12:36 PM
To: Masroor Vettuparambil
Cc: xen-devel@lists.xensource.com
Subject: incompatibility of netfront driver with bonding module

Masroor Vettuparambil wrote:
> Thanks for your work.
> I could manage the live migration using a simple check in netfront 
> while resuming on destination.
> 
> if (!(netdev->flags & IFF_SLAVE))
> 	memcpy(netdev->dev_addr, info->mac, ETH_ALEN); But this will
work 
> only if the interface is enslaved.
> 
> In your patch, should we let to change the MAC while interface is up?
It might be desirable, but I don''t know for now if we can achieve it
easily.
> I need to include the support for arp link monitoring to netfront.
Is link monitoring really necessary?
If you mean you use link monitoring to change the active slave of the
bond0, I think we can do that using ifenslave command inside domU.
  #ifenslave bond0 eth1     -> enslave new netif(eth1) to bonding device
  #ifenslave -d bond0 eth0  -> release old netif(eth0) from bonding

Regards,
   Yosuke
> 
> Regards
> Masroor,
>  
> 
> -----Original Message-----
> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
> Sent: Thursday, April 03, 2008 12:00 PM
> To: Masroor Vettuparambil
> Cc: xen-devel@lists.xensource.com
> Subject: incompatibility of netfront driver with bonding module
> 
> Hi,
> 
> Masroor Vettuparambil wrote:
>> 1. Normally, bonding will inherit the mac from the first slave and 
>> assign it to all the other slaves added later. So the mac of vif will
>> be updated. But during migration, the mac of vif is getting updated 
>> from xenstore(/vm/). So how about having a set_mac_address entry in 
>> netfront that update the xenstore?
> 
>> So I need help, especially on #1. Is it ok to update the xenstore 
>> /vm/
> 
>> keys from domU? how to do this?
> 
> I tried a bit to find out the way to update /vm/ keys from frontend, 
> but didn''t succeed. Either way, I don''t think it is a
good idea to
> update the xenstore key of the vif mac address, because the original 
> mac will be lost and we won''t be able to reset it e.g. after the
guest
reboot.
> 
> So the attached patch adds set mac_address() support to netfront.
> It doesn''t touch xenstore at all, but just preserves the modified
mac
> address in netfront_info structure and keeps using it after migration.
> 
> Thanks,
> -----------------------
> Yosuke Iwamatsu
>          NEC Corporation


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Thank you.

Now I''m curious if you really succeeded the live migration of
passthrough domains, using pci-hotplug and bonding.
(Currently I have one testing machine and can only try
local migration.)

-- Yosuke

Masroor Vettuparambil wrote:
> The attached patch provides the arp link monitoring support for
> netfront.
> Also it prevents setting the mac while the interface is up.
> 
> Regards
> Masroor 
> 
> -----Original Message-----
> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
> Sent: Monday, April 07, 2008 12:36 PM
> To: Masroor Vettuparambil
> Cc: xen-devel@lists.xensource.com
> Subject: incompatibility of netfront driver with bonding module
> 
> Masroor Vettuparambil wrote:
>> Thanks for your work.
>> I could manage the live migration using a simple check in netfront 
>> while resuming on destination.
>>
>> if (!(netdev->flags & IFF_SLAVE))
>> 	memcpy(netdev->dev_addr, info->mac, ETH_ALEN); But this will
> work 
>> only if the interface is enslaved.
>>
>> In your patch, should we let to change the MAC while interface is up?
> 
> It might be desirable, but I don''t know for now if we can achieve
it
> easily.
> 
>> I need to include the support for arp link monitoring to netfront.
> 
> Is link monitoring really necessary?
> If you mean you use link monitoring to change the active slave of the
> bond0, I think we can do that using ifenslave command inside domU.
>   #ifenslave bond0 eth1     -> enslave new netif(eth1) to bonding device
>   #ifenslave -d bond0 eth0  -> release old netif(eth0) from bonding
> 
> Regards,
>    Yosuke
> 
>> Regards
>> Masroor,
>>  
>>
>> -----Original Message-----
>> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
>> Sent: Thursday, April 03, 2008 12:00 PM
>> To: Masroor Vettuparambil
>> Cc: xen-devel@lists.xensource.com
>> Subject: incompatibility of netfront driver with bonding module
>>
>> Hi,
>>
>> Masroor Vettuparambil wrote:
>>> 1. Normally, bonding will inherit the mac from the first slave and 
>>> assign it to all the other slaves added later. So the mac of vif
will
> 
>>> be updated. But during migration, the mac of vif is getting updated
>>> from xenstore(/vm/). So how about having a set_mac_address entry in
>>> netfront that update the xenstore?
>>> So I need help, especially on #1. Is it ok to update the xenstore 
>>> /vm/
>>> keys from domU? how to do this?
>> I tried a bit to find out the way to update /vm/ keys from frontend, 
>> but didn''t succeed. Either way, I don''t think it is a
good idea to
>> update the xenstore key of the vif mac address, because the original 
>> mac will be lost and we won''t be able to reset it e.g. after
the guest
> reboot.
>> So the attached patch adds set mac_address() support to netfront.
>> It doesn''t touch xenstore at all, but just preserves the
modified mac
>> address in netfront_info structure and keeps using it after migration.
>>
>> Thanks,
>> -----------------------
>> Yosuke Iwamatsu
>>          NEC Corporation
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Yes, I could do a live migration of pass-through domain using bonding.  

Regards
Masroor 

-----Original Message-----
From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
Sent: Thursday, April 10, 2008 5:35 PM
To: Masroor Vettuparambil
Cc: xen-devel@lists.xensource.com
Subject: Re: [PATCH] incompatibility of netfront driver with bonding
module

Thank you.

Now I''m curious if you really succeeded the live migration of
passthrough domains, using pci-hotplug and bonding.
(Currently I have one testing machine and can only try local migration.)

-- Yosuke

Masroor Vettuparambil wrote:
> The attached patch provides the arp link monitoring support for 
> netfront.
> Also it prevents setting the mac while the interface is up.
> 
> Regards
> Masroor
> 
> -----Original Message-----
> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
> Sent: Monday, April 07, 2008 12:36 PM
> To: Masroor Vettuparambil
> Cc: xen-devel@lists.xensource.com
> Subject: incompatibility of netfront driver with bonding module
> 
> Masroor Vettuparambil wrote:
>> Thanks for your work.
>> I could manage the live migration using a simple check in netfront 
>> while resuming on destination.
>>
>> if (!(netdev->flags & IFF_SLAVE))
>> 	memcpy(netdev->dev_addr, info->mac, ETH_ALEN); But this will
> work
>> only if the interface is enslaved.
>>
>> In your patch, should we let to change the MAC while interface is up?
> 
> It might be desirable, but I don''t know for now if we can achieve
it
> easily.
> 
>> I need to include the support for arp link monitoring to netfront.
> 
> Is link monitoring really necessary?
> If you mean you use link monitoring to change the active slave of the 
> bond0, I think we can do that using ifenslave command inside domU.
>   #ifenslave bond0 eth1     -> enslave new netif(eth1) to bonding
device
>   #ifenslave -d bond0 eth0  -> release old netif(eth0) from bonding
> 
> Regards,
>    Yosuke
> 
>> Regards
>> Masroor,
>>  
>>
>> -----Original Message-----
>> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
>> Sent: Thursday, April 03, 2008 12:00 PM
>> To: Masroor Vettuparambil
>> Cc: xen-devel@lists.xensource.com
>> Subject: incompatibility of netfront driver with bonding module
>>
>> Hi,
>>
>> Masroor Vettuparambil wrote:
>>> 1. Normally, bonding will inherit the mac from the first slave and 
>>> assign it to all the other slaves added later. So the mac of vif 
>>> will
> 
>>> be updated. But during migration, the mac of vif is getting updated
>>> from xenstore(/vm/). So how about having a set_mac_address entry in
>>> netfront that update the xenstore?
>>> So I need help, especially on #1. Is it ok to update the xenstore 
>>> /vm/ keys from domU? how to do this?
>> I tried a bit to find out the way to update /vm/ keys from frontend, 
>> but didn''t succeed. Either way, I don''t think it is a
good idea to
>> update the xenstore key of the vif mac address, because the original 
>> mac will be lost and we won''t be able to reset it e.g. after
the
>> guest
> reboot.
>> So the attached patch adds set mac_address() support to netfront.
>> It doesn''t touch xenstore at all, but just preserves the
modified mac
>> address in netfront_info structure and keeps using it after
migration.
>>
>> Thanks,
>> -----------------------
>> Yosuke Iwamatsu
>>          NEC Corporation
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Masroor Vettuparambil wrote:
> Yes, I could do a live migration of pass-through domain using bonding.  
That''s great!

Thanks,
   Yosuke
> -----Original Message-----
> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
> Sent: Thursday, April 10, 2008 5:35 PM
> To: Masroor Vettuparambil
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [PATCH] incompatibility of netfront driver with bonding
> module
> 
> Thank you.
> 
> Now I''m curious if you really succeeded the live migration of
> passthrough domains, using pci-hotplug and bonding.
> (Currently I have one testing machine and can only try local migration.)
> 
> -- Yosuke
> 
> Masroor Vettuparambil wrote:
>> The attached patch provides the arp link monitoring support for 
>> netfront.
>> Also it prevents setting the mac while the interface is up.
>>
>> Regards
>> Masroor
>>
>> -----Original Message-----
>> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
>> Sent: Monday, April 07, 2008 12:36 PM
>> To: Masroor Vettuparambil
>> Cc: xen-devel@lists.xensource.com
>> Subject: incompatibility of netfront driver with bonding module
>>
>> Masroor Vettuparambil wrote:
>>> Thanks for your work.
>>> I could manage the live migration using a simple check in netfront 
>>> while resuming on destination.
>>>
>>> if (!(netdev->flags & IFF_SLAVE))
>>> 	memcpy(netdev->dev_addr, info->mac, ETH_ALEN); But this will
>> work
>>> only if the interface is enslaved.
>>>
>>> In your patch, should we let to change the MAC while interface is
up?
>> It might be desirable, but I don''t know for now if we can
achieve it
>> easily.
>>
>>> I need to include the support for arp link monitoring to netfront.
>> Is link monitoring really necessary?
>> If you mean you use link monitoring to change the active slave of the 
>> bond0, I think we can do that using ifenslave command inside domU.
>>   #ifenslave bond0 eth1     -> enslave new netif(eth1) to bonding
> device
>>   #ifenslave -d bond0 eth0  -> release old netif(eth0) from bonding
>>
>> Regards,
>>    Yosuke
>>
>>> Regards
>>> Masroor,
>>>  
>>>
>>> -----Original Message-----
>>> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
>>> Sent: Thursday, April 03, 2008 12:00 PM
>>> To: Masroor Vettuparambil
>>> Cc: xen-devel@lists.xensource.com
>>> Subject: incompatibility of netfront driver with bonding module
>>>
>>> Hi,
>>>
>>> Masroor Vettuparambil wrote:
>>>> 1. Normally, bonding will inherit the mac from the first slave
and
>>>> assign it to all the other slaves added later. So the mac of
vif
>>>> will
>>>> be updated. But during migration, the mac of vif is getting
updated
>>>> from xenstore(/vm/). So how about having a set_mac_address
entry in
>>>> netfront that update the xenstore?
>>>> So I need help, especially on #1. Is it ok to update the
xenstore
>>>> /vm/ keys from domU? how to do this?
>>> I tried a bit to find out the way to update /vm/ keys from
frontend,
>>> but didn''t succeed. Either way, I don''t think it
is a good idea to
>>> update the xenstore key of the vif mac address, because the
original
>>> mac will be lost and we won''t be able to reset it e.g.
after the
>>> guest
>> reboot.
>>> So the attached patch adds set mac_address() support to netfront.
>>> It doesn''t touch xenstore at all, but just preserves the
modified mac
> 
>>> address in netfront_info structure and keeps using it after
> migration.
>>> Thanks,
>>> -----------------------
>>> Yosuke Iwamatsu
>>>          NEC Corporation

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel