crossbow discuss - Jun 2009 - ?: what happens on a MAC collision when creating a VNIC

I took a look at the docs below, and did not see how Solaris handles a 
MAC collision. Since users have several options when creating a VNIC 
regarding the use of or creation of a MAC address, including specifying 
their own IEEE OUI, what happens if there is a collision? Especially 
when it is not being set, but partially or fully random? Does it fail, 
or does it re-try?

Thanks
Steffen

http://opensolaris.org/os/project/crossbow/Docs/crossbow-drivers.pdf
http://opensolaris.org/os/project/crossbow/Docs/crossbow-virt.pdf

On Jun 22, 2009, at 1:51 PM, Steffen Weiberle wrote:
> I took a look at the docs below, and did not see how Solaris handles  
> a MAC collision. Since users have several options when creating a  
> VNIC regarding the use of or creation of a MAC address, including  
> specifying their own IEEE OUI, what happens if there is a collision?  
> Especially when it is not being set, but partially or fully random?  
> Does it fail, or does it re-try?
Steffen,

When a VNIC is created we have checks in place to ensure that the  
address doesn''t conflict with another MAC address defined on top of  
the same underlying NIC. When the MAC address is generated randomly,  
and the generated MAC address conflicts with another VNIC, we  
currently fail the whole operation. We should try another MAC address  
in that case, transparently to the user, I filed CR 6853771 to track  
this.

To reduce the risk of MAC address conflicts with physical NICs on  
other hosts on the network, we use by default a OUI with local bit set  
for random MAC addresses, and we let the administrator use a different  
OUI or prefix if desired. We currently don''t have a mechanism in place
do perform automatic MAC address duplication detection between  
multiple hosts.

Nicolas.

>
> Thanks
> Steffen
>
> http://opensolaris.org/os/project/crossbow/Docs/crossbow-drivers.pdf
> http://opensolaris.org/os/project/crossbow/Docs/crossbow-virt.pdf
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss
-- 
Nicolas Droux - Solaris Kernel Networking - Sun Microsystems, Inc.
nicolas.droux at sun.com - http://blogs.sun.com/droux

On 06/22/09 19:14, Nicolas Droux wrote:
> 
> On Jun 22, 2009, at 1:51 PM, Steffen Weiberle wrote:
> 
>> I took a look at the docs below, and did not see how Solaris handles a 
>> MAC collision. Since users have several options when creating a VNIC 
>> regarding the use of or creation of a MAC address, including 
>> specifying their own IEEE OUI, what happens if there is a collision? 
>> Especially when it is not being set, but partially or fully random? 
>> Does it fail, or does it re-try?
> 
> Steffen,
> 
> When a VNIC is created we have checks in place to ensure that the 
> address doesn''t conflict with another MAC address defined on top
of the
> same underlying NIC. When the MAC address is generated randomly, and the 
> generated MAC address conflicts with another VNIC, we currently fail the 
> whole operation. We should try another MAC address in that case, 
> transparently to the user, I filed CR 6853771 to track this.
> 
> To reduce the risk of MAC address conflicts with physical NICs on other 
> hosts on the network, we use by default a OUI with local bit set for 
> random MAC addresses, and we let the administrator use a different OUI 
> or prefix if desired. We currently don''t have a mechanism in place
do
> perform automatic MAC address duplication detection between multiple hosts.
> 
> Nicolas.
Hi Nicolas,

Thanks. I did not think about internal collisions, so my question was 
about what you addressed in the second part. I did confirm that by 
setting a VNIC MAC address to that of another host, I can prevent 
communications silently, and that confirms that there is no duplication 
detection across hosts.

Regarding within a system, is the check only for conflicts on the same 
NIC, or in the whole system? I read the former, so it might be possible 
to have multiple VNICs on different NICs with the same MAC (similar to 
OBP local-mac-address?=false)?

Steffen

OBP=OpenBoot PROM, which has the option to use only one MAC address for 
a system (for the times when NICs did not deliver a MAC address) or to 
use the addresses delivered with different NICs
>> Thanks
>> Steffen
>>
>> http://opensolaris.org/os/project/crossbow/Docs/crossbow-drivers.pdf
>> http://opensolaris.org/os/project/crossbow/Docs/crossbow-virt.pdf
>> _______________________________________________
>> crossbow-discuss mailing list
>> crossbow-discuss at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss
>

On Jun 23, 2009, at 5:35 AM, Steffen Weiberle wrote:
> On 06/22/09 19:14, Nicolas Droux wrote:
>> On Jun 22, 2009, at 1:51 PM, Steffen Weiberle wrote:
>>> I took a look at the docs below, and did not see how Solaris  
>>> handles a MAC collision. Since users have several options when  
>>> creating a VNIC regarding the use of or creation of a MAC address,
>>> including specifying their own IEEE OUI, what happens if there is  
>>> a collision? Especially when it is not being set, but partially or
>>> fully random? Does it fail, or does it re-try?
>> Steffen,
>> When a VNIC is created we have checks in place to ensure that the  
>> address doesn''t conflict with another MAC address defined on
top of
>> the same underlying NIC. When the MAC address is generated  
>> randomly, and the generated MAC address conflicts with another  
>> VNIC, we currently fail the whole operation. We should try another  
>> MAC address in that case, transparently to the user, I filed CR  
>> 6853771 to track this.
>> To reduce the risk of MAC address conflicts with physical NICs on  
>> other hosts on the network, we use by default a OUI with local bit  
>> set for random MAC addresses, and we let the administrator use a  
>> different OUI or prefix if desired. We currently don''t have a
>> mechanism in place do perform automatic MAC address duplication  
>> detection between multiple hosts.
>> Nicolas.
>
> Hi Nicolas,
>
> Thanks. I did not think about internal collisions, so my question  
> was about what you addressed in the second part. I did confirm that  
> by setting a VNIC MAC address to that of another host, I can prevent  
> communications silently, and that confirms that there is no  
> duplication detection across hosts.
>
> Regarding within a system, is the check only for conflicts on the  
> same NIC, or in the whole system? I read the former, so it might be  
> possible to have multiple VNICs on different NICs with the same MAC  
> (similar to OBP local-mac-address?=false)?
Yes that''s correct, it''s the former, on a per NIC basis.

Nicolas.
>
> Steffen
>
> OBP=OpenBoot PROM, which has the option to use only one MAC address  
> for a system (for the times when NICs did not deliver a MAC address)  
> or to use the addresses delivered with different NICs
>
>>> Thanks
>>> Steffen
>>>
>>>
http://opensolaris.org/os/project/crossbow/Docs/crossbow-drivers.pdf
>>> http://opensolaris.org/os/project/crossbow/Docs/crossbow-virt.pdf
>>> _______________________________________________
>>> crossbow-discuss mailing list
>>> crossbow-discuss at opensolaris.org
>>> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss
>
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss
-- 
Nicolas Droux - Solaris Kernel Networking - Sun Microsystems, Inc.
nicolas.droux at sun.com - http://blogs.sun.com/droux

On 06/23/09 11:54, Nicolas Droux wrote:
> 
> On Jun 23, 2009, at 5:35 AM, Steffen Weiberle wrote:
> 
>> On 06/22/09 19:14, Nicolas Droux wrote:
>>> On Jun 22, 2009, at 1:51 PM, Steffen Weiberle wrote:
>>>> I took a look at the docs below, and did not see how Solaris
handles
>>>> a MAC collision. Since users have several options when creating
a
>>>> VNIC regarding the use of or creation of a MAC address,
including
>>>> specifying their own IEEE OUI, what happens if there is a
collision?
>>>> Especially when it is not being set, but partially or fully
random?
>>>> Does it fail, or does it re-try?
>>> Steffen,
>>> When a VNIC is created we have checks in place to ensure that the 
>>> address doesn''t conflict with another MAC address defined
on top of
>>> the same underlying NIC. When the MAC address is generated
randomly,
>>> and the generated MAC address conflicts with another VNIC, we 
>>> currently fail the whole operation. We should try another MAC
address
>>> in that case, transparently to the user, I filed CR 6853771 to
track
>>> this.
>>> To reduce the risk of MAC address conflicts with physical NICs on 
>>> other hosts on the network, we use by default a OUI with local bit 
>>> set for random MAC addresses, and we let the administrator use a 
>>> different OUI or prefix if desired. We currently don''t
have a
>>> mechanism in place do perform automatic MAC address duplication 
>>> detection between multiple hosts.
>>> Nicolas.
>>
>> Hi Nicolas,
>>
>> Thanks. I did not think about internal collisions, so my question was 
>> about what you addressed in the second part. I did confirm that by 
>> setting a VNIC MAC address to that of another host, I can prevent 
>> communications silently, and that confirms that there is no 
>> duplication detection across hosts.
>>
>> Regarding within a system, is the check only for conflicts on the same 
>> NIC, or in the whole system? I read the former, so it might be 
>> possible to have multiple VNICs on different NICs with the same MAC 
>> (similar to OBP local-mac-address?=false)?
> 
> Yes that''s correct, it''s the former, on a per NIC basis.
> 
> Nicolas.
Thanks! On a related topic, is there a way yet to see how many and what 
factory MAC addresses are delivered and available? I don''t see any 
indication on my NICs, however, they only have one afaik.

It might be worth adding this info to
http://www.opensolaris.org/os/project/crossbow/topics/nic/
and I can work with Raoul on it if it makes sense to put it there.

Steffen

Nicolas Droux wrote:
>
> On Jun 22, 2009, at 1:51 PM, Steffen Weiberle wrote:
>
>> I took a look at the docs below, and did not see how Solaris handles 
>> a MAC collision. Since users have several options when creating a 
>> VNIC regarding the use of or creation of a MAC address, including 
>> specifying their own IEEE OUI, what happens if there is a collision? 
>> Especially when it is not being set, but partially or fully random? 
>> Does it fail, or does it re-try?
>
> Steffen,
>
> When a VNIC is created we have checks in place to ensure that the 
> address doesn''t conflict with another MAC address defined on top
of
> the same underlying NIC. When the MAC address is generated randomly, 
> and the generated MAC address conflicts with another VNIC, we 
> currently fail the whole operation. We should try another MAC address 
> in that case, transparently to the user, I filed CR 6853771 to track 
> this.
>
> To reduce the risk of MAC address conflicts with physical NICs on 
> other hosts on the network, we use by default a OUI with local bit set 
> for random MAC addresses, and we let the administrator use a different 
> OUI or prefix if desired. We currently don''t have a mechanism in
place
> do perform automatic MAC address duplication detection between 
> multiple hosts.
This makes me wonder if having a daemon and some sort of protocol to 
(dynamically?) allocate MAC addresses wouldn''t be such a bad idea.

Darren

On Jun 23, 2009, at 9:09 AM, Steffen Weiberle wrote:
> Thanks! On a related topic, is there a way yet to see how many and  
> what factory MAC addresses are delivered and available? I don''t
see
> any indication on my NICs, however, they only have one afaik.
You can get that info on a running system using ''dladm show-phys
-m''.
> It might be worth adding this info to
> http://www.opensolaris.org/os/project/crossbow/topics/nic/
> and I can work with Raoul on it if it makes sense to put it there.
That would be great. For now the only NICs we''ve seen which supports  
multiple factory MAC addresses are those driven by nxge (Neptune/N2  
NIU), which provide 16 addresses.

Nicolas.

-- 
Nicolas Droux - Solaris Kernel Networking - Sun Microsystems, Inc.
nicolas.droux at sun.com - http://blogs.sun.com/droux

On 06/23/09 15:09, Nicolas Droux wrote:
>
> On Jun 23, 2009, at 9:09 AM, Steffen Weiberle wrote:
>
>> Thanks! On a related topic, is there a way yet to see how many and 
>> what factory MAC addresses are delivered and available? I
don''t see
>> any indication on my NICs, however, they only have one afaik.
>
> You can get that info on a running system using ''dladm show-phys
-m''.
>
>> It might be worth adding this info to
>> http://www.opensolaris.org/os/project/crossbow/topics/nic/
>> and I can work with Raoul on it if it makes sense to put it there.
>
> That would be great. For now the only NICs we''ve seen which
supports
> multiple factory MAC addresses are those driven by nxge (Neptune/N2 
> NIU), which provide 16 addresses.
Don''t forget hxge as well.
>
> Nicolas.
>

-- 
Michael F. Speer
Senior Staff Engineer, Ethernet Connectivity and Fabrics 
Netra Systems and Networking
Sun Microsystems, Inc
17 Network Circle
Menlo Park, CA 94025
Phone: +1-650-786-3200
email: michael.speer at sun.com

On Jun 23, 2009, at 9:28 AM, Darren Reed wrote:
> Nicolas Droux wrote:
>>
>> On Jun 22, 2009, at 1:51 PM, Steffen Weiberle wrote:
>>
>>> I took a look at the docs below, and did not see how Solaris  
>>> handles a MAC collision. Since users have several options when  
>>> creating a VNIC regarding the use of or creation of a MAC address,
>>> including specifying their own IEEE OUI, what happens if there is  
>>> a collision? Especially when it is not being set, but partially or
>>> fully random? Does it fail, or does it re-try?
>>
>> Steffen,
>>
>> When a VNIC is created we have checks in place to ensure that the  
>> address doesn''t conflict with another MAC address defined on
top of
>> the same underlying NIC. When the MAC address is generated  
>> randomly, and the generated MAC address conflicts with another  
>> VNIC, we currently fail the whole operation. We should try another  
>> MAC address in that case, transparently to the user, I filed CR  
>> 6853771 to track this.
>>
>> To reduce the risk of MAC address conflicts with physical NICs on  
>> other hosts on the network, we use by default a OUI with local bit  
>> set for random MAC addresses, and we let the administrator use a  
>> different OUI or prefix if desired. We currently don''t have a
>> mechanism in place do perform automatic MAC address duplication  
>> detection between multiple hosts.
>
> This makes me wonder if having a daemon and some sort of protocol to  
> (dynamically?) allocate MAC addresses wouldn''t be such a bad idea.
Sure, the mechanisms that we provide today can be used to build some  
other protocols to do more complex distributed MAC address allocation  
on a network.

Nicolas.

>
> Darren
>
-- 
Nicolas Droux - Solaris Kernel Networking - Sun Microsystems, Inc.
nicolas.droux at sun.com - http://blogs.sun.com/droux

On Jun 23, 2009, at 3:12 PM, Michael Speer wrote:
> On 06/23/09 15:09, Nicolas Droux wrote:
>>
>> On Jun 23, 2009, at 9:09 AM, Steffen Weiberle wrote:
>>
>>> Thanks! On a related topic, is there a way yet to see how many and
>>> what factory MAC addresses are delivered and available? I
don''t
>>> see any indication on my NICs, however, they only have one afaik.
>>
>> You can get that info on a running system using ''dladm
show-phys -m''.
>>
>>> It might be worth adding this info to
>>> http://www.opensolaris.org/os/project/crossbow/topics/nic/
>>> and I can work with Raoul on it if it makes sense to put it there.
>>
>> That would be great. For now the only NICs we''ve seen which  
>> supports multiple factory MAC addresses are those driven by nxge  
>> (Neptune/N2 NIU), which provide 16 addresses.
> Don''t forget hxge as well.
I''m looking at hxge_main.c and it doesn''t advertise the  
MAC_CAPAB_MULTIFACTADDR capability from hxge_m_getcapab().  
MAC_CAPAB_MULTIFACTADDR is used by drivers to expose their multiple  
factory MAC address values. If hxge support multiple factory MAC  
address values, we should file a CR to add this support to the driver.

Note that I was specifically talking about the factory MAC address  
values, not the unicast MAC address slots. Pre-Crossbow we had only  
one capability to handle both, now the MAC address slots are leveraged  
through MAC_CAPAB_RINGS, and factory MAC address values are exposed  
through MAC_CAPAB_MULTIFACTADDR.

Nicolas.

-- 
Nicolas Droux - Solaris Kernel Networking - Sun Microsystems, Inc.
nicolas.droux at sun.com - http://blogs.sun.com/droux