CentOS - Feb 2015 - C7, igb and DCB support for pause frame ?

Hi there,

I?m working on deploying our new cluster.
Masters have 5?1gbps (i210 and i350, thus using igb.ko), configured  
with mtu 9000, 802.3ad. Works fine *but* I can?t get DCB working  
(pause frame, aka flow control, which is supported by and enabled on  
our switches).

[root at master2 ~]# dcbtool gc eno1 dcb
Command:        Get Config
Feature:        DCB State
Port:           eno1
Status:         Device not capable

(I get the same with ELRepo 5.2.15 kmod).
Intel datasheet says flow control is available.
Can?t find much about it on the web or in kernel git repo. Could  
someone give me a hand ?

Regards,
Laurent.

On Thu, Feb 26, 2015 at 9:18 AM, Laurent Wandrebeck <
l.wandrebeck at quelquesmots.fr> wrote:
> Hi there,
>
> I?m working on deploying our new cluster.
> Masters have 5?1gbps (i210 and i350, thus using igb.ko), configured with
> mtu 9000, 802.3ad. Works fine *but* I can?t get DCB working (pause frame,
> aka flow control, which is supported by and enabled on our switches).
>
> [root at master2 ~]# dcbtool gc eno1 dcb
> Command:        Get Config
> Feature:        DCB State
> Port:           eno1
> Status:         Device not capable
>
> (I get the same with ELRepo 5.2.15 kmod).
> Intel datasheet says flow control is available.
> Can?t find much about it on the web or in kernel git repo. Could someone
> give me a hand ?
>
> Regards,
> Laurent.
>

 DCB requires Priority Flow Control(PFC) aka 802.1Qbb.
"Flow Control" is 802.3x.

The two are often confused and not compatible.

http://www.intel.com/content/www/us/en/ethernet-controllers/ethernet-controller-i350-datasheet.html

Mentions "flow control" several times, but never
PFC/priority-flow-control/802.1Qbb.

PFC capable switches purposefully disable 802.3x flow control. Also PFC has
to negotiate between two devices/switches matching QoS/CoS/no-drop policies.

Some good reading for beginner PFC knowledge:

http://www.cisco.com/c/dam/en/us/solutions/collateral/data-center-virtualization/ieee-802-1-data-center-bridging/at_a_glance_c45-460907.pdf

What exactly are you trying to pause? Typically FCoE/iSCSI is set to
"no-drop" and Ethernet traffic is paused/dropped in favor of storage
traffic. If there is only one type/class/CoS of traffic PFC won't gain much
over regular flow control/802.3x.

Hope that helps.

Steven Tardy <sjt5atra at gmail.com> a ?crit?:
>  DCB requires Priority Flow Control(PFC) aka 802.1Qbb.
> "Flow Control" is 802.3x.
>
> The two are often confused and not compatible.
>
>
http://www.intel.com/content/www/us/en/ethernet-controllers/ethernet-controller-i350-datasheet.html
>
> Mentions "flow control" several times, but never
> PFC/priority-flow-control/802.1Qbb.
>
> PFC capable switches purposefully disable 802.3x flow control. Also PFC has
> to negotiate between two devices/switches matching QoS/CoS/no-drop
policies.
>
> Some good reading for beginner PFC knowledge:
>
>
http://www.cisco.com/c/dam/en/us/solutions/collateral/data-center-virtualization/ieee-802-1-data-center-bridging/at_a_glance_c45-460907.pdf
>
> What exactly are you trying to pause? Typically FCoE/iSCSI is set to
> "no-drop" and Ethernet traffic is paused/dropped in favor of
storage
> traffic. If there is only one type/class/CoS of traffic PFC won't gain
much
> over regular flow control/802.3x.
>
> Hope that helps.
Hello Steven,

You?ve been really helpful !

Our switches indeed do support 802.3x and not 802.1Qbb.

Ethtool telling:
Supported pause frame use: Symmetric
Advertised pause frame use: Symmetric

I guess (I?m more of a sysad guy than netad) we?re on the right track  
and have no need of DCB* and lldpad.

Actually, our masters will be metadata server for the distributed FS  
(RozoFS not to name it), and will export a system image via NFS?to  
nodes (2?1gbps, 802.3ad) which are ? diskless ? (no disk for OS but  
disks for distributed FS storage only).
FC (802.3x) usage is mandatory for RozoFS.
There will be some other traffic due to HTCondor (nodes will be  
execute nodes too), syslog being centralized on masters?
I know, that not the perfect config, but we had to do that way due to  
budget constraints.
Now I need to find how to get a single image for all the nodes :)  
(PXE, dhcpd, dracut and yum --installroot should do the trick I hope).

Thanks again for the head?s up !

Regards,
Laurent.