Pete French
2007-Jul-04 14:59 UTC
should if_lagg balance outbound traffic on an lacp connection ?
Having recently discovered if_lagg in stable I have spent the last couple
of days experimentsin with this and using it to aggregate connections between
a pair of servers and a pair of Cisco switches. It all appears to be
functioning, but looking at the stats I see this:
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
bce0 1500 <Link#1> 00:19:bb:33:00:de 7545268 0 9303330 0
0
bce1 1500 <Link#2> 00:19:bb:33:00:de 7099168 0 126 0
0
lo0 16384 <Link#3> 15377 0 15376 0
0
lo0 16384 fe80:3::1 fe80:3::1 0 - 0 - -
lo0 16384 localhost ::1 60 - 60 - -
lo0 16384 your-net localhost 14222 - 14221 - -
lagg0 1500 <Link#4> 00:19:bb:33:00:de 7545187 0 9303208 58
0
lagg0 1500 10.17.16/20 turpentine 7539218 - 9305974 - -
Note that whilst I am getting evenly split traffic comming in from the switch,
outgoing traffic is all heading out on bce0. The ifconfig output looks
like this:
options=3b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
ether 00:19:bb:33:00:de
media: Ethernet autoselect (1000baseTX <full-duplex>)
status: active
lagg: laggdev lagg0
bce1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=3b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
ether 00:19:bb:33:00:de
media: Ethernet autoselect (1000baseTX <full-duplex>)
status: active
lagg: laggdev lagg0
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
inet 10.17.19.0 netmask 0xfffff000 broadcast 10.17.31.255
ether 00:19:bb:33:00:de
media: Ethernet autoselect
status: active
lagg: laggproto lacp
laggport bce1 =18<COLLECTING,DISTRIBUTING>
laggport bce0 =1c<ACTIVE,COLLECTING,DISTRIBUTING>
On the switch both ports are configured in LACP 'active' mode. Is this
expected behaviour, or have I done something wrong ? My rc.conf file looks
like this:
cloned_interfaces="lagg0"
network_interfaces="lo0 bce0 bce1 lagg0"
ifconfig_bce0="up"
ifconfig_bce1="up"
ifconfig_lagg0="laggproto lacp laggport bce0 laggport bce1"
ipv4_addrs_lagg0="10.17.19.0/20"
Any advice ? Or that simply the expected behaviour ? (I dont actuall mind
if it is - I am aiming for fault tolerance not performance on this
connection, but I am curious for future reference)
cheers,
-pcf.
Pete French
2007-Jul-04 15:14 UTC
should if_lagg balance outbound traffic on an lacp connection ?
Having recently discovered if_lagg in stable I have spent the last couple
of days experimentsin with this and using it to aggregate connections between
a pair of servers and a pair of Cisco switches. It all appears to be
functioning, but looking at the stats I see this:
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
bce0 1500 <Link#1> 00:19:bb:33:00:de 7545268 0 9303330 0
0
bce1 1500 <Link#2> 00:19:bb:33:00:de 7099168 0 126 0
0
lo0 16384 <Link#3> 15377 0 15376 0
0
lo0 16384 fe80:3::1 fe80:3::1 0 - 0 - -
lo0 16384 localhost ::1 60 - 60 - -
lo0 16384 your-net localhost 14222 - 14221 - -
lagg0 1500 <Link#4> 00:19:bb:33:00:de 7545187 0 9303208 58
0
lagg0 1500 10.17.16/20 turpentine 7539218 - 9305974 - -
Note that whilst I am getting evenly split traffic comming in from the switch,
outgoing traffic is all heading out on bce0. The ifconfig output looks
like this:
options=3b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
ether 00:19:bb:33:00:de
media: Ethernet autoselect (1000baseTX <full-duplex>)
status: active
lagg: laggdev lagg0
bce1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=3b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
ether 00:19:bb:33:00:de
media: Ethernet autoselect (1000baseTX <full-duplex>)
status: active
lagg: laggdev lagg0
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
inet 10.17.19.0 netmask 0xfffff000 broadcast 10.17.31.255
ether 00:19:bb:33:00:de
media: Ethernet autoselect
status: active
lagg: laggproto lacp
laggport bce1 =18<COLLECTING,DISTRIBUTING>
laggport bce0 =1c<ACTIVE,COLLECTING,DISTRIBUTING>
On the switch both ports are configured in LACP 'active' mode. Is this
expected behaviour, or have I done something wrong ? My rc.conf file looks
like this:
cloned_interfaces="lagg0"
network_interfaces="lo0 bce0 bce1 lagg0"
ifconfig_bce0="up"
ifconfig_bce1="up"
ifconfig_lagg0="laggproto lacp laggport bce0 laggport bce1"
ipv4_addrs_lagg0="10.17.19.0/20"
Any advice ? Or that simply the expected behaviour ? (I dont actuall mind
if it is - I am aiming for fault tolerance not performance on this
connection, but I am curious for future reference)
cheers,
-pcf.
Andrew Thompson
2007-Jul-04 18:31 UTC
should if_lagg balance outbound traffic on an lacp connection ?
On Wed, Jul 04, 2007 at 03:44:06PM +0100, Pete French wrote:> Having recently discovered if_lagg in stable I have spent the last couple > of days experimentsin with this and using it to aggregate connections between > a pair of servers and a pair of Cisco switches. It all appears to be > functioning, but looking at the stats I see this: > > Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll > bce0 1500 <Link#1> 00:19:bb:33:00:de 7545268 0 9303330 0 0 > bce1 1500 <Link#2> 00:19:bb:33:00:de 7099168 0 126 0 0 > lo0 16384 <Link#3> 15377 0 15376 0 0 > lo0 16384 fe80:3::1 fe80:3::1 0 - 0 - - > lo0 16384 localhost ::1 60 - 60 - - > lo0 16384 your-net localhost 14222 - 14221 - - > lagg0 1500 <Link#4> 00:19:bb:33:00:de 7545187 0 9303208 58 0 > lagg0 1500 10.17.16/20 turpentine 7539218 - 9305974 - - > > Note that whilst I am getting evenly split traffic comming in from the switch, > outgoing traffic is all heading out on bce0. The ifconfig output looks > like this: > > options=3b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU> > ether 00:19:bb:33:00:de > media: Ethernet autoselect (1000baseTX <full-duplex>) > status: active > lagg: laggdev lagg0 > bce1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 > options=3b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU> > ether 00:19:bb:33:00:de > media: Ethernet autoselect (1000baseTX <full-duplex>) > status: active > lagg: laggdev lagg0 > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > inet6 ::1 prefixlen 128 > inet 127.0.0.1 netmask 0xff000000 > lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 > inet 10.17.19.0 netmask 0xfffff000 broadcast 10.17.31.255 > ether 00:19:bb:33:00:de > media: Ethernet autoselect > status: active > lagg: laggproto lacp > laggport bce1 =18<COLLECTING,DISTRIBUTING> > laggport bce0 =1c<ACTIVE,COLLECTING,DISTRIBUTING> >bce1 is a concern here as it is not in the ACTIVE state. On your switch have a look at the lacp stats, here is an example from mine with a 4 port aggregation. c2950#sh lacp neighbor Flags: S - Device is requesting Slow LACPDUs F - Device is requesting Fast LACPDUs A - Device is in Active mode P - Device is in Passive mode Channel group 1 neighbors Partner's information: LACP port Oper Port Port Port Flags Priority Dev ID Age Key Number State Fa0/11 SA 32768 0005.5d71.8db8 4s 0xE6 0x1 0x3D Fa0/12 SA 32768 0005.5d71.8db8 5s 0xE6 0x2 0x3D Fa0/13 SA 32768 0005.5d71.8db8 4s 0xE6 0x3 0x3D Fa0/14 SA 32768 0005.5d71.8db8 3s 0xE6 0x4 0x3D c2950#sh lacp internal Flags: S - Device is requesting Slow LACPDUs F - Device is requesting Fast LACPDUs A - Device is in Active mode P - Device is in Passive mode Channel group 1 LACP port Admin Oper Port Port Port Flags State Priority Key Key Number State Fa0/11 SA bndl 32768 0x1 0x1 0xB 0x3D Fa0/12 SA bndl 32768 0x1 0x1 0xC 0x3D Fa0/13 SA bndl 32768 0x1 0x1 0xD 0x3D Fa0/14 SA bndl 32768 0x1 0x1 0xE 0x3D As you can see all ports are in 'bndl' state which means they have been negoiated. lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 00:05:5d:71:8d:b8 media: Ethernet autoselect status: active lagg: laggproto lacp laggport ste3 =1c<ACTIVE,COLLECTING,DISTRIBUTING> laggport ste2 =1c<ACTIVE,COLLECTING,DISTRIBUTING> laggport ste1 =1c<ACTIVE,COLLECTING,DISTRIBUTING> laggport ste0 =1c<ACTIVE,COLLECTING,DISTRIBUTING> And all ports here are ACTIVE. I have been meaning to add IOCTLS to display more lacp stats to help debug this sort of thing. Andrew