Frank Thommen
2017-Apr-18 14:54 UTC
[CentOS] anaconda/kickstart: bonding device not created as expected
Hi,
I am currently struggling with the right way to configure a bonding
device via kickstart (via PXE).
I am installing servers which have "eno" network interfaces. Instead
of
the expected bonding device with two active slaves (bonding mode is
balance-alb), I get a bonding device with only one active slave and an
independent, non-bonded network device. Also the bonding device gets
its MAC address from the second instead of from the first device.
I appreciate any hint (or rtfm with the name of the correct fm ;-) on
how to achieve the desired setup through kickstart. Please find the
used PXE and kickstart settings and resulting network configuration below.
I did this with CentOS 7.2.1511. We cannot go further due to Infiniband
and lustre drivers which are currently only supported for this CentOS
7.x version
Cheers
frank
--------------------------------------------------
The used PXE configuration is
LABEL CentOS-7
kernel centos-7/vmlinuz
append initrd=centos-7/initrd.img ip=dhcp
nameserver=xx.xx.xx.xx ksdevice=eno1
inst.repo=http://our.mirror.server/7/os/x86_64 inst.ks.sendmac
inst.ks=http://our.kickstart.server/ks.cgi
and the network settings in the kickstart file are
network --device bond0 --bondslaves=eno1,eno2
--bondopts=mode=balance-alb --bootproto=dhcp --hostname=myhost --activate
I would have expected to get a bonding device with eno1 and eno2 as
slave devices, the bonding device inheriting the MAC address from eno1
(otherwise DHCP won't work). Instead the result is a bonding device
with eno2 as - sole - slave device and eno1 as a single active device
with the main IP address of the host:
bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST> mtu 1500
inet6 fe80::42f2:e9ff:fec7:b5f1 prefixlen 64 scopeid 0x20<link>
ether 40:f2:e9:c7:b5:f1 txqueuelen 0 (Ethernet)
RX packets 29 bytes 5274 (5.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 39 bytes 3486 (3.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet xx.xx.xx.xx netmask 255.255.255.0 broadcast xx.xx.xx.xx
inet6 fe80::42f2:e9ff:fec7:b5f0 prefixlen 64 scopeid 0x20<link>
ether 40:f2:e9:c7:b5:f0 txqueuelen 1000 (Ethernet)
RX packets 4303 bytes 798163 (779.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1686 bytes 481585 (470.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16
eno2: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500
ether 40:f2:e9:c7:b5:f1 txqueuelen 1000 (Ethernet)
RX packets 29 bytes 5274 (5.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 39 bytes 3486 (3.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 17
The ifcfg-files look basically ok, but there are two for the eno1 device.
ifcfg of the bonding device:
$ cat ifcfg-bond0
# Generated by parse-kickstart
IPV6INIT="yes"
DHCP_HOSTNAME="myhost"
NAME="Bond connection bond0"
BONDING_MASTER="yes"
BOOTPROTO="dhcp"
BONDING_OPTS="mode=balance-alb"
DEVICE="bond0"
TYPE="Bond"
ONBOOT="yes"
UUID="35910614-4a7c-43c9-8e44-dcf44b783358"
$
ifcfg of the two slave devices
$ cat ifcfg-bond0_slave_1
# Generated by parse-kickstart
NAME="bond0 slave 1"
MASTER="35910614-4a7c-43c9-8e44-dcf44b783358"
HWADDR="40:f2:e9:c7:b5:f0"
TYPE="Ethernet"
ONBOOT="yes"
UUID="f3a0a007-861c-42b6-8264-6efba62232ce"
$
$ cat ifcfg-bond0_slave_2
# Generated by parse-kickstart
NAME="bond0 slave 2"
MASTER="35910614-4a7c-43c9-8e44-dcf44b783358"
HWADDR="40:f2:e9:c7:b5:f1"
TYPE="Ethernet"
ONBOOT="yes"
UUID="ee3f7c84-d4cb-412e-887d-6b1c753eb913"
$
ifcfg of eno1 (which physically has the MAC address 40:f2:e9:c7:b5:f0,
which is the same as ifcfg-bond0_slave_1
$ cat ifcfg-eno1
# Generated by dracut initrd
NAME="eno1"
DEVICE="eno1"
ONBOOT=yes
NETBOOT=yes
UUID="d20645a0-8093-45f3-9630-d0249f76726b"
IPV6INIT=yes
BOOTPROTO=dhcp
TYPE=Ethernet
DNS1="192.55.188.177"
$
Tris Hoar
2017-Apr-19 16:07 UTC
[CentOS] anaconda/kickstart: bonding device not created as expected
On 18/04/2017 15:54, Frank Thommen wrote:> Hi, > > I am currently struggling with the right way to configure a bonding > device via kickstart (via PXE). > > I am installing servers which have "eno" network interfaces. Instead of > the expected bonding device with two active slaves (bonding mode is > balance-alb), I get a bonding device with only one active slave and an > independent, non-bonded network device. Also the bonding device gets > its MAC address from the second instead of from the first device. > > I appreciate any hint (or rtfm with the name of the correct fm ;-) on > how to achieve the desired setup through kickstart. Please find the > used PXE and kickstart settings and resulting network configuration below. > > I did this with CentOS 7.2.1511. We cannot go further due to Infiniband > and lustre drivers which are currently only supported for this CentOS > 7.x version > > Cheers > frank > > -------------------------------------------------- > > The used PXE configuration is > > LABEL CentOS-7 > kernel centos-7/vmlinuz > append initrd=centos-7/initrd.img ip=dhcp nameserver=xx.xx.xx.xx > ksdevice=eno1 inst.repo=http://our.mirror.server/7/os/x86_64 > inst.ks.sendmac inst.ks=http://our.kickstart.server/ks.cgi > > > and the network settings in the kickstart file are > > network --device bond0 --bondslaves=eno1,eno2 > --bondopts=mode=balance-alb --bootproto=dhcp --hostname=myhost --activate > > > I would have expected to get a bonding device with eno1 and eno2 as > slave devices, the bonding device inheriting the MAC address from eno1 > (otherwise DHCP won't work). Instead the result is a bonding device > with eno2 as - sole - slave device and eno1 as a single active device > with the main IP address of the host: > > > bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST> mtu 1500 > inet6 fe80::42f2:e9ff:fec7:b5f1 prefixlen 64 scopeid 0x20<link> > ether 40:f2:e9:c7:b5:f1 txqueuelen 0 (Ethernet) > RX packets 29 bytes 5274 (5.1 KiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 39 bytes 3486 (3.4 KiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 > inet xx.xx.xx.xx netmask 255.255.255.0 broadcast xx.xx.xx.xx > inet6 fe80::42f2:e9ff:fec7:b5f0 prefixlen 64 scopeid 0x20<link> > ether 40:f2:e9:c7:b5:f0 txqueuelen 1000 (Ethernet) > RX packets 4303 bytes 798163 (779.4 KiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 1686 bytes 481585 (470.2 KiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > device interrupt 16 > > eno2: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500 > ether 40:f2:e9:c7:b5:f1 txqueuelen 1000 (Ethernet) > RX packets 29 bytes 5274 (5.1 KiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 39 bytes 3486 (3.4 KiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > device interrupt 17 > > > The ifcfg-files look basically ok, but there are two for the eno1 device. > > ifcfg of the bonding device: > > $ cat ifcfg-bond0 > # Generated by parse-kickstart > IPV6INIT="yes" > DHCP_HOSTNAME="myhost" > NAME="Bond connection bond0" > BONDING_MASTER="yes" > BOOTPROTO="dhcp" > BONDING_OPTS="mode=balance-alb" > DEVICE="bond0" > TYPE="Bond" > ONBOOT="yes" > UUID="35910614-4a7c-43c9-8e44-dcf44b783358" > $ > > > ifcfg of the two slave devices > > $ cat ifcfg-bond0_slave_1 > # Generated by parse-kickstart > NAME="bond0 slave 1" > MASTER="35910614-4a7c-43c9-8e44-dcf44b783358" > HWADDR="40:f2:e9:c7:b5:f0" > TYPE="Ethernet" > ONBOOT="yes" > UUID="f3a0a007-861c-42b6-8264-6efba62232ce" > $ > > > $ cat ifcfg-bond0_slave_2 > # Generated by parse-kickstart > NAME="bond0 slave 2" > MASTER="35910614-4a7c-43c9-8e44-dcf44b783358" > HWADDR="40:f2:e9:c7:b5:f1" > TYPE="Ethernet" > ONBOOT="yes" > UUID="ee3f7c84-d4cb-412e-887d-6b1c753eb913" > $ > > > ifcfg of eno1 (which physically has the MAC address 40:f2:e9:c7:b5:f0, > which is the same as ifcfg-bond0_slave_1 > > $ cat ifcfg-eno1 > # Generated by dracut initrd > NAME="eno1" > DEVICE="eno1" > ONBOOT=yes > NETBOOT=yes > UUID="d20645a0-8093-45f3-9630-d0249f76726b" > IPV6INIT=yes > BOOTPROTO=dhcp > TYPE=Ethernet > DNS1="192.55.188.177" > $ >Hi Frank, This is from my satellite kickstart where I'm building the bond at the point of PXE booting, and using static (I'm working on doing this with DHCP and tagged VLANs but currently cant get to the hardware needed since messing up the BMC config :( ) LABEL linux KERNEL boot/RedHat-7.3-x86_64-vmlinuz APPEND initrd=boot/RedHat-7.3-x86_64-initrd.img ks=http://example.com/host.ks ks.device=bootif network ks.sendmac bond=bond0:eno1,eno2:mode=802.3ad vlan=bond0.10:bond0 ip=10.10.0.2::10.10.0.1:255.255.255.0:host.example.com:bond0.10:none nameserver=10.10.0.1 Then in the KS we have network --bootproto=static --device=link --gateway=10.10.0.1 --hostname=host.example.com --ip=10.10.0.2 --nameserver=10.10.0.1,10.11.0.1 --netmask=255.255.255.0 It should be fairly simple to convert that to use DHCP as you just need to change the IP line in the kernel parameters and ensure you have --device=link in the kickstart. BTW cat /proc/net/bonding/bond0 shows more info on how the bond is actually setup and working. Tris ************************************************************* This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify postmaster at bgfl.org The views expressed within this email are those of the individual, and not necessarily those of the organisation *************************************************************
Frank Thommen
2017-Apr-21 19:19 UTC
[CentOS] anaconda/kickstart: bonding device not created as expected
Hi Tris, On 04/19/2017 06:07 PM, Tris Hoar wrote:> On 18/04/2017 15:54, Frank Thommen wrote: >> Hi, >> >> I am currently struggling with the right way to configure a bonding >> device via kickstart (via PXE). >> >> I am installing servers which have "eno" network interfaces. Instead of >> the expected bonding device with two active slaves (bonding mode is >> balance-alb), I get a bonding device with only one active slave and an >> independent, non-bonded network device. Also the bonding device gets >> its MAC address from the second instead of from the first device. >> >> [...] >> >> The used PXE configuration is >> >> LABEL CentOS-7 >> kernel centos-7/vmlinuz >> append initrd=centos-7/initrd.img ip=dhcp nameserver=xx.xx.xx.xx >> ksdevice=eno1 inst.repo=http://our.mirror.server/7/os/x86_64 >> inst.ks.sendmac inst.ks=http://our.kickstart.server/ks.cgi >> >> >> and the network settings in the kickstart file are >> >> network --device bond0 --bondslaves=eno1,eno2 >> --bondopts=mode=balance-alb --bootproto=dhcp --hostname=myhost --activate >> >> >> [...] >> > > Hi Frank, > > This is from my satellite kickstart where I'm building the bond at the > point of PXE booting, and using static (I'm working on doing this with > DHCP and tagged VLANs but currently cant get to the hardware needed > since messing up the BMC config :( ) > > LABEL linux > KERNEL boot/RedHat-7.3-x86_64-vmlinuz > APPEND initrd=boot/RedHat-7.3-x86_64-initrd.img > ks=http://example.com/host.ks ks.device=bootif network ks.sendmac > bond=bond0:eno1,eno2:mode=802.3ad vlan=bond0.10:bond0 > ip=10.10.0.2::10.10.0.1:255.255.255.0:host.example.com:bond0.10:none > nameserver=10.10.0.1 > > > Then in the KS we have > network --bootproto=static --device=link --gateway=10.10.0.1 > --hostname=host.example.com --ip=10.10.0.2 > --nameserver=10.10.0.1,10.11.0.1 --netmask=255.255.255.0 > > It should be fairly simple to convert that to use DHCP as you just need > to change the IP line in the kernel parameters and ensure you have > --device=link in the kickstart. > > BTW cat /proc/net/bonding/bond0 shows more info on how the bond is > actually setup and working.Finally the issue was a simple and stupid syntax error in the ks config. I had omitted the equal sign between "--device" and "bond0": network --device=bond0 --bondslaves=eno1,eno2 --bondopts=mode=balance-alb --bootproto=dhcp --hostname=myhost --activate works much better :-). Since the PXE config is global for all hosts (being configured with or without bonding device), I cannot use bonding definitions at that point. However the idea of using --device=link is interesting and I might use it in an other context in the future. Thanks a lot frank
Possibly Parallel Threads
- anaconda/kickstart: bonding device not created as expected
- kickstart: dracut-initqueue fails due to unresolvable hostname even though network config looks perfectly ok
- Bridges, VLANs, Bonding on CentOS 7
- nmcli: unwanted secondary ip-address
- nmcli: unwanted secondary ip-address