As part of my initial KVM host on C8 deployment, I decided to set up some HA features on the new host, specifically NIC teaming. Teaming seems to be bond++ of a sort, so I thought I would at least try it.? So here's the scenario: 1.) Server with two gigabit ethernet ports, two Cisco switches. 2.) During install, used the 'Server with GUI' group and added the virtualization packages. 3.) During install, set up team0 to include the two gig-e ports set up active-backup (two switches). 4.) During install, set up three bridges, with the slave devices being VLANs pointed to the team0 subinterfaces (using VLANs 68, 101, and 302; 101 is to be the management bridge for the host, with guests on all three VLANs).? So, for instance, bridge101 has a slave that is VLAN101 that points to team0.101 with a VLAN ID of 101.? The bridge101 interface has a manual IP address, but bridge68 and bridge302 do not (IPv4 disabled; IPv6 Ignore) 5.) After reboot, the bridge101 interface comes up, and I successfully connect to the host, since the install is 8.1.1911, I ran a 'dnf update' up to 8.2.2004, which went well, then I successfully set up and used cockpit, cockpit-bridge, cockpit-machines, again over the IP address on bridge101. Ok, now that the base connectivity is working: 1.) Connect to the host (traffic on bridge101 over team0.101) using virt-manager on my laptop and install a C8 guest, with the network pointed to bridge302, and a manual IP address. 2.) After reboot of guest, there is no IP connectivity to the guest's gateway on VLAN302. 3.) HOWEVER, the gateway's MAC address shows up in the host's bridge fdb for VLAN302, AND in the arp output for the guest; ALSO, the MAC address for the guest shows on the cisco switch 'show mac-address-table' output.? The output of 'ip --br link' looks normal for this configuration, but there's a disconnect somewhere.? So, since I see that VLAN101 is passing traffic to the bridge correctly (since the management IP is on that VLAN), I try to set up a guest on VLAN101; no dice, no work, but the management IP still works fine. So, does anyone here have a working setup with KVM guests connecting to bridges using 802.1q VLANs on top of a team?? Or even on top of a bond (I can reinstall and set it up as a bond easily enough, using active-backup, as far as I know; and, yes, I would reinstall the host from scratch to do this).
Deventer-2, M.S.J. van
2020-Jun-17 13:59 UTC
[CentOS] C8 - KVM on bridge on VLAN on team issues.
Hi, the first thing that comes to mind, did you set ip_forward to enable in /etc/sysctl.conf ? net.ipv4.ip_forward = 1 Should explain why you IP on the bridge works but not on the vms. Regards, Michel On Wed, 2020-06-17 at 09:43 -0400, Lamar Owen wrote:> As part of my initial KVM host on C8 deployment, I decided to set up > some HA features on the new host, specifically NIC teaming. Teaming > seems to be bond++ of a sort, so I thought I would at least try it. > So > here's the scenario: > > 1.) Server with two gigabit ethernet ports, two Cisco switches. > > 2.) During install, used the 'Server with GUI' group and added the > virtualization packages. > > 3.) During install, set up team0 to include the two gig-e ports set > up > active-backup (two switches). > > 4.) During install, set up three bridges, with the slave devices > being > VLANs pointed to the team0 subinterfaces (using VLANs 68, 101, and > 302; > 101 is to be the management bridge for the host, with guests on all > three VLANs). So, for instance, bridge101 has a slave that is > VLAN101 > that points to team0.101 with a VLAN ID of 101. The bridge101 > interface > has a manual IP address, but bridge68 and bridge302 do not (IPv4 > disabled; IPv6 Ignore) > > 5.) After reboot, the bridge101 interface comes up, and I > successfully > connect to the host, since the install is 8.1.1911, I ran a 'dnf > update' > up to 8.2.2004, which went well, then I successfully set up and used > cockpit, cockpit-bridge, cockpit-machines, again over the IP address > on > bridge101. > > > Ok, now that the base connectivity is working: > > 1.) Connect to the host (traffic on bridge101 over team0.101) using > virt-manager on my laptop and install a C8 guest, with the network > pointed to bridge302, and a manual IP address. > > 2.) After reboot of guest, there is no IP connectivity to the > guest's > gateway on VLAN302. > > 3.) HOWEVER, the gateway's MAC address shows up in the host's bridge > fdb > for VLAN302, AND in the arp output for the guest; ALSO, the MAC > address > for the guest shows on the cisco switch 'show mac-address-table' > output. The output of 'ip --br link' looks normal for this > configuration, but there's a disconnect somewhere. So, since I see > that > VLAN101 is passing traffic to the bridge correctly (since the > management > IP is on that VLAN), I try to set up a guest on VLAN101; no dice, no > work, but the management IP still works fine. > > > So, does anyone here have a working setup with KVM guests connecting > to > bridges using 802.1q VLANs on top of a team? Or even on top of a > bond > (I can reinstall and set it up as a bond easily enough, using > active-backup, as far as I know; and, yes, I would reinstall the > host > from scratch to do this). > > _______________________________________________ > CentOS mailing list > CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos-- Michel van Deventer Integratie Specialist | Divisie Laboratoria, Apotheek en Biomedische Genetica, Infra Services & Integration Universitair Medisch Centrum Utrecht | Kamernummer 2.139 Tel. 06-25710398 ------------------------------------------------------------------------------ De informatie opgenomen in dit bericht kan vertrouwelijk zijn en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onterecht ontvangt, wordt u verzocht de inhoud niet te gebruiken en de afzender direct te informeren door het bericht te retourneren. Het Universitair Medisch Centrum Utrecht is een publiekrechtelijke rechtspersoon in de zin van de W.H.W. (Wet Hoger Onderwijs en Wetenschappelijk Onderzoek) en staat geregistreerd bij de Kamer van Koophandel voor Midden-Nederland onder nr. 30244197. Denk s.v.p aan het milieu voor u deze e-mail afdrukt. ------------------------------------------------------------------------------ This message may contain confidential information and is intended exclusively for the addressee. If you receive this message unintentionally, please do not use the contents but notify the sender immediately by return e-mail. University Medical Center Utrecht is a legal person by public law and is registered at the Chamber of Commerce for Midden-Nederland under no. 30244197. Please consider the environment before printing this e-mail.
On 6/17/20 9:59 AM, Deventer-2, M.S.J. van wrote:> Hi, > > the first thing that comes to mind, did you set ip_forward to enable in > /etc/sysctl.conf ? > net.ipv4.ip_forward = 1 > > Should explain why you IP on the bridge works but not on the vms. >First, thanks for the reply and excellent suggestion.? Yeah, I thought about that, and while it's not specifically defined in /etc/sysctl.conf or /etc/sysctl.d/*, if I: [root at c8-kvm-pe1950-1 ~]# cat /proc/sys/net/ipv4/ip_forward 1 It shows as being defined to 1.? I'm going to try adding to sysctl.conf and see if that makes any difference, though.