libvirt users - Dec 2015 - Hypervisor loses network connectivity

I have been running KVM on a CentOS 7.1 host for a few months. I have one
"main" host server plus a "backup" host server. Some time
ago everything
was running fine on "main", but all of a sudden the server became
non-contactable on it's network interface. I could no longer SSH in and
therefore couldn't run virt-manager. I don't know what caused this and I
had to bring all the VMs up on "backup" and they've been running
OK there
ever since.

Over the last few days I've rebuilt "main" and again all was going
well and
I migrated 4 of the 5 VMs back onto it. Then this morning I came to set up
the final VM on "main" which is a Sophos firewall machine with 3 NICs
(I
have 3 physical NICs). Nothing uses 2 of the 3 NICs except the Sophos VM. I
unplugged one of the ethernet cables from "backup" and plugged it into
the
3rd NIC on "main" and all of a sudden "main" froze and again
all network
connectivity was lost. I rebooted it but the problem still remains that
it's not contactable over the network and is therefore totally
non-functional. When I log in at the console and run "ip a" I can see
various network interfaces including br0 with the expected IP address - so
it looks like it's got network connectivity but it hasn't.

Has anyone else experienced this and know why it happens and how to fix it?
I don't think I fancy rebuilding everything a 3rd time as it seems likely
to happen again. Depending on why it's happening I'm thinking of moving
off
CentOS onto Ubuntu as the host OS, or moving away from KVM altogether.

Thanks in advance for any help you can give.

On 23.12.2015 03:56, Phill Edwards wrote:
> I have been running KVM on a CentOS 7.1 host for a few months. I have one
> "main" host server plus a "backup" host server. Some
time ago everything
> was running fine on "main", but all of a sudden the server became
> non-contactable on it's network interface. I could no longer SSH in and
> therefore couldn't run virt-manager. I don't know what caused this
and I
> had to bring all the VMs up on "backup" and they've been
running OK there
> ever since.
> 
> Over the last few days I've rebuilt "main" and again all was
going well and
> I migrated 4 of the 5 VMs back onto it. Then this morning I came to set up
> the final VM on "main" which is a Sophos firewall machine with 3
NICs (I
> have 3 physical NICs). Nothing uses 2 of the 3 NICs except the Sophos VM. I
> unplugged one of the ethernet cables from "backup" and plugged it
into the
> 3rd NIC on "main" and all of a sudden "main" froze and
again all network
> connectivity was lost. I rebooted it but the problem still remains that
> it's not contactable over the network and is therefore totally
> non-functional. When I log in at the console and run "ip a" I can
see
> various network interfaces including br0 with the expected IP address - so
> it looks like it's got network connectivity but it hasn't.
Sounds like a HW problem to me. guest should not freeze when you
(un-)plug a cable.

Can you bring down a firewall for a while and see if that helps? Plenty
of connectivity issues are caused by firewalls. It that won't help I
suspect clashing subnets/routes.

Michal