Hi, I have to host (with KVM) an appliance which does not use its second and third NIC. They have to be present in the guest, but they'd better stay totally disconnected from anything in the host. "Second" and "third" apparently means bus order. Let's consider virtio devices only. I think the best technical solution is adding -device virtio-net-pci,addr=0x3 and similar options to the KVM command line, without any corresponding -netdev options (better ideas welcome). QEMU emits "Warning: nic virtio-net-pci.2 has no peer" messages, but that's expected. I can even do this much using the <qemu:commandline> element, but libvirt assigns the 0x3 address to other virtio devices, leading to collision. Is there a way to "reserve" a bus address for such manually added devices without assigning explicit addresses to all other devices in the configuration? Things I also tried (and found inadequate): * Using "generic ethernet connection" for the dummy NICs. Close, but requires extra permissions for accessing /dev/net/tun, and technically feels a little inferior to using a peerless network device like above. * TCP tunnel server. Even more inferior, does not require extra permissions but leaves even looser ends (listening sockets). Also, the RelaxNG grammal does not let me specify a model for this interface type, so maintaining bus order with respect to the virtio interfaces is impossible. A grammar bug? * Using a dummy VLAN in the bridge. This is what I temporarily settled for, but this requires global agreement and still technically inferior, so I'd like to move away. * A <network> without forwarding. Still inferior, and also requires configuration sharing across the host cluster. -- Thanks, Feri
On 3/11/19 5:05 AM, wferi@niif.hu wrote:> Hi, > > I have to host (with KVM) an appliance which does not use its second and > third NIC. They have to be present in the guest, but they'd better stay > totally disconnected from anything in the host. "Second" and "third" > apparently means bus order. Let's consider virtio devices only. I think > the best technical solution is adding -device virtio-net-pci,addr=0x3 and > similar options to the KVM command line, without any corresponding > -netdev options (better ideas welcome). QEMU emits "Warning: nic > virtio-net-pci.2 has no peer" messages, but that's expected. I can even > do this much using the <qemu:commandline> element, but libvirt assigns > the 0x3 address to other virtio devices, leading to collision. Is there > a way to "reserve" a bus address for such manually added devices without > assigning explicit addresses to all other devices in the configuration?I think qemu is going to be upset by anything that has no backend to the emulated device. As for libvirt reserving addresses that (from its point of view are otherwise unused - no, there's no way to do that; if a PCI address isn't used by a device in the libvirt config, it is considered fair game for assigning to a new device, and we've never considered such an option.> > Things I also tried (and found inadequate): > > * Using "generic ethernet connection" for the dummy NICs. Close, but > requires extra permissions for accessing /dev/net/tun, and technically > feels a little inferior to using a peerless network device like above.What version is your libvirt? extra permissions for qemu using type='ethernet' (beyond what's required for a type='network' or type='bridge') have not been required since libvirt-1.3.3, released on April 6, 2016 (this was the result of commit 9c17d665f). If your libvirt is that old, you *really* should update to something newer. If it's *not* that old, then you're just working with out of date documentation.> > * TCP tunnel server. Even more inferior, does not require extra > permissions but leaves even looser ends (listening sockets). Also, the > RelaxNG grammal does not let me specify a model for this interface > type, so maintaining bus order with respect to the virtio interfaces is > impossible. A grammar bug? > > * Using a dummy VLAN in the bridge. This is what I temporarily settled > for, but this requires global agreement and still technically inferior, > so I'd like to move away. > > * A <network> without forwarding. Still inferior, and also requires > configuration sharing across the host cluster. >Does it matter if the interface is online or not? I would recommend using an expansion of this: <interface type='ethernet'> <link state='down'/> ... Your guest will have a network device in the desired position, qemu will be satisfied that the device has a backend, libvirt will know there is a device there so it won't give the PCI address away to somebody else, and the tap device will be IFF_DOWN, so there will be no possibility of network traffic accidentally leaking into the host (which would already be nearly impossible unless someone separately assigned an IP address to the host side of the tap device).
Laine Stump <laine@redhat.com> writes:> On 3/11/19 5:05 AM, wferi@niif.hu wrote: > >> Is there a way to "reserve" a bus address for such manually added >> devices without assigning explicit addresses to all other devices in >> the configuration? > > no, there's no way to do that; if a PCI address isn't used by a device > in the libvirt config, it is considered fair game for assigning to a > new device, and we've never considered such an option.Understood; thanks for confirming this.>> Things I also tried (and found inadequate): >> >> * Using "generic ethernet connection" for the dummy NICs. Close, but >> requires extra permissions for accessing /dev/net/tun, and technically >> feels a little inferior to using a peerless network device like above. > > What version is your libvirt? extra permissions for qemu using > type='ethernet' (beyond what's required for a type='network' or > type='bridge') have not been required since libvirt-1.3.3, released on > April 6, 2016 (this was the result of commit 9c17d665f). If your > libvirt is that old, you *really* should update to something newer. If > it's *not* that old, then you're just working with out of date > documentation.Yes, it's really that old, 1.2.9 from Debian jessie. Works just fine for our purposes and very stable in our environment. We hit several problems (deadlocks and crashes) with earlier versions before settling here, so I'm not exactly looking forward to upgrading to 5.0, but testing will start in the near future.> Does it matter if the interface is online or not?No, it doesn't matter.> I would recommend using an expansion of this: > > <interface type='ethernet'> > <link state='down'/> > ... > > Your guest will have a network device in the desired position, qemu > will be satisfied that the device has a backend, libvirt will know > there is a device there so it won't give the PCI address away to > somebody else, and the tap device will be IFF_DOWN, so there will be > no possibility of network traffic accidentally leaking into the host > (which would already be nearly impossible unless someone separately > assigned an IP address to the host side of the tap device).Thanks for this, it looks like a good compromise for after the upgrade. Those tap devices are still somewhat ugly, but so is specifying qemu command line options and bus addresses. And who knows, maybe qemu eventually implements the null netdev backend mentioned on IRC and I can have the best possible dummy interfaces for this... appliance. -- Regards, Feri