Ever since I switched from passing a PCI USB controller through to a domU in favour of passing USB devices using the "usbdevice" config parameter, the domU affected seems to have qemu-dm "idling" at about 5% CPU usage in dom0. With the PCI USB controller passed through, this does not happen. Is this normal/expected? Does a similar continuous CPU overhead invisibly apply to PCI passthrough devices as well? Or is this CPU tax only applicable to USB passthrough? Gordan
On Mon, Aug 5, 2013 at 1:28 PM, Gordan Bobic <gordan@bobich.net> wrote:> Ever since I switched from passing a PCI USB controller > through to a domU in favour of passing USB devices using > the "usbdevice" config parameter, the domU affected > seems to have qemu-dm "idling" at about 5% CPU usage in > dom0. With the PCI USB controller passed through, this > does not happen. > > Is this normal/expected? Does a similar continuous CPU > overhead invisibly apply to PCI passthrough devices as > well? Or is this CPU tax only applicable to USB > passthrough?Do you have the emulated USB controller enabled when you''re passing through the PCI USB controller? My understanding was that certain USB devices must, by specification, be polled something like a hundred times per second. QEMU has to emulate every one of these. One of the things XenServer does for really high-density VDI deployments, I believe, is disable USB in the guests entirely. Felipe, any comments? -George
On Mon, 2013-08-05 at 13:28 +0100, Gordan Bobic wrote:> Ever since I switched from passing a PCI USB controller > through to a domU in favour of passing USB devices using > the "usbdevice" config parameter, the domU affected > seems to have qemu-dm "idling" at about 5% CPU usage in > dom0. With the PCI USB controller passed through, this > does not happen. > > Is this normal/expected? Does a similar continuous CPU > overhead invisibly apply to PCI passthrough devices as > well? Or is this CPU tax only applicable to USB > passthrough?I''ve heard that USB host controllers are expensive to virtualise because USB has an effective 100ms (?) poll at the hardware level, which needs to be managed (something to do with Isochronous URBs and/or the USB schedule? I''m not 100% sure -- deeper USB voodoo than I can manage anyway). It had never occurred to me that this might also apply to usbdevice style passthrough but since in that case, AIUI, you are running an emulated host controller with a real device behind it it seems plausible that this configuration would have that sort of overhead too. Ian.
On Mon, 5 Aug 2013 14:57:09 +0100, George Dunlap <dunlapg@umich.edu> wrote:> On Mon, Aug 5, 2013 at 1:28 PM, Gordan Bobic <gordan@bobich.net> > wrote: >> Ever since I switched from passing a PCI USB controller >> through to a domU in favour of passing USB devices using >> the "usbdevice" config parameter, the domU affected >> seems to have qemu-dm "idling" at about 5% CPU usage in >> dom0. With the PCI USB controller passed through, this >> does not happen. >> >> Is this normal/expected? Does a similar continuous CPU >> overhead invisibly apply to PCI passthrough devices as >> well? Or is this CPU tax only applicable to USB >> passthrough? > > Do you have the emulated USB controller enabled when you''re passing > through the PCI USB controller?Yes, but nothing is hanging off it (well, other than maybe the emulated VNC mouse/keyboard).> My understanding was that certain USB devices must, by specification, > be polled something like a hundred times per second. QEMU has to > emulate every one of these. One of the things XenServer does for > really high-density VDI deployments, I believe, is disable USB in the > guests entirely.So it stands to reason that with no physical USB devices passed through the USB controller polling gets optimized out. Gordan
On Mon, Aug 5, 2013 at 3:01 PM, Gordan Bobic <gordan@bobich.net> wrote:> On Mon, 5 Aug 2013 14:57:09 +0100, George Dunlap <dunlapg@umich.edu> wrote: >> >> On Mon, Aug 5, 2013 at 1:28 PM, Gordan Bobic <gordan@bobich.net> wrote: >>> >>> Ever since I switched from passing a PCI USB controller >>> through to a domU in favour of passing USB devices using >>> the "usbdevice" config parameter, the domU affected >>> seems to have qemu-dm "idling" at about 5% CPU usage in >>> dom0. With the PCI USB controller passed through, this >>> does not happen. >>> >>> Is this normal/expected? Does a similar continuous CPU >>> overhead invisibly apply to PCI passthrough devices as >>> well? Or is this CPU tax only applicable to USB >>> passthrough? >> >> >> Do you have the emulated USB controller enabled when you''re passing >> through the PCI USB controller? > > > Yes, but nothing is hanging off it (well, other than maybe the > emulated VNC mouse/keyboard).>> My understanding was that certain USB devices must, by specification, >> be polled something like a hundred times per second. QEMU has to >> emulate every one of these. One of the things XenServer does for >> really high-density VDI deployments, I believe, is disable USB in the >> guests entirely. > > > So it stands to reason that with no physical USB devices passed through > the USB controller polling gets optimized out.You could probably verify this by taking a xentrace and then running xenalyze on it. xenalyze can be found here: http://xenbits.xensource.com/ext/xenalyze.hg There''s a very minimalist .html doc in the repo. You want "--summary --with-mmio-enumeration" I think. It may be that the device you''re passing through is more "active" wrt polling... -George
On Mon, Aug 05, 2013 at 01:28:39PM +0100, Gordan Bobic wrote:> Ever since I switched from passing a PCI USB controller > through to a domU in favour of passing USB devices using > the "usbdevice" config parameter, the domU affected > seems to have qemu-dm "idling" at about 5% CPU usage in > dom0. With the PCI USB controller passed through, this > does not happen. > > Is this normal/expected? Does a similar continuous CPU > overhead invisibly apply to PCI passthrough devices as > well? Or is this CPU tax only applicable to USB > passthrough?I think that is about right. USB1 sucks at virtualization so QEMU has to scan a list of memory regions to figure out if the guest has decided that "hey, I''ve a got a packet of data''. The CPU overhead for PCI passthrough does not happen as the PCI device can detect where the request went and do it stuff. USB2.0 is not any better either. The XHCI on the other hand is - but I am not sure if that is implemented in QEMU yet. Ah, Gerd here has a nice update: http://www.linux-kvm.org/wiki/images/b/be/2012-forum-qemu-usb-status-update.pdf> > Gordan > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel