Linux Virtualization - Oct 2014 - [PATCH RFC 00/11] qemu: towards virtio-1 host support

On 2014-10-22 10:44, Michael S. Tsirkin wrote:
> On Wed, Oct 08, 2014 at 11:04:28AM +0200, Cornelia Huck wrote:
>> On Tue, 07 Oct 2014 18:24:22 -0700
>> Andy Lutomirski <luto at amacapital.net> wrote:
>>
>>> On 10/07/2014 07:39 AM, Cornelia Huck wrote:
>>>> This patchset aims to get us some way to implement virtio-1
compliant
>>>> and transitional devices in qemu. Branch available at
>>>>
>>>> git://github.com/cohuck/qemu virtio-1
>>>>
>>>> I've mainly focused on:
>>>> - endianness handling
>>>> - extended feature bits
>>>> - virtio-ccw new/changed commands
>>>
>>> At the risk of some distraction, would it be worth thinking about a
>>> solution to the IOMMU bypassing mess as part of this?
>>
>> I think that is a whole different issue. virtio-1 is basically done -
we
>> just need to implement it - while the IOMMU/DMA stuff certainly needs
>> more discussion. Therefore, I'd like to defer to the other
discussion
>> thread here.
> 
> I agree, let's do a separate thread for this.
> I also think it's up to the hypervisors at this point.
> People talked about using ACPI to report IOMMU bypass
> to guest.
> If that happens, we don't need a feature bit.
I thought about this again, and I'm not sure anymore if we can use ACPI
to "black-list" the incompatible virtio devices. Reason: hotplug. To
my
understanding, the ACPI DRHD tables won't change during runtime when a
device shows up or disappears. We would have to isolate virtio devices
from the rest of the system by using separate buses for it (and avoid
listing those in any DRHD table) and enforce that they only get plugged
into those buses. I suppose that is not desirable.

Maybe it's better to fix virtio /wrt IOMMUs.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

On Wed, Oct 22, 2014 at 04:17:40PM +0200, Jan Kiszka
wrote:
> On 2014-10-22 10:44, Michael S. Tsirkin wrote:
> > On Wed, Oct 08, 2014 at 11:04:28AM +0200, Cornelia Huck wrote:
> >> On Tue, 07 Oct 2014 18:24:22 -0700
> >> Andy Lutomirski <luto at amacapital.net> wrote:
> >>
> >>> On 10/07/2014 07:39 AM, Cornelia Huck wrote:
> >>>> This patchset aims to get us some way to implement
virtio-1 compliant
> >>>> and transitional devices in qemu. Branch available at
> >>>>
> >>>> git://github.com/cohuck/qemu virtio-1
> >>>>
> >>>> I've mainly focused on:
> >>>> - endianness handling
> >>>> - extended feature bits
> >>>> - virtio-ccw new/changed commands
> >>>
> >>> At the risk of some distraction, would it be worth thinking
about a
> >>> solution to the IOMMU bypassing mess as part of this?
> >>
> >> I think that is a whole different issue. virtio-1 is basically
done - we
> >> just need to implement it - while the IOMMU/DMA stuff certainly
needs
> >> more discussion. Therefore, I'd like to defer to the other
discussion
> >> thread here.
> > 
> > I agree, let's do a separate thread for this.
> > I also think it's up to the hypervisors at this point.
> > People talked about using ACPI to report IOMMU bypass
> > to guest.
> > If that happens, we don't need a feature bit.
> 
> I thought about this again, and I'm not sure anymore if we can use ACPI
> to "black-list" the incompatible virtio devices. Reason: hotplug.
To my
> understanding, the ACPI DRHD tables won't change during runtime when a
> device shows up or disappears. We would have to isolate virtio devices
> from the rest of the system by using separate buses for it (and avoid
> listing those in any DRHD table) and enforce that they only get plugged
> into those buses. I suppose that is not desirable.
That's reasonable I think.
> Maybe it's better to fix virtio /wrt IOMMUs.
> 
> Jan
Yes but this seems unlikely for 2.2:

The wish to run old guests with iommu remains.
So we'll need to support iommu bypass on the host, and so as
a minimum new guest needs to detect such bypass host and fail.

Unrelated: we also need to teach vhost and dataplane virtio
to get mappings from the iommu.
> -- 
> Siemens AG, Corporate Technology, CT RTC ITP SES-DE
> Corporate Competence Center Embedded Linux

On Wed, 2014-10-22 at 16:17 +0200, Jan Kiszka wrote:
> I thought about this again, and I'm not sure anymore if we can use
> ACPI
> to "black-list" the incompatible virtio devices. Reason: hotplug.
To
> my
> understanding, the ACPI DRHD tables won't change during runtime when a
> device shows up or disappears. We would have to isolate virtio devices
> from the rest of the system by using separate buses for it (and avoid
> listing those in any DRHD table) and enforce that they only get
> plugged
> into those buses. I suppose that is not desirable.
> 
> Maybe it's better to fix virtio /wrt IOMMUs.
I always go back to my initial proposal which is to define that current
virtio always bypass any iommu (which is what it does really) and have
it expose via a new capability if that isn't the case. That means fixing
that Xen thingy to allow qemu to know what to expose I assume but that
seems to be the less bad approach.

Cheers,
Ben.

On 2014-10-22 22:34, Benjamin Herrenschmidt wrote:
> On Wed, 2014-10-22 at 16:17 +0200, Jan Kiszka wrote:
>> I thought about this again, and I'm not sure anymore if we can use
>> ACPI
>> to "black-list" the incompatible virtio devices. Reason:
hotplug. To
>> my
>> understanding, the ACPI DRHD tables won't change during runtime
when a
>> device shows up or disappears. We would have to isolate virtio devices
>> from the rest of the system by using separate buses for it (and avoid
>> listing those in any DRHD table) and enforce that they only get
>> plugged
>> into those buses. I suppose that is not desirable.
>>
>> Maybe it's better to fix virtio /wrt IOMMUs.
> 
> I always go back to my initial proposal which is to define that current
> virtio always bypass any iommu (which is what it does really) and have
> it expose via a new capability if that isn't the case. That means
fixing
> that Xen thingy to allow qemu to know what to expose I assume but that
> seems to be the less bad approach.
Just one thing to consider: feature negotiation happens after guest
startup. If we run a virtio device under IOMMU control, what will we
have to do when the guest says it does not support such devices? Simply
reject operation?

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

On Thu, Oct 23, 2014 at 07:34:16AM +1100, Benjamin Herrenschmidt
wrote:
> On Wed, 2014-10-22 at 16:17 +0200, Jan Kiszka wrote:
> > I thought about this again, and I'm not sure anymore if we can use
> > ACPI
> > to "black-list" the incompatible virtio devices. Reason:
hotplug. To
> > my
> > understanding, the ACPI DRHD tables won't change during runtime
when a
> > device shows up or disappears. We would have to isolate virtio devices
> > from the rest of the system by using separate buses for it (and avoid
> > listing those in any DRHD table) and enforce that they only get
> > plugged
> > into those buses. I suppose that is not desirable.
> > 
> > Maybe it's better to fix virtio /wrt IOMMUs.
> 
> I always go back to my initial proposal which is to define that current
> virtio always bypass any iommu (which is what it does really) and have
> it expose via a new capability if that isn't the case. That means
fixing
> that Xen thingy to allow qemu to know what to expose I assume but that
> seems to be the less bad approach.
> 
> Cheers,
> Ben.
> 
OK so how does this work?
If you want to run an existing guest, you use the old device.
And presumably you blacklist virtio for nested virt then,
unless a new capability is present?


-- 
MST