libvirt users - Feb 2020 - Cannot turn iommu on inside guest

So I want to turn iommu support on in a guest (    <type
arch='x86_64'
machine='pc-q35-4.1'>hvm</type>). Per [1] I tried to add to
devices
either

<devices>
  <iommu model='intel'>
    <driver intremap='on'/>
  </iommu>
</devices>

or just a simpler

<devices>
  <iommu model='intel /'>
</devices>

However, when I try to boot it, it crashes with the following error
message (stolen from /var/log/libvirt/qemu/testkvm.log):

2020-02-11T05:13:27.278707Z qemu-system-x86_64: We need to set
caching-mode=on for intel-iommu to enable device assignment with IOMMU
protection.
2020-02-11 05:13:27.577+0000: shutting down, reason=crashed

that sounds like it wants me to do something like

qemu-system-x86_64 [...] -device intel-iommu,intremap=on,caching-mode=on[2]

using virsh. But

<devices>
  <iommu model='intel'>
    <driver intremap='on' caching-mode='on '/>
  </iommu>
</devices>

is not only not mentioned in [1] but also fails

[root@testbox tests]# virsh edit testkvm
error: XML document failed to validate against schema: Unable to
validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element devices in interleave
Element domain failed to validate content

Failed. Try again? [y,n,i,f,?]:

Can anyone show me what I am missing here?

[1] https://libvirt.org/formatdomain.html#elementsIommu
[2] https://wiki.qemu.org/Features/VT-d#Command_Line_Example_2

On Tue, Feb 11, 2020 at 09:09:36AM -0500, Mauricio Tavares
wrote:
> So I want to turn iommu support on in a guest (    <type
arch='x86_64'
> machine='pc-q35-4.1'>hvm</type>). Per [1] I tried to add
to devices
> either
> 
> <devices>
>   <iommu model='intel'>
>     <driver intremap='on'/>
>   </iommu>
> </devices>
> 
> or just a simpler
> 
> <devices>
>   <iommu model='intel /'>
> </devices>
> 
> However, when I try to boot it, it crashes with the following error
> message (stolen from /var/log/libvirt/qemu/testkvm.log):
> 
> 2020-02-11T05:13:27.278707Z qemu-system-x86_64: We need to set
> caching-mode=on for intel-iommu to enable device assignment with IOMMU
> protection.
> 2020-02-11 05:13:27.577+0000: shutting down, reason=crashed
> 
> that sounds like it wants me to do something like
> 
> qemu-system-x86_64 [...] -device intel-iommu,intremap=on,caching-mode=on[2]
> 
> using virsh. But
> 
> <devices>
>   <iommu model='intel'>
>     <driver intremap='on' caching-mode='on '/>
You need   caching_mode="on"  - notice underscore, not hypehn
>   </iommu>
> </devices>
> 
> is not only not mentioned in [1] but also fails
The correctly spelt caching_mode is mentioned there.
> 
> [root@testbox tests]# virsh edit testkvm
> error: XML document failed to validate against schema: Unable to
> validate doc against /usr/share/libvirt/schemas/domain.rng
> Extra element devices in interleave
> Element domain failed to validate content
> 
> Failed. Try again? [y,n,i,f,?]:
> 
> Can anyone show me what I am missing here?
> 
> [1] https://libvirt.org/formatdomain.html#elementsIommu
> [2] https://wiki.qemu.org/Features/VT-d#Command_Line_Example_2
> 
> 
Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On Tue, Feb 11, 2020 at 9:17 AM Daniel P. Berrangé <berrange@redhat.com>
wrote:
>
> On Tue, Feb 11, 2020 at 09:09:36AM -0500, Mauricio Tavares wrote:
> > So I want to turn iommu support on in a guest (    <type
arch='x86_64'
> > machine='pc-q35-4.1'>hvm</type>). Per [1] I tried to
add to devices
> > either
> >
> > <devices>
> >   <iommu model='intel'>
> >     <driver intremap='on'/>
> >   </iommu>
> > </devices>
> >
> > or just a simpler
> >
> > <devices>
> >   <iommu model='intel /'>
> > </devices>
> >
> > However, when I try to boot it, it crashes with the following error
> > message (stolen from /var/log/libvirt/qemu/testkvm.log):
> >
> > 2020-02-11T05:13:27.278707Z qemu-system-x86_64: We need to set
> > caching-mode=on for intel-iommu to enable device assignment with IOMMU
> > protection.
> > 2020-02-11 05:13:27.577+0000: shutting down, reason=crashed
> >
> > that sounds like it wants me to do something like
> >
> > qemu-system-x86_64 [...] -device
intel-iommu,intremap=on,caching-mode=on[2]
> >
> > using virsh. But
> >
> > <devices>
> >   <iommu model='intel'>
> >     <driver intremap='on' caching-mode='on '/>
>
> You need   caching_mode="on"  - notice underscore, not hypehn
      Much happier.  Thanks!
>
> >   </iommu>
> > </devices>
> >
> > is not only not mentioned in [1] but also fails
>
> The correctly spelt caching_mode is mentioned there.
>
> >
> > [root@testbox tests]# virsh edit testkvm
> > error: XML document failed to validate against schema: Unable to
> > validate doc against /usr/share/libvirt/schemas/domain.rng
> > Extra element devices in interleave
> > Element domain failed to validate content
> >
> > Failed. Try again? [y,n,i,f,?]:
> >
> > Can anyone show me what I am missing here?
> >
> > [1] https://libvirt.org/formatdomain.html#elementsIommu
> > [2] https://wiki.qemu.org/Features/VT-d#Command_Line_Example_2
> >
> >
>
> Regards,
> Daniel
> --
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange
:|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com
:|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange
:|
>