Andy Lutomirski
2014-Aug-27 16:19 UTC
[PATCH 0/3] virtio: Clean up scatterlists and use the DMA API
On Wed, Aug 27, 2014 at 9:13 AM, Christopher Covington <cov at codeaurora.org> wrote:> On 08/27/2014 11:50 AM, Andy Lutomirski wrote: >> On Wed, Aug 27, 2014 at 8:45 AM, Michael S. Tsirkin <mst at redhat.com> wrote: >>> On Wed, Aug 27, 2014 at 08:11:15AM -0700, Andy Lutomirski wrote: >>>> On Aug 26, 2014 11:46 PM, "Stefan Hajnoczi" <stefanha at gmail.com> wrote: >>>>> >>>>> On Tue, Aug 26, 2014 at 10:16 PM, Andy Lutomirski <luto at amacapital.net> wrote: >>>>>> There are two outstanding issues. virtio_net warns if DMA debugging >>>>>> is on because it does DMA from the stack. (The warning is correct.) >>>>>> This also is likely to do something unpleasant to s390. >>>>>> (Maintainers are cc'd -- I don't know what to do about it.) >>>>> >>>>> This changes the semantics of vring and breaks existing guests when >>>>> bus address != physical address. >>>>> >>>>> Can you use a transport feature bit to indicate that bus addresses are >>>>> used? That way both approaches can be supported. >>>> >>>> I can try to support both styles of addressing, but I don't think that >>>> this can be negotiated between the device (i.e. host or physical >>>> virtio-speaking device) and the guest. In the Xen case that I care >>>> about (Linux on Xen on KVM), the host doesn't know about the >>>> translation at all -- Xen is an intermediate layer that only the guest >>>> is aware of. In this case, there are effectively two layers of >>>> virtualization, and only the inner one (Xen) knows about the >>>> translation despite the fact that the the outer layer is the one >>>> providing the virtio device. >>>> >>>> I could change virtio_ring to use the DMA API only if requested by the >>>> lower driver (virtio_pci, etc) and to have only virtio_pci enable that >>>> feature. Will that work for all cases? >>>> >>>> On s390, this shouldn't work just like the current code. On x86, I >>>> think that if QEMU ever starts exposing an IOMMU attached to a >>>> virtio-pci device, then QEMU should expect that IOMMU to be used. If >>>> QEMU expects to see physical addresses, then it shouldn't advertise an >>>> IOMMU. Since QEMU doesn't currently support guest IOMMUs, this should >>>> be fine for everything that uses QEMU. >>>> >>>> At least x86's implementation of the DMA ops for devices that aren't >>>> behind an IOMMU should be very fast. >>>> >>>> Are there any other weird cases for which this might be a problem? >>>> >>>>> >>>>> Please also update the virtio specification: >>>>> https://tools.oasis-open.org/version-control/browse/wsvn/virtio/ >>>>> >>>> >>>> I'm not sure it will need an update. Perhaps a note in the PCI >>>> section indicating that, if the host expects the guest to program an >>>> IOMMU, then it should use the appropriate platform-specific mechanism >>>> to expose that IOMMU. >>>> >>>> --Andy >>> >>> If there's no virtio mechanism to negotate enabling/disabling >>> translations, then specification does not need an extension. >> >> It wouldn't shock me if someone wants to negotiate this for >> virtio_mmio some day, but that might be more of a device tree thing. >> I have no idea how that works, but I think it can wait until someone >> wants it. > > At one point I wanted VirtIO-MMIO to not fail miserably on ARM LPAE systems. > > http://lists.infradead.org/pipermail/linux-arm-kernel/2014-March/241613.html >Since I know nothing about LPAE, I'll leave this to you :) But I'll cc you on patch v2 soon, and please tell me whether my code makes sense on ARM. (My attempt to boot-test s390 failed because I have no clue what command-line options to pass to QEMU. If anyone wants to give me some pointers to get a working configuration with -kernel and some kind of console, I can add support to virtme. Alas, I think that no one ever bothered to implement 9p over virtio-ccw in QEMU. Why exactly does the virtio stuff in QEMU require that you instantiate virtio-9p-pci instead of just asking for an appropriate virtio dievice?) --Andy
Christopher Covington
2014-Aug-27 17:34 UTC
[PATCH 0/3] virtio: Clean up scatterlists and use the DMA API
On 08/27/2014 12:19 PM, Andy Lutomirski wrote:> On Wed, Aug 27, 2014 at 9:13 AM, Christopher Covington > <cov at codeaurora.org> wrote: >> On 08/27/2014 11:50 AM, Andy Lutomirski wrote: >>> On Wed, Aug 27, 2014 at 8:45 AM, Michael S. Tsirkin <mst at redhat.com> wrote: >>>> On Wed, Aug 27, 2014 at 08:11:15AM -0700, Andy Lutomirski wrote: >>>>> On Aug 26, 2014 11:46 PM, "Stefan Hajnoczi" <stefanha at gmail.com> wrote: >>>>>> >>>>>> On Tue, Aug 26, 2014 at 10:16 PM, Andy Lutomirski <luto at amacapital.net> wrote: >>>>>>> There are two outstanding issues. virtio_net warns if DMA debugging >>>>>>> is on because it does DMA from the stack. (The warning is correct.) >>>>>>> This also is likely to do something unpleasant to s390. >>>>>>> (Maintainers are cc'd -- I don't know what to do about it.) >>>>>> >>>>>> This changes the semantics of vring and breaks existing guests when >>>>>> bus address != physical address. >>>>>> >>>>>> Can you use a transport feature bit to indicate that bus addresses are >>>>>> used? That way both approaches can be supported. >>>>> >>>>> I can try to support both styles of addressing, but I don't think that >>>>> this can be negotiated between the device (i.e. host or physical >>>>> virtio-speaking device) and the guest. In the Xen case that I care >>>>> about (Linux on Xen on KVM), the host doesn't know about the >>>>> translation at all -- Xen is an intermediate layer that only the guest >>>>> is aware of. In this case, there are effectively two layers of >>>>> virtualization, and only the inner one (Xen) knows about the >>>>> translation despite the fact that the the outer layer is the one >>>>> providing the virtio device. >>>>> >>>>> I could change virtio_ring to use the DMA API only if requested by the >>>>> lower driver (virtio_pci, etc) and to have only virtio_pci enable that >>>>> feature. Will that work for all cases? >>>>> >>>>> On s390, this shouldn't work just like the current code. On x86, I >>>>> think that if QEMU ever starts exposing an IOMMU attached to a >>>>> virtio-pci device, then QEMU should expect that IOMMU to be used. If >>>>> QEMU expects to see physical addresses, then it shouldn't advertise an >>>>> IOMMU. Since QEMU doesn't currently support guest IOMMUs, this should >>>>> be fine for everything that uses QEMU. >>>>> >>>>> At least x86's implementation of the DMA ops for devices that aren't >>>>> behind an IOMMU should be very fast. >>>>> >>>>> Are there any other weird cases for which this might be a problem? >>>>> >>>>>> >>>>>> Please also update the virtio specification: >>>>>> https://tools.oasis-open.org/version-control/browse/wsvn/virtio/ >>>>>> >>>>> >>>>> I'm not sure it will need an update. Perhaps a note in the PCI >>>>> section indicating that, if the host expects the guest to program an >>>>> IOMMU, then it should use the appropriate platform-specific mechanism >>>>> to expose that IOMMU. >>>>> >>>>> --Andy >>>> >>>> If there's no virtio mechanism to negotate enabling/disabling >>>> translations, then specification does not need an extension. >>> >>> It wouldn't shock me if someone wants to negotiate this for >>> virtio_mmio some day, but that might be more of a device tree thing. >>> I have no idea how that works, but I think it can wait until someone >>> wants it. >> >> At one point I wanted VirtIO-MMIO to not fail miserably on ARM LPAE systems. >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2014-March/241613.html >> > > Since I know nothing about LPAE, I'll leave this to you :) But I'll > cc you on patch v2 soon, and please tell me whether my code makes > sense on ARM. > > (My attempt to boot-test s390 failed because I have no clue what > command-line options to pass to QEMU. If anyone wants to give me some > pointers to get a working configuration with -kernel and some kind of > console, I can add support to virtme. Alas, I think that no one ever > bothered to implement 9p over virtio-ccw in QEMU. Why exactly does > the virtio stuff in QEMU require that you instantiate virtio-9p-pci > instead of just asking for an appropriate virtio dievice?)Virtme looks interesting. If it's any use, here is my modest QEMU command line collection. $dir/bin/x86_64-linux-gnu/qemu-system-x86_64 \ -m 1024 \ -kernel $dir/boot/x86_64-linux-gnu/bzImage-x86_64 \ -append 'root=/dev/root init=/sbin/x86_64-linux-gnu/init rootfstype=9p rootflags=trans=virtio,version=9p2000.L ro console=ttyS0' \ -serial stdio \ -fsdev local,id=root,path=$dir,security_model=none \ -device virtio-9p-pci,fsdev=root,mount_tag=/dev/root \ -monitor none $dir/bin/x86_64-linux-gnu/qemu-system-arm \ -M virt \ -cpu cortex-a15 \ -m 1024 \ -kernel $dir/boot/arm-linux-gnueabihf/Image \ -append 'root=/dev/root init=/sbin/arm-linux-gnueabihf/init rootfstype=9p rootflags=trans=virtio,version=9p2000.L ro console=ttyAMA0' \ -serial stdio \ -fsdev local,id=root,path=$dir,security_model=none \ -device virtio-9p-device,fsdev=root,mount_tag=/dev/root \ -monitor none $dir/bin/x86_64-linux-gnu/qemu-system-aarch64 \ -M virt \ -cpu cortex-a57 \ -m 1024 \ -kernel $dir/boot/aarch64-linux-gnu/Image \ -append 'root=/dev/root init=/sbin/aarch64-linux-gnu/init rootfstype=9p rootflags=trans=virtio,version=9p2000.L ro console=ttyAMA0' \ -serial stdio \ -fsdev local,id=root,path=$dir,security_model=none \ -device virtio-9p-device,fsdev=root,mount_tag=/dev/root -monitor none Christopher -- Employee of Qualcomm Innovation Center, Inc. Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the Linux Foundation.
Andy Lutomirski
2014-Aug-27 17:37 UTC
[PATCH 0/3] virtio: Clean up scatterlists and use the DMA API
On Wed, Aug 27, 2014 at 10:34 AM, Christopher Covington <cov at codeaurora.org> wrote:> On 08/27/2014 12:19 PM, Andy Lutomirski wrote: >> On Wed, Aug 27, 2014 at 9:13 AM, Christopher Covington >> <cov at codeaurora.org> wrote: > > Virtme looks interesting. If it's any use, here is my modest QEMU command line > collection. > > $dir/bin/x86_64-linux-gnu/qemu-system-x86_64 \ > -m 1024 \ > -kernel $dir/boot/x86_64-linux-gnu/bzImage-x86_64 \ > -append 'root=/dev/root init=/sbin/x86_64-linux-gnu/init rootfstype=9p > rootflags=trans=virtio,version=9p2000.L ro console=ttyS0' \ > -serial stdio \ > -fsdev local,id=root,path=$dir,security_model=none \ > -device virtio-9p-pci,fsdev=root,mount_tag=/dev/root \ > -monitor noneThis is similar to what virtme does, except that virtme will give you a much more functional system at the cost of a small fraction of a second of additional boot time. And virtme sometimes supports modules :)> > $dir/bin/x86_64-linux-gnu/qemu-system-arm \ > -M virt \ > -cpu cortex-a15 \ > -m 1024 \ > -kernel $dir/boot/arm-linux-gnueabihf/Image \ > -append 'root=/dev/root init=/sbin/arm-linux-gnueabihf/init rootfstype=9p > rootflags=trans=virtio,version=9p2000.L ro console=ttyAMA0' \ > -serial stdio \ > -fsdev local,id=root,path=$dir,security_model=none \ > -device virtio-9p-device,fsdev=root,mount_tag=/dev/root \ > -monitor noneCan you give me some hints as to how to configure a kernel for this? I tried to get anything other than versatilepb working and gave up.> > $dir/bin/x86_64-linux-gnu/qemu-system-aarch64 \ > -M virt \ > -cpu cortex-a57 \ > -m 1024 \ > -kernel $dir/boot/aarch64-linux-gnu/Image \ > -append 'root=/dev/root init=/sbin/aarch64-linux-gnu/init rootfstype=9p > rootflags=trans=virtio,version=9p2000.L ro console=ttyAMA0' \ > -serial stdio \ > -fsdev local,id=root,path=$dir,security_model=none \ > -device virtio-9p-device,fsdev=root,mount_tag=/dev/root -monitor none >Ooh, thanks! I'll add support for this to virtme. --Andy
Seemingly Similar Threads
- [PATCH 0/3] virtio: Clean up scatterlists and use the DMA API
- [PATCH 0/3] virtio: Clean up scatterlists and use the DMA API
- [PATCH 0/3] virtio: Clean up scatterlists and use the DMA API
- Re: Broken OS when booting rootfs from 9p share
- [PATCH 0/3] virtio: Clean up scatterlists and use the DMA API