Hi, We plan to add a new vsock transport based on hypercall (e.g. vmcall on Intel CPUs). It transports AF_VSOCK packets between the guest and host, which is similar to virtio-vsock, vmci-vsock and hyperv-vsock. Compared to the above listed vsock transports which are designed for high performance, the main advantages of hypercall-vsock are: 1) It is VMM agnostic. For example, one guest working on hypercall-vsock can run on either KVM, Hyperv, or VMware. 2) It is simpler. It doesn't rely on any complex bus enumeration (e.g. virtio-pci based vsock device may need the whole implementation of PCI). An example usage is the communication between MigTD and host (Page 8 at https://static.sched.com/hosted_files/kvmforum2021/ef/TDX%20Live%20Migration_Wei%20Wang.pdf). MigTD communicates to host to assist the migration of the target (user) TD. MigTD is part of the TCB, so its implementation is expected to be as simple as possible (e.g. bare mental implementation without OS, no PCI driver support). Looking forward to your feedbacks. Thanks, Wei -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linuxfoundation.org/pipermail/virtualization/attachments/20211110/2fe44df6/attachment-0001.html>
On Wed, Nov 10, 2021 at 07:12:36AM +0000, Wang, Wei W wrote:> We plan to add a new vsock transport based on hypercall (e.g. vmcall on Intel CPUs). > It transports AF_VSOCK packets between the guest and host, which is similar to > virtio-vsock, vmci-vsock and hyperv-vsock. > > Compared to the above listed vsock transports which are designed for high performance, > the main advantages of hypercall-vsock are: > > 1) It is VMM agnostic. For example, one guest working on hypercall-vsock can run on > > either KVM, Hyperv, or VMware. > > 2) It is simpler. It doesn't rely on any complex bus enumeration > > (e.g. virtio-pci based vsock device may need the whole implementation of PCI). > > An example usage is the communication between MigTD and host (Page 8 at > https://static.sched.com/hosted_files/kvmforum2021/ef/TDX%20Live%20Migration_Wei%20Wang.pdf). > MigTD communicates to host to assist the migration of the target (user) TD. > MigTD is part of the TCB, so its implementation is expected to be as simple as possible > (e.g. bare mental implementation without OS, no PCI driver support).AF_VSOCK is designed to allow multiple transports, so why not. There is a cost to developing and maintaining a vsock transport though. I think Amazon Nitro enclaves use virtio-vsock and I've CCed Andra in case she has thoughts on the pros/cons and how to minimize the trusted computing base. If simplicity is the top priority then VIRTIO's MMIO transport without indirect descriptors and using the packed virtqueue layout reduces the size of the implementation: https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/virtio-v1.1-cs01.html#x1-1440002 Stefan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: <http://lists.linuxfoundation.org/pipermail/virtualization/attachments/20211110/a0784d63/attachment.sig>
On Wed, Nov 10, 2021 at 07:12:36AM +0000, Wang, Wei W wrote:> Hi, > > > > We plan to add a new vsock transport based on hypercall (e.g. vmcall on Intel > CPUs). > > It transports AF_VSOCK packets between the guest and host, which is similar to > > virtio-vsock, vmci-vsock and hyperv-vsock. > > > > Compared to the above listed vsock transports which are designed for high > performance, > > the main advantages of hypercall-vsock are: > > 1) It is VMM agnostic. For example, one guest working on hypercall-vsock > can run on > > either KVM, Hyperv, or VMware.hypercalls are fundamentally hypervisor dependent though. Assuming you can carve up a hypervisor independent hypercall, using it for something as mundane and specific as vsock for TDX seems like a huge overkill. For example, virtio could benefit from faster vmexits that hypercalls give you for signalling. How about a combination of virtio-mmio and hypercalls for fast-path signalling then?> 2) It is simpler. It doesn?t rely on any complex bus enumeration > > (e.g. virtio-pci based vsock device may need the whole implementation of PCI). >Next thing people will try to do is implement a bunch of other device on top of it. virtio used pci simply because everyone implements pci. And the reason for *that* is because implementing a basic pci bus is dead simple, whole of pci.c in qemu is <3000 LOC.> > An example usage is the communication between MigTD and host (Page 8 at > > https://static.sched.com/hosted_files/kvmforum2021/ef/ > TDX%20Live%20Migration_Wei%20Wang.pdf). > > MigTD communicates to host to assist the migration of the target (user) TD. > > MigTD is part of the TCB, so its implementation is expected to be as simple as > possible > > (e.g. bare mental implementation without OS, no PCI driver support). > >Try to list drawbacks? For example, passthrough for nested virt isn't possible unlike pci, neither are hardware implementations.> Looking forward to your feedbacks. > > > > Thanks, > > Wei >
On Wed, Nov 10, 2021 at 07:12:36AM +0000, Wang, Wei W wrote:>Hi, > >We plan to add a new vsock transport based on hypercall (e.g. vmcall on Intel CPUs). >It transports AF_VSOCK packets between the guest and host, which is similar to >virtio-vsock, vmci-vsock and hyperv-vsock. > >Compared to the above listed vsock transports which are designed for high performance, >the main advantages of hypercall-vsock are: > >1) It is VMM agnostic. For example, one guest working on hypercall-vsock can run on > >either KVM, Hyperv, or VMware. > >2) It is simpler. It doesn't rely on any complex bus enumeration > >(e.g. virtio-pci based vsock device may need the whole implementation of PCI). > >An example usage is the communication between MigTD and host (Page 8 at >https://static.sched.com/hosted_files/kvmforum2021/ef/TDX%20Live%20Migration_Wei%20Wang.pdf). >MigTD communicates to host to assist the migration of the target (user) >TD. >MigTD is part of the TCB, so its implementation is expected to be as simple as possible >(e.g. bare mental implementation without OS, no PCI driver support).Adding Andra and Sergio, because IIRC Firecracker and libkrun emulates virtio-vsock with virtio-mmio so the implementation should be simple and also not directly tied to a specific VMM. Maybe this fit for your use case too, in this way we don't have to maintain another driver. Thanks, Stefano